Skip to content

perf(ci): parallelize monorepo build via mise DAG dependencies#203

Open
scottschreckengaust wants to merge 3 commits into
mainfrom
fix/201-mise-parallel
Open

perf(ci): parallelize monorepo build via mise DAG dependencies#203
scottschreckengaust wants to merge 3 commits into
mainfrom
fix/201-mise-parallel

Conversation

@scottschreckengaust
Copy link
Copy Markdown
Contributor

@scottschreckengaust scottschreckengaust commented May 27, 2026

Summary

Restructures the monorepo build from sequential execution to parallel DAG-based execution using mise's native dependency resolution. Also adds dynamic runner selection via repo variable or PR label.

Changes (2 files + build.yml runner config)

Root mise.toml:

  • Build task changes from sequential run = [cdk, cli, docs] to parallel depends = [agent, cdk, cli, docs]
  • Removes redundant //docs:sync (already a dep of //docs:build)

cdk/mise.toml:

  • test now declares depends = [":compile"] — waits for tsc before running Jest
  • synth:quiet now declares depends = [":compile"] — waits for tsc before synthesizing
  • eslint has no compile dependency — starts immediately (operates on .ts source)
  • build depends on [":test", ":eslint", ":synth:quiet"] — compile is pulled in transitively

build.yml runner selection:

# Priority: vars.DEFAULT_RUNNER_LABEL > PR label 'self-hosted' > PR label 'ubuntu-latest-4-cores' > 'ubuntu-latest'
runs-on: >-
  ${{ vars.DEFAULT_RUNNER_LABEL || ... || 'ubuntu-latest' }}

Resulting DAG

//:build
├── //agent:quality              (starts immediately)
├── //cdk:build
│   ├── //cdk:compile            (starts immediately)
│   ├── //cdk:eslint             (starts immediately)
│   ├── //cdk:test → compile     (waits for compile)
│   └── //cdk:synth:quiet → compile  (waits for compile)
├── //cli:build                  (starts immediately)
└── //docs:build → docs:sync     (starts immediately)

Also fixes

  • Latent race condition: Previously test and synth ran in parallel with compile (no ordering guarantee). Now they explicitly depend on compile completing first.
  • Redundant docs:sync: Was called twice. Removed duplicate.

Test plan

  • mise tasks deps build shows correct DAG (verified locally)
  • mise run build --dry-run shows parallel execution
  • CI: mise run build passes (all tests, synth, lint, docs green)
  • ~~Verify build time drops from ~12min to ~3-4min in CI~~ — Cannot verify: ubuntu-latest has only 2 vCPU, so parallel tasks contend for CPU rather than running concurrently. The DAG is correct but the speedup requires larger runners (4+ cores). See dynamic runner selection above — set vars.DEFAULT_RUNNER_LABEL or add PR label ubuntu-latest-4-cores when available.
  • Verify mise //cdk:test still works standalone — confirmed via successful CI build (DAG pulls in compile first)
  • Verify mise //cdk:synth still works standalone — confirmed via successful CI build (DAG pulls in compile first)

Refs #201

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

Restructures task execution from sequential to parallel:

Root mise.toml: changes build from sequential `run = [...]` to
parallel `depends = [agent, cdk, cli, docs]`. All packages now
build concurrently.

cdk/mise.toml: adds `depends = [":compile"]` to test and synth:quiet
so they wait for TypeScript compilation, while eslint runs immediately.
Removes compile from build's depends (it's now pulled in transitively).

DAG execution order:
  t=0:  agent:quality + cdk:compile + cdk:eslint + cli:build + docs:build
  t=35: cdk:test + cdk:synth (after compile finishes)
  t=185: all done

Critical path: cdk:compile (35s) → cdk:synth (150s) = ~3min
vs. previous serial: ~12min

Also removes redundant //docs:sync (already a dependency of //docs:build).

Refs #201

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Probes whether larger runners are available to the org. If the job
queues and runs, mise parallelism should actually benefit from 4
real cores vs 2. If it fails, we know larger runners aren't enabled.

Adds nproc/RAM logging and SECONDS timing to measure the difference.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant