Table of contents

• Selective builds and test execution
• Dependency graph analysis
• Test splitting and parallelization
• Caching strategies
• Scalable pipeline design
• Managing PRs with Graphite
• Conclusion

Optimizing CI pipelines for monorepos is essential for keeping builds fast and scalable. This guide covers language-agnostic best practices that apply to any team structure, from early-stage startups to large engineering orgs.

Selective builds and test execution

In a monorepo, you shouldn't rebuild or retest the entire codebase for every change. Instead, set up your pipeline to only build and test what's impacted.

Most CI tools support path filtering. For example, in GitHub Actions:

on:
  push:
    paths: ['services/service-a/**']

You can go further by writing a small script that uses git diff to determine what has changed and which tests or builds to run. This approach is especially helpful if your CI tool has limited native path filtering support.

Selective builds reduce waste and speed up feedback, especially in larger repos with many independent modules or services.

Dependency graph analysis

Selective builds are even more powerful when paired with dependency graph analysis (see: How to implement CI/CD strategies for monorepos). Many tools (like Bazel, Nx, or Pants) support graph-based execution, where CI only rebuilds affected nodes and their dependents.

If you update a shared library, these tools automatically identify which apps or modules depend on it and trigger only those builds and tests. For example, changing a utility function might trigger builds for two services that depend on it, but nothing else.

Even without advanced build tools, you can script this yourself by modeling dependencies in a JSON or YAML file and using it to compute the affected components during each CI run.

Test splitting and parallelization

Monorepos often accumulate thousands of tests. Test splitting allows you to run them concurrently across multiple CI jobs, cutting total runtime dramatically (see this guide: Monorepo with GitHub Actions).

For example, in GitHub Actions:

strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: npm test -- --shard ${{ matrix.shard }}/4

This method divides the test suite across four workers. Some test frameworks (like Jest or PyTest) support sharding or test filtering out of the box.

You can also split by directory, module, or even runtime duration if your CI system supports historical timing data (e.g., CircleCI or Buildkite). The more parallelism you add (up to your infrastructure limits), the shorter your builds.

Caching strategies

Caching reduces repeated work between runs. You should cache:

• Dependency installs (e.g., node_modules, ~/.m2, ~/.cache)
• Build artifacts (e.g., compiled binaries)
• Intermediate steps in the pipeline

In GitHub Actions:

- uses: actions/cache@v3
  with:
    path: ~/.m2/repository
    key: maven-${{ hashFiles('**/pom.xml') }}

This restores Maven dependencies if the relevant files haven't changed.

For larger teams or more complex setups, Bazel and Nx offer remote build caches that can be shared across machines and users. This is especially effective in monorepos where multiple services may share common libraries.

A good rule of thumb: cache anything that takes longer to compute than it takes to download.

Scalable pipeline design

Monorepos require CI pipelines that scale with the codebase. Best practices include:

• Multiple workflows: Create separate workflows per app, service, or domain. Each workflow should have its own trigger conditions to avoid unnecessary runs.
• Job fan-out/fan-in: Split the pipeline into parallel jobs (fan-out), then merge results for a shared integration or deploy step (fan-in).
• Template reuse: Use YAML includes, anchors, or reusable CI commands/actions so that projects can share common logic without duplication.
• Dynamic configuration: Some CI tools support generating config at runtime. For example, a pre-job could analyze the diff and determine which jobs to run.

This approach keeps the pipeline lean and responsive, even as the number of projects grows.

Managing PRs with Graphite

Monorepos often involve large or multi-module changes, which complicates pull request (PR) workflows. Graphite helps by offering a stacked PR workflow, merge queue automation, and CI-aware optimizations.

• Stacked PRs: Developers can break large changes into smaller, dependent PRs. These stacks are easier to review and test. With Graphite, stacks are automatically rebased and tracked, reducing merge conflicts.
• Merge queue: Graphite’s merge queue keeps the main branch green by rebasing and retesting each PR before merging. This is especially helpful in fast-moving monorepos with many contributors.
• CI optimization: For stacked PRs, Graphite can detect overlap and avoid running the full test suite for each PR individually. Instead, it consolidates runs where possible, cutting redundant CI time.

These features help teams maintain fast CI cycles and high code quality even with a high volume of contributions.

Conclusion

Efficient CI for monorepos means running less, reusing more, and scaling with your codebase. Use selective execution, dependency graphs, test sharding, and caching to speed things up. Organize your pipeline config for maintainability, and consider tools like Graphite to streamline pull request workflows. With these strategies in place, your CI pipeline can remain fast and reliable—even as your monorepo grows.