Live streamJoin us for "State of code review 2024" on July 18thRegister today
Optimize CI (beta)
Learn CI optimizations & best practices for stacked pull requests.


Stacking leads to developers creating smaller easier-to-review pull requests, which can lead to more CI runs unless you optimize your CI for stacking. Organizations with fewer than 10 stackers are unlikely to see any difference in CI wait times or runs.

Additional CI runs occur when stacking PRs in part due to to the additional PRs created, and due to behind-the-scenes rebasing to keep stacked branches up to date.

To solve this, Graphite offers first-class integrations with Buildkite and GitHub Actions to let you customize which PRs in stacks you want to run CI on. See the blog post for additional details.

There are two ways to configure Graphite to optimize your Buildkite pipelines:


Note

In this Buildkite configuration, you create a new Graphite CI optimizer pipeline that runs before your repo's other pipeline(s). It has the advantage of explicitly showing PR authors that some of their CI did not run when the optimizer skips CI.


In Buildkite, create a new “Stack CI Optimizer” pipeline that runs before all other CI pipelines. This new pipeline determines if CI should run for this PR, and triggers the other pipeline(s) if so.

Getting started

  1. Create a new CI optimization in Graphite settings and copy the pipeline YAML

  2. In Buildkite, create a new pipeline for the same repo you configured in the previous step

  3. Paste the YAML copied from Graphite into the Buildkite pipeline configuration UI or into your repo's .buildkite/ directory as a new pipeline. Remember to update the trigger step in the pasted YAML so the CI optimizer pipeline can call your own pipeline after it decides whether to optimize CI for your PR.

  4. In your own pipeline settings under GitHub > GitHub Settings, check the box Skip builds with existing commits. This ensures the Graphite optimizer pipeline runs first and conditionally triggers your existing pipelines.

Note: you can test Buildkite pipeline changes in branches/PRs before merging them into your main trunk branch, so you can verify the optimizer is configured correctly before enabling it for your repo.


Note

In this Buildkite configuration, you add a job to the start of your pipeline that your others wait for. It has the disadvantage of showing the overall pipeline status as green on GitHub, even when the CI optimizer decides to skip tests. The Buildkite and Graphite UIs show the accurate skip statuses.


Getting started

  1. Create a new CI optimization in Graphite settings and copy the pipeline YAML

  2. Add the following YAML to the beginning of your repo's pipeline(s), including the wait step (pipelines are typically stored in .buildkite/). Replace graphite_token with the token from the first step.

yaml
steps:
- name: ":graphite: Graphite CI optimizer"
soft_fail: true
plugins:
withgraphite/graphite-ci#main:
graphite_token: "xxxxxxxxxxxxxxxxxxxxxxx"
- wait
# the rest of your jobs in the pipeline
- label: "Your first job to run after the optimizer"
command: echo "hello"

Note: You should securely pass your token to the Graphite plugin instead of storing it in your pipeline configuration.

  1. Create a new CI optimization in Graphite settings and copy the pipeline YAML

  2. Add the following to your GitHub Actions workflows (typically stored in .github/workflows/). Replace graphite_token with the token from the first step.

yaml
jobs:
optimize_ci:
runs-on: ubuntu-latest # or whichever runner you use for your CI
outputs:
skip: ${{ steps.check_skip.outputs.skip }}
steps:
- name: Optimize CI
id: check_skip
uses: withgraphite/graphite-ci-action@main
with:
graphite_token: ${{ secrets.GRAPHITE_CI_OPTIMIZER_TOKEN }}
your_first_job:
...
your_second_job:
...

Then for each job in the workflow you want to optimize, add the following YAML:

yaml
job_name:
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
...

This ensures the optimized jobs only run when the CI optimizer gives them the signal.

Graphite's Buildkite and GitHub Actions integrations are configured to "fail open" so that outages and errors still result in your CI running.

Google recommends breaking up your tests from one CI job into many which run at different points. One recommended split is:

  • CI that runs on all PRs

  • CI that runs on PRs, excluding upstack

  • CI that runs after PRs merge to main

In GitHub actions, and other providers, you can do this by creating multiple workflows and setting different triggers for them.

If your organization doesn’t already have one set up, a dependency management tool (such as Bazel or Buck) can be really helpful. These tools look at what code changed, and determine which tests need to run as a result. This prevents unnecessary CI runs with stacking by skipping any tests that were unaffected.

In a similar vein, workflow orchestration tools with caching can create a similar effect. For example, Turborepo caches CI results based on the hash of the project’s files. Unlike Bazel and Buck, all tests will still run, but if the input files were unchanged across a stack, the test will hit the cache and make the cost negligible.

If your organization has branch protection rules turned on, and you are not running CI on upstack PRs, you may see upstack PR as “missing required CI”. This is because the CI job that is required has not yet run on that PR (because it’s waiting for dependencies to be merged).

Lastly, the Graphite Merge Queue can help you save on CI cost when merging. The merge queue allows various configurations that help reduce the number of CI runs:

  1. Batching: CI will run just once per batch size stacks, where batch size can be configured in our UI. This results in a saving of batch size * stack height CI runs.

  2. Parallel CI: Users can configure CI to run just once per stack, allowing users to save stack height CI runs.

This is can be very useful for organizations that merge a lot of stacks.

Give your PR workflow
an upgrade today

Stack easier | Ship smaller | Review quicker

Or install our CLI.
Product Screenshot 1
Product Screenshot 2