CI Optimizations

Stacking leads to developers creating smaller easier-to-review pull requests, which can lead to more CI runs unless you optimize your CI for stacking. Organizations with fewer than 10 stackers are unlikely to see any difference in CI wait times or runs.

Additional CI runs occur when stacking PRs in part due to the additional PRs created, and due to behind-the-scenes rebasing to keep stacked branches up to date.

To solve this, Graphite offers an API endpoint for your CI workflows to query, allowing you to customize which PRs in stacks you want to run CI on.

​

How CI Optimizations work

CI Optimizations is set up per-repository, and requires a very small amount of configuration, namely:

• How many PRs at the bottom of each stack should run CI?

• Should CI be run at the top of the stack?

In addition to this configuration, you’ll add a step to your CI that calls the Graphite Buildkite/GitHub actions step. These CI steps are open-source, so you know exactly what is running in your CI pipeline. The details on adding these steps can be found below.

Each step is a wrapper script for a very simple API call that responds with a boolean: whether CI should be skipped. You don’t need to worry about handling the API response manually or querying any additional data about the PR/stack from GitHub to decide whether to run CI – just add the step to the beginning of your configuration.

• If the request to our API is malformed or errors for any reason, we will not skip CI

• If CI Optimizations have been disabled, we will not skip CI

• If the PR is in a merge queue or merging as a stack with Graphite, we will not skip CI

​

How to set up CI Optimizations

To begin setting up CI Optimizations, click the “Add new” button on its settings page on the Graphite dashboard.

You’ll see a panel as in the below screenshot, which will guide you through adding the step for your CI provider, and configuring when CI should run.

Once you’re done, a card for your repository will appear on the settings page. You can edit the settings at any time.

​

Setting up the Buildkite step

There are two ways to configure Graphite to optimize your Buildkite pipelines:

​

Option 1: Graphite pipeline runs first (recommended)

In Buildkite, create a new “Stack CI Optimizer” pipeline that runs before all other CI pipelines. This new pipeline determines if CI should run for this PR, and triggers the other pipeline(s) if so.

Getting started

1. Create a new CI optimization in Graphite settings and copy the pipeline YAML

2. In Buildkite, create a new pipeline for the same repo you configured in the previous step

3. Paste the YAML copied from Graphite into the Buildkite pipeline configuration UI or into your repo’s .buildkite/ directory as a new pipeline. Remember to update the trigger step in the pasted YAML so the CI optimizer pipeline can call your own pipeline after it decides whether to optimize CI for your PR.

4. In your own pipeline settings under GitHub > GitHub Settings, check the box Skip builds with existing commits. This ensures the Graphite optimizer pipeline runs first and conditionally triggers your existing pipelines.

Note: you can test Buildkite pipeline changes in branches/PRs before merging them into your main trunk branch, so you can verify the optimizer is configured correctly before enabling it for your repo.

​

Option 2: Graphite job runs first

Getting started

1. Create a new CI optimization in Graphite settings and copy the pipeline YAML

2. Add the following YAML to the beginning of your repo’s pipeline(s), including the wait step (pipelines are typically stored in .buildkite/). Replace graphite_token with the token from the first step.

steps:
  - name: ":graphite: Graphite CI optimizer"
    soft_fail: true
    plugins:
      withgraphite/graphite-ci#main:
        graphite_token: "xxxxxxxxxxxxxxxxxxxxxxx"


  - wait

  # the rest of your jobs in the pipeline
  - label: "Your first job to run after the optimizer"
    command: echo "hello"

Note: You should securely pass your token to the Graphite plugin instead of storing it in your pipeline configuration.

​

Setting up the GitHub Actions step

1. Create a new CI optimization in Graphite settings and copy the pipeline YAML

2. Add the following to your GitHub Actions workflows (typically stored in .github/workflows/). Replace graphite_token with the token from the first step.

jobs:
  optimize_ci:
    runs-on: ubuntu-latest # or whichever runner you use for your CI
    outputs:
      skip: ${{ steps.check_skip.outputs.skip }}
    steps:
      - name: Optimize CI
        id: check_skip
        uses: withgraphite/graphite-ci-action@main
        with:
          graphite_token: ${{ secrets.GRAPHITE_CI_OPTIMIZER_TOKEN }}

  your_first_job:
    ...

  your_second_job:
    ...

Then for each job in the workflow you want to optimize, add the following YAML:

job_name:
  needs: optimize_ci
  if: needs.optimize_ci.outputs.skip == 'false'
  ...

This ensures the optimized jobs only run when the CI optimizer gives them the signal.

​

Error handling

Graphite’s Buildkite and GitHub Actions integrations are configured to “fail open” so that outages and errors still result in your CI running.

​

Other ways to optimize CI

​

Breaking up CI

Google recommends breaking up your tests from one CI job into many which run at different points. One recommended split is:

• CI that runs on all PRs

• CI that runs on PRs, excluding upstack

• CI that runs after PRs merge to main

In GitHub actions, and other providers, you can do this by creating multiple workflows and setting different triggers for them.

​

Dependencies and test caching

If your organization doesn’t already have one set up, a dependency management tool (such as Bazel or Buck) can be really helpful. These tools look at what code changed, and determine which tests need to run as a result. This prevents unnecessary CI runs with stacking by skipping any tests that were unaffected.

In a similar vein, workflow orchestration tools with caching can create a similar effect. For example, Turborepo caches CI results based on the hash of the project’s files. Unlike Bazel and Buck, all tests will still run, but if the input files were unchanged across a stack, the test will hit the cache and make the cost negligible.

​

Required CI

If your organization has branch protection rules turned on for all base branches (instead of only trunk, for example), and you are not running CI on upstack PRs, you may see upstack PR as “missing required CI”. This is because the CI job that is required has not yet run on that PR (because it’s waiting for dependencies to be merged).

​

Reducing CI runs with the Graphite Merge Queue

Lastly, the Graphite Merge Queue can help you save on CI cost when merging. The merge queue allows various configurations that help reduce the number of CI runs:

1. Batching: CI will run just once per batch size stacks, where batch size can be configured in our UI. This results in a saving of batch size * stack height CI runs.

2. Parallel CI: Users can configure CI to run just once per stack, allowing users to save stack height CI runs.

This can be very useful for organizations that merge a lot of stacks.