Building Graphite with Graphite

At Graphite, we fervently believe in dogfooding - using our own tool to build our product. It's essential we experience the pain points firsthand, and it’s satisfying solving our own problems.

Recently, I refactored our 404 error handling to use standardized logging and responses. In the past, this sort of project would have led to at least one monolithic, dangerous pull request. Using the Graphite workflow, I was able to create a continuous stream of safer PRs while avoiding ever getting blocked. We've been seeing a lot of feedback, questions and requests asking for more clarification on day-to-day Graphite workflow and usage, so hopefully this helps!

The initial problem space

While working on our Express endpoint handlers, I noticed our 404 error handling was inconsistent across endpoints. Each developer manually checked for auth permissions, resulting in differing implementations. Our responses lacked consistency on 400, 401, 404s, and sometimes even 500s - not to mention consistent internal server logging around access failures.

Note: I have trimmed down some of the code details for the sake of brevity.

// Old pattern
export const setMacroEndpoint = authedEndpoint(
  mutations.setMacro,
  async (user, requestContext, req, res) => {
    if (
      !(await requestContext.authedClients.github.query.getOrgMembership({
        org: req.body.org,
      }))
    ) {
      return res.status(404).end();
    }
    ...

It was clear we needed to standardize our 404 handling with

• Consistent 404 status codes

• Standardized logging for debugging

• Simple helper functions to avoid repetitive code and improve dev-ex

The goal? Create a standardized and flexible pattern for authentication.

Creating the core components

First, I wrote an Authorization helper class to simplify common permissions checks.

export class Authorization {
  ...
  public async ensureCanAccessRepo(repo: {
    name: string;
    org: string;
  }): Promise<void>
  public async ensureCanAccessOrg({ org }: { org: string })
  private authOrRaiseError(isAccessible: boolean)
}

Pre-Graphite Greg would have continued coding. But I appreciate the value of small changes, and now was a valid checkpoint where I could create my first branch.

$ gt branch create --all -m "feat(auth): add initial helper class"

Next, I created a second change introducing error-handling middleware:

// New middleware for consistent response status codes
export const authorizationErrorMiddleware: ErrorRequestHandler = (
  err: Error,
  req: express.Request,
  res: express.Response,
  next: Function
) => {
  const requestContext = getRequestContext(res);
  const logger = requestContext.splog;
  if (err instanceof AuthorizationError) {
    splog.error({
      message: "Authorization error",
      tags: {
        path: req.path,
      },
      err,
    });

    if (!res.headersSent) {
      res.status(404).send();
    }
  }
  next(err);
};

And created another branch:

$ gt branch create --all -m "feat(auth): create auth error middleware"

Lastly, I created a third branch composing an Authorization class instance into our per-request context object, and adding the middleware to the server. Essentially, a change that wired up my two new components.

$ gt branch create --all -m "feat(auth): connect helper and middleware"

At this point, I had the core components - a helper class, middleware - coded and integrated as the first three PRs in my stack. The small standalone changes were easy to review and provided an opportunity for early feedback.

I submitted all three branches for review with one command:

$ gt stack submit

These three changes built off one another. The Authorization class introduced a custom error that was needed to implement the middleware. Both needed to exist before I could integrate them into the existing server initialization. By stacking, I avoided creating a single large PR, thereby increasing the speed at which I’d get reviewed and decreasing the likelihood of a bug getting introduced.

Applying middleware to an endpoint

With the foundation in place, it was time to start migrating endpoints. Without stacking, I would need to wait until my previous changes were reviewed and merged. With stacking, however, I was free to keep coding.

// Old endpoint pattern
export const setMacroEndpoint = authedEndpoint(
  mutations.setMacro,
  async (user, requestContext, req, res) => {
    if (
      !(await requestContext.authedClients.github.query.getOrgMembership({
        org: req.body.org,
      }))
    ) {
      return res.status(404).end();
    }
    ...

Our setMacro endpoint for removing PRs was a great candidate. It already checked for 404 conditions like PR not existing or user lacking access - it just needed standardization. I updated it to use the new Authorize middleware.

// New pattern
export const setMacroEndpoint = authedEndpoint(
  mutations.setMacro,
  async (user, requestContext, req, res) => {
    // Use a composed `auth` object to standardize checks.
    await requestContext.auth.ensureCanAccessOrg({ org: req.body.org });
    ...

Before turning myself into a refactoring machine, I wanted to put this PR up as early as possible in case folks had feedback on the dev-ex.

$ gt bc -am "feat(auth): add pattern to first endpoint"
$ gt ss

At this point, my stack was four PRs deep - all up for review and now passing CI. Time for lunch.

Incorporating feedback

By the time I returned from lunch, the first two PRs in my stack had been reviewed, and feedback received :

• Add more info to debug logging

• Feature flag areas we throw errors so we can toggle it off if something unexpected occurs during the rollout.

• Ensure we correctly handle access to public repos, not just private repos.

With raw git, editing downstack branches would have destroyed me. I'd have to update my existing commits, perform multiple manual three-point rebases, and carefully force-push each outstanding PR. But Graphite's stacking helped me incorporate changes to the bottom of the stack with ease.

# Navigate to the first PR in the stack
$ gt stack bottom
# Update with feedback, and stage changes
$ gt add .
# Ammend my commit because I prefer single-commit branches
# gt automatically handles rebasing up the stack
$ gt commit amend
# Resubmit the stack, causing all four PRs to update with my changes
$ gt stack submit

With the feedback incorporated, I marked the outstanding comments on the Graphite website as “resolved” and re-requested review.

At this point, I still haven't merged any changes. Without stacking, I’d definitely be blocked.

Refactoring more endpoints

While my bottom-three PRs needed re-review, the fourth PR containing my first migrated endpoint had been approved. With the general go-ahead on the pattern, I was ready to start migrating more endpoints.

I checked out the top of my stack and started refactoring, updating about three endpoints at a time to keep my PRs small.

$ gt stack top
# Update three endpoints
$ gt bc -am "feat(auth): add checks to notif endpoints"
# Update three endpoints
$ gt bc -am "feat(auth): add checks to insights endpoints"
$ gt stack submit

Merging in a partial stack

By the time I had opened two more stacked pull requests, my bottom four changes had finally been approved.

The top of my stack was still running CI and awaiting feedback, but I had an afternoon with no meetings, and it felt like a great time to try merging. Using the Graphite website, I navigated to the fourth PR in my stack and enqueued four changes to our merge queue.

Our queue would re-run CI, merge the PRs into trunk, deploy to staging, and then eventually promote to prod.

I wanted to stay alert as the deployment rolled out in case anything went wrong, but I also now had 30-60 minutes to kill. I returned to my stack, working on a 6th pull request migrating more endpoints.

Eventually, I received a Slack notification letting me know the bottom four pull requests had been successfully merged. I then ran a sync command to delete the branches that had now been merged and restack everything else because I’m a bit of a clean freak.

$ gt bc -am "feat(auth): refactor another three endpoints"
$ gt repo sync
# Graphite deletes my bottom three PRs, restacking the remaining onto main
$ gt stack submit

Lastly, I resubmitted my automatically-rebased stack, to ensure that everything looked as fresh as possible to reviewers. I now had four PRs merged, and three more outstanding PRs.

At this point, I went home for the day. This refactor would continue on for likely a week of my time, and would result in over twenty small PRs by the time I finished.

Reflecting on workflow parallelization

By leveraging Graphite's stacking capabilities, I turned a monumental refactor into a continuous stream of incremental changes. I’m was formerly an engineer at Airbnb using vanilla GitHub. Having switched to a stacking workflow, the improvements are palpable:

• Small PRs enabled bite-sized reviews

• Stacking allowed constant development and feedback

• Merging lower PRs let me deploy piecewise while continuing to code

• Syncing and restacking kept my stack tidy and rebased

The true power of stacking is the parallelization is affords. I was writing code, merging code, and having code reviewed all at the same time. And my peers had the pleasure of only needing to review small PRs rather than one refactor with an obscure mix of risk and boilerplate. Hopefully, my anecdote helps inspire you to think about how you could break up your work into a continuous stream of development and achieve a new level of velocity!