Optimize for optionality and build towards checkpoints

In a project management-themed Hackers Incorporated episode, Adam Wathan introduced derisking projects with save points. The entire episode is definitely worth your time, but that specific piece of advice has changed the way I work as a developer and make decisions as a project manager.

In practice, it has taught me to optimize for optionality, not efficiency.

The monolithic branch

To illustrate what it means, we'll build a user account section as an example. The new account section will be shipped to users as one big launch. A user can…

Update their basic profile information
Add their address with Google Maps autocomplete
Upload a profile picture
Connect social login accounts
Reset their password

Let's make an estimate:

Task	Estimate
Basic information	6h
Address autocomplete	12h
Profile picture	4h
Social login	8h
Reset password	4h

Summing up: 6 + 12 + 4 + 4 + 8 = 34, that's a 40-hour estimate. (Gotta add that margin!) We'll checkout a new feature/user-account branch, implement, and ship. The final logs in our time tracker:

Task	Estimate	Actual
Basic information	6h	3h
Address autocomplete	12h	14h
Profile picture	4h	6h
Social login	8h	18h
Reset password	4h	2h

3 + 14 + 6 + 18 + 2 = 43. Not too far off of our (padded) estimate! Some tasks were easier to implement than expected, some took a lot longer. But we didn't ship 43 working hours later, it took 4 weeks to reach production. Why?

We realized our application needed to be approved by our social login provider, that took a week. After that it took a lot longer than expected to implement.
When we started to implement the new reset password flow we found a few issues in the design, so the design team had to update the Figma file. The implementation went smoother than expected, but we had to wait for the changes.
Of course, the usual bugs and small feature requests that creep into our schedule pushed this project further down the road.

That users had to wait longer than estimated for the new account feature isn't a problem. This post isn't a manifesto to ship incremental changes over big bang launches—I'll leave that decision to the product managers. Shoving everything into one branch is plain easy. Because we weren't going to ship anything individually, we were allowed to! What I care about, is how it affects everything else. Because meanwhile…

Another developer needed an address autocompletion on the shipping page. Now we have two implementations and need to trash one.
Someone else refactored the user model and related code, causing a bunch of merge conflicts throughout the 4 weeks of development.
Another timeline-like feature design relies on profile pictures, but we wanted to wait until this branch was merged so we had access to the profile picture components that are already set up.

Checkpoints

Let's treat each task as an individual project. It will probably take longer—at least on our timesheet. We need to create a separate branch for every task and can't implement changes in bulk. Instead of one big feature/user-account merge, we created, reviewed, and merged a bunch of smaller branches over the weeks.

feature/user-basic-information-views
refactor/share-google-maps-api-credentials
feature/google-maps-autocomplete
feature/profile-picture
refactor/user-authentication-changes
feature/social-login
fix/profile-picture-upload-bug
feature/reset-password-update
feature/reset-password-update-2

But what have we gained from this approach?

If the address autocompletion was merged earlier, we wouldn't have ended up with two implementations. Even if the first wasn't shipped to production yet, it would have been a usable component in the codebase.
We wouldn't have spent as much time on merge conflicts because frequent merges keep them small or non-existent.
We could have decided postpone social login if we knew it was such a big investment. We didn't because it was too entangled in the monolithic branch.

Working towards mergeable chunks doesn't only give the developer more optionality, it makes the entire organization more flexible.

Hindsight is 20/20. It could have gone differently: maybe working in the dark for a prolonged time didn't affect anyone. But you never know this ahead of time. Priorities change, and you don't have control of the external factors that force them to. If you were halfway the monolithic user account branch and a critical bug or competitor forces your hand, all of your code is held hostage until you get back to it. Even if you don't release it to your users, it's more valuable to have in your main branch.

When code is in a feature branch it doesn't contribute to the rest of the codebase. This 40-hour project is an innocent example. We often fall into the same trap for 250-hour projects. That's a month or two of work—a month or two of work in progress code dilly-dallying. Code rots over time. Work in progress code rots significantly faster.

To paraphrase Eliyahu M. Goldratt (I'm a sucker for the theory of constraints): large amounts of work in progress masks inefficiencies and bottlenecks in the production process. Reducing work in progress can improve cash flow. As work in progress is converted to finished goods and sold more quickly, it accelerates the cash conversion cycle. In our language: code only becomes an asset after it's merged.

Action items

To make this plea actionable: treat each chunk of work as something that should be merged by the end of the week. That doesn't mean it needs to be "done" or available to the end user, it needs to become a citizen of The Codebase.

The hard part is finding your checkpoints. Identify the critical path. What can you strip from a feature while keeping it useful? Do that last. take a step back every few hours, ask yourself what the least amount of work would be to make what you're doing mergeable. Have a bug-free staging environment from day one and keep it that way to ensure frequent merges don't affect quality (read about the broken window theory).

It takes a while to get used to, and will feel uncomfortable at first. But do this enough, and you'll see checkpoints all over the place. Working in small chunks means individual tasks may take longer. But in the long term you and the team as a whole will see gains in flexibility, optionality, and efficiency. These benefits vastly outweigh the time you would win from a bulk discount.

Some desert wisdom to close:

Arrakis teaches the attitude of the knife—chopping off what's incomplete and saying: "Now, it's complete because it's ended here."