Setting up code contribution workflows is one of the very first steps for many software engineering organizations / departments. With proper setup, effective collaboration and integration can be enabled, along with traceability to what has been delivered. Here I’d like to share my own insights on this topic. Please note that the terminologies used in this article are based on GitHub/GitHub Enterprise, while you should be able to find equivalents in other code hosting / collaboration platforms.
Choose a Git-based Platform
Git has become the most widely adopted source code management system in the last decade, and there are a number of free, paid, SaaS and on-premises options on the market. Make the choice based on your budget and operational model —— a free platform usually requires self-hosting (meaning you need an SRE or TechOps team), while choosing a SaaS platform would require careful consideration of code security.
Once the platform is in place, onboard your existing code repository to prepare for the next steps.
The Collaboration Modes for Repositories
The second thing is to set up a collaboration mode for repositories. If you take GitHub as an example, there are two modes: Branch-and-Pull and Fork-and-Pull.
No matter which model you choose, it is recommended that you always do the following:
- Submit code changes via Pull Requests so that automated gating and manual reviews can be performed.
- Implement Continuous Integration with small commits and frequent merges, avoiding long-lived feature branches as possible.
The following sections provide a quick comparison between these two modes.
Branch-and-Pull
The branch-and-pull model is based on different branches of a same repository, which is common in the enterprise.
Advantages
- Easy to collaborate on a feature branch.
Disadvantages
- Can result in long-lived feature branches, which would be a challenge to manage repositories if retention is not implemented. I once cleaned up a monolithic code repository with more than 3,000 feature branches and suffered a lot.
- Repository permissions should be fine-grained to protect key branches (main, dev, release, prod, etc.) from accidental pushes.
Fork-and-Pull
The fork-and-pull model is quite common in open source communities. Contributors must create a clone of the resource repository in their own namespace before coding.
Advantages
- Better protection of the upstream source repository (no need to give contributors write access to the repository).
Disadvantages
- It is not convenient for people to work on the same feature branch.
- Increased resource consumption if you host the platform yourself.
The Branching Model
There are a number of models you can find in the public domain, including Git-Flow, GitHub Flow, Trunk-based Development, and many others.
Git-Flow covers a variety of scenarios, from enterprise application development to Web/SaaS-type applications, and is suitable for large teams that need to work on multiple releases (via release branches) and/or multiple environments simultaneously. On the other hand, its adaptability also brings complexity, making it overkill for smaller projects.
GitHub Flow can be considered as a simplified version of Git-Flow with a main branch and a feature branch, suitable for small teams and Web/SaaS-type development. If you need to support multiple releases simultaneously, you need to extend this model with release branches and cherrypicks.
Trunk-based Flow is a more “extreme” version of Git-Flow, strictly requiring small and frequent commits into a single main branch. Implementing this model requires engineering teams to build stronger delivery capabilities, including Continuous Integration, Continuous Delivery, and architectural capabilities like feature toggles to fine control the blast radius of system functionality.
It is important to tailor the models to your own needs, based on the characteristics of your business (delivering infrastructure software is usually different from hosting an e-business site) and your teams. Keep in mind that the technical and management difficulty of branching cannot be underestimated. The fewer branches (or branches that are active at the same time) the better —— if deliveries can be enabled and collaborated with two active branches, never introduce a third one. Always consider frequent integration and short-lived feature branches.
When I see misuse or abuse of branching, I often see tactical diligence and strategic laziness behind the scenes, which in the long run will cause irreversible damage to the code and the system.
The Traceability
Traceability tries to answer the following questions:
- People should be able to tell what code is shipped with specific product builds, using information such as commit SHAs.
- People should be able to tell if certain features and/or defect fixes are shipped with certain product builds.
Here are the short answers.
The Versioning
Versioning is another key part of your delivery model that should be universally defined and communicated. Semantic versioning is a perfect reference to start with at the repository level, applied to git tags to version your code.
Going further, if your product consists of multiple repositories or services, some mechanism should be put in place to link the git tags to product versions, as they may follow different schemes. It is not uncommon to see product versions such as 01.02.01.03 or 2022H2. In order to associate 2022H2 with these x.y.z component versions, you will need to generate and persist appropriate information from the build systems for future queries, so that the git tag of any component can be traced nack to a speicific product version.
Linking Code to Requirements
The final piece is to link the code to the project management system (like JIRA) via the commit/pull request template. An easy way to do this is to extend Conventional Commit with automated checks to ensure the linkage between commits and corresponding story/defect numbers from the project management system. The example below shows a feature commit / pull request that adds a new button to the home page is to deliver story JIRA-123:
feat(ui): [JIRA-123] add a new button on the home page
Summary
Adopting a code collaboration platform to manage your software delivery is essential. Once the infrastructure is in place, you need to consider establishing a set of standardized and easy-to-follow rules, with automated tasks/checks to ensure strong compliance from each development team, including:
- How to collaborate on repositories.
- How to use branches.
- Defining the tags and versions.
- Linking code to requirements.
While this is quite ideal, in the real world, especially if you are managing the software delivery for an enterprise, you need to think and answer: Does the code collaboration process need to be strictly uniform across the board, or can different teams and/or projects make their own choices? What flexibility do you want to provide?
This is the challenging and interesting part of real-world DevOps, which I will try to explore in more detail with my own experiences in each area. Stay tuned.