Self-service management of GitHub Repositories at Coop Norge SA

Coop Norge SA believes in using infrastructure as code to manage cloud resources. This means storing everything in git repositories, which gives engineers a structured view of their resources, an audit log through commit history, and the ability to automate validation from the start and ongoing. This also includes how GitHub repositories ⧉ are created.

The old process

Engineers used to create repositories using Terraform, managed by the platform team, with the following setup:

Terraform

module "helloworld_github_repo" {
  source     = "spacelift.io/coopnorge/repos/github"
  version    = "6.5.0"
  teams_data = module.teams.teams_data

  name = "helloworld"

  allow_squash_merge = true
  allow_merge_commit = false
  allow_rebase_merge = false

  require_code_owner_reviews      = true
  required_approving_review_count = 1
  required_linear_history         = true

  status_checks = [
    "build",
  ]

  has_issues   = true
  has_projects = false
  has_wiki     = false

  teams = {
    "team-a" = {
      permission = "push"
    }
  }
}

This was great for keeping track of changes, validating setups, and setting good defaults. However, as more people started using GitHub, it became clear that the platform team couldn't keep up with approving repository creation PRs. They ended up rubber-stamping them, causing delays and errors. The team needed a better solution.

Addressing the challenges

The platform team wanted to address the following questions:

• Who has the right context to create repositories?
• Should repository creation be gated?
• How to ensure users stay within their privilege boundaries?
• Who can approve changes to repository settings?
• How to maintain an audit log?
• How to ensure good defaults?
• Should Terraform be used?

The team quickly realized that repository creation shouldn't be gated. The teams owning the repositories should approve changes. The platform team also wanted to keep the audit log.

The API approach

To improve user experience and maintain safe boundaries, the platform team moved to an API-based approach. The team created a simple API for defining repository configurations, limiting the configuration options to ensure correct boundaries. The team used Kubernetes Custom Resource Definitions (CRDs) for the APIs. Reconciliation in Kubernetes ensures that the repository configuration is always in sync. Also when a engineer updates the configuration it is immediately synced. Combining this with GitOps provides Git traceability. The platform team chose Crossplane, which manages infrastructure like Terraform but operates within Kubernetes and uses reconciliation.

The GitHub provider

To configure resources with Crossplane, a provider is needed. Since no GitHub provider was available, the platform team developed their own, which is open source and available on https://github.com/crossplane-contrib/provider-upjet-github ⧉

The initial user experience

Coop was already using Backstage ⧉ for team documentation. Backstage's Software Template System to create and configure GitHub repositories. Backstage sets up new repositories and adds the configuration file for Kubernetes to manage. This way, repository owners maintain configurations without support from the platform team.

The new experience

Now, creating a new repository is easy:

1. Go to the internal developer portal Inventory, based on Backstage ⧉.
2. Click create
3. Select the type of repository they need
4. Fill out some minor details
5. Hit create
6. Done

Backstage creates the repository and adds the necessary file for Crossplane to manage:

repoconfig.yaml

apiVersion: github.coop.no/v1alpha1
kind: RepositoryConfig
metadata:
  name: repository-config
spec:
  description: "Hello, World - a sample repo"

This setup enables branch protection by default, suiting most repositories at Coop Norge SA. Specific options are available through the API. This method retains the audit log and good defaults, while repository owners control settings changes. Good defaults remain enforced.

Here is a graphical overview of the process:

Next steps

The platform team is expanding on this idea to offer complete self-service control over other essential resources in a similar fashion. Here are some of the potential capabilities:

• Rapid creation and deployment of a basic app to production within minutes.
• Setting up library repositories with comprehensive CI integration.

The objective is to empower software engineers to swiftly begin iterating on creating value, rather than investing time in initial setup processes.

Notes

A Special thanks to Harald Skoglund ⧉

At October 7, 2024 the provider-github project as been donated to the crossplane-contrib organization. This is great news.