Go is designed to make concurrency easier than it is in other languages, but at Sourcegraph, I still found myself running into the same problems repeatedly when writing concurrent code.

In particular, it is difficult to write concurrent code that operates reasonably in the face of panics. We don’t want the entire process to crash when a panic occurs in a spawned goroutine, and we want to avoid other problems that can be triggered by panics like deadlocks or leaked goroutines. Go does not provide an easy way to do this natively.

So I built conc, a library that makes writing concurrent code more elegant and reduces the amount of boilerplate code. The code below shows how much boilerplate you can reduce when using conc instead of the Go standard library.

Some background: Solving the same problem many times and building an internal library

I’ve often found myself writing similar code to handle common concurrency patterns and panics in Go. In many Sourcegraph projects, we rely heavily on concurrent Go code to efficiently search billions of lines of code, so after the Go 1.18 release in March 2022 introduced generics, I realized that this could fix a lot of the frustrations I’d had before and make a concurrency library a lot more succinct.

Previously when I’d written helper functions to better handle concurrency, I needed different functions for different data types. This meant writing separate functions for strings, tasks, or any other data type that I was working with. Now that generics are available, it’s easy to operate over whatever data types are present in a consistent way.

Around six months ago, I started working on an internal library to generalize how we use goroutines. Internally, we’ve used this library extensively already, but now I’ve cleaned up the code, documented everything, and released it as an open-source package for other developers and teams to use too.

I found time to do the majority of this cleaning and documenting while I was trapped at an airport over the Christmas break due to flight delays across the US in December. I wrote the README, added and cleaned up the comments and docstrings, and added and moved the code to a separate repository with a permissive license. I even had time to throw together a colorful logo that represents the spawning and joining of goroutines before I finally got to board my plane.

Diving into goroutines, concurrency, panics, and scope

With Go, concurrency is already a first-class citizen, so it’s generally a pretty low bar to add concurrent code to a Go project. But doing it correctly can still be hard and there are lots of mistakes I’ve seen (and made) in concurrent Go code, such as: