A Reliable, simple & efficient distributed task queue in Golang

A Golang library for queueing tasks and processing them asynchronously with workers. It’s backed by Redis and designed to be scalable yet easy to start.

A high-level overview of how the library works:

  • The client puts tasks on a queue
  • The server pulls tasks off queues and starts a golang worker goroutine for each task
  • Tasks are processed concurrently by multiple workers

The task queues are used as a mechanism to distribute work across multiple machines. A system can consist of multiple worker servers and brokers, giving way to horizontal scaling and high availability.

Example use case with some golang workers

Task Queue Diagram

Features, implemented with Golang