A Golang library for queueing tasks and processing them asynchronously with workers. It’s backed by Redis and designed to be scalable yet easy to start.
A high-level overview of how the library works:
- The client puts tasks on a queue
- The server pulls tasks off queues and starts a golang worker goroutine for each task
- Tasks are processed concurrently by multiple workers
The task queues are used as a mechanism to distribute work across multiple machines. A system can consist of multiple worker servers and brokers, giving way to horizontal scaling and high availability.
Example use case with some golang workers
Features, implemented with Golang
- Guaranteed at least one execution of a task
- Scheduling of tasks
- Retries of failed tasks
- Low latency to add a task since writes are fast in Redis with Golang
- De-duplication of tasks using unique option
- Allow timeout and deadline per task
- Allow aggregating group of tasks to batch multiple successive operations
- Flexible handler interface with support for middlewares
- Ability to pause queue to stop processing tasks from the queue
- Periodic Tasks
- Support Redis Cluster for automatic sharding and high availability
- Support Redis Sentinels for high availability
- Integration with Prometheus to collect and visualize queue metrics
- Web UI to inspect and remote-control queues and tasks
- CLI to inspect and remote-control queues and tasks
- Automatic recovery of tasks in the event of a worker crash
- Weighted priority queues
- Strict priority queues
There is no ads to display, Please add some