A Golang library for queueing tasks and processing them asynchronously with workers. It’s backed by Redis and designed to be scalable yet easy to start.

A high-level overview of how the library works:

• The client puts tasks on a queue
• The server pulls tasks off queues and starts a golang worker goroutine for each task
• Tasks are processed concurrently by multiple workers

The task queues are used as a mechanism to distribute work across multiple machines. A system can consist of multiple worker servers and brokers, giving way to horizontal scaling and high availability.

Example use case with some golang workers

Features, implemented with Golang

• Guaranteed at least one execution of a task
• Scheduling of tasks
• Retries of failed tasks
• Low latency to add a task since writes are fast in Redis with Golang
• De-duplication of tasks using unique option
• Allow timeout and deadline per task
• Allow aggregating group of tasks to batch multiple successive operations
• Flexible handler interface with support for middlewares
• Ability to pause queue to stop processing tasks from the queue
• Periodic Tasks
• Support Redis Cluster for automatic sharding and high availability
• Support Redis Sentinels for high availability
• Integration with Prometheus to collect and visualize queue metrics
• Web UI to inspect and remote-control queues and tasks
• CLI to inspect and remote-control queues and tasks
• Automatic recovery of tasks in the event of a worker crash
• Weighted priority queues
• Strict priority queues

There is no ads to display, Please add some