Difficulty scaling Laravel Horizon across multiple instances (ECS / Auto Scaling) #57524
Unanswered
iagosilva1
asked this question in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hello everyone 👋
I’m deploying a Laravel application in a containerized architecture (AWS ECS), where I’ve split services into three separate tasks:
Web (HTTP requests)
Horizon (job processing)
Scheduling (cron tasks)
Each service has its own auto scaling policy, so new instances can be created dynamically as demand increases.
The issue I’m facing is that when trying to scale Horizon horizontally (with multiple containers running simultaneously), there’s almost no documentation or discussion on how Horizon is expected to behave in this scenario.
From what I’ve understood, when I set for example --max-processes=10, it means each instance of Horizon can process up to 10 jobs concurrently. However, there’s no coordination between instances, which causes several problems:
Horizon doesn’t balance jobs evenly across containers;
While a new instance is starting up, existing ones may become overloaded;
Increasing the process count may lead to resource contention (CPU/RAM) on current instances while new ones are not yet active.
In short, there doesn’t seem to be an official strategy for Horizon to operate efficiently across multiple instances with auto scaling.
It appears to assume a static, single-node execution model, which doesn’t adapt well to elastic environments.
❓ Question
Is there any recommended or official approach to make Laravel Horizon work properly in a multi-instance, auto-scaled environment, where each Horizon container runs an independent number of processes without having to define a fixed global limit?
If not, is there any suggested workaround or best practice (for example using Redis coordination, an external Supervisor, or ECS lifecycle hooks) to ensure job distribution and scaling remain predictable and efficient?
💡 Context
Laravel 11
Horizon (latest version)
Redis as queue driver
Deployment: ECS Fargate with auto scaling based on queue metrics
Thanks in advance for any guidance!
I believe this is a fairly common question for those running Horizon in modern, container-based infrastructures with elastic scaling.
Beta Was this translation helpful? Give feedback.
All reactions