Alert on queue depth and consumer lag
A queue nobody is watching, given alerts on depth and lag that are back-tested against a real backlog.
The ticket
Add alerting on queue depth and consumer lag.
Acceptance criteria
- Both depth and lag are alerted separately
- The rules fire when replayed against a real backlog window
- They stay quiet across a normal window
- They route to the rotation that owns the consumer
What lands as proof
Replays against a backlog window and a normal one, showing the rules distinguish between them.
Why teams defer it
- Queues absorb problems quietly, which is their job and also why nobody notices they are drowning.
- Choosing thresholds needs a normal baseline, and nobody has measured one.
Questions
- What does the agent actually change?
- The ticket is scoped to one outcome: add alerting on queue depth and consumer lag. Work that serves that outcome is in scope, and anything outside it is left for a separate ticket, so the pull request stays reviewable.
- How do I know the work is done?
- The pull request carries the evidence, not only the diff. Here that means alert fires against a replayed backlog, so a reviewer can confirm the result without reproducing the work locally.
- How much oversight does this need?
- The run stops once the plan is written. Somebody reads the approach and approves it before any code exists, which is the cheapest moment to redirect the work.