Redis on 0AndWild_log

Designing a Product Ranking System

Fri, 17 Jul 2026 16:20:00 +0900

TL;DR

I designed a daily product ranking system based on user behavior events.

My first approach published product views, likes, and successful payments to Kafka. commerce-streamer consumed the events and updated ranking scores in a Redis Sorted Set in real time. Redis was a good fit as a serving store because it supports fast Top N queries and rank lookups for individual products.

As I worked through the design, though, I began to question whether Redis should be the ranking system’s only Source of Truth. Expired or lost data is difficult to recover, and changing weights requires historical metrics if we want to recalculate past scores. Supporting hourly, weekly, and monthly rankings makes retaining those source metrics even more important.

This post starts with the Redis design, then explores an alternative that stores source metrics in an RDB and uses Redis to serve only the Top N results needed for queries.

My first reading of the requirements

Initially, ranking sounded like a simple matter of calculating a score for each product and sorting the results.

On closer inspection, the more important question was how to interpret different user actions.

A product detail view signals interest, but less strongly than a purchase. A like is stronger than a view, but doesn’t necessarily lead to revenue. A successful payment is the strongest signal, yet using the sales amount directly could let expensive products dominate the ranking.

I used the following signals:

Event	Meaning	Ranking effect
Product detail view	Mild interest	Increase view score
Like	Explicit interest	Increase like score
Unlike	Withdrawn interest	Decrease like score
Successful payment	Purchase conversion	Increase sales score

Views, likes, and sales use different units, so I kept their metrics separate and applied weights when calculating the final score.

score = carry
 + viewCount * viewWeight
 + likeCount * likeWeight
 + ln(1 + salesAmount) * salesWeight

The sales contribution is based on price * quantity, with a logarithm applied. Using the raw amount could give expensive or already popular products an excessive, persistent advantage once they reach the top.

First design: real-time Redis rankings

My first design used Redis as the real-time ranking store.

The flow is roughly:

A user views a product, likes it, or pays for an order.
commerce-api records a domain event.
The event is stored in a transaction outbox.
An outbox relay publishes it to a Kafka topic.
commerce-streamer consumes the event.
It updates the metrics and final score in the daily Redis ranking keys according to the event type.
The ranking API reads ranks and scores from Redis, adds product information from MySQL, and returns the response.

The outbox addresses the gap between the API transaction and Kafka publication. If an order or like change commits but its event fails to publish, ranking data can drift from the actual state. Recording the outbox row in the same transaction as the domain change lets a relay publish it separately.

Why use date-specific Redis keys?

These are daily rankings, so the keys include a date.

ranking:metric:view:{yyyyMMdd}
ranking:metric:like:{yyyyMMdd}
ranking:metric:sales:{yyyyMMdd}
ranking:metric:raw-sales-amount:{yyyyMMdd}
ranking:metric:carry:{yyyyMMdd}
ranking:all:{yyyyMMdd}
ranking:processed:{yyyyMMdd}

I separated the metric keys because weights can change.

If we store only the final score, changing viewWeight or likeWeight leaves us without enough information to reinterpret the old value. Separate view, like, and sales metrics let us recalculate ranking:all:{date} using the current weights.

ranking:processed:{date} tracks duplicate events. Kafka consumers may process messages at least once, which means the same event can arrive again. If an eventId has already been processed, it must not increase the score a second time.

What worked well about Redis

A Redis Sorted Set felt like a natural match for ranking: it stores members with scores and supports fast retrieval in score order.

ZINCRBY ranking:metric:view:20260717 1 productId
ZREVRANGE ranking:all:20260717 0 19 WITHSCORES
ZREVRANK ranking:all:20260717 productId

Redis commands cover both Top N queries and rank lookups for a specific product.

The API doesn’t need to run expensive aggregations. It reads ranks and scores from Redis, then retrieves display information such as product and brand names from MySQL.

Another benefit is freshness. Scores change as soon as events are consumed, so user actions are reflected quickly. A batch-only design introduces at least the scheduling delay; here, Kafka consumer lag accounts for most of the delay.

End-of-day carry-over

Starting every product at zero each day would feel abrupt. Products that were popular yesterday shouldn’t necessarily disappear at midnight.

I planned a carry-over at 23:50 each day, using that day’s Top 100 products to seed the next day’s scores.

Read the Top 100 from ranking:all:{D}
Multiply each final score by 0.1
Apply to ranking:metric:carry:{D+1}
Also apply to ranking:all:{D+1}

The Top 100 limit controls Redis memory usage. Carrying every product forward would become expensive as product counts and daily keys accumulate.

Keys also have a TTL. Together, expiry and the carry-over limit keep Redis from becoming an indefinite historical store.

Limitations of the first design

The Redis design is simple and fast for daily real-time rankings. Looking at it from an operational perspective reveals several limitations.

1. Can Redis be the Source of Truth?

This was my biggest concern.

Redis works well for serving ranking data, but this design needs more support for preserving the source information. TTL expiry removes data. After an outage or operational mistake, we need enough evidence to reconstruct rankings, and a Redis-centered design alone doesn’t provide that.

Retaining Kafka topics and replaying events is an option. But replaying all events just to recalculate one historical period can be costly. If the event schema or consumer logic has changed, reproducing the original result may also be difficult.

2. Changing weights and recalculating scores

The first design’s separate metric ZSETs allow recalculation while the daily data remains in Redis.

After the TTL expires, that option disappears. A request a month later to recalculate last week’s rankings with today’s weights cannot rely on the remaining Redis data alone.

Runtime weight changes make the distinction between metrics and scores important. A score is the result of a policy; the metrics describe what happened.

3. Rankings across different periods

Date-based keys look sufficient for daily rankings. Hourly, weekly, monthly, and yearly rankings make the key strategy more complicated.

ranking:all:daily:20260717
ranking:all:hourly:2026071713
ranking:all:weekly:2026W29
ranking:all:monthly:202607

We could update every period’s keys when consuming each event. But then a single event modifies multiple keys, and weight changes require recalculating more sets of scores.

At that point, retaining source metrics elsewhere and publishing each period’s Top N to Redis becomes a more natural design.

4. Should every product live in Redis?

The memory question also becomes more important as the catalog grows.

Storing 100,000 products is something we can test readily. At one million or ten million products, multiplied across daily keys, Redis memory cost becomes harder to ignore.

Most ranking API requests need only the Top N. Keeping those results in Redis and the complete metrics elsewhere gives each store a clearer role.

Revised design: RDB Metric SOT + Redis Top N

To address those limits, I explored storing source metrics in an RDB and serving only query-ready Top N results from Redis.

Redis changes roles in this design. In the first version, it was effectively the Source of Truth for real-time rankings. Here, it is a read model for fast queries.

Source metrics live in an RDB such as MySQL.

ranking_daily_metric
- metric_date
- product_id
- view_count
- like_count
- sales_amount

Weights are deliberately left out of those records. Views, likes, and sales amounts describe recorded activity; weights are policy, and policy can change. Storing metrics before weights are applied makes recalculation easier.

Event consumption

The event pipeline stays largely the same. User actions reach Kafka, and RankingMetricConsumer consumes them. Instead of incrementing Redis scores directly, it upserts metric rows in the RDB.

PRODUCT_VIEWED -> view_count + 1
PRODUCT_LIKED -> like_count + 1
PRODUCT_UNLIKED -> like_count - 1
PAYMENT_SUCCEEDED -> sales_amount + price * quantity

Rankings can now be rebuilt from RDB metrics even if Redis is empty.

Weekly rankings can be computed by summing daily metrics over the required period, and monthly rankings work the same way. Neither requires replaying every original event.

Score calculation

A separate batch or scheduler calculates scores. For example, it can read recent metrics every five minutes and apply the current weights.

score = carry
 + view_count * currentViewWeight
 + like_count * currentLikeWeight
 + ln(1 + sales_amount) * currentSalesWeight

It then loads only the Top N products into a Redis ZSET.

ranking:top:daily:20260717

Redis contains the results the API needs rather than the entire catalog. It serves as the fast serving layer, while the source data remains elsewhere.

What this design improves

The biggest improvement is recoverability. If Redis data is lost, the Top N can be rebuilt from RDB metrics. If incorrect weights are deployed, the same metrics can be used to calculate corrected scores.

The second improvement is support for different ranking periods. With daily metrics, weekly, monthly, and yearly rankings become period aggregation problems. At larger scale, an RDB alone may not be enough; an OLAP store or separate aggregate tables could be needed. Retaining the source metrics leaves those options open.

Third, Redis memory usage becomes easier to control. Only the Top N needed for queries lives there. Full product metrics remain in the RDB.

What it costs

This design isn’t automatically better in every respect.

Rankings become less immediate. A five-minute calculation schedule can introduce roughly five minutes of update delay.

The system also becomes more involved. It needs metric tables, upsert logic, score calculation batches, batch failure recovery, and Redis loading logic. There is more to manage than a direct ZINCRBY on a ZSET.

RDB write load matters, too. Every user action can lead to a metric upsert. At higher traffic volumes, buffering, batch inserts, time-bucket aggregation, Kafka Streams, or an OLAP store may become worth considering.

Comparing the designs

Perspective	Real-time Redis rankings	RDB Metric SOT + Redis Top N
Priority	Immediate updates	Recovery and recalculation
Redis role	Real-time ranking store	Query read model
Source metrics	Stored in Redis keys	Persisted in the RDB
Weight changes	Recalculate only periods still in Redis	Recalculate from retained RDB metrics
Multiple periods	More keys add complexity	Extend through metric aggregation
Memory usage	Grows when all products are loaded	Controlled by loading only Top N
Implementation complexity	Relatively simple	Requires batch and recovery policies
Update delay	Low	Depends on the batch interval

The first design is a good starting point for fast daily rankings. User actions are reflected almost immediately, and the API read path is simple.

As recovery, recalculation, and rankings across longer periods become operational requirements, the RDB Metric SOT design becomes more compelling.

How should we choose?

This exercise made me realize that the first question shouldn’t be which database to use. More useful questions are:

How quickly must this ranking reflect new activity?
Can we tolerate losing the ranking data?
Do historical rankings need to be recalculated?
Could the weight policy change frequently?
Do all products need to remain in the ranking dataset?
Will rankings stay daily, or expand to weekly and monthly periods?

If the answers prioritize freshness, a Redis-centered design is simple and fast. If recovery, auditing, recalculation, and broader time periods matter, the system needs to retain source metrics.

Evolving the design

The real-time Redis design initially seemed sufficient. Daily rankings needed to respond quickly to user behavior, and Sorted Sets supported both Top N queries and individual product ranks well.

I placed several limits on that design:

Use date-specific keys.
Give keys a TTL.
Carry over only the Top 100 products.
Keep metric ZSETs separate so weights can be recalculated for retained daily data.
Track processed events to avoid duplicate updates.

These choices make it a starting point that favors freshness and simplicity. Once historical recovery or weekly and monthly rankings enter the requirements, evolving toward RDB Metric SOT + Redis Top N makes more sense.

Closing thoughts

Designing rankings kept bringing me back to questions beyond sorting: which actions count as strong signals, how long to retain them, whether scores must be recalculated when policy changes, and whether an empty ranking after Redis failure is acceptable.

I began by focusing on how quickly Redis Sorted Sets could provide rankings. As the design grew, deciding what to preserve as source data became more important than deciding what to put in Redis.

The first design is a starting point for real-time daily rankings. RDB Metric SOT + Redis Top N is a possible next step as operational requirements grow.

I don’t think a good design has to implement every future requirement from the start. But I do want to be able to explain the limits of today’s choice and how it could evolve when those limits matter.

Handling Order Surges: Designing a Redis Waiting Queue

Fri, 10 Jul 2026 16:59:39 +0900

TL;DR

Instead of sending every request straight to the order API, I first placed users in a Redis Sorted Set.

A scheduler removes up to 50 users from the queue every second and issues entry tokens valid for five minutes. Users poll their position until they receive a token, and the order API accepts only requests with a valid X-Entry-Token.

After an order commits successfully, a local event listener receives OrderEvent.Created and deletes the token. If the order fails, the token remains so the user can try again.

This is an initial design for controlling admission rate, rather than a complete production waiting queue. Atomic queue entry, recovery during token issuance, global throughput across instances, and polling load still need more work.

Overview

Goals:
- Keep a sudden burst of order requests from reaching the order database all at once.
- Let users check their queue position and estimated wait time.
- Issue entry tokens to a limited number of users who can then call the order API.
- Remove entry tokens only after a successful order transaction.
- Keep the waiting queue and order domains independent of Redis implementation details.
Settings used in this implementation:
- Scheduler fixed delay: 1 second
- Scheduler batch size: 50 users
- Entry token TTL: 300 seconds
- Rank: zero-based, using the Redis ZRANK result directly

Why put a queue in front of the order API?

A modest rise in order traffic may be handled by scaling out application servers or tuning the database connection pool.

A sharp burst at the start of an event is different. Even with more application instances, requests converge on the same database to lock inventory and coupon rows and save orders. Accepting more requests at the application tier can make database connection demand and lock waits grow even faster.

The queue’s purpose is to admit requests at a rate the downstream system can handle, while other users wait outside that processing path.

I chose not to store order request payloads in the server-side queue. The queue contains only memberId; once admitted, the user sends the original order request again.

This avoids managing the state and retry policy of queued order payloads on the server. The client instead needs to poll its position and call the order API after reaching READY.

Overall architecture

The flow has three parts:

Register users in a Redis ZSET and expose their current status.
Let the scheduler remove users from the front and issue entry tokens.
Validate tokens at the order API and delete them through a local event after the order commits.

The layers have distinct responsibilities:

Layer	Responsibility
Interface	Waiting queue and order APIs, plus the order completion event listener
Application	User authentication, queue status assembly, token issuance, and order admission validation
Domain / Port	The `WaitingQueuePosition` state model and Repository interfaces that abstract Redis
Infrastructure	Repository port implementations using Spring Data Redis
Redis	A ZSET for waiting order and a String entry token for each admitted user

1. Representing queue order with a Redis Sorted Set

The queue needs to answer three main questions:

Is the user in the queue?
What is their current position?
How many users are waiting in total?

A Redis Sorted Set can represent all three.

key : queue:waiting
member : memberId
score : enteredAt

Because memberId is unique within the ZSET, adding the same user repeatedly doesn’t create duplicate members. Using enteredAt as the score places earlier users ahead of later ones. ZRANK returns their position, and ZPOPMIN removes users from the front.

I use the ZRANK value directly, so the first user’s rank is 0. Keeping both the internal model and API response zero-based avoids +1 and -1 conversions at every boundary.

Preserve the position on repeated entry

A waiting user may refresh the page or call the entry API again. Overwriting the score with the current time would send them to the back.

The current implementation checks for an existing score and adds only users who aren’t already registered.

override fun enterIfAbsent(memberId: Long, score: Double) {
 val member = memberId.toString()
 if (redisTemplate.opsForZSet().score(WAITING_QUEUE_KEY, member) == null) {
 redisTemplate.opsForZSet().add(WAITING_QUEUE_KEY, member, score)
 }
}

This preserves the score and rank across sequential repeated calls.

There is an important limitation: ZSCORE and ZADD are separate commands. Two simultaneous first-entry requests for the same user can both see no score and then write different scores. Strict idempotency requires a single atomic Redis operation such as ZADD NX or addIfAbsent.

2. One response model for waiting status

After entering, clients poll the same position API. It returns one of three states:

WAITING: the user is in the ZSET and has no entry token yet.
READY: the user has an entry token and can call the order API.
NOT_ENTERED: the user is neither in the queue nor holding a valid entry token.

Status checks look for the entry token first. The scheduler removes users from the ZSET before issuing tokens, so a READY user is no longer in the ZSET. Checking rank first could incorrectly classify an admitted user as NOT_ENTERED.

stateDiagram-v2
 [*] --> NOT_ENTERED
 NOT_ENTERED --> WAITING: enter() / register in queue
 WAITING --> WAITING: position polling
 WAITING --> READY: scheduler / issue token
 READY --> READY: order rollback / retain token
 READY --> NOT_ENTERED: order commit / delete token
 READY --> NOT_ENTERED: TTL expires

WaitingQueuePosition uses the retrieved rank and total waiting count to calculate these response values:

status
rank
currentTotalWaitingCount
estimatedWaitSeconds
pollingIntervalSeconds
entryToken

The estimated wait is the current rank divided by an assumed throughput of 50 users per second. The polling interval is one second with fewer than 100 users ahead, three seconds with fewer than 1,000, and five seconds otherwise.

Users near admission receive faster updates, while users further back make fewer Redis queries. Since the throughput assumption is fixed, this is only a rough estimate; it doesn’t reflect actual order processing speed.

3. Controlling admission rate with a scheduler

Sending all queued users to the order API at once would defeat the purpose of the queue.

WaitingQueueScheduler calls issueNextEntries() every second. The service uses ZPOPMIN to remove up to 50 users with the lowest scores and issues a token to each.

fun issueNextEntries(batchSize: Long): List<String> {
 return waitingQueueRepository.popNext(batchSize)
 .mapNotNull { memberId ->
 val token = generateToken()
 entryTokenRepository.issue(memberId, token, properties.entryTokenTtl)
 entryTokenRepository.find(memberId)
 }
}

Each token is a URL-safe Base64 encoding of 32 bytes generated by SecureRandom.

key : queue:entry-token:{memberId}
value : generated token
TTL : 300 seconds

The application doesn’t return the generated token directly. It stores it in Redis and reads it back, so a token that wasn’t stored isn’t treated as a valid admission credential.

The TTL serves two purposes. It prevents a token from granting indefinite access to the order API, and it eventually cleans up tokens even if deletion after order completion fails.

A very short TTL could expire while a user is preparing an order. A very long TTL leaves users eligible to enter long after they have stopped trying. The current 300-second value is the assignment’s default policy; a production value should be based on measured user behavior and target throughput.

4. Requiring an entry token at the order API

Order requests include an X-Entry-Token header.

After authentication, OrderFacade.placeOrder() compares the request token with the one stored in Redis. A missing, mismatched, or expired token results in 401 Unauthorized before order processing begins.

I put this validation in the application layer because eligibility to place an order is a precondition of the use case, beyond simply validating an HTTP header’s format.

This is a fail-closed design: when Redis doesn’t respond, orders are blocked. I chose to stop accepting orders instead of letting traffic bypass the queue and overwhelm downstream systems. That protects the server, but makes a Redis outage an order outage as well.

5. Deleting tokens outside the order transaction

I initially considered deleting the token directly inside the order creation method.

That would mix an external Redis operation into the database transaction flow. Deleting the token before commit is also problematic: if the order rolls back, the user loses admission even though the order failed.

The order flow already publishes OrderEvent.Created on success, so I moved token deletion into a listener that receives this local event at AFTER_COMMIT.

@TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT)
fun handle(event: OrderEvent.Created) {
 waitingQueueService.deleteEntryToken(event.memberId)
}

This gives us two useful properties:

A rolled-back order leaves the token available for another attempt.
A token cleanup failure cannot roll back an already committed order.

A local event is not a durable message, however. If the process exits just after commit, or Redis deletion fails, the listener has no automatic recovery. The TTL provides eventual cleanup for now. Immediate cleanup would require a retryable event record or a separate cleanup job.

AFTER_COMMIT also does not automatically mean asynchronous execution or exception isolation. The current listener runs in the same call flow as the order request. If a Redis deletion exception propagates, the HTTP response can fail even though the order has committed. Production handling needs an explicit policy for logging and alerting on those exceptions, alongside retryable cleanup work.

Technical decisions

Design area	Choice	Reason
Waiting order	Redis Sorted Set	Unique members, score ordering, rank, and total count in one structure
ZSET member	`memberId`	One waiting entry per user
ZSET score	Entry timestamp in milliseconds	Give earlier arrivals priority
Admission control	Fixed-delay scheduler + batch size	Limit the rate at which users reach downstream processing
Admission credential	Per-user Redis String token + TTL	Separate waiting from readiness and expire old credentials automatically
Status updates	Client polling	Expose positions through a simple HTTP API without a separate push channel
Order validation	Start of `OrderFacade`	Check admission before expensive order processing
Token cleanup	`AFTER_COMMIT` listener for `OrderEvent.Created`	Clean up only successful orders and separate deletion from order rollback

Trade-offs

Admission rate is controlled, but concurrent orders are not strictly bounded

Issuing 50 tokens per second doesn’t guarantee that no more than 50 orders run at once.

Tokens remain valid for 300 seconds. Users admitted across several seconds can wait and then submit orders at the same moment. This design controls the token issuance rate, not actual in-flight order concurrency.

A strict downstream concurrency limit would need active slots, token claims, a semaphore, or another control. Another option is to admit users based on actual order completion rate.

Polling is simple, but creates read traffic

HTTP polling is easy to implement on both sides and doesn’t require long-lived connections. But more waiting users mean more repeated GET token, ZRANK, and ZCARD calls. Varying the polling interval by rank reduces that load.

At larger scale, I would consider client-side jitter to spread requests out, longer polling intervals, Redis read replicas rather than CDN caching for these reads, or SSE-based push.

Separate cleanup does not guarantee immediate cleanup

An AFTER_COMMIT listener reduces the responsibilities inside the order transaction. Its failures happen after the order succeeds, though, so the order cannot be rolled back. A local event also leaves no retry record.

The TTL eventually resolves the leftover-token state, but the token can remain valid until it expires.

A Redis outage stops orders

Fail-closed behavior protects the database from requests bypassing the queue, while making Redis a required dependency of the entire order flow. Production design needs replication, Sentinel or Cluster, persistence, and failover policies as well.

Remaining limitations and work beyond the assignment

1. Concurrent entry by the same user must be atomic

ZSCORE followed by ZADD is a check-then-act sequence. I verified concurrent entry by eight distinct users and sequential repeated entry by the same user. That does not establish atomicity for simultaneous first-entry requests from one user.

This should use the single ZADD NX command.

2. Users can be lost between ZPOPMIN and token storage

The scheduler removes a user from the ZSET before saving the token. If the process exits or the Redis write fails between those operations, the user has neither a queue entry nor a token and becomes NOT_ENTERED.

Reading the token back after storage cannot recover a user who has already been popped.

A production design could atomically move users into a processing ZSET, then acknowledge them after token issuance succeeds. A recovery job could return users without an acknowledgment to the waiting ZSET after a timeout.

3. Batch size is not a global limit across application instances

If every instance runs the scheduler, each can pop 50 users per second. ZPOPMIN prevents duplicate removal of the same user, but four instances can issue up to 200 tokens per second in total.

Using batch size as a global throughput limit requires scheduler leader election, a distributed lock, a dedicated worker, or a Redis-based global rate limiter.

4. The token grants temporary admission, not exactly one order

Token deletion happens after order commit. Two concurrent requests from the same user with the same token can both pass validation.

If one token must authorize only one order, validation and claiming it must be atomic. A simple GETDEL at order start also removes the token when the order later rolls back. A more complete design needs states such as READY → CLAIMED → CONSUMED, plus a release policy on rollback. Order-level idempotency keys deserve separate consideration, too.

5. Estimated wait time doesn’t reflect real throughput

The current formula is rank / 50. It matches the scheduler setting but ignores database latency, order success rate, users who receive tokens without ordering, and outages.

A moving average of recent token issuance and order completion rates, together with batch-size adjustments based on operational metrics, would provide a more realistic estimate.

6. Arrivals within the same millisecond are not strictly FIFO

Scores use application timestamps in milliseconds. Multiple users can arrive in the same millisecond, and clock differences across application servers can change score order relative to actual arrival order.

For equal scores, Redis ZSET orders members lexicographically, which can also differ from arrival order. Strict FIFO would require a design using Redis TIME with a sequence, or a separate incrementing value in the score.

What I verified

Using Testcontainers Redis and API E2E tests, I checked:

First entry returns rank 0 and the total waiting count.
Sequential repeated entry preserves the user’s rank.
Concurrent entry by distinct users assigns unique ranks.
Responses represent WAITING, READY, and NOT_ENTERED.
Tokens are unavailable after their TTL expires.
One scheduler run issues no more than the batch size, even with more users waiting.
Orders without an entry token are rejected.
A successful order commit deletes the entry token.

These tests verify functional contracts. They don’t establish maximum capacity or the right batch size. To choose production settings, I would use a tool such as k6 to generate entry bursts and position polling together, while observing application latency, Redis command throughput, CPU, memory, and connection counts.

Alternatives considered

Option	Benefits	Costs
Delete the token inside the order transaction	The full flow is visible in the order code	Redis cleanup becomes part of the transaction’s responsibilities, and rollback handling becomes more complicated
Selected: delete through an `AFTER_COMMIT` local event	Clean up successful orders and separate post-processing from the transaction	Listener failures aren’t automatically recovered; tokens can remain until TTL expiry

The central decision was that token deletion should not determine whether an order succeeds.

Once an order has committed, preserving that success and cleaning up the token separately makes more sense than presenting a failure because cleanup failed.

The local event alone doesn’t provide that reliability, though. It separates responsibilities, while failure recovery still relies on the TTL. A cleaner structure and a more reliable production system are separate outcomes.

Closing thoughts

At first, I thought adding users to a Redis ZSET and returning their rank would complete the queue implementation.

Most of the difficult questions turned out to concern when a user becomes eligible to order, when that eligibility ends, and which state they should return to after a failure.

In this design, the ZSET represents waiting and the TTL token represents readiness. The order API checks the token, and an event after order commit triggers cleanup. This made the boundary between queue responsibilities and the order transaction clearer.

It also exposed gaps: atomic registration, recovery after pop, global throughput across instances, one-time token use, and wait estimates based on actual throughput. The assignment didn’t force all of these issues into view, but production use would require addressing them.

A waiting queue is an admission control system. It needs to move users at a pace the system can handle and preserve their state transitions even when something fails.