Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execute less Redis commands while dequeuing #51

Open
oli-h opened this issue Jul 10, 2018 · 0 comments
Open

Execute less Redis commands while dequeuing #51

oli-h opened this issue Jul 10, 2018 · 0 comments

Comments

@oli-h
Copy link

oli-h commented Jul 10, 2018

Situation

RedisQues executes the following sequence of eight Redis commands for dequeue of one message.
Example with a queue named "oli":

1531139048.907426 "GET"     "redisques:consumers:oli"
1531139048.907666 "EXPIRE"  "redisques:consumers:oli" "20"
1531139048.907805 "GET"     "redisques:consumers:oli"
1531139048.907939 "HEXISTS" "redisques:locks" "oli"
1531139048.908095 "LINDEX"  "redisques:queues:oli" "0"
1531139048.908418 "ZADD"    "redisques:queues" "1.531139048908E12" "oli"
1531139048.909548 "LPOP"    "redisques:queues:oli"
1531139048.909864 "LLEN"    "redisques:queues:oli"

Most of them are only executed after the previous one's response is received. RedisQues therefor suffers more from higher roundtrip times towards Redis server.

Some ideas to improve this

  • Remove the call to EXPIRE redisques:consumers:oli 20 from the dequeuing sequence. RedisQues already refreshes consumer registration independently with a timer. No need to repeat this while dequeuing.
  • Remove the seconds GET redisques:consumers:oli - Method RedisQues.consume(String queue) is only called for the current registered consumer - no need to check this here again
  • The LPOP redisques:queues:oli is only used to remove the head-of-queue after successful message delivery - Redis response is not used. Better use LTRIM ...queue 1 -1 to remove head-of-queue without returning the message content.
  • Don't call LLEN redisques:queues:oli to check if queue is empty before starting the next consumption. Just do it, avoid this extra Redis command. Later, the LINDEX ... will detect the empty queue and stop processing
  • Do we really need to lookup 'who is' the consumer of a queue while 'we' are consuming this queue at the moment? In this case would easily assume that 'we' are still the consumer - remember that 'we' have a timer which refreshes that 'we' are the consumer.
    In other words: No need for GET redisques:consumers:oli while we are in the consumption race. Perhaps we can have a look at myQueues to cross-check if we're still the consumer of this queue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant