What are the Slack Archives?

It’s a history of our time together in the Slack Community! There’s a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.

Because this space is not active, you won’t be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..

Hi guys Most of the time messages remain for long time in event queue … even if I ran multi process

UM9F81RCP
UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet
in Help

Hi guys

Most of the time messages remain for long time in event queue …
even if I ran multi processes of queue tasks

vendor/bin/console queue:task:start event

it still takes so long! Sometimes running more processes makes the situation even worse !

What am I missing on the application level configuration?

Side note:
The server where rabbitmq is running is not fully utilised (CPU average does not go beyond 38%)

Comments

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    Do you see any redelivered activity on the event queue? Sometimes if an event listener is slow the process will timeout, and the event is returned to the queue. So your server is really busy but it doesn't get anywhere...

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    and then running more processes just makes things slower, so more timeouts

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    a good strategy is often to reduce the batch size of the event queue.

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    by batch size you mean EventConstants::EVENT_CHUNK ?

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    For us it's defined in EventConfig::getEventQueueMessageChunkSize

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    It might be a config value now 🙂

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    Yes sorry - chunk size 🙂

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    I see

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    ok thank you very much

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    The default is 1000, but it's worth reducing to 100

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    and then see if it helps

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    in your opinion … if I do this and increase the number of workers .. would it be better ?

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    If you still have CPU and memory that would make sense

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    I guess then it makes sense

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    I want to utilise the resources 😄 🔱

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    Get your money's worth 😉
    But it really depends on the events - these changes may help some events, but make others a bit slower

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    So it's a bit of a balancing act.

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    exactly

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    I noticed that

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    thank you Rick

  • UKEP86J66
    UKEP86J66 Posts: 208 🧑🏻‍🚀 - Cadet

    np, hope it helps :crossed_fingers:

  • Andriy Netseplyayev
    Andriy Netseplyayev Domain Lead Solution Architecture Sprykee Posts: 519 🧑🏻‍🚀 - Cadet

    processor is not always the case of a slow message consumption. It can be also related to the lack of memory. If memory is over and swap burns your disk, message processing becomes may become very slow, and adding more works can make it even worse.

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    my problem is the opposite 🙂 … my problem is not over-utilisation of cpu … it is under-utilisation … my CPU is more free than it should be

  • Andriy Netseplyayev
    Andriy Netseplyayev Domain Lead Solution Architecture Sprykee Posts: 519 🧑🏻‍🚀 - Cadet

    what’s about memory though?

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    memory is also free

  • UM9F81RCP
    UM9F81RCP Posts: 516 🧑🏻‍🚀 - Cadet

    I used the recommendation from @UKEP86J66 and RabbitMQ seems to go better now

  • Andriy Netseplyayev
    Andriy Netseplyayev Domain Lead Solution Architecture Sprykee Posts: 519 🧑🏻‍🚀 - Cadet

    okay! Then yes, finding a good balance between chunk-size and amount of workers - is the thing to try 👍

Categories