What are the Slack Archives?
It’s a history of our time together in the Slack Community! There’s a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.
Because this space is not active, you won’t be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..
Hi guys Most of the time messages remain for long time in event queue … even if I ran multi process
Hi guys
Most of the time messages remain for long time in event queue …
even if I ran multi processes of queue tasks
vendor/bin/console queue:task:start event
it still takes so long! Sometimes running more processes makes the situation even worse !
What am I missing on the application level configuration?
Side note:
The server where rabbitmq is running is not fully utilised (CPU average does not go beyond 38%)
Comments
-
Do you see any
redeliveredactivity on the event queue? Sometimes if an event listener is slow the process will timeout, and the event is returned to the queue. So your server is really busy but it doesn't get anywhere...0 -
and then running more processes just makes things slower, so more timeouts
0 -
a good strategy is often to reduce the batch size of the event queue.
0 -
by batch size you mean EventConstants::EVENT_CHUNK ?
0 -
For us it's defined in
EventConfig::getEventQueueMessageChunkSize0 -
It might be a config value now 🙂
0 -
Yes sorry - chunk size 🙂
0 -
I see
0 -
ok thank you very much
0 -
The default is 1000, but it's worth reducing to 100
0 -
and then see if it helps
0 -
in your opinion … if I do this and increase the number of workers .. would it be better ?
0 -
If you still have CPU and memory that would make sense
0 -
I guess then it makes sense
0 -
I want to utilise the resources 😄 🔱
0 -
Get your money's worth 😉
But it really depends on the events - these changes may help some events, but make others a bit slower0 -
So it's a bit of a balancing act.
0 -
exactly
0 -
I noticed that
0 -
thank you Rick
0 -
np, hope it helps :crossed_fingers:
0 -
processor is not always the case of a slow message consumption. It can be also related to the lack of memory. If memory is over and swap burns your disk, message processing becomes may become very slow, and adding more works can make it even worse.
0 -
my problem is the opposite 🙂 … my problem is not over-utilisation of cpu … it is under-utilisation … my CPU is more free than it should be
0 -
what’s about memory though?
0 -
memory is also free
0 -
I used the recommendation from @UKEP86J66 and RabbitMQ seems to go better now
0 -
okay! Then yes, finding a good balance between chunk-size and amount of workers - is the thing to try 👍
0
Categories
- All Categories
- 42 Getting Started & Guidelines
- 7 Getting Started in the Community
- 8 Additional Resources
- 7 Community Ideas and Feedback
- 83 Spryker News
- 972 Developer Corner
- 825 Spryker Development
- 91 Spryker Dev Environment
- 362 Spryker Releases
- 3 Oryx frontend framework
- 36 Propel ORM
- 68 Community Projects
- 3 Community Ideation Board
- 30 Hackathon
- 3 PHP Bridge
- 6 Gacela Project
- 30 Job Opportunities
- 3.2K 📜 Slack Archives
- 116 Academy
- 5 Business Users
- 370 Docker
- 551 Slack General
- 2K Help
- 75 Knowledge Sharing
- 6 Random Stuff
- 4 Code Testing
- 33 Product & Business Questions
- 70 Spryker Safari Questions
- 50 Random