What are the Slack Archives?
Itโs a history of our time together in the Slack Community! Thereโs a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.
Because this space is not active, you wonโt be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..
[SOLVED] Hi everyone ๐ค, I have some configuration for queue: ```// ---------- Queue $conf
[SOLVED] Hi everyone ๐ค,
I have some configuration for queue:
// ---------- Queue $config[QueueConstants::QUEUE_ADAPTER_CONFIGURATION_DEFAULT] = [ QueueConfig::CONFIG_QUEUE_ADAPTER => RabbitMqAdapter::class, QueueConfig::CONFIG_MAX_WORKER_NUMBER => 12, ]; $config[QueueConstants::QUEUE_ADAPTER_CONFIGURATION][ProductPageSearchConfig::PUBLISH_PRODUCT_ABSTRACT_PAGE][QueueConfig::CONFIG_MAX_WORKER_NUMBER] = 6;
As u see I defined a specific CONFIG_MAX_WORKER_NUMBER
for queue PUBLISH_PRODUCT_ABSTRACT_PAGE
because it has an high usage of memory (known issue!).
Since this configuration the worker does not start more the task event for this specific queue.
I debugged \Spryker\Zed\Queue\Business\Worker\Worker::startProcesses
to see if the right number of maxWorkerNumbers is read and calculated and everything is fine, but no processes are started.
Can anyone help me to understand the problem? Thank u in advance.!!!
Comments
-
PUBLISH_PRODUCT_ABSTRACT_PAGE
looks like the key for the queue messages - itโs not a queue itself.0 -
I think this config only works with the queue names that you see in the RabbitMQ UI queues page, eg
sync.search.product
0 -
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
the queue name is right... and the queue is also there..
0 -
ah interesting - Iโve not seen the publish queue split like this before
0 -
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
and the configuration is according the name of queue too...
0 -
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
I have every queue splitted ... i have ca. 100 queues...
0 -
Wow
0 -
Iโve just checked our config for this, itโs slightly different to yours
0 -
0
-
Maybe the adapter class is relevant?
0 -
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
yes.. u are right.. maybe this one... i will try
0 -
Otherwise Iโm out of ideasโฆ
0 -
Out of interest is the publish queue splitting a Spryker feature, or something custom you have done?
0 -
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
Spryker standard and some customizations
0 -
I you're running into memory issues with RabbitMq, have a look at lazy queues:
0 -
They can reduce the memory footprint significantly
0 -
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
@U01T075RRHD the problem about memory issue is related to process consumed from spryker to publish the data in the spy_product_abstract_page_search ... Is related to a lot of data loaded and to propel gc too...
what do u mean with rabbitMq memory issue? In RMQ Server Ram used?
0 -
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
@UKEP86J66 it works!!! Thanks! U saved my day ๐ ...
0 -
๐ฅณ great to hear!
0 -
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
I just forgot to specify the adapter class.. in this case the worker was not able to retrieve the message and start the process
0 -
By the way we had issues with memory for product page search and found there are 2 columns for data in that table. One is for debugging purposes and can be left blank which will certainly reduce the propel memory.
0 -
I asked about it in Spryker slack but I cannot find the link though ๐
0 -
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
I think that the structured_data field (this that u left blank) is thought for the refresh command..
0 -
@UL65CH0MC Yes, my suggestion targeted RabbitMq memory usage ... so I guess I got you all wrong ๐ . Happy you could work out a solution
0 -
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
But i can check if the memory usage is related to this field..
0 -
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
@U01T075RRHD thanks.. But i am interested how did u changed the queues to lazy via spryker... did u overwrite the queue:setup ?
0 -
You can provide it in the queue setup using
RabbitMqOptionTransfer::setArguments()
0 -
-
giovanni.piemontese Technical Lead @ Lรถffelhardt Spryker Solution Partner Posts: 871 ๐ง๐ปโ๐ - Cadet
@U01T075RRHD so ?
/** * @param string $queueName * @param string $routingKey * * @return \Generated\Shared\Transfer\RabbitMqOptionTransfer */ protected function createQueueOptionTransfer($queueName, $routingKey = '') { $queueOptionTransfer = parent::createQueueOptionTransfer($queueName, $routingKey); $queueOptionTransfer->setArguments(['queue-mode' => 'lazy']); return $queueOptionTransfer; }
Or the arguments should be passed in another way?
0 -
It should be
x-queue-mode
I guess0
Categories
- All Categories
- 42 Getting Started & Guidelines
- 7 Getting Started in the Community
- 8 Additional Resources
- 7 Community Ideas and Feedback
- 73 Spryker News
- 910 Developer Corner
- 770 Spryker Development
- 87 Spryker Dev Environment
- 361 Spryker Releases
- 3 Oryx frontend framework
- 34 Propel ORM
- 68 Community Projects
- 3 Community Ideation Board
- 30 Hackathon
- 3 PHP Bridge
- 6 Gacela Project
- 25 Job Opportunities
- 3.2K ๐ Slack Archives
- 116 Academy
- 5 Business Users
- 370 Docker
- 551 Slack General
- 2K Help
- 75 Knowledge Sharing
- 6 Random Stuff
- 4 Code Testing
- 32 Product & Business Questions
- 69 Spryker Safari Questions
- 50 Random