What are the Slack Archives?
Itβs a history of our time together in the Slack Community! Thereβs a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.
Because this space is not active, you wonβt be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..
[SOLVED] Hi everyone π€, I have some configuration for queue: ```// ---------- Queue $conf
[SOLVED] Hi everyone π€,
I have some configuration for queue:
// ---------- Queue
$config[QueueConstants::QUEUE_ADAPTER_CONFIGURATION_DEFAULT] = [
QueueConfig::CONFIG_QUEUE_ADAPTER => RabbitMqAdapter::class,
QueueConfig::CONFIG_MAX_WORKER_NUMBER => 12,
];
$config[QueueConstants::QUEUE_ADAPTER_CONFIGURATION][ProductPageSearchConfig::PUBLISH_PRODUCT_ABSTRACT_PAGE][QueueConfig::CONFIG_MAX_WORKER_NUMBER] = 6;
As u see I defined a specific CONFIG_MAX_WORKER_NUMBER for queue PUBLISH_PRODUCT_ABSTRACT_PAGE because it has an high usage of memory (known issue!).
Since this configuration the worker does not start more the task event for this specific queue.
I debugged \Spryker\Zed\Queue\Business\Worker\Worker::startProcesses to see if the right number of maxWorkerNumbers is read and calculated and everything is fine, but no processes are started.
Can anyone help me to understand the problem? Thank u in advance.!!!
Comments
-
PUBLISH_PRODUCT_ABSTRACT_PAGElooks like the key for the queue messages - itβs not a queue itself.0 -
I think this config only works with the queue names that you see in the RabbitMQ UI queues page, eg
sync.search.product0 -
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
the queue name is right... and the queue is also there..
0 -
ah interesting - Iβve not seen the publish queue split like this before
0 -
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
and the configuration is according the name of queue too...
0 -
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
I have every queue splitted ... i have ca. 100 queues...
0 -
Wow
0 -
Iβve just checked our config for this, itβs slightly different to yours
0 -
0
-
Maybe the adapter class is relevant?
0 -
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
yes.. u are right.. maybe this one... i will try
0 -
Otherwise Iβm out of ideasβ¦
0 -
Out of interest is the publish queue splitting a Spryker feature, or something custom you have done?
0 -
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
Spryker standard and some customizations
0 -
I you're running into memory issues with RabbitMq, have a look at lazy queues:
0 -
They can reduce the memory footprint significantly
0 -
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
@U01T075RRHD the problem about memory issue is related to process consumed from spryker to publish the data in the spy_product_abstract_page_search ... Is related to a lot of data loaded and to propel gc too...
what do u mean with rabbitMq memory issue? In RMQ Server Ram used?
0 -
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
@UKEP86J66 it works!!! Thanks! U saved my day π ...
0 -
π₯³ great to hear!
0 -
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
I just forgot to specify the adapter class.. in this case the worker was not able to retrieve the message and start the process
0 -
By the way we had issues with memory for product page search and found there are 2 columns for data in that table. One is for debugging purposes and can be left blank which will certainly reduce the propel memory.
0 -
I asked about it in Spryker slack but I cannot find the link though π
0 -
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
I think that the structured_data field (this that u left blank) is thought for the refresh command..
0 -
@UL65CH0MC Yes, my suggestion targeted RabbitMq memory usage ... so I guess I got you all wrong π . Happy you could work out a solution
0 -
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
But i can check if the memory usage is related to this field..
0 -
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
@U01T075RRHD thanks.. But i am interested how did u changed the queues to lazy via spryker... did u overwrite the queue:setup ?
0 -
You can provide it in the queue setup using
RabbitMqOptionTransfer::setArguments()0 -
-
giovanni.piemontese Technical Lead @ LΓΆffelhardt Spryker Solution Partner Posts: 871 π§π»βπ - Cadet
@U01T075RRHD so ?
/** * @param string $queueName * @param string $routingKey * * @return \Generated\Shared\Transfer\RabbitMqOptionTransfer */ protected function createQueueOptionTransfer($queueName, $routingKey = '') { $queueOptionTransfer = parent::createQueueOptionTransfer($queueName, $routingKey); $queueOptionTransfer->setArguments(['queue-mode' => 'lazy']); return $queueOptionTransfer; }Or the arguments should be passed in another way?
0 -
It should be
x-queue-modeI guess0
Categories
- All Categories
- 42 Getting Started & Guidelines
- 7 Getting Started in the Community
- 8 Additional Resources
- 7 Community Ideas and Feedback
- 83 Spryker News
- 965 Developer Corner
- 819 Spryker Development
- 91 Spryker Dev Environment
- 362 Spryker Releases
- 3 Oryx frontend framework
- 36 Propel ORM
- 68 Community Projects
- 3 Community Ideation Board
- 30 Hackathon
- 3 PHP Bridge
- 6 Gacela Project
- 29 Job Opportunities
- 3.2K π Slack Archives
- 116 Academy
- 5 Business Users
- 370 Docker
- 551 Slack General
- 2K Help
- 75 Knowledge Sharing
- 6 Random Stuff
- 4 Code Testing
- 33 Product & Business Questions
- 70 Spryker Safari Questions
- 50 Random