Hi, I am currently facing issues with messages in RMQ, which aren't processed. When I have a look in

ULL0N440J

Hi, I am currently facing issues with messages in RMQ, which aren't processed.
When I have a look into Jenkins, I see that the job queue:worker:start is running endlessly. Could this be a lack of memory?

As an example: I see this behaviour when I add a bigger category to a productlist, this is creating up to 200.000 messages.

rshopin

Hello Daniel, can you try to run job from command line with console q:w:s -vvv ?

rshopin

Also is there some messages in *error queues?

ULL0N440J

Error queues are empty.
The messages stay in Redis as unacked.

Valerii Trots

I remember one of the projects had this kind of error on staging due to not default path to the PHP.

Valerii Trots

I will try to find now details and conclusion from that thread.

ULL0N440J

It took about 5 minutes until q:w:s -vvv was completed

now both queues sync.storage.product and sync.search.product where filled with messages (typical behaviour for productlists)

ULL0N440J

Rerunning q:w:s now processes all messages which where in RMQ as intended

rshopin

So it works when you run command from cli?

ULL0N440J

Yes, it works when I run it (twice) from cli. I've disabled Jenkins while testing.

1: event queue is processed, probably just all the categories
2: processing assigned products in search and storage

rshopin

In this case please check what command you have inside Jenkins and try to execute it from cli, it's autogenerated, maybe something is wrong

rshopin

like PHP path or something else

rshopin

Other strange thing that execution of q:w:s takes more that 59 seconds

rshopin

Or you runned it 5 times?

ULL0N440J

the progressbar really took the 59 seconds - but it took another ~4 Minutes until I could enter new line in CLI

Valerii Trots

Custom PHP_BIN paths from config/Zed/cronjobs/cron.conf gets ignored by the command invoked tasks and will only be used by the top-level command like "vendor/bin/console queue:worker:start". Why?
If "php" is not in the default path like /usr/bin/php (which is quite often the case on multi php version hosting setups), the custom PATH lookup from ~/.bash_profile or ~/. profile with /usr/bin/env always works only when the process is spawned from a login shell. But as you might know, this is not the way how Jenkins triggers the jobs.


My workaround was:



$ cat deploy/vars

...

PATH="$PATH:/usr/local/software/php/7.3.5/bin/"

Valerii Trots

This is the conclusion from the problem I mentioned above.

ULL0N440J

This seems to be the normal q:w:s right?

Valerii Trots

Yes.

ULL0N440J

Got an error message and I am debugging it now 🙂 Thanks guys!

[2020-04-02 14:33:24] application_[events.INFO](http://events.INFO): [async] Failed to handle "Entity.spy_product_list.update" for listener "Spryker\Zed\ProductListStorage\Communication\Plugin\Event\Listener\ProductListStorageListener". Exception: "Argument 1 passed to Spryker\Zed\ProductListStorage\Business\ProductListProductAbstractStorage\ProductListProductAbstractStorageWriter::getProductAbstractProductListStorageEntity() must be of the type integer, null given, called in /data/shop/development/current/vendor/spryker/product-list-storage/src/Spryker/Zed/ProductListStorage/Business/ProductListProductAbstractStorage/ProductListProductAbstractStorageWriter.php on line 59",

ULL0N440J

So, turns out that the

getProductAbstractListIdsByProductAbstractIds(array[] productIds): array

can return null as an index of the array.

This may be a part of the previous behaviour I faced: there are *_storage-tables which aren't filled (for example the spy_product_list_product_concrete).

Anyway; shouldn't there be a check, if $idProductAbstract is null before
vendor/spryker/product-list-storage/src/Spryker/Zed/ProductListStorage/Business/ProductListProductAbstractStorage/ProductListProductAbstractStorageWriter.php::59?

Valerii Trots

Why spy_product_list_product_concrete isn't filled in the first place?
I checked yesterday this point as well - it's filled with a data as soon as product list is created.

ULL0N440J

I don't know. It isn't filled as long as I don't add any products to a productlist.

So it's empty when I add categories

Valerii Trots

Let me check once again.

Valerii Trots

Ok, sorry, finally got the same exception even though didn't have it yesterday while testing the whole functionality. My bad.

Valerii Trots

Maybe this is because yesterday I was using spryker-shop/b2b-demo-shop and now spryker-shop/suite. 🤔

ULL0N440J

Are all your messages processed? Even if there's no null given, they are remaining in event queue for me

Valerii Trots

I've got this exception only when I assign a root\parent category that doesn't have products assigned to the product list.

In case I assign not a root\parent category but some child categories that have products assigned to them everything works fine. All messages are consumed without errors.

Valerii Trots

Probably I will need to install your project once again tomorrow to see how is it going with your data.

ULL0N440J

That could be. I add every category, even the parent one. Will remove it for testing.

Valerii Trots

Ok.