What are the Slack Archives?

It’s a history of our time together in the Slack Community! There’s a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.

Because this space is not active, you won’t be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..

Hi, I am currently facing issues with messages in RMQ, which aren't processed. When I have a look in

ULL0N440J
ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet
edited April 2020 in Help

Hi, I am currently facing issues with messages in RMQ, which aren't processed.
When I have a look into Jenkins, I see that the job queue:worker:start is running endlessly. Could this be a lack of memory?

As an example: I see this behaviour when I add a bigger category to a productlist, this is creating up to 200.000 messages.

«1

Comments

  • rshopin
    rshopin Senior Solution Architect Sprykee Posts: 32 🧑🏻‍🚀 - Cadet

    Hello Daniel, can you try to run job from command line with console q:w:s -vvv ?

  • rshopin
    rshopin Senior Solution Architect Sprykee Posts: 32 🧑🏻‍🚀 - Cadet
    edited April 2020

    Also is there some messages in *error queues?

  • ULL0N440J
    ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet

    Error queues are empty.
    The messages stay in Redis as unacked.

  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    I remember one of the projects had this kind of error on staging due to not default path to the PHP.

  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    I will try to find now details and conclusion from that thread.

  • ULL0N440J
    ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet

    It took about 5 minutes until q:w:s -vvv was completed

    now both queues sync.storage.product and sync.search.product where filled with messages (typical behaviour for productlists)

  • ULL0N440J
    ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet
    edited April 2020

    Rerunning q:w:s now processes all messages which where in RMQ as intended

  • rshopin
    rshopin Senior Solution Architect Sprykee Posts: 32 🧑🏻‍🚀 - Cadet
    edited April 2020

    So it works when you run command from cli?

  • ULL0N440J
    ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet
    edited April 2020

    Yes, it works when I run it (twice) from cli. I've disabled Jenkins while testing.

    1: event queue is processed, probably just all the categories
    2: processing assigned products in search and storage

  • rshopin
    rshopin Senior Solution Architect Sprykee Posts: 32 🧑🏻‍🚀 - Cadet

    In this case please check what command you have inside Jenkins and try to execute it from cli, it's autogenerated, maybe something is wrong

  • rshopin
    rshopin Senior Solution Architect Sprykee Posts: 32 🧑🏻‍🚀 - Cadet

    like PHP path or something else

  • rshopin
    rshopin Senior Solution Architect Sprykee Posts: 32 🧑🏻‍🚀 - Cadet

    Other strange thing that execution of q:w:s takes more that 59 seconds

  • rshopin
    rshopin Senior Solution Architect Sprykee Posts: 32 🧑🏻‍🚀 - Cadet

    Or you runned it 5 times?

  • ULL0N440J
    ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet

    the progressbar really took the 59 seconds - but it took another ~4 Minutes until I could enter new line in CLI

  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice
    Custom PHP_BIN paths from config/Zed/cronjobs/cron.conf gets ignored by the command invoked tasks and will only be used by the top-level command like "vendor/bin/console queue:worker:start". Why?
    If "php" is not in the default path like /usr/bin/php (which is quite often the case on multi php version hosting setups), the custom PATH lookup from ~/.bash_profile or ~/. profile with /usr/bin/env always works only when the process is spawned from a login shell. But as you might know, this is not the way how Jenkins triggers the jobs.
    
    
    My workaround was:
    
    
    
    $ cat deploy/vars
    
    ...
    
    PATH="$PATH:/usr/local/software/php/7.3.5/bin/"
    
  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice
    edited April 2020

    This is the conclusion from the problem I mentioned above.

  • ULL0N440J
    ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet

    This seems to be the normal q:w:s right?

  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    Yes.

  • ULL0N440J
    ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet
    edited April 2020

    Got an error message and I am debugging it now 🙂 Thanks guys!

    [2020-04-02 14:33:24] application_[events.INFO](http://events.INFO): [async] Failed to handle "Entity.spy_product_list.update" for listener "Spryker\Zed\ProductListStorage\Communication\Plugin\Event\Listener\ProductListStorageListener". Exception: "Argument 1 passed to Spryker\Zed\ProductListStorage\Business\ProductListProductAbstractStorage\ProductListProductAbstractStorageWriter::getProductAbstractProductListStorageEntity() must be of the type integer, null given, called in /data/shop/development/current/vendor/spryker/product-list-storage/src/Spryker/Zed/ProductListStorage/Business/ProductListProductAbstractStorage/ProductListProductAbstractStorageWriter.php on line 59",
    
  • ULL0N440J
    ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet
    edited April 2020

    So, turns out that the

    getProductAbstractListIdsByProductAbstractIds(array[] productIds): array
    

    can return null as an index of the array.

    This may be a part of the previous behaviour I faced: there are *_storage-tables which aren't filled (for example the spy_product_list_product_concrete).

    Anyway; shouldn't there be a check, if $idProductAbstract is null before
    vendor/spryker/product-list-storage/src/Spryker/Zed/ProductListStorage/Business/ProductListProductAbstractStorage/ProductListProductAbstractStorageWriter.php::59?

  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    Why spy_product_list_product_concrete isn't filled in the first place?
    I checked yesterday this point as well - it's filled with a data as soon as product list is created.

  • ULL0N440J
    ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet

    I don't know. It isn't filled as long as I don't add any products to a productlist.

    So it's empty when I add categories

  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    Let me check once again.

  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    Ok, sorry, finally got the same exception even though didn't have it yesterday while testing the whole functionality. My bad.

  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice
    edited April 2020

    Maybe this is because yesterday I was using spryker-shop/b2b-demo-shop and now spryker-shop/suite. 🤔

  • ULL0N440J
    ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet
    edited April 2020

    Are all your messages processed? Even if there's no null given, they are remaining in event queue for me

  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    I've got this exception only when I assign a root\parent category that doesn't have products assigned to the product list.

    In case I assign not a root\parent category but some child categories that have products assigned to them everything works fine. All messages are consumed without errors.

  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    Probably I will need to install your project once again tomorrow to see how is it going with your data.

  • ULL0N440J
    ULL0N440J Posts: 184 🧑🏻‍🚀 - Cadet
    edited April 2020

    That could be. I add every category, even the parent one. Will remove it for testing.

  • Valerii Trots
    Valerii Trots SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    Ok.