What are the Slack Archives?

It’s a history of our time together in the Slack Community! There’s a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.

Because this space is not active, you won’t be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..

Hello, we experienced an issue today that might concern us in the future. Our customer has abstract

Posts: 509 πŸ§‘πŸ»β€πŸš€ - Cadet

Hello,
we experienced an issue today that might concern us in the future.
Our customer has abstract products that get drilled down to a concrete product using many super attributes. Today we had 9 in sum, with two options each … which is not that much in my opinion.
However, with 8 attributes we had no problems before, but today with 9 we got a memory allocation issue in the AttributePermutationGenerator while generating the attribute permutations for the product detail page. The limit of 1024MB was exceeded.
So, yes, this one case is solvable increasing the memory limit. But I am worry about further limitations since it’s very likely that we get additional attributes.
Also, this permutation generation stuff seems not to progress linear because the problem at all is not a linear problem. The memory consumption will explode at a specific point.

So my question here: What are the limitations? Any experiences from other projects handling many super attributes?

Best

Welcome!

It looks like you're new here. Sign in or register to get started.
Β«1

Comments

  • Lead Product Manager Posts: 130 πŸ§‘πŸ»β€πŸš€ - Cadet

    @UPWG9AYH2 could you create a support ticket for it, we will need to solve this problem.
    cc: @valerii.trots

  • I can confirm that issue

  • Posts: 123 πŸ§‘πŸ»β€πŸš€ - Cadet
    edited November 2020

    @UPWG9AYH2 try to make super attributes as simple text types. Without options. If possible. I had the same problem and as i remember it helped

  • Posts: 287 πŸ§‘πŸ»β€πŸš€ - Cadet

    I'm facing the same issue. Changing the super attributes' type doesn't help.
    What's the progress on this one? Is there a ticket number I can refer to?

  • SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice
  • Posts: 287 πŸ§‘πŸ»β€πŸš€ - Cadet

    Doesn't work. After updating the packages, I'm getting redis connect errors. So I tried to resetup the docker env and ended up with this:

    Store: AT | Environment: docker.dev
    Build Yves frontend
    
    
    
      spryker-b2b-demo-shop@1.0.0 yves /data
  • Posts: 287 πŸ§‘πŸ»β€πŸš€ - Cadet

    Seems that the shop-ui module depends on a missing npm package.

  • Posts: 509 πŸ§‘πŸ»β€πŸš€ - Cadet
  • SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    Thanks Ingo, you were faster. πŸ™‚

  • Posts: 287 πŸ§‘πŸ»β€πŸš€ - Cadet

    So the recommended solution is to downgrade the shop-ui and lose updatability? Am I getting it right?

  • Posts: 509 πŸ§‘πŸ»β€πŸš€ - Cadet

    As far as i can see, the solution is to add the missing dependency to your project level package.json

  • SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    You need to fix package.json as

    Unfortunately we cannot force npm dependencies update using composer.
    
  • Posts: 241 πŸ§‘πŸ»β€πŸš€ - Cadet

    thanks for the link @valerii.trots

  • Posts: 287 πŸ§‘πŸ»β€πŸš€ - Cadet
    edited February 2021

    Thanks. We're one step further now. But now the Jenkins jobs aren't set up anymore.

  • Posts: 509 πŸ§‘πŸ»β€πŸš€ - Cadet

    Whats the error message? Seems to me like the usual stuff that happen when updates gettin applied πŸ™‚

  • Posts: 287 πŸ§‘πŸ»β€πŸš€ - Cadet

    No error message. When I open scheduler.spryker.local, there are no jobs. Even after running docker/sdk console scheduler:setup manually.

  • SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice
    edited February 2021

    docker/sdk console scheduler:setup -vvv?

  • Posts: 287 πŸ§‘πŸ»β€πŸš€ - Cadet

    Seems to work now. Here's what I did:

    docker/sdk cli composer require filp/whoops spryker/product-storage:"^1.31.0" spryker-shop/product-detail-page:"^3.12.0" spryker/product:"^6.22.1" spryker/product-attribute:"^1.8.1"
    docker/sdk cli npm install password-validator
    

    It didn't work until I ran the composer update without the --update-with-dependencies option.
    I needed to add the filp/whoops package, too, since it was moved to dev dependencies recently, which obviously is wrong.

  • SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    which obviously is wrong

    Nope. It's just a part of some updates\migration path that isn't taken for the project yet.

  • Posts: 287 πŸ§‘πŸ»β€πŸš€ - Cadet

    At first glance, it looked as if it would work, but it doesn't. Still failing at the same place:

    [console] [cli] - [2021-02-09T09:01:29.657080+00:00] Zed.CRITICAL: ErrorException - FATAL ERROR - Allowed memory size of 4294967296 bytes exhausted (tried to allocate 20480 bytes) in "/data/vendor/spryker/product/src/Spryker/Zed/Product/Business/Product/Variant/AttributePermutationGenerator.php::55" {"exception":"[object] (ErrorException(code: 0): FATAL ERROR - Allowed memory size of 4294967296 bytes exhausted (tried to allocate 20480 bytes) at /data/vendor/spryker/product/src/Spryker/Zed/Product/Business/Product/Variant/AttributePermutationGenerator.php:55)
    
  • SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice
    edited February 2021

    How many super attributes do you use?

  • Posts: 287 πŸ§‘πŸ»β€πŸš€ - Cadet

    There are products with seven super attributes. There might be even more.

  • SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    This "fix" works with at least 9 super attributes.
    Could you please provide exact steps to reproduce?

    @jeremy.fourna FYI.

  • Posts: 287 πŸ§‘πŸ»β€πŸš€ - Cadet

    I don't think that it's safe to say works with at least x super attributes. It's not only about the number of super attributes, but also the amount of different values. You could have for example two super attributes only, but if each of them is having ten values, things will go crazy again.
    And that's my actual issue. The combination of super attributes and their values makes the php memory consumption grow exponentially.

    The steps that I'm doing are pretty basic. Once I'm having everything in the db, I simply call docker/sdk console publish:trigger-events. In rabbitmq I can see the publish messages coming in and being processed. Once it gets to the abstract product publish messages, it will exceed the memory. The queue is getting stuck then, i. e. it's picking the same bulk of 500 messages over and over again and retries forever.

  • Posts: 509 πŸ§‘πŸ»β€πŸš€ - Cadet

    My β€œexperience” with this issue was, that the problem is growing more worse with different attributes than with different values.

  • SRE @ Spryker Sprykee Posts: 1,654 ✨ - Novice

    I may be wrong but this "fix" has been developed specifically for Ingo's case as a quick win. Proper fix will follow as a big refactoring of the area.

    @jeremy.fourna could you confirm this?

  • Lead Product Manager Posts: 130 πŸ§‘πŸ»β€πŸš€ - Cadet
  • Lead Product Manager Posts: 130 πŸ§‘πŸ»β€πŸš€ - Cadet

    @valerii.trots the solution is not supposed to be a quick fix, we normally solved the root cause

  • Posts: 287 πŸ§‘πŸ»β€πŸš€ - Cadet

    Can't confirm, unfortunately.

Welcome!

It looks like you're new here. Sign in or register to get started.