Hello, we experienced an issue today that might concern us in the future. Our customer has abstract

UPWG9AYH2

Hello,
we experienced an issue today that might concern us in the future.
Our customer has abstract products that get drilled down to a concrete product using many super attributes. Today we had 9 in sum, with two options each … which is not that much in my opinion.
However, with 8 attributes we had no problems before, but today with 9 we got a memory allocation issue in the AttributePermutationGenerator while generating the attribute permutations for the product detail page. The limit of 1024MB was exceeded.
So, yes, this one case is solvable increasing the memory limit. But I am worry about further limitations since it’s very likely that we get additional attributes.
Also, this permutation generation stuff seems not to progress linear because the problem at all is not a linear problem. The memory consumption will explode at a specific point.

So my question here: What are the limitations? Any experiences from other projects handling many super attributes?

Best

Jeremy Fourna

@UPWG9AYH2 could you create a support ticket for it, we will need to solve this problem.
cc: @valerii.trots

Unknown

I can confirm that issue

U0145CP9Q6Q

@UPWG9AYH2 try to make super attributes as simple text types. Without options. If possible. I had the same problem and as i remember it helped

U01LKKBK97T

I'm facing the same issue. Changing the super attributes' type doesn't help.
What's the progress on this one? Is there a ticket number I can refer to?

Valerii Trots

Check this release please - https://api.release.spryker.com/release-group/3210.

U01LKKBK97T

Doesn't work. After updating the packages, I'm getting redis connect errors. So I tried to resetup the docker env and ended up with this:

Store: AT | Environment: docker.dev
Build Yves frontend



  spryker-b2b-demo-shop@1.0.0 yves /data

U01LKKBK97T

Seems that the shop-ui module depends on a missing npm package.

UPWG9AYH2

Thats some other issue. See https://sprykercommunity.slack.com/archives/CKJRJM5FG/p1607950686484600 for a solution

Valerii Trots

Thanks Ingo, you were faster. 🙂

U01LKKBK97T

So the recommended solution is to downgrade the shop-ui and lose updatability? Am I getting it right?

UPWG9AYH2

As far as i can see, the solution is to add the missing dependency to your project level package.json

Valerii Trots

You need to fix package.json as

Unfortunately we cannot force npm dependencies update using composer.

Valerii Trots

https://github.com/spryker-shop/suite/pull/692/files#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519

U01LE4BMBK7

thanks for the link @valerii.trots

U01LKKBK97T

Thanks. We're one step further now. But now the Jenkins jobs aren't set up anymore.

UPWG9AYH2

Whats the error message? Seems to me like the usual stuff that happen when updates gettin applied 🙂

U01LKKBK97T

No error message. When I open scheduler.spryker.local, there are no jobs. Even after running docker/sdk console scheduler:setup manually.

Valerii Trots

docker/sdk console scheduler:setup -vvv?

U01LKKBK97T

Seems to work now. Here's what I did:

docker/sdk cli composer require filp/whoops spryker/product-storage:"^1.31.0" spryker-shop/product-detail-page:"^3.12.0" spryker/product:"^6.22.1" spryker/product-attribute:"^1.8.1"
docker/sdk cli npm install password-validator

It didn't work until I ran the composer update without the --update-with-dependencies option.
I needed to add the filp/whoops package, too, since it was moved to dev dependencies recently, which obviously is wrong.

Valerii Trots

which obviously is wrong

Nope. It's just a part of some updates\migration path that isn't taken for the project yet.

U01LKKBK97T

At first glance, it looked as if it would work, but it doesn't. Still failing at the same place:

[console] [cli] - [2021-02-09T09:01:29.657080+00:00] Zed.CRITICAL: ErrorException - FATAL ERROR - Allowed memory size of 4294967296 bytes exhausted (tried to allocate 20480 bytes) in "/data/vendor/spryker/product/src/Spryker/Zed/Product/Business/Product/Variant/AttributePermutationGenerator.php::55" {"exception":"[object] (ErrorException(code: 0): FATAL ERROR - Allowed memory size of 4294967296 bytes exhausted (tried to allocate 20480 bytes) at /data/vendor/spryker/product/src/Spryker/Zed/Product/Business/Product/Variant/AttributePermutationGenerator.php:55)

Valerii Trots

How many super attributes do you use?

U01LKKBK97T

There are products with seven super attributes. There might be even more.

Valerii Trots

This "fix" works with at least 9 super attributes.
Could you please provide exact steps to reproduce?

@jeremy.fourna FYI.

U01LKKBK97T

I don't think that it's safe to say works with at least x super attributes. It's not only about the number of super attributes, but also the amount of different values. You could have for example two super attributes only, but if each of them is having ten values, things will go crazy again.
And that's my actual issue. The combination of super attributes and their values makes the php memory consumption grow exponentially.

The steps that I'm doing are pretty basic. Once I'm having everything in the db, I simply call docker/sdk console publish:trigger-events. In rabbitmq I can see the publish messages coming in and being processed. Once it gets to the abstract product publish messages, it will exceed the memory. The queue is getting stuck then, i. e. it's picking the same bulk of 500 messages over and over again and retries forever.

UPWG9AYH2

My “experience” with this issue was, that the problem is growing more worse with different attributes than with different values.

Valerii Trots

I may be wrong but this "fix" has been developed specifically for Ingo's case as a quick win. Proper fix will follow as a big refactoring of the area.

@jeremy.fourna could you confirm this?

Jeremy Fourna

@UPKAYJJ2U

Jeremy Fourna

@valerii.trots the solution is not supposed to be a quick fix, we normally solved the root cause

U01LKKBK97T

Can't confirm, unfortunately.