What are the Slack Archives?
It’s a history of our time together in the Slack Community! There’s a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.
Because this space is not active, you won’t be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..
Good Morning, does anybody in this round has some practical experience with performance of P&S p
Good Morning,
does anybody in this round has some practical experience with performance of P&S process for on-demand requested items?
How long takes it in real to get the data from saving the database entry to redis/elasticsearch and is this timing stable? So for my theoretical understanding, the data could “wait” up to one minute until a new worker instance for p&s is started. But in practical it often does not feel that long.
Is there are also a way to speed this process up or to “enforce” a “synchronized” way?
Background: Fetching a customer specific price from an external system for a product triggered by a frontend call should immediately be available for further processes in redis/elasticsearch, so that the next frontend action can work with this prices from redis/elasticsearch. Here it is reasonable to have longer waiting time in frontend until the price is fetched. In my opinion this conflicts with the async behaviour of the whole spryker p&s concept. But maybe there is any concept to solve this problem.
Does anybody has experience with this?
Best
Comments
-
Hi Ingo, can i ask why do u want save/sync the customer prices for products in redis/ES? We have a similar concept, but we save the customer prices in own db redis after fetching (for some ttl) and then we have in post-save-plugins in cart just a plugin expander that read these data from redis if exists, otherwise perforn a new sap call, and then the price in cart are just the correct customer prices according logged customer.
Of course, there is more than one expander plugin that u have to write.. i.e. also for quick-order price-resolver etc...0 -
Hi Giovanni, we also have/had this idea to save it for a while in an extra space in redis. But my idea here would be to have minimal implementation effort and use spryker ootb concepts as much as possible. So one idea would be to save customer specific prices as a new price dimension in spryker. But this requires the whole process from saving it in the db and publish it to the storage/es … this takes time and you cant ensure that on the next call the price is already published and ready to use … but if it would be there, everything else would work out of the box i think.
0 -
Yes, understood... But u have to think/analyse it in according other aspects...
For example we have ca. 50K customers and ca. 3M Products...
In the case i choice for ootb solution with price dimension i think i have in a couple of months sure some problems... .. And.. don't forget also that in most cases the customer price can change every day (as in our case)... it means that the problem is not only bound to p+s for publish, but also for unpublish case..0 -
sounds reasonable in your case … but if i think of the code that must be changed to work spryker as expected … for example what about adding this product with this price to the cart? the complete cart calculation stack must be deactivated and post/pre plugins must set the correct price for the item?
0 -
and in worst case i have in db saved 50 000 * 3 000 000 = 150Miliard Kombination... Not possible with some frameworks and db engine..
No. the price calculation stack works further... we have in db saved only for all 3M products the Brutto price and in post save stack run own plugins that update the item prices according sap price data.. and then spryker (standard) after post save plugin run again the calculation... we have always right calculated cart....
-> The only place to get central in your code the price applying in cart is in postSavePlugins Stack in CartDependencyProvider..p.s.: of course in the case u have some additional cost from shipment or payment etc... it will be a little bit complicated...
0 -
and in worst case i have in db saved 50 000 * 3 000 000 = 150Miliard Kombination... Not possible with some frameworks and db engine..
Would be the case if every combination is requested in the end. In our case, a customer just get the specific price if he enter the pdp … but yes, in long term, doesn’t seem feasible
No. the price calculation stack works further... we have in db saved only for all 3M products the Brutto price and in post save stack run own plugins that update the item prices according sap price data..
So you use the merchant related prices feature on top?
spryker (standard) after post save plugin run again the calculation... we have always right calculated cart....
That means, there is also calculation logic on the spryker side? In our case the prices that get requested are calculated by the erp and dont need to pass calculation stack … but only if these are requested in the end. if not, the standard prices should be calculated (with calculation stack eventually)
0 -
- Ok
- We don't use merchant related prices. We have just the
DEFAULT
Price imported to avoid to change some business logic. But thanks the post save plugin the price are always right and customer specific. - In my case i need to pass the calculation stack (standard spryker do it after post save plugins) because as said the prices from erp (sap) can have an impact on shipment surcharge, payment surcharge and some other related prices... In the case, for some reason, sap middleware just not reply, the system will calculate everything with default prices saved in db (in our case brutto price).
p.s.: we are full in B2B concept and closed shop.
0 -
i.e, addToCart Operation (the same is for removeFromCart and for ReloadCart etc...)
0 -
Okay, i hope i got it now:
1. load the prices into a custom redis k:v structure when making an ERP request (like on pdp page)
2. on cart actions like addtocart, tell a postsaveplugin to fetch these prices from the custom redis structure instead of loading it from the default spryker db (?)0 -
Can you detect if the prices in the cart are now sap or spryker default prices?
0 -
I could also think of a unfavorable situation when there are mixed prices (default for some items, sap related for some other)
0 -
- yes
- make a erp call... (without to know if there is a cache or not). Business logic there has a complex cache logic.. in the case the request is full cached then return just the data from redis, otherwise make the erp call and save the cache from response..
p.s..: we have bulk request but we split the cache on single product
ps.2: yes i have a flag...
0 -
okay, sounds reasonable to me and since the cart just calculates only when there is a action on it, the effort seems to be moderate at all
0 -
yes.. 🙂 if u want we can speak about it in details in private.. i can show u how is implemented.. maybe u can save a lot of concept etc...
0 -
very interesting thread, indeed 🙂
saving ERP prices into redis - how fast did you come to this approach? Sounds like an optimisation, some shops try to keep it “on the fly” if response times are good though! In that case, do you specify/manage the ttl of the stored key in redis?That could also be interesting in a multi-store environment
0 -
Hi Andriy,
yes, we have a SapMiddlewareApi Bundle that in our System is the heart bundle... Then we have another bundle called SapMiddleareApiCache...
In the Request Class we have of course some stack plugins registered..
Every Request is just configurable to enable the cache and give a TTL for Redis..
For example for price we have 1day, for some other requests we have i.e. just 2 hours or similar..Of course we extended the Redis Adapter to allow TTL in
setex
and combination ofsetMulti
+expire
and to properly separate the configuration of other redis (KV, Storage) from this redis sap.It is very a huge of code, very difficult to explain just in few words...
About Mult-Store Env I think that is just the cache key generation to take care on Store (or maybe just configure a relation 1-1 store-db_redis
0 -
Have you considered other options like using DB spy_price*_storage calls directly by PriceProductStorageClient from Postgres/MariaDB?
0 -
Hi Stanislav,
we have ca, 3Mio Products and ca. 50K Customers.. in worst case I have ca. 150Miliarde Prices... Und die Preise ändern sich fast jeden Tag von SAP...
it means that I see a potential bottleneck using the storage table... and (important): the writing in this redis db (sap cache) must be sync otherwise (in case of worker problems or to much events) i have a lot of request to SAP for prices already requested, but not yes synced in redis.
0 -
don't forget (VERY IMPORTANT) -> SAP is transactional system and slow for real-time request.. it means that to much request to sap can be a bottleneck problem...
0 -
Hey @UL65CH0MC @UPWG9AYH2 I hope your SAP integrations work well ☀ ! I just wanted to check with you - which PHP libraries are you using in your integrations, maybe you could recommend any or the other way round - to stay away from? Thanks in advance!
0 -
Hi Andriy,
it is very complicated about library etc..
(i will create a Docker Image on Docker Hub for all, this is my goal).But of course we can have a call and i can explain u what we have done.. and what libraries we use in our middleware project..
0 -
Hi @UKJSE6T47, we are currently on other things and had no time working on this topic in the meantime. So, no useful input from me yet. But I’ll come back later to this topic again 🙂
0
Categories
- All Categories
- 42 Getting Started & Guidelines
- 7 Getting Started in the Community
- 8 Additional Resources
- 7 Community Ideas and Feedback
- 75 Spryker News
- 924 Developer Corner
- 783 Spryker Development
- 88 Spryker Dev Environment
- 362 Spryker Releases
- 3 Oryx frontend framework
- 35 Propel ORM
- 68 Community Projects
- 3 Community Ideation Board
- 30 Hackathon
- 3 PHP Bridge
- 6 Gacela Project
- 25 Job Opportunities
- 3.2K 📜 Slack Archives
- 116 Academy
- 5 Business Users
- 370 Docker
- 551 Slack General
- 2K Help
- 75 Knowledge Sharing
- 6 Random Stuff
- 4 Code Testing
- 32 Product & Business Questions
- 70 Spryker Safari Questions
- 50 Random