Hello, we want to extend the spryker customer page area, especially the order history part. We want

UPWG9AYH2

Hello,
we want to extend the spryker customer page area, especially the order history part. We want to import additional information from our ERP system to spryker and show it to the customer.
We noticed, that already existing information here is fetched from Zed backend (like orders) and never from redis/elastic. Since we want to make parts of the customer area searchable in future and extend it much more we are not sure were we should provide which data then. Zed or Redis/Elasticsearch (?) …
So, what is the “rule” oder technical considerations when we should consider providing data using the P&S Flow to redis/elasticsearch and when it should “remain” in backend … also considering performance
Best

Alberto Reyer

Mainly the amount of data, the access pattern and the update frequency should be considered for the decision. Another important point is the data security.
All data that is way more often read than written should be put into redis. I personally would keep all customer individual data inside Zed and not put them into redis/elasticsearch for a few reasons.

1. Data security requirements are much higher for customer data (order address vs. product name)
2. If only a small subset of the data is accessed in parallel (e.g.: orders) a DB search with a good index will be way more efficient and less complex than P&S
3. Orders are a lot more data than products so your Elasticsearch/redis will grow a lot over time, without a huge benefit gain from it
4. P&S all order data (some of the orders are updated very often at least for a short timeframe, e.g.: as long as the order is open) will cause a lot of events which might require you to scale the messagequeue as well

Alberto Reyer

Those non functional requirements should be compared against the benefits you expect from putting them into redis/ES. If the only reason is to be able to have a fuzzy search, this can be achieved in a DB as well (https://mariadb.com/kb/en/full-text-index-overview/), which is way more efficient especially if only a subset of the data needs to be searched, as it is the case for order search were only the orders of one customer/company need to be searched.

UPWG9AYH2

Hi Alberto,
thanks for your response.
In terms of read/write ratio … i would rate it as almost read-only which speaks for Redis/Elasticsearch … in terms of performance my first guess was, that its the way around … to have more performance using Redis/Elasticsearch especially for huge data amounts and search functionality compared to db access … in terms of security … seems that elastic/redis is not secure (?) But its definitely more exposed than db layer, i agree …
With complexity, i fully agree. Zed is much less complex than implementing P&S.

Since the default spryker customer area is very simple, i could imagine that the decision for using Zed was made just to not overengineer it for such a simple case. But when we use it more extensive, the case might be another …

However, i think it can be also a good idea to start with Zed backend solution and put P&S later on top when performance/system load is not sufficient

Best regards

Alberto Reyer

i would rate it as almost read-only which speaks for Redis/Elasticsearch

Along the whole lifetime of a product vs. order how often is it read vs. written? I would guess that a product through his whole lifecycle might be written a few times (maybe 1000 times) vs. read a lot more (guess >100x).
Same for an order, depending on the process it is updated a few times (assuming 5x for the example) and maybe viewed 10-20 times (factor 2-4). Also consider who is accessing the data, a product will be accessed by a wide range of customers vs. an order that is only accessed by a specific customer.

to have more performance using Redis/Elasticsearch especially for huge data amounts

That's true when you need to search though a big part of the data all the time. But this is not the case for orders as you already can pre filter all orders to those that are related to the customer and only apply the search filters to those subresult and even on those this is not applied to all fields but to a subset of fields.

However, i think it can be also a good idea to start with Zed backend solution

That's what I suggest. If it this is not sufficient have a look on the querys on the DB, it might already by solved by better indizes.
If not, maybe DB read replicas are a way to solve it. Offloading order data into the Elasticsearch would be one of the last steps I would suggest.

U018XELUZS9

We're loading that also via Zed, just make sure to profile that page with a customer that has multiple orders. We found some bottlenecks in the database queries and were able to increase the performance by a huge factor, thanks to detailed insights via tideways.

I would stop optimizing the query if the execution time is below 1ms. Some consider a query "fast enough" when it takes less than 100ms, but that's already pretty slow from my point of view.