What is the relationship with `publish:trigger-events` and `sync:data` ? If I run `publish:trigger-e

U03T08LE3CP

What is the relationship with publish:trigger-events and sync:data ? If I run publish:trigger-events when do I need to run sync:data?

U01BZ7Q3XRV

publish:trigger-events triggers all propel entity events (as far as I understand, probably the update event for all entities) which leads to the first step of publish / synchronize being executed: the denormalized data is written to the _storage and _search tables in the database. This then automatically triggers the events that lead to the sync part of the publish / sync: writing all the data from _storage tables to redis and all the data from _search tables to elasticsearch. the sync:data command only does the last part, transfering the data from _storage and _search tables to redis and elasticsearch (while _storage and _search are only naming conventions. I think they can be named differently, and also there are at least 1 or 2 tables with _search suffix that do something different)

U01BZ7Q3XRV

so if you only want to sync for example from spy_product_concrete_page_search to elasticsearch, sync:data (-r product_concrete I think also works) would be sufficient. if spy_product_concrete_page_search does not contain the latest data, you need publish:trigger-events (-r product_concrete) to publish the data from spy_product to spy_product_concrete_page_search

U03T08LE3CP

so if I do say, manipulations to image sets in the database directly, should it suffice to publish:trigger-events -r product_abstract_image?

U01BZ7Q3XRV

yes, it should 👍 (I think 😛 )

U03T08LE3CP

I'm starting to get really hesitant to make any SQL changes since I cannot seem to get the indices update cleanly..

U03T08LE3CP

now the imageSets value in glue abstract_products is empty and I don't know why

U01BZ7Q3XRV

hm ok. So you can go step by step from the database side, and look at the spy_product_abstract_image_storage table and see if it contains the correct data

U01BZ7Q3XRV

and if yes, the next step would be to look into the storage (probably redis). you can get the redis key for this from the spy_product_abstract_image_storage table

U01BZ7Q3XRV

or one question: are you querying one specific product or a list of products? if it is a list it might be _search table and elasticsearch instead

U03T08LE3CP

queyring ong product

U03T08LE3CP

/abstract-products/001_4059729207241?include=concrete-products,abstract-product-image-sets,concrete-product-prices,concrete-product-availabilities,category-nodes,concrete-product-image-sets

U03T08LE3CP

something like that

U01BZ7Q3XRV

then the data should come from the redis storage which is (or should be) synced from the spy_product_abstract_image_storage table. I'm not 100% sure about the glue part (because I did not use glue myself, yet): You included the image sets, maybe there is also another include needed for the actual image data?

U03T08LE3CP

I know this is right it just doesn't reflect the db changes

U01BZ7Q3XRV

while there is only one storage table for abstract images so probably they are combined during publish

U01BZ7Q3XRV

then you should look at the spy_product_abstract_image_storage table and search for the product to see if this is correct

U01BZ7Q3XRV

you might also want to look at rabbitmq, if there are any messages on one of the publish.error queues. If so, you can look at the messages in the rabbitmq UI and see if they contain an error message (if something is going wrong during publish)

U03T08LE3CP

storage looks right

U03T08LE3CP

error queues are empty

U01BZ7Q3XRV

then the problem is probably that the data is not updated to redis (so the sync:data part). Depending on your infrastructure you might be able to use some client to connect to it and look at the values. But there is also a UI in Zed to access it under maintenance -> storage I think

U01BZ7Q3XRV

you can use the key for the data set from the spy_product_abstract_image_storage table and search for it in the storage UI

U01BZ7Q3XRV

(in Zed it must be an exact match of the key I think)

U03T08LE3CP

yeah it's redis 🙂

U03T08LE3CP

but I have tinkered so that I have cli access to it as well

U01BZ7Q3XRV

ok, I'm not very familiar with the cli (other then deleting everything 😄 ), but you should be able to search for the key there quite easily

U01BZ7Q3XRV

I think its just GET ... 😛

U03T08LE3CP

the root cause might be that I didn't quite get how the product-image connection works

U01BZ7Q3XRV

in glue?

U03T08LE3CP

in the database

U01BZ7Q3XRV

ah ok