Hi I'm trying to figure out why `spy_product_abstract_category_storage` table is always empty and th

U03T08LE3CP

Hi I'm trying to figure out why spy_product_abstract_category_storage table is always empty and thus redis also has no product_abstract_category values. I run vendor/bin/console publish:trigger-events -r product_abstract_category but nothing happens. Ideas?

fsmeier

I know you likely have already checked, but just to be sure: have you set a breakpoint in vendor/spryker/product-category-storage/src/Spryker/Zed/ProductCategoryStorage/Communication/Plugin/Publisher/ProductCategoryPublisherTriggerPlugin.php and checked what exactly happens and which condition might fail?

U03T08LE3CP

that file has no conditions 🤔

U03T08LE3CP

something inside $this->createFilterTransfer($offset, $limit); ?

fsmeier

maybe not related, but maybe it is: i remember a flaw i was running into in a project: they removes some db entries by hand from the db. so the IDs started not with 1 but somewhere 101 or so. The spryker ootb iteration was via id’s but since the first pagination was resulting in no entries the whole process did not continue. I think this was also something about pub&sync, maybe just sync

U03T08LE3CP

this is really unknown territory for me, but I think(?) it does publish the all the products

U03T08LE3CP

where do I break to see the queue being read?

U03T08LE3CP

or was it db first then queue 🤔

fsmeier

The ProductCategoryPublisherTriggerPlugin should normally read from the db first (the original entity) and then perform the publish the entity. this should create a sync message. and this all should happen in bulks.

fsmeier

so i would set a breakpoint in getData() first

U03T08LE3CP

it reads 2000+ ids just fine, does ?? and then ??

fsmeier

if getData does return something then my thought about the missing ids would not be valid here.

U03T08LE3CP

I think it writes it to sync.storage.product?

fsmeier

oh, i think i was wrong. I have not set a breakpoint or anything (working on sth else right now) but by looking into the Plugin i think it DOES trigger publish events ProductCategory.category.publish as of getEventName() in the plugin

fsmeier

do you see right now with your breakpoint which class/method calls getData() ?

U03T08LE3CP

ProductCategoryPublisherTriggerPlugin.php:37, Spryker\Zed\ProductCategoryStorage\Communication\Plugin\Publisher\ProductCategoryPublisherTriggerPlugin->getData()
EventResourceRepositoryBulkPluginIterator.php:36, Spryker\Zed\EventBehavior\Business\Model\EventResourceRepositoryBulkPluginIterator->updateCurrent()
AbstractEventResourcePluginIterator.php:101, Spryker\Zed\EventBehavior\Business\Model\AbstractEventResourcePluginIterator->rewind()
EventResourceRepositoryManager.php:138, Spryker\Zed\EventBehavior\Business\Model\EventResourceRepositoryManager->processEventsForRepositoryBulkPlugins()
EventResourceRepositoryManager.php:105, Spryker\Zed\EventBehavior\Business\Model\EventResourceRepositoryManager->processEventsForBulkRepositoryPlugins()
EventResourceRepositoryManager.php:63, Spryker\Zed\EventBehavior\Business\Model\EventResourceRepositoryManager->processResourceEvents()
EventResourcePluginResolver.php:135, Spryker\Zed\EventBehavior\Business\Model\EventResourcePluginResolver->processResourceEvents()
EventResourcePluginResolver.php:81, Spryker\Zed\EventBehavior\Business\Model\EventResourcePluginResolver->executeResolvedPluginsBySources()
EventBehaviorFacade.php:142, Spryker\Zed\EventBehavior\Business\EventBehaviorFacade->executeResolvedPluginsBySources()
PublisherToEventBehaviorFacadeBridge.php:34, Spryker\Zed\Publisher\Dependency\Facade\PublisherToEventBehaviorFacadeBridge->executeResolvedPluginsBySources()
PublisherTriggerEventsConsole.php:103, Spryker\Zed\Publisher\Communication\Console\PublisherTriggerEventsConsole->execute()
Command.php:298, Symfony\Component\Console\Command\Command->run()
Application.php:1058, Symfony\Component\Console\Application->doRunCommand()
Application.php:301, Symfony\Component\Console\Application->doRun()
ConsoleBootstrap.php:114, Spryker\Zed\Console\Communication\Bootstrap\ConsoleBootstrap->doRun()
Application.php:171, Symfony\Component\Console\Application->run()
console:27, include()
console:120, {main}()

fsmeier

https://github.com/spryker/event-behavior/blob/master/src/Spryker/Zed/EventBehavior/Business/Model/EventResourceRepositoryManager.php#L210 yeah, so it triggers the publish event - and depending on how it is setup it should write some event message to a publish-queue or to the event-queue if its the older way

U03T08LE3CP

I'm lost what happens after that. Does it go into a RabbitMQ queue named "publish"?

fsmeier

You have to follow the publish event -> it is defined in a constant with the name PRODUCT_CATEGORY_PUBLISH -> i found it in vendor/spryker/product-category-storage/src/Spryker/Zed/ProductCategoryStorage/Communication/Plugin/Publisher/ProductCategory/ProductCategoryWriteForPublishingPublisherPlugin.php and this is plugged-in in src/Pyz/Zed/Publisher/PublisherDependencyProvider.php and no extra queue name defined so it will be put to the default “publish” queue.

fsmeier

Long story short: at least in the latest ootb b2b-demoshop it should be put to that queue by the default settings once you execute vendor/bin/console publish:trigger-events -r product_abstract_category

U03T08LE3CP

I traced the issue to ProductCategoryStorageRepository::getAllCategoryNodeAggregationsOrderedByDescendant , this query returns nothing 😓

SELECT (CASE WHEN spy_category_closure_table.depth = 0 THEN 1 ELSE 0 END) AS boostedDepth, spy_category_node.id_category_node AS idCategoryNode,
 spy_category_closure_table.fk_category_node_descendant AS idCategoryNodeDescendant, spy_category_node.fk_category AS idCategory,
 spy_category_attribute.name AS name, spy_url.url AS url, spy_locale.locale_name AS locale, spy_store.name AS store FROM spy_category_node
LEFT JOIN spy_url ON (spy_category_node.id_category_node=spy_url.fk_resource_categorynode)
INNER JOIN spy_category_closure_table ON (spy_category_node.id_category_node=spy_category_closure_table.fk_category_node)
INNER JOIN spy_category ON (spy_category_node.fk_category=spy_category.id_category)
LEFT JOIN spy_category_store ON (spy_category.id_category=spy_category_store.fk_category)
INNER JOIN spy_store ON (spy_category_store.fk_store=spy_store.id_store)
LEFT JOIN spy_category_attribute ON (spy_category.id_category=spy_category_attribute.fk_category)
INNER JOIN spy_locale ON (spy_category_attribute.fk_locale=spy_locale.id_locale)
WHERE spy_url.fk_locale = spy_category_attribute.fk_locale AND
      spy_category_closure_table.depth IS NOT NULL
      AND spy_category_node.is_root=false
ORDER BY boostedDepth DESC,spy_category_closure_table.depth ASC,spy_category_closure_table.fk_category_node_descendant DESC,spy_category_node.node_order DESC

U03T08LE3CP

what is this closure table?

fsmeier

tbh i did not know before by myself. I colleague of mine just told me that it allows you to get fast and easy all child categories via a single query. it’s like all relations of the category tree represented as a flat table

U03T08LE3CP

yeah I figured that out but who and what updates it?

U03T08LE3CP

we were forced to do some manual spy_url changes, I bet that data is stale

U03T08LE3CP

or hmm 🤔 I think I can fix the data

fsmeier

who and what updates it

This i dont know, sry 😕

U03T08LE3CP

re-importing categories, and reindexing redis seems to halve solved this. Something was off in spy_url data

U03T08LE3CP

.