What are the Slack Archives?

It’s a history of our time together in the Slack Community! There’s a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.

Because this space is not active, you won’t be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..

Hello, currently having some trouble creating/updating categories in via import. It seems that in th

UPWG9AYH2
UPWG9AYH2 Posts: 509 🧑🏻‍🚀 - Cadet
in Help

Hello,
currently having some trouble creating/updating categories in via import. It seems that in the CategoryRepository the getCategoryNodeUrlPathParts() method has a problem with $nodeQuery->find()->toArray() … while $nodeQuery->find() returns a result (4 in this case), the toArray runs into memory exception after a while … any known issues here? Might this be a propel related issue?
Best regards

Comments

  • Alberto Reyer
    Alberto Reyer Posts: 690 🪐 - Explorer

    Only a guess, but maybe inside the toArray method all (which I assume is using the ArrayFormatter behind the scenes) all relations are loaded?
    As far as I understood the getCategoryNodeUrlPathParts it queries the spy_category_closure table, which can cause a recursive loading of categories.

    But be aware that this is only a speculation, haven't tested it.

  • UPWG9AYH2
    UPWG9AYH2 Posts: 509 🧑🏻‍🚀 - Cadet

    Yep, i think this is the right direction, it seems recursive loading somehow breaks everything. Without toArray, lazy loading might prevent the memory error … I am currently looking if there is a circular reference between the nodes. Hopefully yes, because otherwise … hmmm …

  • UPWG9AYH2
    UPWG9AYH2 Posts: 509 🧑🏻‍🚀 - Cadet

    Hmmm … seems not to be any circular reference … but also closure seems not to be the bad guy since i commented out any additional useXyzQuery … only

    SpyCategoryNodeQuery::create($modelAlias, $criteria)

    seems to cause the memory exception. Still no clue here

  • UPWG9AYH2
    UPWG9AYH2 Posts: 509 🧑🏻‍🚀 - Cadet

    Running out of ideas … seems to me a bug. At least in the node hierarchy theres no circular reference.

  • UPWG9AYH2
    UPWG9AYH2 Posts: 509 🧑🏻‍🚀 - Cadet

    Okay, i tried today to follow the recursion tree and it seems legit what toArray() is doing in a technical way … however, i guess in the end, toArray() tries to load the half database into an array since everything is connected to each other … category_node, category_template, … at least a 32GB Macbook Pro stops also with memory_limit -1 in the end …

  • UPWG9AYH2
    UPWG9AYH2 Posts: 509 🧑🏻‍🚀 - Cadet

    One step forward: I changed something in the backoffice for the exact same node id which ends up in the exact same sql query and i am wondering why from backoffice the toArray() will work and from update method from our import it wont even though they have the same criteria filter… then i noticed that the difference is in the dataset coming back from

    $nodeQuery->find()

    while from backoffice the additional columns aCategory etc. is empty, from our import, this data is filled … there must be somewhere a difference when getting the connection, data filter, lazy loading etc … i still have no clue but i am on it … any help welcome …

  • UPWG9AYH2
    UPWG9AYH2 Posts: 509 🧑🏻‍🚀 - Cadet

    Okay, next update: Seems the problem that there are already instances on propel instance pool which does (in the case of the import call) leads to this error since aCategories, aParentCategories etc. is (unlike the backoffice based call) filled … i added temporary InstancePoolingTrait to the CategoryRepository and deactivated it with $this->disableInstancePooling() and tada … its working …

  • UPWG9AYH2
    UPWG9AYH2 Posts: 509 🧑🏻‍🚀 - Cadet

    But this means, there is a dependency to previous Queries, which should imo not resolved by deactivating the instance pool

Categories