What are the Slack Archives?

It’s a history of our time together in the Slack Community! There’s a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.

Because this space is not active, you won’t be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..

Hello all, I am trying to extend DataImport by creating my own Importer Module that imports data fr

U048WDEP3R7
U048WDEP3R7 Posts: 217 🧑🏻‍🚀 - Cadet

Hello all,

I am trying to extend DataImport by creating my own Importer Module that imports data from the Queue. Everything works fine except the triggering of the Publishing event $this->addPublishEvents(.... Here is the code for the writer step:

class AntelopeWriterStep extends PublishAwareStep implements DataSetWriterInterface
{
    public const KEY_NAME = 'name';
    public const KEY_COLOR = 'color';

    use DataSetWriterPersistenceStateAwareTrait;

    /**
     * @throws PropelException
     * @throws AmbiguousComparisonException
     */
    public function write(DataSetInterface $dataSet)
    {
        $this->setDataSetWriterPersistenceState(false);

        $antelopeEntity = PyzAntelopeQuery::create()->filterByName($dataSet[static::KEY_NAME])->findOneOrCreate();

        $antelopeEntity->setColor($dataSet[static::KEY_COLOR]);

        if ($antelopeEntity->isNew() || $antelopeEntity->isModified()) {
            $antelopeEntity->save();

            // for publish event
            $this->addPublishEvents(
                AntelopeEvents::ENTITY_PYZ_ANTELOPE_CREATE,
                $antelopeEntity->getIdAntelope()
            );
        }
    }

    public function flush()
    {
        $this->setDataSetWriterPersistenceState(true);
    }
}

And here is how I am adding the writer step in the factory :

public function getAntelopeQueueDataImporter(): DataImporter
    {
        /** @var DataImporter $dataImporter */

        $dataImporter = $this->createQueueDataImporter(
            $this->getConfig()->getAntelopeDataImporterConfiguration()
        );


        $dataSetStepBroker = $this->createTransactionAwareDataSetStepBroker();

        // validation step
        $dataSetStepBroker->addStep( $this->getAntelopeValidatorStep());

        // publisher step
        $dataSetStepBroker->addStep( $this->getAntelopePublisherStep());

        $dataImporter->addDataSetStepBroker($dataSetStepBroker);

        // writing step
        $dataImporter->setDataSetWriter(new AntelopeWriterStep());

        return $dataImporter;
    }

Can you please tell me if this the right way to do it?

Comments

  • fsmeier
    fsmeier Senior Software Engineer & Developer Enablement Advocate Sprykee Posts: 1,072 ⚖️ - Guardians (admin)

    Hey, how do you see that publishing is not working? Have you checked if your save() method is being called?

  • U048WDEP3R7
    U048WDEP3R7 Posts: 217 🧑🏻‍🚀 - Cadet
    edited November 2022

    @florian.scholz I have another importer module that uses CSV file to import, this works fine with it. I mean this exact line of code:

     // for publish event
                $this->addPublishEvents(
                    AntelopeEvents::ENTITY_PYZ_ANTELOPE_CREATE,
                    $antelopeEntity->getIdAntelope()
                );
            }
    
  • fsmeier
    fsmeier Senior Software Engineer & Developer Enablement Advocate Sprykee Posts: 1,072 ⚖️ - Guardians (admin)

    wrong Florian mentioned 😉

  • fsmeier
    fsmeier Senior Software Engineer & Developer Enablement Advocate Sprykee Posts: 1,072 ⚖️ - Guardians (admin)

    Ok, can you check if the condition if ($antelopeEntity->isNew() || $antelopeEntity->isModified()) is being met? i mean in the writer-step you posted here. It feels like it is not even reached

  • U048WDEP3R7
    U048WDEP3R7 Posts: 217 🧑🏻‍🚀 - Cadet

    Sorry about the wrong mention. Let me check and I will get back to you.

  • U048WDEP3R7
    U048WDEP3R7 Posts: 217 🧑🏻‍🚀 - Cadet
    edited November 2022

    @florian.scholz It seems like it works with

    class AntelopeValidatorStep extends PublishAwareStep implements DataImportStepInterface
    {
       public function execute(DataSetInterface $dataSet)
        {
    ...
    

    But not with

    class AntelopeWriterStep extends PublishAwareStep implements DataSetWriterInterface
    {
       public function write(DataSetInterface $dataSet)
        {
    ...
    

    This will work for me 🙂

  • fsmeier
    fsmeier Senior Software Engineer & Developer Enablement Advocate Sprykee Posts: 1,072 ⚖️ - Guardians (admin)

    Interesting, tbh I dont know what is missing there right now - I sadly dont have time to debug further by myself the next two days but I will definitely look into this later that week because i am very curious about that.
    Maybe another community member can help you. If not, please create a support ticket

  • U048WDEP3R7
    U048WDEP3R7 Posts: 217 🧑🏻‍🚀 - Cadet

    Thank you very much 🙂

  • fsmeier
    fsmeier Senior Software Engineer & Developer Enablement Advocate Sprykee Posts: 1,072 ⚖️ - Guardians (admin)

    Hey, I finally had time to take a small look:
    why do you have $this->setDataSetWriterPersistenceState(true); inside the flush() instead of DataImporterPublisher::triggerEvents();?

    see:
    https://github.com/spryker-shop/b2b-demo-shop/blob/202211.0/src/Pyz/Zed/DataImport[…]bstractStore/Writer/ProductAbstractStorePropelDataSetWriter.php