Hello all, I am trying to extend DataImport by creating my own Importer Module that imports data fr

U048WDEP3R7

Hello all,

I am trying to extend DataImport by creating my own Importer Module that imports data from the Queue. Everything works fine except the triggering of the Publishing event $this->addPublishEvents(.... Here is the code for the writer step:

class AntelopeWriterStep extends PublishAwareStep implements DataSetWriterInterface
{
    public const KEY_NAME = 'name';
    public const KEY_COLOR = 'color';

    use DataSetWriterPersistenceStateAwareTrait;

    /**
     * @throws PropelException
     * @throws AmbiguousComparisonException
     */
    public function write(DataSetInterface $dataSet)
    {
        $this->setDataSetWriterPersistenceState(false);

        $antelopeEntity = PyzAntelopeQuery::create()->filterByName($dataSet[static::KEY_NAME])->findOneOrCreate();

        $antelopeEntity->setColor($dataSet[static::KEY_COLOR]);

        if ($antelopeEntity->isNew() || $antelopeEntity->isModified()) {
            $antelopeEntity->save();

            // for publish event
            $this->addPublishEvents(
                AntelopeEvents::ENTITY_PYZ_ANTELOPE_CREATE,
                $antelopeEntity->getIdAntelope()
            );
        }
    }

    public function flush()
    {
        $this->setDataSetWriterPersistenceState(true);
    }
}

And here is how I am adding the writer step in the factory :

public function getAntelopeQueueDataImporter(): DataImporter
    {
        /** @var DataImporter $dataImporter */

        $dataImporter = $this->createQueueDataImporter(
            $this->getConfig()->getAntelopeDataImporterConfiguration()
        );


        $dataSetStepBroker = $this->createTransactionAwareDataSetStepBroker();

        // validation step
        $dataSetStepBroker->addStep( $this->getAntelopeValidatorStep());

        // publisher step
        $dataSetStepBroker->addStep( $this->getAntelopePublisherStep());

        $dataImporter->addDataSetStepBroker($dataSetStepBroker);

        // writing step
        $dataImporter->setDataSetWriter(new AntelopeWriterStep());

        return $dataImporter;
    }

Can you please tell me if this the right way to do it?

fsmeier

Hey, how do you see that publishing is not working? Have you checked if your save() method is being called?

U048WDEP3R7

@florian.scholz I have another importer module that uses CSV file to import, this works fine with it. I mean this exact line of code:

 // for publish event
            $this->addPublishEvents(
                AntelopeEvents::ENTITY_PYZ_ANTELOPE_CREATE,
                $antelopeEntity->getIdAntelope()
            );
        }

fsmeier

wrong Florian mentioned 😉

fsmeier

Ok, can you check if the condition if ($antelopeEntity->isNew() || $antelopeEntity->isModified()) is being met? i mean in the writer-step you posted here. It feels like it is not even reached

U048WDEP3R7

Sorry about the wrong mention. Let me check and I will get back to you.

U048WDEP3R7

@florian.scholz It seems like it works with

class AntelopeValidatorStep extends PublishAwareStep implements DataImportStepInterface
{
   public function execute(DataSetInterface $dataSet)
    {
...

But not with

class AntelopeWriterStep extends PublishAwareStep implements DataSetWriterInterface
{
   public function write(DataSetInterface $dataSet)
    {
...

This will work for me 🙂

fsmeier

Interesting, tbh I dont know what is missing there right now - I sadly dont have time to debug further by myself the next two days but I will definitely look into this later that week because i am very curious about that.
Maybe another community member can help you. If not, please create a support ticket

U048WDEP3R7

Thank you very much 🙂

fsmeier

Hey, I finally had time to take a small look:
why do you have $this->setDataSetWriterPersistenceState(true); inside the flush() instead of DataImporterPublisher::triggerEvents();?

see:
https://github.com/spryker-shop/b2b-demo-shop/blob/202211.0/src/Pyz/Zed/DataImport[…]bstractStore/Writer/ProductAbstractStorePropelDataSetWriter.php