What are the Slack Archives?

It’s a history of our time together in the Slack Community! There’s a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.

Because this space is not active, you won’t be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..

Hey guys, do you know how to enlarge the usable memory for a single data import console command? `d

michael.schmetter
michael.schmetter Posts: 119 🧑🏻‍🚀 - Cadet
edited February 2023 in Help

Hey guys,
do you know how to enlarge the usable memory for a single data import console command?

docker/sdk console data:import:my-stuff

I get the following error processing a quite large dataset:
PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted

Comments

  • UP8T3GKFH
    UP8T3GKFH Posts: 56 🧑🏻‍🚀 - Cadet

    Hello Michael, you could increase the global memory limit in your deploy.*.yml like in this screenshot.

  • michael.schmetter
    michael.schmetter Posts: 119 🧑🏻‍🚀 - Cadet

    I've seen that, but is there any chance to enlarge it only for this command (One time import)

  • UP8T3GKFH
    UP8T3GKFH Posts: 56 🧑🏻‍🚀 - Cadet

    If you can start the import manually you can switch to docker/sdk cli and then try php -d memory_limit=4096M vendor/bin/console data:import:my-stuff

  • UP8T3GKFH
    UP8T3GKFH Posts: 56 🧑🏻‍🚀 - Cadet

    or try
    docker/sdk cli php -d memory_limit=4096M vendor/bin/console data:import:my-stuff

  • michael.schmetter
    michael.schmetter Posts: 119 🧑🏻‍🚀 - Cadet

    Worked like a charm! Thanks!

  • Alberto Reyer
    Alberto Reyer Lead Spryker Solution Architect / Technical Director Posts: 690 🪐 - Explorer

    Another option would be to fix the exhaustive memory usage.
    Wild guess, you did not disabled the propel instance pooling, which means every entity loaded during the process run is kept in memory.

    Propel::disableInstancePooling();

    at the beginning of your import will solve this issue.
    Downside is a slight increase of the runtime

Categories