Sudden random fail whales and gateway exceptions when running app in debug mode

filip.galic10
filip.galic10 Spryker Solution Partner Posts: 9 🧑🏻‍🚀 - Cadet

Since 1 day ago, my entire team has been experiencing random fail whales and gateway issues when running app in debug mode. I am not aware of any specific update in the code that could cause this. The only common thing is that we are all on docker and mac os.
This happens usually when there is an interaction between yves and zed.

Is there any way to troubleshoot this further, or maybe somebody has experienced something simillar.
The exception is always different depending on what interaction with gateway is happening.

Tagged:

Comments

  • filip.galic10
    filip.galic10 Spryker Solution Partner Posts: 9 🧑🏻‍🚀 - Cadet

    This solved the issue (disabling in the deploy yml file). Just weird since it was there for the past some months, and just suddenly started to cause issues for us.
    Thanks!

  • fsmeier
    fsmeier Senior Software Engineer & Developer Enablement Advocate Sprykee Posts: 1,051 ⚖️ - Guardians (admin)

    Heyhey,

    I just heard about this from other devs - thats why I wanted to check. Once I know more I will update here. If you have access to the support portal, please file an official ticket.

    All the best,

    Florian

  • fsmeier
    fsmeier Senior Software Engineer & Developer Enablement Advocate Sprykee Posts: 1,051 ⚖️ - Guardians (admin)
    edited February 7

    Heyhey,
    just another idea to make sure its nothing with cache: if you enable blackfire again, does it still work or is it broken again?

    All the best,

    Florian

  • sebastian.wagner
    sebastian.wagner Spryker Solution Partner Posts: 9 🧑🏻‍🚀 - Cadet

    Greets @fsmeier :)
    Since yesterday i could experience the same issue. I just switched to docker-sdk 1.60.0 (+ jenkins2.442).
    In the logs I could witness quiet some of those:

    [zed-backoffice] [backoffice_eu] - NOTICE: PHP message: Xdebug: [Step Debug] Could not connect to debugging client. Tried: host.docker.internal:9003 (through xdebug.client_host/xdebug.client_port).
    [zed-backoffice] [backoffice_eu] - [06-Feb-2024 15:00:54] WARNING: [pool debugger] child 20 exited on signal 11 (SIGSEGV) after 12.399835 seconds from start
    [zed-backoffice] [backoffice_eu] - [06-Feb-2024 15:00:54] NOTICE: [pool debugger] child 22 started
    [zed-backoffice] [backoffice_eu] - [06-Feb-2024 15:00:57] WARNING: [pool debugger] child 21 exited on signal 11 (SIGSEGV) after 10.540334 seconds from start
    [06-Feb-2024 15:00:57] NOTICE: [pool debugger] child 23 started
    [zed-backoffice] [backoffice_eu] - NOTICE: PHP message: Xdebug: [Step Debug] Could not connect to debugging client. Tried: host.docker.internal:9003 (through xdebug.client_host/xdebug.client_port).

    It only happens on every first klick/navigation in the backoffice, while further trials succeeded navigating the BO. Also Ajax calls for tables seem to succeed.
    Disabling xdebug cookies in the browser fixes it for now.
    Also mind that my IDE would allow up to 3 concurrent connections.
    I think it could be some timing issue or even new internal host name determination failing.

    Am I missing some config changes introduced along SDK 1.60.0?

  • sebastian.wagner
    sebastian.wagner Spryker Solution Partner Posts: 9 🧑🏻‍🚀 - Cadet

    I can affirm, that disabling blackfire in the deploy file resolves spurious failwhales for me on docker-sdk 1.60.0+master with even xdebug active.

    [zed-backoffice] [backoffice_eu] - NOTICE: PHP message: Xdebug: [Step Debug] Could not connect to debugging client. Tried: host.docker.internal:9003 (through xdebug.client_host/xdebug.client_port).
    [zed-backoffice] [backoffice_eu] - NOTICE: PHP message: Xdebug: [Step Debug] Could not connect to debugging client. Tried: host.docker.internal:9003 (through xdebug.client_host/xdebug.client_port).
    [zed-backoffice] [backoffice_eu] - NOTICE: PHP message: Xdebug: [Step Debug] Could not connect to debugging client. Tried: host.docker.internal:9003 (through xdebug.client_host/xdebug.client_port).

    Messages like those still reside, but no new warnings occur.

  • Hidran Arias
    Hidran Arias Senior Technical Trainer Sprykee Posts: 81 🏛 - Council (mod)

    The Xdebug messages come out when you have the debug cookie or url param but you're not listening for connections on your IDE.

    Could you please verify if by turning on the listen for connection on your IDE if the messages disappear?

    I see them when I execute : docker/sdk cli -x

    and I'm not listening for Debug connections

  • oezkan.yilmaz
    oezkan.yilmaz Posts: 2 🧑🏻‍🚀 - Cadet

    Hey there,

    I am just hooking into this topic.

    Currently we are also facing gateway timeout issues. I can reproduce it when starting containers with sdk/cli -x or sending XDEBUG_SESSION=PHPSTORM in postman.

  • simo.savonen
    simo.savonen Spryker Solution Partner Posts: 8 🧑🏻‍🚀 - Cadet

    The topic could include "502 bad gateway" to make it easier to find.

    It helped to remove blackfire from deploy.dev.yml

    docker/sdk boot deploy.dev.yml shows a new red error about bad boolean value, but that I can live with.

  • fsmeier
    fsmeier Senior Software Engineer & Developer Enablement Advocate Sprykee Posts: 1,051 ⚖️ - Guardians (admin)

    Seems like xdebug fixed it: https://github.com/xdebug/xdebug/releases/tag/3.3.2

    I will bring this to the internal team to react asap :) Will keep you updated!

    All the best,

    Florian

  • fsmeier
    fsmeier Senior Software Engineer & Developer Enablement Advocate Sprykee Posts: 1,051 ⚖️ - Guardians (admin)

    Heyhey,

    somebody still has the problem with blackfire extension loaded?

    I can not reproduce it anymore and Spryker uses already xdebug 3.3.2. For me it may was solved by doing a docker/sdk pull to get the latest images.

    If you still have the problem, can you please provide me the following information before you execute the docker/sdk pull ?

    Execute docker images , look for the TAG "dev-backoffice_eu" and copy the image-id, then execute docker inspect 7ec6558133ea and tell me the date of your image "LastTagTime".

    cc @mohammed.shamim

    All the best,

    Florian