What are the Slack Archives?

It’s a history of our time together in the Slack Community! There’s a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.

Because this space is not active, you won’t be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..

Hi! We currently have an issue with out of sync elastic search, changes like cms placeholder do not

U01F7P3D9NH
U01F7P3D9NH Posts: 60 πŸ§‘πŸ»β€πŸš€ - Cadet
edited May 2021 in Help

Hi! We currently have an issue with out of sync elastic search, changes like cms placeholder do not appear on the frontend and some keys f.e. cms_pages are duplicated (with old and new version of the page) in the ES indexes. Trigger events not helped and sync:data fails with broken pipe in StreamIO.php on write. Any ideas?

Comments

  • UKEP86J66
    UKEP86J66 Posts: 208 πŸ§‘πŸ»β€πŸš€ - Cadet

    Maybe your RabbitMQ instance is flooded with too many messages, or it has some other problem?

  • UKEP86J66
    UKEP86J66 Posts: 208 πŸ§‘πŸ»β€πŸš€ - Cadet

    Sometimes if RMQ has memory issues it will block incoming connections. You can check the status in the UI on the 'connections' tab.

  • U01F7P3D9NH
    U01F7P3D9NH Posts: 60 πŸ§‘πŸ»β€πŸš€ - Cadet

    Thanks, there where a lot of msgs in the queues, connections looks fine and are alive as long the workers are running. Would you recommend to run sync:data again after queues are empty? Still have a bad feeling that index wont be updated correctly

  • UKEP86J66
    UKEP86J66 Posts: 208 πŸ§‘πŸ»β€πŸš€ - Cadet

    If you know specific things are wrong on your site you can re-run sync:data but limit it to the resource type to avoid flooding everything. eg sync:data cms_page_search

  • UKEP86J66
    UKEP86J66 Posts: 208 πŸ§‘πŸ»β€πŸš€ - Cadet

    You can also specify the cms page ids to limit it further.

  • UKEP86J66
    UKEP86J66 Posts: 208 πŸ§‘πŸ»β€πŸš€ - Cadet

    Another thing to check is where the sync:data command pulls data from - in this case it would be spy_cms_page_search

  • UKEP86J66
    UKEP86J66 Posts: 208 πŸ§‘πŸ»β€πŸš€ - Cadet

    If the data is incorrect there then the problem will be in the original data or the event trigger part.

  • U01F7P3D9NH
    U01F7P3D9NH Posts: 60 πŸ§‘πŸ»β€πŸš€ - Cadet

    Great, did not know about the possibility to filter! Also found now an error on publishing with duplicate entry in spy_url_storage that may explains the troubles. Thanks Rick!

  • UKEP86J66
    UKEP86J66 Posts: 208 πŸ§‘πŸ»β€πŸš€ - Cadet
    edited May 2021

    No problem, glad you have a suspect now. Incidentally I've seen duplicate key SQL errors on that table that might arise from lock contention, so running pub sync with the id parameter might help

  • U01LE4BMBK7
    U01LE4BMBK7 Posts: 241 πŸ§‘πŸ»β€πŸš€ - Cadet

    @U01F7P3D9NH check Jenkins

  • U01LE4BMBK7
    U01LE4BMBK7 Posts: 241 πŸ§‘πŸ»β€πŸš€ - Cadet

    more specifically the XX__queue-worker-start job

  • U01LE4BMBK7
    U01LE4BMBK7 Posts: 241 πŸ§‘πŸ»β€πŸš€ - Cadet

    over time your jenkins instance can run out of storage (hard drive)

  • U01F7P3D9NH
    U01F7P3D9NH Posts: 60 πŸ§‘πŸ»β€πŸš€ - Cadet

    The workers run fine, the real issue is a duplicated key for a cms_page in ES. Since none of the sync jobs helped to cleanup the index, I will drop and rebuildt it to prevent future publishing troubles as well

Categories