Hi spryker developers & architects, we have an interesting point regarding spryker scaling and

UMVENKCBF

Hi spryker developers & architects,

we have an interesting point regarding spryker scaling and rollouts of shops in Spryker Paas environments.

Let's assume we have the following setup:
• Spryker PaaS environment at AWS
• 15 shops (AT, DE, CZ, CH, ....)
• shared database persistence
• as in Spryker PaaS designed 1 redis instance
• usage of Jenkins as scheduler
• many custom jobs which are triggered in Jenkins nearly at them same time because of dependencies to the backend systems (like SAP, PIM, ...)
• default Jenkins scale at Spryker PaaS with 2vCPU and 4GB memory
Let's go in deeper:
• each Jenkins job CAN CONSUME 1 vCPU caused by PHP architecture
• 15 shops will trigger 15 Jenkins Jobs for processing the queue (jobs like XX__queue-worker-start) with a maximum of 1 vCPU
◦ Savepoint: 15 jenkins jobs
• for concurrency it will be nice to have a second Jenkins job (worker) for another jobs which will be triggered (like imports, exports, ..., sync with external systems)
◦ Savepoint: 15 jenkins jobs
In summary we can plan (minimum worst case scenario) 30 concurrent running Jenkins jobs on one virtual machine.

Our thoughts are - this setup does not scale with additional shops and Jenkins jobs. There will be a problem in future with concurrency and it's not the task from development team to plan Jenkins job concurrency - system infrastructure should scale with new shops.
Jenkins jobs will block each other or jobs can consume the amount a maximum of cpu power.
In addition we should note the default usage of jenkins which consumes cpu power....

Whats your opinion? What are your experiences regarding this topic in Spryker Paas? How many shops running in parallel with the default setup?

Valerii Trots

My personal experience and recommendation would be to have 1 vCPU per store.
So in case of 15 stores 16 vCPU instance should be enough to deal with the load if the application is optimized (like imports run only for deltas, etc).

UMVENKCBF

In theory: yes. Means one jenkins job will block or reserve the cpu power from 1 vCPU.
If you have long running job(s) you might ran in problems.
Worst case: if 15 workers for importing products will ran with full cpu power there is no power left for running the queue worker jobs.

Scaling in Spryker PaaS for jenkins is in dependency to rolled out shops, right? Or are there any other scaling options like jenkins master / slave concept with multiple virtual machines (ec2 instances)?

Valerii Trots

Didn't see a case where there are 15 import jobs in parallel.
Also queue:worker console command has -s option so it does't run at idle for 1 minute if the queue is empty.

The only option of scaling for Jenkins in the cloud is vertical.
Locally you have Jenkins that curls to zed\backoffice and zed\backoffice executes php.
In PaaS Jenkins has application code inside itself and runs php on its own.

U013EK8RD7A

@valerii.trots 😄, special for You

U013EK8RD7A

we have 20 executors, so 5 are available for regular jobs (such as queue consumer) all these jobs consume 60% - 70% of CPU power of t3.xlarge EC2 instance.

@UMVENKCBF, for Jenkins queue worker You could configure this:

$config[QueueConstants::QUEUE_ADAPTER_CONFIGURATION] = [
    EventConstants::EVENT_QUEUE => [
        QueueConfig::CONFIG_QUEUE_ADAPTER => RabbitMqAdapter::class,
        QueueConfig::CONFIG_MAX_WORKER_NUMBER => 5,
    ],
    PublisherConfig::PUBLISH_QUEUE => [
        QueueConfig::CONFIG_QUEUE_ADAPTER => RabbitMqAdapter::class,
        QueueConfig::CONFIG_MAX_WORKER_NUMBER => 10,
    ],
];

UMVENKCBF

Okay thanks @U013EK8RD7A - how it's working with your case / configuration.
Nice to see that there is some tweaking and that instance typ assigned to your landscape.

UP8T3GKFH

Hello @U013EK8RD7A, could you please explain to me how this works with the 10 workers, when a t3.xlarge only has 4 vCPUs available. My knowledge is that you should not enter more workers than CPU cores, as a worker reserves a core for itself. This would create a massive queue of workers, wouldn't it?