i took some time to find out why my spryker installation is so incredibly slow on my machine. from p

U0251S9J5LP

i took some time to find out why my spryker installation is so incredibly slow on my machine. from previous investigations i already knew that it’s mainly because of NFS syncing which is quite slow on mac. but this doesn’t explain why it’s sometimes extremely slow, like gateway timeout slow… and i also already knew that dumping composer autoload helps. but this is actually more a production thingy… for me it was annoying to always run that dump command again after i created a new or moved a class … and still i didn’t know why it’s compared to other projects so super slow….

now i think i found out…

usually in PHP and composer each composer module has it’s own core namespace. this makes sense as using the same namespace has the risk that two files from different modules have the same namespace AND same class name, which would fail because of duplicate definition.
there’“s a way to create a “class name to file name” composer cache (“dumping the autoload”). this is fast, but like i said this caching mechanism may be a bit bothering when used during development as u need to execute it, if you delete/add/move classes and files. so usually it’s not done locally.

still - even on local environments without dumping the autoload - composer creates some cache - but it does only contain a list of directories where to start searching for each namespace from. this is done so composer doesn’t need to parse every composer.json of every installed vendor package again and again with each request.
so the cache would look like this:

• If namespace of a class starts with “\Foo” look for it in “vendor/foo/foo-module” and it’s sub dirs
• If namespace of a class starts with “\Twig” look for it in “vendor/twig/twig” and it’s sub dirs
• and so on
  so if every vendor module has it’s own namespace there should be just one directory mapped to one namespace. this is usually the case. right? right??? well, yes, but no. there’s an exception: frameworks like Spryker (or laravel, or symfony).
  here several modules for the same framework share the same core namespace.

usually frameworks don’t have a performance issue here but spryker is doing a bit different compared to the others: spryker registers for every spryker module in vendor/spryker the namespace \Spryker. so if composer searches for a file starting with namespace \Spryker (we have a loooot of them) it has to search every directory in /vendor/spryker/* until it finds the one where the file is located. with luck the file is located in one of the first directories. in bad case it’s in the last directory - in worst case the file doesn’t exist at all (happens during factory/facade… resolving sometimes). on bad days on my machine it took like 0.4s until it found \Spryker\Shared\Config\Environment. Just for this one single class!
what spryker could do to improve performance is to only register the module-specific namespaces... so config module should register \Spryker\Shared\Config, \Spryker\Yves\Config... not \Spryker. symfony is doing it like this, e.g here: https://github.com/symfony/security-http/blob/5.3/composer.json
it’s

"psr-4": { "Symfony\\Component\\Security\\Http\\": "" },
instead of
"psr-4": { "Symfony\\": "" },

or laravel:
https://github.com/laravel/ui/blob/3.x/composer.json

"Laravel\\Ui\\": "src/",
instead of
"Laravel\\": "src/",

but what can we do right now to speed it up without the need of “dumping the autoload”?
the following solution will create a “class to file name” cache for the Spryker (and SprykerShop) namespace only. if we create or move files from/to other namespaces we don’t have to bust the cache. changing Spryker files will only happen if we do composer install/update which will rebuild the cache.

in the project’s composer.json file add this to the “autoload-dev” property:

"classmap": [ "vendor/spryker", "vendor/spryker-shop" ]

then execute composer install

my whole autoload property looks like this:

"autoload-dev": {
    "psr-4": {
      ...
    },
    "classmap": ["vendor/spryker", "vendor/spryker-shop"]
  },

please mind:
• i know because of caching functionalities like composer dump-autoloading & enabling unresovable class cache in spryker performance will be much better on non-development machines
• i’ m assuming without this it would also be slow in docker but i only tested via vagrant
• i don’t know what changes did composer in version 2 regarding autoloading. maybe performance is better using v2.
the classmap-solution is just a workaround. @spryker maybe u can consider changing the composer.json namespace registerings in all your modules. besides the performance issue it’s good practise to don’t allow a module to declare classes in the core namespaces.

Chemaclass

Wow, this is a crazy good feedback 🙂
@valerii.trots, do know the best person to bring this message to? Maybe @UJ8Q2T36E?

sebastian.wagner

did you try dump-autoload -o ?

U01T075RRHD

But as far as I understood, composer is not searching for files if you configure PSR-4 autoloading. A file's path will be derived directly from its FQCN so no searching is needed. In that case it would be perfectly fine to have only the root namespace registered.

U01T075RRHD

I think the issue is with Spryker's resolving mechanism where it tries to look up files in various localtions (project leven namespace, project level store specific namespace, ...). This can result in quite some disk IO for a single file depending on how many core and project namespaces you have configured in Spryker.

Unknown

The main challenge, also for historical reasons this way regarding namespacing, is the mono repo approach for internal dev - https://github.com/spryker/spryker-core
It would need to be solved in a way that it works here still, while having the proper "psr4" convention then for each sub-module.
If this can be solved, we can for sure change this.

PS: Also testing namespace (and codeception) is far from ideal and throws lots of warnings. Here we are dependent to some extend on codecept way, they would also need to fix up their conventions.

U0251S9J5LP

@UPJJ1TM8S did u read my post? 😄 @U01T075RRHD it will search by the namespace registerings in the composer.json- i debugged that… adding the classmap in composer.json drastically improved performance on my and other machines

U01T075RRHD

What do you mean by search? Does it use some sort of glob patterns? I didn't do any debugging on this myself and am very curious about your findings.

U01T075RRHD

Will definitly give that classmap configuration a try

U0251S9J5LP

no, it doesn’t use glob… it can’t use a glob like /vendor/spryker/{moduleName}/ as there could be at least one module inside vendor/spryker (in theory) which does not register the sprykercore namespace. composer only collects the namespace registerings in composer.json of all packages and iterates over them… most of the magic happens here: \Composer\Autoload\ClassLoader::findFileWithExtension …$this->prefixLengthsPsr4[$first] will contain just the number of characters of the namespace + \ (“Spryker\” = 8)… $this->prefixDirsPsr4[$search] does contain all registered directories for that namespace… so for “Spryker” usually all spryker module dirs… these directories will be iterated… first file_exist will break the loop and return file the file name

U0251S9J5LP

the project i’m working on right now has > 440 spryker modules.. so there will be hundreds, maybe thousands of file_exists done - on a slow file system

U0251S9J5LP

@UQK3ZPJEN thanks for considering it! had a quick look into that repo and i think it should still work for more specific namespace registerings. so take that first module, Acl, for example. right now registering is this:

"autoload": {
        "psr-4": {
            "Spryker\\": "src/Spryker/",
            "SprykerTest\\Zed\\Acl\\Helper\\": "tests/SprykerTest/Zed/Acl/_support/Helper/",
            "SprykerTest\\Zed\\Acl\\PageObject\\": "tests/SprykerTest/Zed/Acl/_support/PageObject/"
        }
    },
    "autoload-dev": {
        "psr-4": {
            "SprykerTest\\": "tests/SprykerTest/"
        }
    },

could probably be changed to this:

"autoload": {
        "psr-4": {
            "Spryker\\Zed\\Acl": "src/Spryker/Zed/Acl", 
            "Spryker\\Shared\\Acl": "src/Spryker/Shared/Acl", 
            "SprykerTest\\Zed\\Acl\\Helper\\": "tests/SprykerTest/Zed/Acl/_support/Helper/",
            "SprykerTest\\Zed\\Acl\\PageObject\\": "tests/SprykerTest/Zed/Acl/_support/PageObject/"
        }
    },
    "autoload-dev": {
        "psr-4": {
            "Spryker\\Zed\\Acl": "src/Spryker/Zed/Acl", 
            "Spryker\\Shared\\Acl": "src/Spryker/Shared/Acl", 
        }
    },

U0251S9J5LP

i didn’t test it but it should work

U01T075RRHD

I just had a closer look at the composer autoloading and ... oh my ... this is painful. In one of the projects I'm working in we have 562 modules in the Spryker namespace, 112 modules in the SprykerShop namespace, and a couple more in the remaining SprykerEco, SprykerMiddleware, and SprykerSdk namespaces. This is especially bad when Spryker resolves files like e.g. a Facade where it probes agains all namespaces that could possibly contain that class file. It will go trough all that composer resolving functionality and could hit the filesystem hundreds of times for a single file. This explains why enabling the resolvable cache has such a big impact on performance 💡.
@U0251S9J5LP thanks so much for bringing that to everybody's attention

Denis Turkov

@UQK3ZPJEN could you please register this as a feature. Let’s check what we can do in Core to address this problem.

Denis Turkov

Thank you @U0251S9J5LP for great investigation!
We will use it to plan future module updates.