Skip to content

Add significant performance improvements for composer v2 - #608

Open
xvilo wants to merge 13 commits into
repman-io:masterfrom
xvilo:better-v2-implementation
Open

Add significant performance improvements for composer v2#608
xvilo wants to merge 13 commits into
repman-io:masterfrom
xvilo:better-v2-implementation

Conversation

@xvilo

@xvilo xvilo commented Sep 6, 2022

Copy link
Copy Markdown
Contributor

Currently, Repman is quite slow on the /packages.json endpoint, this due it's requesting all JSON files for every package from S3. From composer 2.0 and onwards, the API has been changed, it does not need to have all packages available any more. But, it now may provide the providers-url and metadata-url. This is where information for a specific package can be found. Thus, it does not have to wait for all packages to be resolved AND it's able to request info about the needed packages only.

If people are still using composer 1.0 it will add back the old data, so performance impact will only be solved for people using v2. It will however, show an extra warning.

Curl request statistics for /packages.json repo endpoint:

Currently:

{
"time_redirect": 0.000000,
"time_namelookup": 0.193779,
"time_connect": 0.203508,
"time_appconnect": 0.229232,
"time_pretransfer": 0.230318,
"time_starttransfer": 19.032120,
"time_total": 23.855557,
"size_request": 107,
"size_upload": 0,
"size_download": 1269753,
"size_header": 352
}

with these changes:

{
"time_redirect": 0.000000,
"time_namelookup": 0.018870,
"time_connect": 0.022539,
"time_appconnect": 0.042854,
"time_pretransfer": 0.043032,
"time_starttransfer": 0.260770,
"time_total": 0.260906,
"size_request": 177,
"size_upload": 0,
"size_download": 3345,
"size_header": 352
}

Looking at the total request time for our internal production deployment, we're going from a request time of 24 seconds, to around 260 milliseconds.

@xvilo xvilo changed the title Better v2 implementation Draft: Add significant performance improvements for composer v2 Sep 6, 2022
@xvilo
xvilo force-pushed the better-v2-implementation branch from 809616d to e2c5173 Compare September 6, 2022 14:22
@xvilo
xvilo force-pushed the better-v2-implementation branch from e2c5173 to 0d7119f Compare September 6, 2022 14:23
@xvilo
xvilo force-pushed the better-v2-implementation branch from 318a74c to b851200 Compare September 6, 2022 14:46
@codecov

codecov Bot commented Sep 6, 2022

Copy link
Copy Markdown

Codecov Report

Merging #608 (180897b) into master (f82251f) will decrease coverage by 0.01%.
The diff coverage is 96.96%.

@@             Coverage Diff              @@
##             master     #608      +/-   ##
============================================
- Coverage     99.16%   99.14%   -0.02%     
- Complexity     1910     1917       +7     
============================================
  Files           301      303       +2     
  Lines          6072     6092      +20     
============================================
+ Hits           6021     6040      +19     
- Misses           51       52       +1     
Impacted Files Coverage Δ
src/Controller/RepoController.php 98.70% <95.00%> (-1.30%) ⬇️
src/Service/Composer/ComposerEnvironment.php 100.00% <100.00%> (ø)
...rc/Service/Composer/ComposerEnvironmentFactory.php 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@xvilo xvilo changed the title Draft: Add significant performance improvements for composer v2 Add significant performance improvements for composer v2 Sep 6, 2022
@xvilo

xvilo commented Sep 6, 2022

Copy link
Copy Markdown
Contributor Author

Possible todo: add test for composer v1 and V2 package endpoint differences. Also test this with an invalid user agent. This will fix the missing coverage

@akondas

akondas commented Sep 10, 2022

Copy link
Copy Markdown
Member

I'm willing to accept this ... but can you add more tests for that? I'm not sure if i'm follow (sorry) and tests will give me more confidence 😉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants