Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invoke the resolver in the same process as pipenv rather than utilzing subprocess. #5787

Merged
merged 12 commits into from
Jul 11, 2023

Conversation

matteius
Copy link
Member

@matteius matteius commented Jul 10, 2023

  • Invoke the resolver in the same process as pipenv rather than utilizing subprocess.
  • Restore accidentally commented out part of pip validations.
  • Make a test case faster (fastapi is actually got a lot of dependencies).
  • Spend several hours trying to track down why the tests have random failures in parallelism:
    ** Completely trim down the conftest.py and remove uneeded arguments
    ** try to ensure the directory is totally unique for the test
    ** Realize that click's CliRunner is actually to blame and even says in the docs that its not thread safe.
    ** Convert away from using CliRunner (mostly) for some reason the graph --reverse test is a special beast with unicode characters and a sub-process in a sub-process, so just use CliRunner there only.
  • Made a few edits to requirementslib but trying to keep that minimal as I intend to do more work there in other PRs.

The issue

In thinking more about the issue that arose with the resolver using a different python version, and modules maybe being out of scope, I began to wonder why we were kicking it off as a subprocess. This reduces the number of files we have to write and read back in as well as eliminate the sub-process that we simply were waiting for anyway.

The checklist

  • A news fragment in the news/ directory to describe this fix with the extension .bugfix.rst, .feature.rst, .behavior.rst, .doc.rst. .vendor.rst. or .trivial.rst (this will appear in the release changelog). Use semantic line breaks and name the file after the issue number or the PR #.

…g subprocess. Restore accidentally commented out part of pip validations.
@matteius matteius requested a review from oz123 July 10, 2023 19:05
@matteius matteius closed this Jul 11, 2023
@matteius matteius reopened this Jul 11, 2023
Copy link
Contributor

@oz123 oz123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really great.
As we already spoke about this, I wonder why it was in the beginning subprocess.
Maybe it was a way of doing some parallelization?
We removed some subprocess parallelization code in the past.

In any case, this makes pipenv even faster.

@oz123 oz123 merged commit 70b77ae into main Jul 11, 2023
19 checks passed
@oz123 oz123 deleted the resolver-no-subprocess branch July 11, 2023 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants