Automate a periodic check of FawltyDeps on popular Python projects/packages

[jherland]

For an ongoing Tweag blog post, we (Maria and Zhihan in particular) have done a significant amount of work to run FawltyDeps across a large collection of third-party projects. It would be smart to leverage this work in an automated periodic action to verify that FawltyDeps keeps working on a large and relevant collection of third-party projects.

Reusing the PyPI experiment code that was written for the blog post is a priority here.

What do we mean by "keeps working"?

Obviously we cannot guarantee that third-party projects won't introduce undeclared/unused deps in future releases, so we're not looking for exit code 0 when running FawltyDeps on these projects. But we do want to verify that FawltyDeps at least does not crash.

Which project? What do we mean by "a large and relevant collection"?

A good first approximation would be the top 100 PyPI packages, e.g. as reported by https://hugovk.github.io/top-pypi-packages/, and also make sure to test the latest version of these.

How often?

https://hugovk.github.io/top-pypi-packages/ updates monthly, and that seems like a

Which mechanism?

GitHub Action seems like the obvious choice here, but we need to research if it can be scheduled not on a per-commit basis, but rather on a time basis: E.g. At the start of each month, use the latest version of our master branch to run this experiment.

Extra bonus points if (parts of) this can be written as a Nox action, which makes it easy to reproduce failures locally.

Security must also be considered here: We're downloading an untrusted third-party projects. Might be best to run the analysis in a sandbox or similar.

Other questions:

• How long does it take to run this action?