Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JQ download autoselection causes HTTP 403 (rate limit) #9

Open
lalten opened this issue Aug 27, 2024 · 4 comments
Open

JQ download autoselection causes HTTP 403 (rate limit) #9

lalten opened this issue Aug 27, 2024 · 4 comments

Comments

@lalten
Copy link

lalten commented Aug 27, 2024

WARNING: Download from https://api.github.com/repos/stedolan/jq/releases failed: class java.io.IOException GET returned 403 rate limit exceeded
INFO: Repository jq_linux64 instantiated at:
  /__w/repo/repo/WORKSPACE.bzlmod:81:23: in <toplevel>
  /mnt/data/bazel-user-root/repo_out/external/rules_jq/jq/repositories.bzl:93:24: in jq_register_toolchains
Repository rule jq_repositories defined at:
  /mnt/data/bazel-user-root/repo_out/external/rules_jq/jq/repositories.bzl:71:34: in <toplevel>
ERROR: An error occurred during the fetch of repository 'jq_linux64':
   Traceback (most recent call last):
	File "/mnt/data/bazel-user-root/repo_out/external/rules_jq/jq/repositories.bzl", line 37, column 28, in _jq_repo_impl
		repository_ctx.download(
Error in download: java.io.IOException: Error downloading [https://api.github.com/repos/stedolan/jq/releases] to /mnt/data/bazel-user-root/repo_out/external/jq_linux64/versions.json: GET returned 403 rate limit exceeded

The issue comes from querying the API in

repository_ctx.download(
url = ["https://api.github.com/repos/stedolan/jq/releases"],
output = "versions.json",
)

Probably it would be better to blindly accept a jq version as input and just try download that

@chickenandpork
Copy link
Contributor

chickenandpork commented Sep 3, 2024

I see your point; the goal was to allow the user to get the latest with some protections for checksum even if the user didn't choose.

I think the specification of a version should take priority, but the checksum will still be problematic.

TBH, the releases can be simply merged as a component of this archive so there isn't second download.

I'll look into that (but it may take until Thursday before I get time to look into it)

@lalten
Copy link
Author

lalten commented Sep 3, 2024

thanks. I worked around this by getting a jq release with rules_download and running that binary in a ctx.actions.run_shell, so no rules_jq involved.

@chickenandpork
Copy link
Contributor

thanks. I worked around this by getting a jq release with rules_download

Cool. Keep in mind: you'll have rate-limited downloads in GitHub unless you're using a valid token to authenticate to GitHub. I've run into this when I'm building using the default token.

I'm not trying to be confrontational, but I wanted to warn you that this happens outside of this project. I can help you when you run into that.

@lalten
Copy link
Author

lalten commented Sep 19, 2024

Thanks for the heads-up. We use a combination of persistent runners and vendor mode to avoid this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
@chickenandpork @lalten and others