feat(ampup): add token resolution, rate limiter, and --jobs CLI flag

[mitchhs12]

Part 1 of edgeandnode/amp#1762 (bounded-concurrency parallel downloads)

Summary

• Add GitHub token resolution with fallback chain: explicit token → gh auth token → unauthenticated
• Add --jobs / -j CLI flag to install and update commands for controlling concurrent downloads (default: 4)
• Add GitHubRateLimiter that parses X-RateLimit-Remaining, X-RateLimit-Reset, and Retry-After headers per GitHub REST API docs
• On 429 or rate-limit 403, sets a global pause shared across all concurrent downloads and retries once
• Preemptively pauses when remaining hits 0 (avoids burning requests that will fail)
• Integrates rate limiter into all GitHubClient HTTP methods via send_with_rate_limit
• Make GitHubClient cloneable for future DownloadManager task spawning

Changes

Token resolution (token.rs — new)

• resolve_github_token(): explicit → gh auth token → None
• Security note: --github-token values visible in ps aux, prefer env var or gh

Rate limiter (rate_limiter.rs — new)

• GitHubRateLimiter with tokio::sync::Mutex-protected state
• wait_if_paused(): blocking gate, drops lock before sleeping
• update_from_response() → update_state(): testable state machine
• 429 and 403 (with rate-limit signal) trigger global backoff
• Retry-After default: 60s per GitHub docs when header absent
• extend_pause(): only extends pauses, never shortens

GitHub client (github.rs — modified)

• send_with_rate_limit(): wraps all HTTP requests with rate-limit awareness and one retry on 429
• RateLimited error variant with actionable user message
• #[derive(Clone)] on GitHubClient (cheap — reqwest::Client and Arc<GitHubRateLimiter> are both Arc-backed)

CLI (main.rs — modified)

• --jobs / -j flag on install and update commands

Test plan

• 10 rate limiter unit tests (blocking gate, state machine edge cases, real HTTP header parsing via mock TCP server)
• 2 token resolution unit tests
• All 23 ampup tests pass
• Zero clippy warnings

[LNSD]

The preemptive pause path can silently block for up to 60 minutes (unauthenticated) with no output after the initial warning.

The sequence:

1. A 200 response arrives with X-RateLimit-Remaining: 0
2. update_state sets paused_until to the reset timestamp (up to an hour away per https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api)
3. User sees: ⚠ GitHub API rate limit exhausted, subsequent requests will be paused until reset
4. The current request completes normally
5. The next request enters wait_if_paused() -> tokio::time::sleep(duration) with no further output

From the user's perspective (UX), the process hangs: no countdown, no spinner, no way to know how long the wait will be.

Suggestion: if Retry-After is absent or exceeds 60 seconds, fail immediately with an actionable error instead of sleeping. For unauthenticated users, the error should prompt them to provide a GITHUB_TOKEN to access the 5,000 requests/hour limit. Only sleep-and-retry for short pauses (≤60s) where waiting is reasonable.

[mitchhs12]

Suggestion: if Retry-After is absent or exceeds 60 seconds, fail immediately with an actionable error instead of sleeping. For unauthenticated users, the error should prompt them to provide a GITHUB_TOKEN to access the 5,000 requests/hour limit. Only sleep-and-retry for short pauses (≤60s) where waiting is reasonable.

Good suggestion, makes perfect sense.

[LNSD]

LGTM ✅

[LNSD]

We should avoid introducing dead code. Introducing dead code should be exceptional and justified (e.g., introducing a feature flag for a feature under development, establishing the foundations on which multiple features will depend, etc.). In general, for coordination purposes.

We should aim for self-contained PRs. Where all the changes have the same goal.

Could you remove the CLI options and this dead code, and include them in the downloads manager PR?