-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
Problem
The Ginkgo acceptance tests currently require a valid PIVNET_TOKEN to run because they test against the live Pivnet API. This means:
- Acceptance tests cannot run in CI without storing the token as a GitHub secret
- Tests are network-dependent and subject to rate limits
- Tests fail if Pivnet is down or slow
Current Workaround
We're skipping acceptance tests in CI and running only unit tests. Acceptance tests run manually locally with make acceptance-test-with-token PIVNET_TOKEN=...
Proposed Solution
Add VCR (HTTP recording/replay) fixtures:
- Record fixtures once using real Pivnet API with valid token
- Commit fixtures to repository (YAML files with HTTP request/response pairs)
- Replay in CI without needing PIVNET_TOKEN
- Refresh quarterly or when Pivnet API changes
Implementation Options
Option 1: HTTP Proxy Wrapper (go-vcr)
- Create HTTP proxy server using go-vcr
- Set
HTTP_PROXYenv var when running tile-diff subprocess - Most seamless, but requires custom proxy implementation
Option 2: Manual Recording (mitmproxy/httptest)
- Use external tool to record HTTP traffic
- Convert to simple JSON fixtures
- Build simple HTTP server to replay fixtures
- More control, but more custom code
Option 3: Different Testing Strategy
- Test tile-diff as library (not subprocess)
- Inject VCR-wrapped HTTP client directly
- Requires refactoring tile-diff to expose testable APIs
Success Criteria
- Acceptance tests run in CI without PIVNET_TOKEN
- Fixtures are small enough to commit (<5MB total)
- Easy to re-record fixtures when needed
- Documentation for maintaining fixtures
References
- Design doc:
../tile-diff-internal-docs/plans/2025-12-14-ginkgo-acceptance-tests-design.md - Current implementation:
test/acceptance_test.go - go-vcr library: https://github.com/dnaeon/go-vcr
Metadata
Metadata
Assignees
Labels
No labels