Add health insurance premiums to local area calibration, a matrix builder function, Modal model fitting #473
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
baogorek
commented
Jan 22, 2026
baogorek
changed the title
juaristi22
approved these changes
Jan 22, 2026
policyengine_us_data/datasets/cps/local_area_calibration/build_calibration_matrix.py
Outdated
Show resolved
Hide resolved
policyengine_us_data/datasets/cps/local_area_calibration/fit_calibration_weights.py
Outdated
Show resolved
Hide resolved
- Add build_calibration_matrix.py runner for building X_sparse matrix with SNAP + health_insurance_premiums_without_medicare_part_b targets - Add test_sparse_matrix_builder.py with tests for person-level aggregation and combined target support - Update changelog_entry.yaml Closes #472 Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Add fit_calibration_weights.py for L0-regularized weight optimization - Add l0-python as optional dependency under [project.optional-dependencies] - Uses SparseCalibrationWeights from l0.calibration package - Builds calibration matrix for all 436 congressional districts - Filters to achievable targets before fitting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace sparsity_log with calibration_log that tracks per-target performance (estimate, target, error metrics) across epochs. Update Modal runner to return both weights and log files. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Avoid confusion with fit_calibration_weights.py which contains the actual fitting logic. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
baogorek
force-pushed
the
health-insurance-premiums
branch
from
4545c14 to
ba25e3e
Compare
The versioning workflow bumps the version in pyproject.toml but never updated uv.lock, causing Modal builds to fail with "uv sync --locked" since the lockfile was stale. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…nate dense reweighting Remove the dense optimization path from enhanced CPS (only sparse L0 weights are needed), eliminating the mv/cp workaround in the Makefile. Remove both TEST_LITE and LOCAL_AREA_CALIBRATION environment variable flags, generating all datasets unconditionally in a single pass. Merge the data-local-area target into the data target. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Close the h5 file handle and release the Microsimulation reference before writing to the same file path. Also fix pre-existing black formatting in 3 files. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
9 tasks
- Add "snap" to variables filter in fit_calibration_weights.py to include National SNAP targets (in different stratum_group_id than 4) - Remove redundant test_sparse_matrix_builder.py (tests consolidated in test_matrix_national_variation.py) - Remove redundant build_calibration_matrix.py (functionality covered by fit_calibration_weights.py) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This has really morphed into a hodge-podge of updates. Technically, the health insurance premium variable is included in the matrix builder, but this PR doesn't include any specific testing of the variable. Thus, @juaristi22 's PR #475 will go in first to cover the testing of this variable.
I'm also going to hold off on linting until #475 goes in, so that the "Files changed" doesn't increase unnecessarily.
build_calibration_matrix.pyrunner script for building sparse calibration matrices with combined SNAP and health insurance premium targetstest_sparse_matrix_builder.pywith tests for person-level aggregation and combined target supportDetails
The
health_insurance_premiums_without_medicare_part_bvariable is a person-level target that needs to be aggregated to household level. The existingsparse_matrix_builder.pyalready handles this through PolicyEngine'smap_to="household"parameter, which automatically sums person-level values to household level while preserving totals.Closes #472, Closes #429
Test plan
pytest policyengine_us_data/tests/test_sparse_matrix_builder.py- 3 tests passpython build_calibration_matrix.pyfrom the local_area_calibration directory🤖 Generated with Claude Code