add a tutlial for oregon health insurance experiment #81
Conversation
|
@okiner-3 Thank you so much for the PR! Sorry if it was not clear in the ticket description, but we want to focus on the local distributional treatment effects (LDTE) and LPTE since incomplete compliance was observed in the experiment. In the Oregon dataset, the |
|
@TomeHirata |
… to Non-Compliance
There was a problem hiding this comment.
Pull Request Overview
This PR adds a comprehensive tutorial demonstrating the use of Local Distribution Treatment Effects (LDTE) analysis with the Oregon Health Insurance Experiment dataset. The tutorial showcases how to handle non-compliance scenarios using instrumental variable approaches when not all participants assigned to treatment actually enrolled.
Key changes:
- New comprehensive tutorial file analyzing emergency department costs and visits using local distribution treatment effects
- Tutorial demonstrates both simple and ML-adjusted estimators for handling non-compliance
- Includes stratified analysis by household registration patterns to examine treatment effect heterogeneity
Reviewed Changes
Copilot reviewed 2 out of 11 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| docs/source/tutorials/oregon.rst | Comprehensive tutorial implementing LDTE analysis for the Oregon Health Insurance Experiment with non-compliance handling |
| docs/source/tutorials.rst | Added reference to the new Oregon tutorial in the documentation index |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
Hi, @okiner-3. How's the progress so far? Feel free to let me know if you have any questions! |
|
@TomeHirata |
|
|
||
| **2. Covariate Adjustment Effects and Confidence Intervals** | ||
|
|
||
| The confidence intervals remain wide for both estimators, though ML adjustment shows slightly more consistent patterns in the moderate cost range. The limited precision suggests: (1) substantial heterogeneity in treatment effects within cost bins, (2) limited predictive power of covariates for specific cost levels, or (3) relatively small sample sizes within individual bins. |
There was a problem hiding this comment.
Did we observe no precision gain?
docs/source/tutorials/oregon.rst
Outdated
| - **"Signed self up" stratum**: Confidence intervals remain wide but manageable for both estimators, showing similar patterns to the overall population. | ||
| - **"Signed self up + others" stratum**: | ||
|
|
||
| - Extreme estimation instability, particularly for ML-adjusted estimator |
There was a problem hiding this comment.
The ML estimation is a bit odd, have you tried using different ml models or fold number?
There was a problem hiding this comment.
Changing the ML estimation model stabilized the confidence intervals.
docs/source/tutorials/oregon.rst
Outdated
|
|
||
| Stratified analysis uncovers dramatic treatment effect heterogeneity: single-person households ("signed self up") show moderate effects (LDTE ≈ -0.18 to -0.20), while multi-person households ("signed self up + others") exhibit 3-4x larger effects (LDTE ≈ -0.55). This suggests household structure is a critical moderator—insurance enables care-seeking for multiple family members when households include dependents. | ||
|
|
||
| **4. Limited Efficiency Gains from ML Adjustment** |
There was a problem hiding this comment.
We should figure out the way to increase the efficiency gain instead of listing its difficulty here.
There was a problem hiding this comment.
No implementation errors were found.
Some exploration was conducted by adding other features, but no improvement was observed.
Add a tutorial with Oregon Health Insurance Experiment.