Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
c3c9aea
Add Antweiler-Freyberger (2025) iterative quadrature estimator.
hmgaudecker Apr 15, 2026
193b785
Wire transition function into AF likelihood with shock integration.
hmgaudecker Apr 15, 2026
9397fa6
Add long_running test comparing AF and CHS on MODEL2 data.
hmgaudecker Apr 15, 2026
7967277
Use same uninformed start values for AF and CHS in comparison tests.
hmgaudecker Apr 15, 2026
b03ad70
Add transition constraints, LogSumExp, fix MODEL2 convergence.
hmgaudecker Apr 15, 2026
cb93617
Implement AF investment equation for endogenous factors.
hmgaudecker Apr 15, 2026
04e2d54
Wire start_params argument through AF estimation pipeline.
hmgaudecker Apr 15, 2026
766ad09
Add posterior state extraction for AF via get_filtered_states().
hmgaudecker Apr 15, 2026
765d1c6
Fix 5 issues from code review.
hmgaudecker Apr 15, 2026
852f9ab
Add income-conditional initial draws and translog smoke test.
hmgaudecker Apr 22, 2026
e5b9176
Add fixed_params to AF estimation for time-invariant latent factors.
hmgaudecker Apr 22, 2026
a72e9cc
Use FixedConstraint instead of bounds clamping in AF; add fixed_param…
hmgaudecker Apr 22, 2026
43ce258
Pin optimagic at the probability-allow-fixed-entries branch.
hmgaudecker Apr 22, 2026
907462a
Add MATLAB AF CES + translog reproduction scaffold.
hmgaudecker Apr 22, 2026
8068b7e
Batch the AF likelihood across observations so large grids fit a GPU.
hmgaudecker Apr 22, 2026
2b5f539
Use a joint Halton draw in the AF transition likelihood.
hmgaudecker Apr 22, 2026
cadb799
Harmonise AF CNLSY reproduction: treat investment as a regular latent.
hmgaudecker Apr 23, 2026
c58e495
Add CES reparameterisation helper with level-shift accounting.
hmgaudecker Apr 23, 2026
cbaf02c
Compare skillmodels loglike vs MATLAB's initial-period params.
hmgaudecker Apr 23, 2026
2e5d85d
Match MATLAB normalisation: skills period 0 only, investment never.
hmgaudecker Apr 23, 2026
74c117c
Extend MATLAB loglike comparison to all three periods.
hmgaudecker Apr 23, 2026
564bdc3
Add has_production_shock and has_initial_distribution to FactorSpec.
hmgaudecker Apr 23, 2026
faed06f
Use new FactorSpec flags in MATLAB CES repro and fix translation bugs.
hmgaudecker Apr 23, 2026
6fd7502
Add block-diagonal sandwich standard errors for AF (Phase 1).
hmgaudecker Apr 24, 2026
ab87767
Add full cross-period Newey-McFadden sandwich for AF SEs (Phase 2).
hmgaudecker Apr 24, 2026
8927953
Address code-review feedback on AF SE implementation.
hmgaudecker Apr 24, 2026
d6cd2cb
Hold-last-value imputation for log_income at period 2 in CNLSY loader.
hmgaudecker Apr 24, 2026
b85132f
Vendor CNLSY data file in test directory.
hmgaudecker Apr 27, 2026
4f45d01
Use 20k Halton draws in AF-vs-CHS CNLSY comparison.
hmgaudecker Apr 27, 2026
61607c2
Match MATLAB normalisations exactly in MATLAB-CES/translog comparison.
hmgaudecker Apr 29, 2026
4b34034
Use pinv for AF inference's information-matrix inversion.
hmgaudecker Apr 30, 2026
aea7b86
Bound log_ces phi from above to prevent gradient overflow.
hmgaudecker May 4, 2026
a1a232f
Fix full-sandwich SE: pinv must not assume Hermitian on asymmetric A.
hmgaudecker May 4, 2026
5e8e785
Fix two stale assertions in matlab_ces_repro tests.
hmgaudecker May 4, 2026
83682bd
Add score-resampling cluster bootstrap to AF inference.
hmgaudecker May 4, 2026
281ff84
Add log_ces_with_constant production function.
hmgaudecker May 4, 2026
f558f27
Add Snellius SLURM script for translog AF sim sweep.
hmgaudecker May 6, 2026
8ec7b6d
Snellius script: also run CHS for translog comparison.
hmgaudecker May 6, 2026
7325837
AF: importance-sample carry-over (MATLAB likelihood_12 style).
hmgaudecker May 6, 2026
619286a
Snellius: add conda-flavoured translog sweep script.
hmgaudecker May 6, 2026
dfc2dbf
Use $CONDA_PREFIX/bin/python explicitly in conda slurm script
hmgaudecker May 6, 2026
3b279ee
Snellius slurm: skip 'conda activate' (no conda.sh on H100 nodes); se…
hmgaudecker May 6, 2026
a8e6061
Snellius slurm: trim resources to actually-used (16 CPUs, 96G mem, 8h)
hmgaudecker May 6, 2026
809551c
Skip identity constraint at the final aug-period of each meas-type.
hmgaudecker May 7, 2026
8ffa34c
Snellius slurm: trim wall time to 5h.
hmgaudecker May 7, 2026
ed52604
AF transition: drop prev-period inv-meas from chained-sample weight.
hmgaudecker May 7, 2026
e7d0aac
AF transition: rebuild chained sample from joint Halton (fixes sigma_…
hmgaudecker May 7, 2026
f9d4652
Snellius slurm: drop CHS workers, trim wall time to 3.5h.
hmgaudecker May 8, 2026
4da39c1
Add Marvin (Bonn HPC) slurm script for translog AF sweep.
hmgaudecker May 8, 2026
494c9ac
AF: add moment-based initialization strategy (Phase A, opt-in).
hmgaudecker May 9, 2026
ce5371e
AF: add two-stage measurement system option (Phase B, opt-in).
hmgaudecker May 9, 2026
d4711cb
AF: consolidate inference around the score bootstrap (AF 2025 §4.2).
hmgaudecker May 9, 2026
cebb9cb
AF: flip initialization_strategy default to moment_based; require exp…
hmgaudecker May 9, 2026
960140a
Add Marvin slurm script for translog 3-way comparison at n_halton=2000.
hmgaudecker May 9, 2026
a088278
Marvin slurm: write 3-way comparison results to fresh estimates_3way_…
hmgaudecker May 9, 2026
4d006bf
Add moment-based start values for CHS, hybrid Spearman+OLS, cuda13 env
hmgaudecker May 10, 2026
b58fb12
AF: propagate cross-period equality constraints through the chain
hmgaudecker May 10, 2026
7c0539a
AF validate: relax measurement count to >=2, warn below 3
hmgaudecker May 11, 2026
5844104
Add `pool_equality_groups` helper for moment-init starts
hmgaudecker May 11, 2026
db66382
tests: bump joint-Halton sigma_prod_1 tolerance 30% -> 35%
hmgaudecker May 11, 2026
081fbe6
Add AMN (Attanasio-Meghir-Nix 2020) standalone estimator
hmgaudecker May 11, 2026
a614c3d
Remove scripts/ and docs/superpowers/ from skillmodels
hmgaudecker May 11, 2026
7aefedf
Move CHS Kalman-filter modules into a `skillmodels.chs` subpackage
hmgaudecker May 11, 2026
22a12b5
AF likelihood: mask NaN measurements so missing data doesn't poison g…
hmgaudecker May 11, 2026
20d2688
Move tests/matlab_ces_repro into the parent workspace
hmgaudecker May 11, 2026
506828b
Add portable environment files for the af-estimator branch
hmgaudecker May 11, 2026
2871ecf
AF transition: bridge `@register_params` user functions to AF's calli…
hmgaudecker May 11, 2026
20a63b6
AF result: drop per-period importance samples before returning
hmgaudecker May 11, 2026
7ef2694
Remove Spearman as a final-estimator option
hmgaudecker May 11, 2026
fb3f15e
AF result: materialise every JAX array to numpy before returning
hmgaudecker May 11, 2026
ed6f40c
AF result: free JIT/XLA caches before materialising arrays to numpy
hmgaudecker May 11, 2026
5cebcb2
Reorganise package into common / chs / af / amn subpackages
hmgaudecker May 11, 2026
00b4171
AF result: free samples_per_component before materialising the rest
hmgaudecker May 11, 2026
e509451
Add the full Attanasio-Meghir-Nix (2020) estimator
hmgaudecker May 11, 2026
052e5e4
Make AMN the default start-value strategy for CHS and AF
hmgaudecker May 11, 2026
d0b8d2a
Add scikit-learn dependency for AMN mixture EM
hmgaudecker May 11, 2026
f0df40b
Add scikit-learn to deployment env files
hmgaudecker May 11, 2026
4a517c6
Fall back to Spearman when AMN cannot handle the model
hmgaudecker May 11, 2026
852b095
AMN Stage 3: support every transition function via generic jax.vmap NLS
hmgaudecker May 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,10 @@ When writing new public-facing code, always accept and return `period`. Convert

## Testing

- pytest with markers: `wip`, `unit`, `integration`, `end_to_end`
- pytest with markers: `wip`, `unit`, `integration`, `end_to_end`, `long_running`
- Test files mirror source structure in `tests/`
- Memory profiling available via pytest-memray (Unix only)
- MATLAB AF CES / translog reproduction tests live in the parent workspace at
`../matlab_ces_repro/` (alongside `sim_repro/`), not in this library. They depend on
reference data at `/home/hmg/sciebo/Skill estimation/` and the CNLSY xls bundled
beside them. Run from the workspace root.
2 changes: 1 addition & 1 deletion docs/getting_started/tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
"import pandas as pd\n",
"\n",
"from skillmodels import get_maximization_inputs\n",
"from skillmodels.config import TEST_DATA_DIR\n",
"from skillmodels.common.config import TEST_DATA_DIR\n",
"from skillmodels.test_data.model2 import MODEL2"
]
},
Expand Down
4 changes: 2 additions & 2 deletions docs/how_to_guides/how_to_simulate_dataset.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@
"source": [
"import pandas as pd\n",
"\n",
"from skillmodels.config import REGRESSION_VAULT, TEST_DATA_DIR\n",
"from skillmodels.simulate_data import simulate_dataset\n",
"from skillmodels.common.config import REGRESSION_VAULT, TEST_DATA_DIR\n",
"from skillmodels.common.simulate_data import simulate_dataset\n",
"from skillmodels.test_data.model2 import MODEL2"
]
},
Expand Down
8 changes: 4 additions & 4 deletions docs/how_to_guides/how_to_visualize_correlations.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
"source": [
"import pandas as pd\n",
"\n",
"from skillmodels.config import REGRESSION_VAULT, TEST_DATA_DIR\n",
"from skillmodels.correlation_heatmap import (\n",
"from skillmodels.common.config import REGRESSION_VAULT, TEST_DATA_DIR\n",
"from skillmodels.common.correlation_heatmap import (\n",
" get_measurements_corr,\n",
" get_quasi_scores_corr,\n",
" get_scores_corr,\n",
Expand Down Expand Up @@ -167,7 +167,7 @@
"metadata": {},
"outputs": [],
"source": [
"from skillmodels.visualize_transition_equations import (\n",
"from skillmodels.common.visualize_transition_equations import (\n",
" _get_parsed_params,\n",
" _set_index_params,\n",
")"
Expand All @@ -179,7 +179,7 @@
"metadata": {},
"outputs": [],
"source": [
"from skillmodels.process_model import process_model"
"from skillmodels.common.process_model import process_model"
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,16 @@
"import numpy as np\n",
"import pandas as pd\n",
"\n",
"from skillmodels.config import REGRESSION_VAULT, TEST_DATA_DIR\n",
"from skillmodels.maximization_inputs import get_maximization_inputs\n",
"from skillmodels.simulate_data import simulate_dataset\n",
"from skillmodels.test_data.model2 import MODEL2\n",
"from skillmodels.visualize_factor_distributions import (\n",
"from skillmodels.chs.maximization_inputs import get_maximization_inputs\n",
"from skillmodels.common.config import REGRESSION_VAULT, TEST_DATA_DIR\n",
"from skillmodels.common.simulate_data import simulate_dataset\n",
"from skillmodels.common.visualize_factor_distributions import (\n",
" bivariate_density_contours,\n",
" bivariate_density_surfaces,\n",
" combine_distribution_plots,\n",
" univariate_densities,\n",
")"
")\n",
"from skillmodels.test_data.model2 import MODEL2"
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@
"source": [
"import pandas as pd\n",
"\n",
"from skillmodels.config import REGRESSION_VAULT, TEST_DATA_DIR\n",
"from skillmodels.test_data.model2 import MODEL2\n",
"from skillmodels.visualize_transition_equations import (\n",
"from skillmodels.common.config import REGRESSION_VAULT, TEST_DATA_DIR\n",
"from skillmodels.common.visualize_transition_equations import (\n",
" combine_transition_plots,\n",
" get_transition_plots,\n",
")"
")\n",
"from skillmodels.test_data.model2 import MODEL2"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion docs/how_to_guides/model_specs.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ Fine-tune the estimation:
Define custom transition equations using the `@register_params` decorator:

```python
from skillmodels.decorators import register_params
from skillmodels.common.decorators import register_params

@register_params(params=["lincoeff"])
def my_linear(fac, params):
Expand Down
67 changes: 67 additions & 0 deletions environment-cuda.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
# Conda/mamba environment for the skillmodels `af-estimator` branch
# **with CUDA 12 GPU support**.
#
# Same package set as `environment.yml`, except JAX is pulled in via the
# `jax[cuda12]` PyPI extra and `cuda-nvcc` is added on the conda side.
# Requires the host system to provide a CUDA 12 toolkit; see
# https://jax.readthedocs.io/en/latest/installation.html for details.
#
# Usage:
# mamba env create -f environment-cuda.yml
# mamba activate skillmodels-af-cuda
name: skillmodels-af-cuda
channels:
- conda-forge
- nodefaults
dependencies:
# CUDA toolchain (required for the cuda12 JAX wheel below)
- cuda-nvcc >=12
# Python + core scientific stack
- python ~=3.14.0
- scipy >=1.16.0
- h5py >=3.16.0,<4
# Skillmodels conda deps
- filterpy *
- ipykernel >=6.29.5
- jupyterlab *
- nbformat >=5.10.4
- networkx *
- pybaum >=0.1.3
- scikit-learn >=1.5 # AMN Stage 1 (mixture EM)
# Test / profiling tooling
- pytest >=8.4.1
- pytest-cov >=6.2.1
- pytest-xdist >=3.8.0
- pytest-memray *
- snakeviz *
- xlrd >=2
- prek *
# Downstream-only conda deps (skane-struct-bw / health-cognition):
- deepdiff >=8.5.0
- memray >=1.17.2
- statsmodels >=0.14.5
- tabulate >=0.9.0
- seaborn *
- pip
- pip:
# Skillmodels project deps (PyPI), with CUDA-12 JAX wheel
- dags>=0.5.1
- jax[cuda12]>=0.9
- jupyter-book>=2
- kaleido>=1.2
- numpy>=2.4
- pandas>=3
- plotly>=6.6
- pytask>=0.5.8
- pytask-parallel>=0.5.2
- pdbp
# Pinned to the optimagic branch the AF estimator relies on.
# yamllint disable-line rule:line-length
- optimagic @ git+https://github.com/optimagic-dev/optimagic.git@probability-allow-fixed-entries
# Downstream-only PyPI deps:
- fides>=0.7.8
- statadict>=1.1.0
# Skillmodels itself, from the af-estimator branch.
# yamllint disable-line rule:line-length
- skillmodels @ git+https://github.com/OpenSourceEconomics/skillmodels.git@af-estimator
70 changes: 70 additions & 0 deletions environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
# Conda/mamba environment for the skillmodels `af-estimator` branch.
#
# Installs every package needed to run the skillmodels test suite **and**
# the two downstream research applications (`skane-struct-bw`,
# `health-cognition`) -- minus the two applications themselves, which
# are supplied separately by their respective project teams.
#
# CPU-only JAX. For an environment with CUDA-12 support use
# `environment-cuda.yml` instead.
#
# Usage:
# mamba env create -f environment.yml
# mamba activate skillmodels-af
name: skillmodels-af
channels:
- conda-forge
- nodefaults
dependencies:
# Python + core scientific stack
- python ~=3.14.0
- scipy >=1.16.0
- h5py >=3.16.0,<4
# Skillmodels conda deps
- filterpy *
- ipykernel >=6.29.5
- jupyterlab *
- nbformat >=5.10.4
- networkx *
- pybaum >=0.1.3
- scikit-learn >=1.5 # AMN Stage 1 (mixture EM)
# Test / profiling tooling (skillmodels' tests-cpu feature)
- pytest >=8.4.1
- pytest-cov >=6.2.1
- pytest-xdist >=3.8.0
- pytest-memray *
- snakeviz *
- xlrd >=2
- prek *
# Downstream-only conda deps (not used by skillmodels itself; required
# to run skane-struct-bw / health-cognition pipelines):
- deepdiff >=8.5.0 # health-cognition + skane: snapshot / diff utilities
- memray >=1.17.2 # health-cognition + skane: memory profiling
- statsmodels >=0.14.5 # health-cognition + skane: regression diagnostics
- tabulate >=0.9.0 # health-cognition + skane: table formatting in reports
- seaborn * # health-cognition: figure styling
- pip
- pip:
# Skillmodels project deps (PyPI)
- dags>=0.5.1
- jax>=0.9
- jupyter-book>=2
- kaleido>=1.2
- numpy>=2.4
- pandas>=3
- plotly>=6.6
- pytask>=0.5.8
- pytask-parallel>=0.5.2
- pdbp
# Pinned to the optimagic branch the AF estimator relies on
# (`probability-allow-fixed-entries`). The PyPI release does not
# yet carry the required `FixedConstraintWithValue` semantics.
# yamllint disable-line rule:line-length
- optimagic @ git+https://github.com/optimagic-dev/optimagic.git@probability-allow-fixed-entries
# Downstream-only PyPI deps (not used by skillmodels itself):
- fides>=0.7.8 # health-cognition + skane: optimagic algorithm
- statadict>=1.1.0 # health-cognition + skane: Stata variable labels
# The library itself, from the af-estimator branch.
# yamllint disable-line rule:line-length
- skillmodels @ git+https://github.com/OpenSourceEconomics/skillmodels.git@af-estimator
Loading
Loading