Skip to content

Add JOSS paper for submission#264

Draft
vahid-ahmadi wants to merge 22 commits intomainfrom
joss-paper
Draft

Add JOSS paper for submission#264
vahid-ahmadi wants to merge 22 commits intomainfrom
joss-paper

Conversation

@vahid-ahmadi
Copy link

@vahid-ahmadi vahid-ahmadi commented Mar 23, 2026

Summary

  • Adds paper.md and paper.bib for submission to the Journal of Open Source Software (JOSS)
  • Paper covers all 6 required JOSS sections: Summary, Statement of Need, State of the Field, Software Design, Research Impact Statement, and AI Usage Disclosure
  • Cites adoption by 10 Downing Street, HM Treasury (Algorithmic Transparency Record), NIESR UK Living Standards Review 2025, INET Oxford (Seventh Carbon Budget), IEA reports, and House of Lords parliamentary debate
  • Funded by the Nuffield Foundation

What is JOSS?

JOSS is a peer-reviewed open-access journal for research software. It publishes short papers (750–1,750 words) describing open-source software with research impact. There are no submission fees. The review process happens on GitHub via openjournals/joss-reviews. See the submission guide and paper format for full details.

JOSS requirements checklist

  • OSI-approved open-source license (AGPL-3.0)
  • Public repository with browsable source code
  • Issue tracker readable without registration
  • Public development history > 6 months
  • Demonstrated research impact (government, academic, policy citations)
  • Good open-source practices (CI, tests, docs, PRs, tagged releases)
  • Iterative development over time
  • paper.md with all 6 required sections
  • paper.bib with BibTeX references
  • AI Usage Disclosure
  • Funding acknowledgement (Nuffield Foundation)
  • ORCID IDs for all authors (see below)

Steps before submission

  • Complete the full name of all authors in paper.md
  • ORCID IDs — Each author needs a real ORCID. Register for free at https://orcid.org/register (takes ~2 minutes, use personal email so it follows you across institutions). Update the 0000-0000-0000-0000 placeholders in paper.md:
    • Max Ghenis
    • Nikhil Woodruff
    • Pavel Makarchuk
    • Vahid Ahmadi — 0009-0004-1093-6272
  • All co-authors confirm agreement to be listed
  • Review and approve paper text
  • Merge this PR to main
  • Submit at https://joss.theoj.org/papers/new with the repo URL
  • Disclose conflicts of interest on the submission form (e.g., all authors employed by PolicyEngine)
  • Note related publications on the submission form (NIESR report, No 10 Innovation Fellows work, IEA reports)

After submission

  1. A JOSS editor will open a pre-review issue at joss-reviews
  2. Two reviewers will be assigned; review happens via GitHub issues
  3. Respond to reviewer feedback within 2 weeks, complete changes within 4–6 weeks
  4. Upon acceptance: create a tagged release, deposit to Zenodo for a DOI, update the review thread

Test plan

  • Verify paper.md renders correctly with JOSS preview tools
  • Confirm all paper.bib references are valid and resolve correctly

🤖 Generated with Claude Code

Adds paper.md and paper.bib for submission to the Journal of Open Source
Software (JOSS). Paper covers all 6 required JOSS sections and cites
adoption by 10 Downing Street, HM Treasury, NIESR, INET Oxford, IEA,
and House of Lords parliamentary debate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
vahid-ahmadi and others added 3 commits March 23, 2026 12:07
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rewrites paper.md to describe the policyengine package as a unified
multi-country (UK + US) microsimulation framework rather than the
UK-only policyengine-uk package. Updates title, summary, statement of
need, state of the field, software design, and acknowledgements to
reflect the country-agnostic architecture. Adds TAXSIM and US data
references to paper.bib.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds congressional citation (Young Adult Tax Credit Act, H.R.7547),
NBER partnership (TAXSIM emulator MOU), Niskanen Center CTC report,
NTA conference presentation, and Georgetown Beeck Center collaboration
to the Research Impact section. Adds corresponding BibTeX entries.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
vahid-ahmadi and others added 15 commits March 23, 2026 12:27
- Convert all British spellings to American English (organised →
  organized, programme → program, labour → labor, behavioural →
  behavioral, modelling → modeling, licence → license, etc.)
- Fix Beeck Center citation: was pointing to a 2022 event but labeled
  as 2024. Replace with correct Better Government Lab collaboration
  reference (pe_bgl)
- Soften Youngman et al. claim: confirmed paper cites PolicyEngine UK
  data but removed unverifiable direct quote
- Remove unused beeck2024rac bib entry

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Change country order to "the US and the UK"
- Remove "via the Simulation class" from Summary
- Remove inline dataset names (Enhanced FRS, CPS) from Summary
- Make Statement of Need more specific about access barriers for
  existing models (UKMOD requires institutional application, TAXSIM
  covers only taxes not benefits, HMT/IFS/CBO/TPC are fully
  proprietary)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Update install instructions to match repo docs: pip install
  policyengine, policyengine[uk], policyengine[us], and uv pip
  install -e .[dev] for development
- Remove backtick code styling from "Simulation class" in Statement
  of Need
- Reorder State of the Field: US models first, then UK models

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Merge "Multi-country" and "Fully open-source" bullets into one
- Add specific US program count (over 11) with full list: federal
  income tax, payroll taxes, state income taxes, SNAP, SSI, Social
  Security, Medicare, Medicaid, EITC, CTC, and TANF
- Remove "Integration with the PolicyEngine web application" bullet

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Merge "Distributional analysis outputs" and "Labor supply dynamics"
  into single "Economic analysis" bullet
- Change "policyengine package" to "policyengine.py package"
- Delete "All code examples in the documentation are automatically
  re-executed" sentence
- Remove "built 10ds-microsim on top of" reference, simplify to
  "used PolicyEngine"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Academic research:
- USC/HHS collaboration on effective marginal tax rates (Unrath)
- DeBacker behavioral response modeling (Arnold Ventures)
- Beeck Center rules-as-code reports (2023, 2025)

Policy research:
- DC District Child Tax Credit (first local CTC in US history)
- Senator Booker Keep Your Pay Act calculator on Senate website
- Reorder US citations before UK in both sections

Acknowledgements:
- Add US funders: Arnold Ventures, NEO Philanthropy ($200K grant),
  Gerald Huff Fund for Humanity, NSF POSE program
- Keep Nuffield Foundation for UK

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Atlanta Fed Policy Rules Database MOU for multi-model validation
  in Institutional partnership section
- Add NSF POSE Phase I award (2518372, $299,974) with nsf.gov URL
- Update NBER TAXSIM MOU reference to use taxsim.nber.org instead of
  policyengine.org link
- Update Arnold Ventures note to mention congressional district focus
- Use external/official URLs for all references where available

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consistently place US content before UK content in every section:
Summary, Statement of Need, Software Design, Research Impact
(Government adoption, Congressional/Parliamentary, Academic,
Policy research), and Acknowledgements.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…-installable to Python

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
vahid-ahmadi and others added 3 commits March 23, 2026 13:53
- Remove "None of these tools provide a unified framework" sentence
- Merge "Programmatic reform API" and "Economic analysis" into one
  bullet: "Programmatic reform and economic analysis"
- Fix duplicate "Python Python" typo
- Now 3 bullets: framework, program coverage, reform and analysis

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Delete country-specific models paragraph from Software Design
- ghenis2026no10: gov.uk Innovation Fellowship page
- ghenis2024nta: ntanet.org conference page
- pe_usc: irp.wisc.edu extramural grants page
- pe_dctc: dccouncil.gov bill B25-0190
- pe_keepyourpay: booker.senate.gov press release
- pe_bgl and neo_philanthropy kept as PE links (no external alt exists)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@vahid-ahmadi
Copy link
Author

JOSS Paper Review: Comparison with Top Economics/Microsimulation JOSS Papers

I reviewed the most notable JOSS papers in economics and microsimulation — including AgentPy (39 citations, most-cited in the space), QuantEcon.py (9 pages, Nobel laureate co-author), neworder (dynamic microsimulation), BoARIO (indirect economic cost modeling), MEDUSA (distributional policy impacts), openmpp (R microsimulation), Economiccomplexity (9 citations), and BCEA (cost-effectiveness analysis).

Key finding: PolicyEngine would be the first JOSS paper for an open-source tax-benefit microsimulation model. Neither Tax-Calculator, OpenFisca, OG-Core, nor any PSL tool has a JOSS paper. The Research Impact section is unmatched by anything in comparable papers.


Suggested Improvements

1. Add an Architecture Diagram (inspired by AgentPy — 39 citations)

AgentPy's clean figure showing Agents → Environments → Models → Experiments → Output is likely a factor in its high citation count. The Software Design section describes 5 component groups in prose — a figure showing Dataset + Policy → Simulation → Output classes with the country-package delegation would make this immediately graspable. JOSS explicitly encourages figures.

2. Add a Code Example (inspired by QuantEcon.py)

QuantEcon's paper is mostly working code snippets. This paper has zero code. Even a short example would make it more concrete:

from policyengine import Simulation

sim = Simulation(country="us", reform={...})
sim.calculate("household_net_income")

3. Sharpen Competitive Differentiation (inspired by neworder & BCEA)

The neworder microsimulation paper does a systematic dimension-by-dimension comparison against MODGEN and LIAM2. Consider adding a comparison table:

Feature PolicyEngine TAXSIM UKMOD OpenFisca
Open source Partial
Multi-country US+UK US only UK only FR (+ forks)
Tax + benefits Both Tax only Both Both
Web interface Partial

4. Frame Around Reproducibility (inspired by Economiccomplexity — 9 citations)

Economiccomplexity's strongest move is framing open-source software as a scientific imperative for reproducibility. The Statement of Need focuses on access barriers, which is good — but adding a sentence about reproducibility of policy analysis (CBO/TPC results can't be independently verified) would resonate with JOSS reviewers who care deeply about open science.

5. Consider Trimming the Research Impact Section

This is the paper's strongest section — no other JOSS paper comes close. But it's ~60% of the body. JOSS papers typically run 3–5 pages. The government adoption and congressional citation paragraphs are the strongest; the policy research paragraph could potentially be condensed.

6. Fix Placeholder ORCIDs

Three authors have 0000-0000-0000-0000 as their ORCID. These need real ORCIDs or should be removed — JOSS reviewers will flag this immediately.

7. Engage More with Microsimulation Literature (inspired by BoARIO — 21 references)

BoARIO demonstrates deep engagement with its field's academic literature. Consider citing foundational microsimulation methodology papers (Orcutt 1957, Bourguignon & Spadaro 2006 survey) to position PolicyEngine within the academic tradition, not just against current tools.

8. State What PolicyEngine Does NOT Do (inspired by BCEA)

BCEA's "do one thing well" philosophy is effective. A brief scope statement (e.g., "PolicyEngine does not model macroeconomic feedback effects or general equilibrium") would help reviewers understand the design boundary.


Priority Summary

Priority Action Effort
🔴 High Add architecture diagram Medium
🔴 High Fix placeholder ORCIDs Low
🔴 High Add a minimal code example Low
🟡 Medium Add comparison table in State of the Field Low
🟡 Medium Add reproducibility framing to Statement of Need Low
🟡 Medium Add scope/limitations sentence Low
🟢 Lower Condense policy research paragraph Low
🟢 Lower Add foundational microsimulation references Low

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant