Lifetime income current by justin-ven · Pull Request #545 · simpaths/SimPaths

justin-ven · 2026-10-24T20:21:39Z

updates model for progress implementing lifetime incomes

Update of main (pre-September26-release)

…current

…en/SimPaths into lifetime-income-current

…current

Copilot

🟡 Changes recommended

Startup failures, inconsistent income matching and exports, and a broken Stata continuation remain unresolved.

6 open issues

What changed in this PR

This PR advances SimPaths’ lifetime-income implementation by replacing persisted birth-cohort donors with shared, in-memory normalised income profiles.

Changes:

Generate template profiles and match them to the initial simulation population.
Add survey-sample matching, mortality projection, and income exports.
Update age-specific calibration, supporting model APIs, and package documentation.

File	Description
`src/main/java/simpaths/model/SimPathsModel.java`	Integrates template generation and population imputation.
`src/main/java/simpaths/model/Person.java`	Adds template matching and income reconstruction.
`src/main/java/simpaths/model/lifetime_incomes/WhiteNoiseEstimate.java`	Uses integer observation ages.
`src/main/java/simpaths/model/lifetime_incomes/SampleIndividual.java`	Adds survey matching and mortality projection.
`src/main/java/simpaths/model/lifetime_incomes/RegressionProxy.java`	Supplies age and GDP regression inputs.
`src/main/java/simpaths/model/lifetime_incomes/ManagerProjectLifetimeIncomes.java`	Removes the database-backed projection manager.
`src/main/java/simpaths/model/lifetime_incomes/ManagerLifetimeIncomes.java`	Coordinates template generation, matching, and exports.
`src/main/java/simpaths/model/lifetime_incomes/LifetimeIncomeImputation.java`	Removes legacy cohort-based matching.
`src/main/java/simpaths/model/lifetime_incomes/IndividualComparator.java`	Removes the legacy donor comparator.
`src/main/java/simpaths/model/lifetime_incomes/Individual.java`	Removes the persisted donor entity.
`src/main/java/simpaths/model/lifetime_incomes/ImputedIncome.java`	Adds age-specific income export records.
`src/main/java/simpaths/model/lifetime_incomes/ExportCSV.java`	Removes the legacy cohort exporter.
`src/main/java/simpaths/model/lifetime_incomes/EquivalisedIncomeCDF.java`	Removes the legacy distribution helper.
`src/main/java/simpaths/model/lifetime_incomes/BirthCohort.java`	Removes the persisted cohort entity.
`src/main/java/simpaths/model/lifetime_incomes/AnnualIncome.java`	Removes the persisted annual-income entity.
`src/main/java/simpaths/model/lifetime_incomes/AdjustmentFactors.java`	Revises fixed-effect and age-specific noise factors.
`src/main/java/simpaths/model/enums/Gender.java`	Adds boolean conversion helpers.
`src/main/java/simpaths/model/BenefitUnit.java`	Adds an income getter and clarifies a scale comment.
`src/main/java/simpaths/data/Parameters.java`	Caps observed-income lookups at the maximum age.
`input/InitialPopulations/compile/07_was_wealth_data.do`	Changes whitespace around a continuation marker.
`documentation/wiki/developer-guide/repository-guide.md`	Renames the manager in the wiki package table.
`documentation/repository-guide.md`	Renames the manager in the package table.
`.gitignore`	Excludes annual lifetime-income survey samples.

_{Review effort: Balanced}

justin-ven · 2026-10-31T17:21:43Z

        this.demEnterSample = demEnterSample;

-        if (originalPerson.ltIncomeDonor!=null) {
+        if (demEnterSample != SampleEntry.InputData && Parameters.lifetimeIncomeImpute) {


justin-ven · 2026-10-31T17:21:22Z

+        if (lifetimeIncomeImpute) {
+            ManagerLifetimeIncomes.generateNormalisedProfiles(log, lifetimeIncomeCapAge, lifetimeIncomeEndAge, lifetimeIncomeCohortSize, lifetimeIncomeTemplates,
+                    lifetimeIncomeWriteToCSV, lifetimeIncomeRandomSeed, outputPath);
+            boolean projectSample = true;


This internal control is deliberate at present, as the pull request involves functionality that is not yet fully developed

justin-ven · 2026-10-31T17:24:23Z

+            double dispIncomePerYear = person.getBenefitUnit().getYDispMonth() * 12.0;
+            double equivScale = person.getBenefitUnit().getEquivalisedWeight();
+            this.observedIncome = dispIncomePerYear / equivScale;


justin-ven · 2026-10-31T17:26:33Z

+            templateIndividuals.sort(Comparator.comparingDouble(individual -> individual.getNormIncome(ageHere).getZValue()));
+            final double[] sortedZ = templateIndividuals.stream().mapToDouble(individual -> individual.getNormIncome(ageHere).getZValue()).toArray();
+            boolean agePlus = (age>=capAge);


This cap on age is due to the cap applied by the FRS - age 80 donors are best understood as aged 80+

justin-ven · 2026-10-31T17:29:04Z

+                }
+            }
+            try {
+                String path = outputDir + File.separator + "imputed_incomes.csv";


dav-sonn · 2026-10-27T17:20:22Z

🟡 Changes recommended

Startup failures, inconsistent income matching and exports, and a broken Stata continuation remain unresolved.

6 open issues

Restore whitespace before Stata continuation comment · New

Avoid copying uninitialized lifetime income for processed input · New

Make survey analysis independently opt-in · New

Use matching household income for exported observations · New

Use consistent age capping for donor matching · New

Prevent imputation exports from overwriting each other · New

What changed in this PR
This PR advances SimPaths’ lifetime-income implementation by replacing persisted birth-cohort donors with shared, in-memory normalised income profiles.

Changes:

Generate template profiles and match them to the initial simulation population.

Add survey-sample matching, mortality projection, and income exports.

Update age-specific calibration, supporting model APIs, and package documentation.

File Description
src/main/java/simpaths/model/SimPathsModel.java Integrates template generation and population imputation.
src/main/java/simpaths/model/Person.java Adds template matching and income reconstruction.
src/main/java/simpaths/model/lifetime_incomes/WhiteNoiseEstimate.java Uses integer observation ages.
src/main/java/simpaths/model/lifetime_incomes/SampleIndividual.java Adds survey matching and mortality projection.
src/main/java/simpaths/model/lifetime_incomes/RegressionProxy.java Supplies age and GDP regression inputs.
src/main/java/simpaths/model/lifetime_incomes/ManagerProjectLifetimeIncomes.java Removes the database-backed projection manager.
src/main/java/simpaths/model/lifetime_incomes/ManagerLifetimeIncomes.java Coordinates template generation, matching, and exports.
src/main/java/simpaths/model/lifetime_incomes/LifetimeIncomeImputation.java Removes legacy cohort-based matching.
src/main/java/simpaths/model/lifetime_incomes/IndividualComparator.java Removes the legacy donor comparator.
src/main/java/simpaths/model/lifetime_incomes/Individual.java Removes the persisted donor entity.
src/main/java/simpaths/model/lifetime_incomes/ImputedIncome.java Adds age-specific income export records.
src/main/java/simpaths/model/lifetime_incomes/ExportCSV.java Removes the legacy cohort exporter.
src/main/java/simpaths/model/lifetime_incomes/EquivalisedIncomeCDF.java Removes the legacy distribution helper.
src/main/java/simpaths/model/lifetime_incomes/BirthCohort.java Removes the persisted cohort entity.
src/main/java/simpaths/model/lifetime_incomes/AnnualIncome.java Removes the persisted annual-income entity.
src/main/java/simpaths/model/lifetime_incomes/AdjustmentFactors.java Revises fixed-effect and age-specific noise factors.
src/main/java/simpaths/model/enums/Gender.java Adds boolean conversion helpers.
src/main/java/simpaths/model/BenefitUnit.java Adds an income getter and clarifies a scale comment.
src/main/java/simpaths/data/Parameters.java Caps observed-income lookups at the maximum age.
input/InitialPopulations/compile/07_was_wealth_data.do Changes whitespace around a continuation marker.
documentation/wiki/developer-guide/repository-guide.md Renames the manager in the wiki package table.
documentation/repository-guide.md Renames the manager in the package table.
.gitignore Excludes annual lifetime-income survey samples.
Review effort: Balanced

@justin-ven: some issues reported here are flagged with high priority. Could you briefly check whether they're relevant? Thank you!
I'm happy to proceed with the merge after your confirmation.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

justin-ven · 2026-10-31T17:37:44Z

Hi David, I’ve finally managed to review the pull request – all done and ready to merge. Best, Justin. From: David Sonnewald ***@***.***> Sent: 27 October 2026 18:21 To: simpaths/SimPaths ***@***.***> Cc: justin-ven ***@***.***>; Mention ***@***.***> Subject: Re: [simpaths/SimPaths] Lifetime income current (PR #545) [Image removed by sender.]dav-sonn left a comment (simpaths/SimPaths#545)<#545 (comment)> 🟡 Changes recommended Startup failures, inconsistent income matching and exports, and a broken Stata continuation remain unresolved. 6 open issues * [Image removed by sender. High severity] <https://camo.githubusercontent.com/3a45afe3f70ffe1b8bd7e55cd010b3cfdea3251a5d0661ebd6237ecb6846bcb6/68747470733a2f2f6769746875622e6769746875626173736574732e636f6d2f7374617469632f696d616765732f69636f6e732f636f70696c6f742d636f64652d7265766965772f686967682d76322d6c696768742e706e67> Restore whitespace before Stata continuation comment<#545 (comment)> · New * [Image removed by sender. High severity] <https://camo.githubusercontent.com/3a45afe3f70ffe1b8bd7e55cd010b3cfdea3251a5d0661ebd6237ecb6846bcb6/68747470733a2f2f6769746875622e6769746875626173736574732e636f6d2f7374617469632f696d616765732f69636f6e732f636f70696c6f742d636f64652d7265766965772f686967682d76322d6c696768742e706e67> Avoid copying uninitialized lifetime income for processed input<#545 (comment)> · New * [Image removed by sender. High severity] <https://camo.githubusercontent.com/3a45afe3f70ffe1b8bd7e55cd010b3cfdea3251a5d0661ebd6237ecb6846bcb6/68747470733a2f2f6769746875622e6769746875626173736574732e636f6d2f7374617469632f696d616765732f69636f6e732f636f70696c6f742d636f64652d7265766965772f686967682d76322d6c696768742e706e67> Make survey analysis independently opt-in<#545 (comment)> · New * [Image removed by sender. Medium severity] <https://camo.githubusercontent.com/61bc3f51460c23a17b60ad45bd0941ac833960ebf1f0f46c74b915e68f750e93/68747470733a2f2f6769746875622e6769746875626173736574732e636f6d2f7374617469632f696d616765732f69636f6e732f636f70696c6f742d636f64652d7265766965772f6d656469756d2d76322d6c696768742e706e67> Use matching household income for exported observations<#545 (comment)> · New * [Image removed by sender. Medium severity] <https://camo.githubusercontent.com/61bc3f51460c23a17b60ad45bd0941ac833960ebf1f0f46c74b915e68f750e93/68747470733a2f2f6769746875622e6769746875626173736574732e636f6d2f7374617469632f696d616765732f69636f6e732f636f70696c6f742d636f64652d7265766965772f6d656469756d2d76322d6c696768742e706e67> Use consistent age capping for donor matching<#545 (comment)> · New * [Image removed by sender. Medium severity] <https://camo.githubusercontent.com/61bc3f51460c23a17b60ad45bd0941ac833960ebf1f0f46c74b915e68f750e93/68747470733a2f2f6769746875622e6769746875626173736574732e636f6d2f7374617469632f696d616765732f69636f6e732f636f70696c6f742d636f64652d7265766965772f6d656469756d2d76322d6c696768742e706e67> Prevent imputation exports from overwriting each other<#545 (comment)> · New What changed in this PR This PR advances SimPaths’ lifetime-income implementation by replacing persisted birth-cohort donors with shared, in-memory normalised income profiles. Changes: * Generate template profiles and match them to the initial simulation population. * Add survey-sample matching, mortality projection, and income exports. * Update age-specific calibration, supporting model APIs, and package documentation. File Description src/main/java/simpaths/model/SimPathsModel.java Integrates template generation and population imputation. src/main/java/simpaths/model/Person.java Adds template matching and income reconstruction. src/main/java/simpaths/model/lifetime_incomes/WhiteNoiseEstimate.java Uses integer observation ages. src/main/java/simpaths/model/lifetime_incomes/SampleIndividual.java Adds survey matching and mortality projection. src/main/java/simpaths/model/lifetime_incomes/RegressionProxy.java Supplies age and GDP regression inputs. src/main/java/simpaths/model/lifetime_incomes/ManagerProjectLifetimeIncomes.java Removes the database-backed projection manager. src/main/java/simpaths/model/lifetime_incomes/ManagerLifetimeIncomes.java Coordinates template generation, matching, and exports. src/main/java/simpaths/model/lifetime_incomes/LifetimeIncomeImputation.java Removes legacy cohort-based matching. src/main/java/simpaths/model/lifetime_incomes/IndividualComparator.java Removes the legacy donor comparator. src/main/java/simpaths/model/lifetime_incomes/Individual.java Removes the persisted donor entity. src/main/java/simpaths/model/lifetime_incomes/ImputedIncome.java Adds age-specific income export records. src/main/java/simpaths/model/lifetime_incomes/ExportCSV.java Removes the legacy cohort exporter. src/main/java/simpaths/model/lifetime_incomes/EquivalisedIncomeCDF.java Removes the legacy distribution helper. src/main/java/simpaths/model/lifetime_incomes/BirthCohort.java Removes the persisted cohort entity. src/main/java/simpaths/model/lifetime_incomes/AnnualIncome.java Removes the persisted annual-income entity. src/main/java/simpaths/model/lifetime_incomes/AdjustmentFactors.java Revises fixed-effect and age-specific noise factors. src/main/java/simpaths/model/enums/Gender.java Adds boolean conversion helpers. src/main/java/simpaths/model/BenefitUnit.java Adds an income getter and clarifies a scale comment. src/main/java/simpaths/data/Parameters.java Caps observed-income lookups at the maximum age. input/InitialPopulations/compile/07_was_wealth_data.do Changes whitespace around a continuation marker. documentation/wiki/developer-guide/repository-guide.md Renames the manager in the wiki package table. documentation/repository-guide.md Renames the manager in the package table. .gitignore Excludes annual lifetime-income survey samples. Review effort: Balanced @justin-ven<https://github.com/justin-ven>: some issues reported here are flagged with high priority. Could you briefly check whether they're relevant? Thank you! I'm happy to proceed with the merge after your confirmation. — Reply to this email directly, view it on GitHub<#545?email_source=notifications&email_token=AKJL7NCXWR6UX3RLNN3LGZT5WDKOZA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTMNBUHA3DKMJUGIZ2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-6448651423>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKJL7NC6DUFAHMWKJ3P7DH35WDKOZAVCNFSNUABFKJSXA33TNF2G64TZHM3TANBQGE2DEMJYHNEXG43VMU5TMMBRG4ZDANBRG422C5QC>. Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS<https://github.com/notifications/mobile/ios/AKJL7NGF63C3NOKKYN2A27L5WDKOZA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTMNBUHA3DKMJUGIZ2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJKTGN5XXIZLSL5UW64Y> and Android<https://github.com/notifications/mobile/android/AKJL7NGAPTW4W2AH526VTZT5WDKOZA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTMNBUHA3DKMJUGIZ2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLTGN5XXIZLSL5QW4ZDSN5UWI>. Download it today! You are receiving this because you were mentioned.Message ID: ***@***.***>

justin-ven added 10 commits August 31, 2026 09:37

Merge pull request simpaths#518 from simpaths/develop

b649c69

Update of main (pre-September26-release)

parameterisation of lifetime income analysis

74574be

intermediate

0cff4f4

Merge branch 'develop' into lifetime-income-current

7d42be9

Merge branch 'simpaths:main' into lifetime-income-current

c68903d

Merge remote-tracking branch 'upstream/develop' into lifetime-income-…

93860cb

…current

Merge branch 'lifetime-income-current' of https://github.com/justin-v…

b7cf2bf

…en/SimPaths into lifetime-income-current

lifetime income analysis

36cfb66

Merge remote-tracking branch 'upstream/develop' into lifetime-income-…

328feaf

…current

omit lifetime incomes here

43e9869

dav-sonn requested a balanced review from Copilot October 27, 2026 15:53

Copilot started reviewing on behalf of dav-sonn October 27, 2026 15:54

View session

Copilot AI reviewed Oct 27, 2026

View reviewed changes

justin-ven and others added 2 commits October 31, 2026 18:14

Fix formatting of debt variables in wealth data

74a5690

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

responses to copilot comments

20d548b

Navigation Menu

Repository navigation

Lifetime income current - #545

Lifetime income current#545
justin-ven wants to merge 12 commits into
simpaths:developfrom
justin-ven:lifetime-income-current

justin-ven commented Oct 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

justin-ven Oct 31, 2026

Uh oh!

justin-ven Oct 31, 2026

Uh oh!

justin-ven Oct 31, 2026

Uh oh!

justin-ven Oct 31, 2026

Uh oh!

justin-ven Oct 31, 2026

Uh oh!

dav-sonn commented Oct 27, 2026

🟡 Changes recommended

Uh oh!

justin-ven commented Oct 31, 2026 via email

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

justin-ven commented Oct 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

justin-ven Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

justin-ven Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

justin-ven Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

justin-ven Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

justin-ven Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

dav-sonn commented Oct 27, 2026

🟡 Changes recommended

Uh oh!

justin-ven commented Oct 31, 2026 via email

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants