Skip to content

Update Labour to get specific properties when getting person#1829

Open
tamuri wants to merge 12 commits intomasterfrom
tamuri/labour-performance
Open

Update Labour to get specific properties when getting person#1829
tamuri wants to merge 12 commits intomasterfrom
tamuri/labour-performance

Conversation

@tamuri
Copy link
Collaborator

@tamuri tamuri commented Feb 23, 2026

Profiling finds that repeated df.loc[[person_id]], where all properties are retrieved, is a bottleneck in the Labour module. This PR updates the module to only retrieve those properties necessary.

Joe - although each labour linear model only uses a few of these properties, grabbing everything necessary for all of them is an easy change for now.

I'm running profiling again to see if further improvements can be made.

Copy link
Collaborator

@tbhallett tbhallett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes good sense to me.

- the decorator allows us to list none or more properties needed to evaluate the function
- does not change the logic inside the function, but is a bit of metadata that can be used by callers
@tamuri
Copy link
Collaborator Author

tamuri commented Feb 27, 2026

Trying a different approach to speed this up further. This is has reduced runtime of predict from 4% to <0.3%

A decorator person_properties in labour_lm adds metadata to the function specifying exactly which person properties from the population dataframe (if any) are needed to evaluate the function. If nothing it needed, it is set to an empty list.

The labour module uses this metadata to grab only those specified properties from the population dataframe.

Sometimes the required modules are not available and we have to remove their properties from the list.

Sorry, @tbhallett @joehcollins, do you mind looking at this again.

@tamuri tamuri requested a review from tbhallett February 27, 2026 22:16
… then new simulations in the same process may fail.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants