Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Local authorities calibration #27

Merged
merged 16 commits into from
Dec 6, 2024
Merged

Local authorities calibration #27

merged 16 commits into from
Dec 6, 2024

Conversation

vahid-ahmadi
Copy link
Contributor

@vahid-ahmadi vahid-ahmadi commented Nov 29, 2024 •
edited
Loading

The calibration follows the constituency approach.

  1. 'calibrate.py' initiates calibration through target matrices. The script calls create_local_authority_target_matrix and create_national_target_matrix from 'loss.py'. The matrices contain income, age, and employment metrics.
  2. The script sets weights as PyTorch tensor (360 x 100180). Initial weights derive from household_weight divided by local authority count (360). The process applies log transformation to weights.
  3. Loss combines local and national components. Local loss calculates metric predictions using weights and measures deviation from targets. The national loss ensures aggregate predictions match national totals. The function returns a combined mean squared error.

Fixes #32

@MaxGhenis
Copy link

A few suggestions (feel free to mark as draft if it's not for review yet):

  1. I see there's no direct download link to turn this into a shell or Python script. Could you please add a README.md to document the procedure to retrieve the file?
  2. Could you confirm this is a many:many mapping?
  3. What does the notebook do?

Please also file an issue to connect this PR to.

@vahid-ahmadi
Copy link
Contributor Author

  1. "mapping_2010_to_LAD2024.py": This file generates direct mapping from 2010 constituencies to 2024 local authority districts. Now we don't need two-step mapping (constituency mapping and then LD mapping). So, simply you can apply "mapping_2010_to_LAD2024.csv" on weights.h5 file.

  2. "Local_Authority_example.ipynb": This file is an example file of to how use this mapping to generate a UK income map.

  3. Download the shapefile from here: https://drive.google.com/file/d/1ZDmZ69LvojYTzclDKZAbB5oKWrTIOMmR/view?usp=sharing

@nikhilwoodruff nikhilwoodruff changed the title constituencies to local authorities Constituencies to local authorities Dec 3, 2024
@nikhilwoodruff nikhilwoodruff changed the title Constituencies to local authorities Map constituencies to local authorities Dec 3, 2024
@vahid-ahmadi vahid-ahmadi changed the title Map constituencies to local authorities Local authorities calibration Dec 5, 2024
nikhilwoodruff
nikhilwoodruff approved these changes Dec 6, 2024
View reviewed changes
local_authority/Vahid_test_files/LA_income_target.csv Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean up this file to be numeric and e.g. exclude tab breaks?

@nikhilwoodruff nikhilwoodruff merged commit 3394913 into PolicyEngine:main Dec 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nikhilwoodruff nikhilwoodruff Awaiting requested review from nikhilwoodruff

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Local authorities calibration
3 participants
@vahid-ahmadi @MaxGhenis @nikhilwoodruff