Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add food affordability data #1381

Merged
merged 4 commits into from
Jul 26, 2023
Merged

Add food affordability data #1381

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
999c987
chore: Install WB python package
pabloarosado Jul 24, 2023
9069840
feat(harmonize): Include country code as another alias in harmonize tool
pabloarosado Jul 25, 2023
a54453f
feat(wb): Add snapshot, meadow, garden and grapher steps for food aff…
pabloarosado Jul 25, 2023
4e32135
Implement Pablo As suggestions
pabloarosado Jul 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions dag/agriculture.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,12 @@ steps:
#
data://explorers/agriculture/2023-06-12/crop_yields:
- data://grapher/agriculture/2023-06-12/attainable_yields
#
# Food prices for nutrition - World Bank based on FAO.
#
data://meadow/wb/2023-07-24/food_prices_for_nutrition:
- snapshot://wb/2023-07-24/food_prices_for_nutrition.csv
data://garden/wb/2023-07-24/food_prices_for_nutrition:
- data://meadow/wb/2023-07-24/food_prices_for_nutrition
data://grapher/wb/2023-07-24/food_prices_for_nutrition:
- data://garden/wb/2023-07-24/food_prices_for_nutrition
2 changes: 2 additions & 0 deletions etl/harmonize.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ def __init__(self) -> None:
if code in aliases_s.index:
for alias in aliases_s.loc[[code]]:
aliases[alias.lower()] = name
# Include the region code itself as another alias.
aliases[code.lower()] = name

self.aliases = aliases
self.valid_names = valid_names
Expand Down
188 changes: 188 additions & 0 deletions etl/steps/data/garden/wb/2023-07-24/food_prices_for_nutrition.countries.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
{
"ABW": "Aruba",
"AGO": "Angola",
"AIA": "Anguilla",
"ALB": "Albania",
"ARE": "United Arab Emirates",
"ARG": "Argentina",
"ARM": "Armenia",
"ATG": "Antigua and Barbuda",
"AUS": "Australia",
"AUT": "Austria",
"AZE": "Azerbaijan",
"BDI": "Burundi",
"BEL": "Belgium",
"BEN": "Benin",
"BFA": "Burkina Faso",
"BGD": "Bangladesh",
"BGR": "Bulgaria",
"BHR": "Bahrain",
"BHS": "Bahamas",
"BIH": "Bosnia and Herzegovina",
"BLR": "Belarus",
"BLZ": "Belize",
"BMU": "Bermuda",
"BOL": "Bolivia",
"BRA": "Brazil",
"BRB": "Barbados",
"BRN": "Brunei",
"BTN": "Bhutan",
"BWA": "Botswana",
"CAF": "Central African Republic",
"CAN": "Canada",
"CHE": "Switzerland",
"CHL": "Chile",
"CHN": "China",
"CIV": "Cote d'Ivoire",
"CMR": "Cameroon",
"COD": "Democratic Republic of Congo",
"COG": "Congo",
"COL": "Colombia",
"COM": "Comoros",
"CPV": "Cape Verde",
"CRI": "Costa Rica",
"CUW": "Curacao",
"CYM": "Cayman Islands",
"CYP": "Cyprus",
"CZE": "Czechia",
"DEU": "Germany",
"DJI": "Djibouti",
"DMA": "Dominica",
"DNK": "Denmark",
"DOM": "Dominican Republic",
"DZA": "Algeria",
"ECU": "Ecuador",
"EGY": "Egypt",
"ESP": "Spain",
"EST": "Estonia",
"ETH": "Ethiopia",
"FIN": "Finland",
"FJI": "Fiji",
"FRA": "France",
"GAB": "Gabon",
"GBR": "United Kingdom",
"GHA": "Ghana",
"GIN": "Guinea",
"GMB": "Gambia",
"GNB": "Guinea-Bissau",
"GNQ": "Equatorial Guinea",
"GRC": "Greece",
"GRD": "Grenada",
"GUY": "Guyana",
"HKG": "Hong Kong",
"HND": "Honduras",
"HRV": "Croatia",
"HTI": "Haiti",
"HUN": "Hungary",
"IDN": "Indonesia",
"IND": "India",
"IRL": "Ireland",
"IRN": "Iran",
"IRQ": "Iraq",
"ISL": "Iceland",
"ISR": "Israel",
"ITA": "Italy",
"JAM": "Jamaica",
"JOR": "Jordan",
"JPN": "Japan",
"KAZ": "Kazakhstan",
"KEN": "Kenya",
"KGZ": "Kyrgyzstan",
"KHM": "Cambodia",
"KNA": "Saint Kitts and Nevis",
"KOR": "South Korea",
"KWT": "Kuwait",
"LAO": "Laos",
"LBR": "Liberia",
"LCA": "Saint Lucia",
"LKA": "Sri Lanka",
"LSO": "Lesotho",
"LTU": "Lithuania",
"LUX": "Luxembourg",
"LVA": "Latvia",
"MAR": "Morocco",
"MDA": "Moldova",
"MDG": "Madagascar",
"MDV": "Maldives",
"MEX": "Mexico",
"MKD": "North Macedonia",
"MLI": "Mali",
"MLT": "Malta",
"MMR": "Myanmar",
"MNE": "Montenegro",
"MNG": "Mongolia",
"MOZ": "Mozambique",
"MRT": "Mauritania",
"MSR": "Montserrat",
"MUS": "Mauritius",
"MWI": "Malawi",
"MYS": "Malaysia",
"NAM": "Namibia",
"NER": "Niger",
"NGA": "Nigeria",
"NIC": "Nicaragua",
"NLD": "Netherlands",
"NOR": "Norway",
"NPL": "Nepal",
"NZL": "New Zealand",
"OMN": "Oman",
"PAK": "Pakistan",
"PAN": "Panama",
"PER": "Peru",
"PHL": "Philippines",
"POL": "Poland",
"PRT": "Portugal",
"PRY": "Paraguay",
"PSE": "Palestine",
"QAT": "Qatar",
"ROU": "Romania",
"RUS": "Russia",
"RWA": "Rwanda",
"SAU": "Saudi Arabia",
"SDN": "Sudan",
"SEN": "Senegal",
"SGP": "Singapore",
"SLE": "Sierra Leone",
"SLV": "El Salvador",
"SRB": "Serbia",
"STP": "Sao Tome and Principe",
"SUR": "Suriname",
"SVK": "Slovakia",
"SVN": "Slovenia",
"SWE": "Sweden",
"SWZ": "Eswatini",
"SXM": "Sint Maarten (Dutch part)",
"SYC": "Seychelles",
"TCA": "Turks and Caicos Islands",
"TCD": "Chad",
"TGO": "Togo",
"THA": "Thailand",
"TJK": "Tajikistan",
"TTO": "Trinidad and Tobago",
"TUN": "Tunisia",
"TUR": "Turkey",
"TWN": "Taiwan",
"TZA": "Tanzania",
"UGA": "Uganda",
"URY": "Uruguay",
"USA": "United States",
"VCT": "Saint Vincent and the Grenadines",
"VGB": "British Virgin Islands",
"VNM": "Vietnam",
"ZAF": "South Africa",
"ZMB": "Zambia",
"ZWE": "Zimbabwe",
"BON": "Bonaire (WB)",
pabloarosado marked this conversation as resolved.
Show resolved Hide resolved
"EAS": "East Asia & Pacific (WB)",
"ECS": "Europe & Central Asia (WB)",
"HIC": "High-income countries",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general we are using the hyphen only for lower-middle or upper-middle. I don't think I have seen high"-"income or low"-"income or lower-middle"-"income much.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is currently how we name income groups:

    MAPPING_CLASSIFICATION = {
        "..": np.nan,  # no available classification for country-year (maybe country didn't exist yet/anymore)
        "L": "Low-income countries",
        "H": "High-income countries",
        "UM": "Upper-middle-income countries",
        "LM": "Lower-middle-income countries",
        "LM*": "Lower-middle-income countries",
    }

For consistency, those are the names we should use. We could consider changing them (but it would be a significant refactor, similar to the one of East Timor).

"LCN": "Latin America & Caribbean (WB)",
"LIC": "Low-income countries (WB)",
"LMC": "Lower-middle-income countries",
"MEA": "Middle East & North Africa (WB)",
"NAC": "North America (WB)",
"SAS": "South Asia (WB)",
"SSF": "Sub-Saharan Africa (WB)",
"UMC": "Upper-middle-income countries",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assignment of (WB) seems inconsistent between the WB income groups: Low-income countries has it, but not High income or Upper-middle or Lower-middle income

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! By the way, I'm assuming that their income groups are consistent (and up-to-date) in all World Bank datasets (and hence consistent with OWID), let me know if that's not a good assumption.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that as well, but it will depend on when that dataset was released. If it's sufficiently old it can have a previous definition of income groups. I see they change every year.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that you say it, it should be that groups are dynamically changing per year, so that would be consistent. I don't know that with 100% certainty.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they assume a definition of income groups that changes over the years, then those income groups are inconsistent with ours. Because we assume the current definition of income groups. But for this dataset (with just a few years of data) I suppose it should not matter much.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I wasn't sure about that. I would assume theirs is dynamic, but I don't know

"WLD": "World"
}
Loading