weather.md

Weather data

As part of time varying units, OED wants to track weather data and use to normalize reading data. OED may also do graphics where usage and temperature are shown together. This will mean having two different y-axis scales, something being done for line compare.

At a minimum, OED will want to have temperature data. It may want other information. Exactly what needs to be determined. For now, only temperature is being considered.

See issue #1291 about this.

Getting weather data

Update 7/24: A team created code in this GitHub repo on branch frontend-team to create a weather location and get the temperature data. While working, it needs the following:

• Several translation keys are not done: weather.input.error, weather.failed.to.create.location & weather.successfully.create.location where others should be checked.
• There is a fetchData npm command in package.json. It is not working, the name should be changed and there should be a way to specify dates. See src/server/services/weather/fetchData.js.
• A system needs to be set up to get weather data for a location. This should be via a admin interface and cron jobs to do automatically.
• The code should be reviewed and updated as needed. See src/client/app/components/weather/ for the UI code. Note @huss did a review of one version and their notes can be asked for.
• The SQL code in src/server/sql/weather_data/ and src/server/sql/weather_location/ does not have a migration in src/server/migrations/1.0.0-1.1.0 (or whatever version is needed).

In addition, the graphic with meter/group data and weather data needs to be done.

It has become clearer that normalizing by temperature is more complex:

• Many resources should not be normalized by temperature (such as mass for recycling). OED needs to work out how admins will set which ones can be done.
• Normalizing probably requires time of year info since, for example, electricity may differ in hot vs cold months. It is possible a model will automatically take this into account.
• The model seems to need other information such as building nominal temperature where it does not heat or cool. If so, how will this relate to meters/groups.
• Many other factors may be needed.

Given this, researching and testing the models will need to happen before it is incorporated into OED. If possible, it is desired to use previous data to create a model to then predict future usage based on temperature. That could be used, for example, to see how much a change in a building actually saved. Currently OED could show this but not taking into account temperature changes between years (for example).

Original

OED need to acquire weather data to support this feature. In a perfect world, there would be a single source of data for all sites. This may not be possible so having a standard format with modules to talk to different weather sources (similarly to how meter data is done) would be desirable. Demonstrating with two sources (or one if a standard source exists) would be good for the initial work.

Here are some possible sources where more may exist. The value and appropriateness of any options needs to be evaluated.

• U.S. National Centers for Environmental Information with world-wide data & historical
  • all data
  • land data including weather
• seems to have lots of historical and current data world-wide with API
• U.S. NOAA info with weather/degree days, etc. Need to see what has and if API.
• U.S. National Weather Service API to get data. Unclear if historical and probably not world-wide.
  • FAQ
  • https://www.weather.gov/documentation/services-web-api Documentation
• question about getting weather data but dated
• others are available if needed and find with a web search

Admin control of weather data

Admins need to be able to set if and how (if needed) weather data is acquired. The exact design depends on how weather data is acquired. Its page may be the general admin panel if it is simple or a newly linked page.

Use weather data to normalize values

Once appropriate weather data is available, it needs to be used to normalize the meter data. Here are some sources of information on using weather data to normalize usage with basic ideas to modeling. Others are available. The possibilities need to be analyzed, a choice made, tested against real data and implemented within OED.

• PDF with how to use historical data to get energy normalized data.
• basic CDD/HDD
• research paper
• normalization info and methods from company
• Heating/cooling days and normalizationdegree-days-and-weather-normalization/
• free online CDD/HDD calculator

Database changes

The exact changes relate to the sections above as the needed data may change. The database and models need to be created. This is a first cut at the DB setup:

weather location

id, identifier, gps, note

meter

The field weather_location_id is added (foreign key into weather location table)

group

Similar to meter

weather data

weather_location_id (foreign key into weather location table), start_time, end_time, temp

May need other values depending on choices made.

UI interface

This will be decided after other choices are made.

• It seems a good idea to have a group check the location of the underlying meters and warn if they differ from the group location. This is different but has some similarities to the area calculation for groups.
• Depending on how normalization is done, we may want DB views for hour, half day, full day or something else to optimize normalizing by weather.
• UI choices may be similar to area to turn on/off. Change y-axis label to include "normalized by weather" (internationalized) assuming it fits.
• Need to decide if compatible with area normalization (seems to be) or any other graphic choices.
• There limitations on what data can weather normalized. Need to specify precisely.