Retrieve and merge current weather information from stations close to each other

[ugur175]

Hello,

I am trying to locate nearest dwd sensors from a list, therefore wanted to install the new library of dwd. However when I try to run the sample code seen at the Usage as a library documentation, I get this error.

dw = DwdWeather() # Sample code 
No schema information for resolution "None" found in knowledge base.
An exception has occurred, use %tb to see the full traceback.

SystemExit: 1

/home/halden/anaconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3333: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.
  warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)

Edit: Apparently I forgot to specify the resolution parameter. It was a problem from my end. therefore I closed the topic.
I googled possible solutions but nothing worked so far. Any ideas about troubleshooting?

[amotl]

Dear @ugur175,

thanks for writing in. We will be happy to look into this issue. As you already closed it again, may we humbly ask if things have been resolved already?

With kind regards,
Andreas.

[amotl]

Apparently I forgot to specify the resolution parameter. It was a problem from my end. therefore I closed the topic.

I see, thanks. Good luck for the next steps. Please let us know if you experience any other issues.

I googled possible solutions but nothing worked so far. Any ideas about troubleshooting?

Do you have any more troubles after resolving this beforehand? Will be happy to help...

[ugur175]

Dear @amotl,

I successfully gathered nearest sensors and their id number. However I encountered additional problem. For example I have sensor ids such as "15810, 5758, 6098" and when I implement import_measures function such as:

dwd.import_measures(station_id = 15810, latest = True, historic = False)

I get this this:

Station "15810" has no data for category "air_temperature"
Station "15810" has no data for category "cloud_type"
Station "15810" has no data for category "cloudiness"
Station "15810" has no data for category "dew_point"
Station "15810" has no data for category "extreme_temperature"
Station "15810" has no data for category "extreme_wind"
Importing "precipitation" data from "https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/10_minutes/precipitation/recent/10minutenwerte_nieder_15810_akt.zip/produkt_zehn_min_rr_20180322_20190922_15810.txt" not implemented yet
Station "15810" has no data for category "pressure"
Station "15810" has no data for category "soil_temperature"
Station "15810" has no data for category "solar"
Station "15810" has no data for category "sun"
Station "15810" has no data for category "visibility"
Station "15810" has no data for category "wind"
Station "15810" has no data for category "wind_synop"

I checked them manually, https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/10_minutes/air_temperature/recent/ has no sensor named 15810, which I need to get temperature and humidity results. However sensor 15810 has entries in https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/10_minutes/precipitation/recent/

Is there a way to get the nearest sensor results for air temperature segment instead of precipitation?

[amotl]

Dear @ugur175,

thanks for your detailed answer. Let me see how we could help here.

I checked them manually, https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/10_minutes/air_temperature/recent/ has no sensor named 15810

Sorry to hear that. I believe there's nothing we can do about this, right?

However sensor 15810 has entries in https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/10_minutes/precipitation/recent/

I see from the excerpt of your log that this measurement is not implemented yet. Are you (also) asking to implement this guy?

Is there a way to get the nearest sensor results for air temperature segment instead of precipitation?

If the program is telling us Station "15810" has no data for category "air_temperature", this is probably true unless we prove otherwise.

What comes to my mind about this is:
a) Maybe choose a different station not too far away from the designated 15810.
b) Maybe also look at the "hourly" category. It might well be different to the "10_minutes" category you have requested within your trials.

Feel free to come up with further ideas how we could support you.

With kind regards,
Andreas.

[amotl]

I would like to confirm that indeed there seems to be no data available from stations 15810, 5758 and 6098.

However, the program works flawlessly on a different station where data is actually available from.

dwdweather weather 7368 20190901T17

Maybe @wetterfrosch knows anything about the lost stations?

[amotl]

when I try to run the sample code seen at the Usage as a library documentation, I get this error.

dw = DwdWeather() # Sample code 
No schema information for resolution "None" found in knowledge base.

We've just fixed this error, see #15.

[amotl]

Dear @ugur175,

while we unlocked retrieval of the "now" timerange [1] by #13, we are still not able to find any current data for the named stations 5758, 6098 or 15810. Bummer!

Do you believe we are missing something here or is there actually no data available for these stations at all?

With kind regards,
Andreas.

[1] From e.g. https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/10_minutes/air_temperature/now/

[ugur175]

Dear @amotl,

Since above mentioned sensors had no data for temperature and humidity, I wrote a script from by myself. What I did was:

1- Get the sensor id, lat and long results for temp. sensors under here: https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/10_minutes/air_temperature/now/zehn_now_tu_Beschreibung_Stationen.txt

2- Calculate the nearest sensor locations to my sensor locations. (I used haversine formula, however while thinking I thought nearest sensor by what? Some people might want the nearest sensor just by kilometer, but maybe in switzerland an user might want to consider the height of the sensor while calculating the nearest sensor too, a sensor in the lower heights might not have representative data compared to sensor on the top of alps.)

3- I saved the nearest sensor ids into a list and scan through them from here: https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/10_minutes/air_temperature/now/

4- Cached the data, deleted everything except sensor id, date, temp and RH values,

5- Appended them to my dataframe.

For sensors 5758, 6098 or 15810. I think those weather stations only measure precipitation, and even though they were the nearest sensors, I needed the temp. and RH. data. Therefore I limited the search only by the sensor list I gathered from the link above.

I think it will be better to specifiy temperature, solar or precipitation in the dwdweather2 connector, so the system can automatically detect the nearest sensor by the sensor list in those folders.

[amotl]

Dear @ugur175,

thanks for outlining the process of widening the "search scope" from a fixed list of station identifiers in order to get the desired data from the most relevant stations closest to the original list (probably by the means of different strategies, as you mentioned). This sounds like a totally sensible fallback-like solution to tackle that problem where the original stations just don't deliver the requested data.

However, this process will involve multiple steps to achieve the goal like you outlined above. I will definitively take that into consideration when being able to continue our work on the codebase. I see that as a kind of advanced/smart filtering feature.

Thanks again for bringing this up and good to hear you already found a solution on your own behalf. Let's keep this open in order to track this feature request appropriately.

With kind regards,
Andreas.

[amotl]

The feature requested here might have been implemented by @JohannesRol through JohannesRol/dwdopendata@83c5290 already. See also #3.

[amotl]

Dear @ugur175,

I would like to announce that development moved on to contribute to the Wetterdienst package. It is the successor library to dwdweather and dwdweather2 and is becoming more and more popular these days. We have been trying to get this en par with all the features offered by dwdweather2 and we have come pretty far these days. Enjoy!

Regarding the gist of this issue: We will be happy to hear back from you if and how easy your requirements can be solved using Wetterdienst for the job. When there will be things to improve, we will definitively take them into consideration.

With kind regards,
Andreas.

cc @gutzbenj