Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix PsdmTimeBasedWeatherValueFactory #267

Closed
johanneshiry opened this issue Dec 17, 2020 · 3 comments · Fixed by #329 or #561
Closed

Fix PsdmTimeBasedWeatherValueFactory #267

johanneshiry opened this issue Dec 17, 2020 · 3 comments · Fixed by #329 or #561
Assignees
Labels
bug Something isn't working
Milestone

Comments

@johanneshiry
Copy link
Member

johanneshiry commented Dec 17, 2020

The column names of the PsdmTimeBasedWeatherValueFactory are currently invalid.

  • time -> time
  • coordinate -> coordinateid
  • diffuseirradiation -> diffuseirradiance
  • directirradiation -> directirradiance
  • winddirection -> winddirection
  • windvelocity -> windvelocity
  • temperature -> temperature

Furthermore the datum column of theIconTimeBasedWeatherValueFactory might be renamed to time as well. (considering #175)

Furthermore, Psdm� should be renamed to Cosmo.

@johanneshiry johanneshiry added the bug Something isn't working label Dec 17, 2020
@johanneshiry johanneshiry added this to the Version 2.0 milestone Dec 17, 2020
@johanneshiry johanneshiry self-assigned this Dec 17, 2020
@ckittl
Copy link
Member

ckittl commented Mar 31, 2021

As of discussion today: The factory does not reflect the actual scheme in the weather database. Therefore, either the factory has to be altered or at the same time the scheme of the data base.

An argument for changing the data scheme is, that InfluxDB requires the time information to be called time. On the other hand, not changing the scheme, would not lead to breaking other applications.

Decision:

  • Alter the scheme for the COSMO model within the data base, as this kind of is legacy data as well.
  • Rename PsdmTimeBaseWeatherValueFactory to CosmoTimeBasedWeatherValueFactory
  • Alter the scheme of ICON weather data as well (datum -> time, coordinate -> coordinateid; cf. Adapt date column name DWDWeatherTools#1) -- change the column name here as well

@ckittl
Copy link
Member

ckittl commented Apr 6, 2021

Even more issues than expected...

  • For some sources, the actual column names within the data base or file are needed. However, within the factories only the
    "iron case" names of the fields (neither of the known cases, all small letters, no underscores or hyphens) are apparent. These approaches are out there:
    • CouchbaseWeatherSource takes name of coordinate column from constructor -> possibly inconsistent to what the factory expects
    • InfluxDbWeatherSource takes the iron case field name from factory -> query does not work if the column is called coordinate_id
    • SqlWeatherSource queries meta data of table, "irons" the column name and checks for a match against the factory
  • Field names in Couchbase (the actual source) are iron and not snake case as in other sources -> can it be harmonized?

Some reasoning about the next steps:

  • Building of queries / way of determining the column name across sources shall be harmonized
  • Currently all supported technologies support snake case, however, this is not necessarily the case for future technologies
    • Column name is determined by technology (case) and data scheme ("case less" value)
  • The "case less" value of the column name should be invariant (therefore, the harmonization of data schemes)
  • Holding the "case less" value in a way, that allow for technology specific transfer between different cases, e.g. in snake case, within the factories

@johanneshiry We had several discussion about this, would you mind to have one more? ^^

@ckittl
Copy link
Member

ckittl commented Jan 28, 2022

Breaking down the above discussion to actual tasks:

sebastian-peter added a commit that referenced this issue Mar 2, 2022
t-ober added a commit that referenced this issue Mar 3, 2022
…me-adaptions

Adapt to changed scheme of COSMO weather data
sebastian-peter added a commit that referenced this issue Mar 3, 2022
# Conflicts:
#	CHANGELOG.md
#	src/main/java/edu/ie3/datamodel/io/source/couchbase/CouchbaseWeatherSource.java
#	src/main/java/edu/ie3/datamodel/io/source/influxdb/InfluxDbWeatherSource.java
#	src/test/groovy/edu/ie3/datamodel/io/source/couchbase/CouchbaseWeatherSourceCosmoIT.groovy
t-ober added a commit that referenced this issue Mar 4, 2022
…e-adaptions

Adapt column scheme of ICON weather data 2.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants