Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Units issue with isentropic_interpolation_as_dataset and CONUS404 data #3315

Closed
spacekace opened this issue Dec 12, 2023 · 10 comments · Fixed by #3324
Closed

Units issue with isentropic_interpolation_as_dataset and CONUS404 data #3315

spacekace opened this issue Dec 12, 2023 · 10 comments · Fixed by #3324
Labels
Area: Units Pertains to unit information Area: Xarray Pertains to xarray integration Type: Bug Something is not working like it should
Milestone

Comments

@spacekace
Copy link

spacekace commented Dec 12, 2023

What went wrong?

I am trying to use metpy.calc.isentropic_interpolation_as_dataset to interpolate CONUS404 data to isentropic levels. The CONUS404 data is produced using WRF v3.9.1.1. As such, I am using xarray and xWRF to read and process the data for use with MetPy. Unfortunately, I am receiving an error from this function stating:

isentropic_interpolation given arguments with incorrect units: pressure requires "[pressure]" but given "dimensionless"

Pressure is not an input for isentropic_interpolation_as_dataset and therefore I have no control over the pressure units referred to here. It looks like this function calls on the isentropic_interpolation function of metpy.calc, which does require pressure as an input, so perhaps however isentropic_interpolation_as_dataset is calculating pressure from the temperature values provided does not work with WRF/CONUS404 data. I have previously used isentropic_interpolation_as_dataset AND isentropic_interpolation without issue, but that was with ERA5 atmospheric reanalysis GRIB data, which is CF compliant and allows for the use of metpy.parse_cf.

Any insight into how I can fix this issue and complete the interpolation would be greatly appreciated. Thank you!!

Note: You will notice that the "CLAT" coordinate is dropped when reading in variables. This is because isentropic_interpolation_as_dataset would not use variables with two latitude dimensions (i.e., XLAT and CLAT).

Operating System

Linux

Version

1.5.1

Python Version

3.9.16

Code to Reproduce

import metpy
import metpy.calc as mpcalc
from metpy.units import units
import xarray as xr
import xwrf
import numpy as np

#------------------------------------------------------------------------------
#--- Open the NetCDF file -----------------------------------------------------
filename = "/glade/work/kshourd/29_June_2012/wrfout_d01_2012-06-29_12:00:00"
#--- xWRF postprocess & grid destaggering
ds = xr.open_dataset(filename).xwrf.postprocess()
ds = ds.xwrf.destagger()

lat = ds["XLAT"].drop_vars("CLAT")
lon = ds["XLONG"].drop_vars("CLAT")

#--- Extract variables
z = ds['geopotential_height'].drop_vars("CLAT")
z = z.metpy.quantify()
print(z)

ua = ds['wind_east'].drop_vars("CLAT")
ua = ua.metpy.quantify()
va = ds['wind_north'].drop_vars("CLAT")
va = va.metpy.quantify()

pressure = ds["air_pressure"].drop_vars("CLAT")
pressure = pressure.metpy.quantify()

theta = ds["air_potential_temperature"].drop_vars("CLAT")
theta = theta.metpy.quantify()
temp  = mpcalc.temperature_from_potential_temperature(pressure, theta)
temp  = temp.metpy.quantify()

# Compute the geostrophic wind
print("Geostrophic wind calculation...")
ugeo, vgeo = mpcalc.geostrophic_wind(z)
ugeo = ugeo.metpy.quantify()
vgeo = vgeo.metpy.quantify()

#--- Isentropic interpolation w/ MetPy
thlv     = np.arange(298,350,1)
isen_lev = thlv*units.kelvin

isentropic = mpcalc.isentropic_interpolation_as_dataset(isen_lev.squeeze(), temp.squeeze(), ugeo.squeeze(), vgeo.squeeze(), ua.squeeze(), va.squeeze())

print(list(isentropic.variables))

Errors, Traceback, and Logs

/glade/work/kshourd/testing123.py:53: UserWarning: More than one time coordinate present for variable .
  isentropic = mpcalc.isentropic_interpolation_as_dataset(isen_lev.squeeze(), temp.squeeze(), ugeo.squeeze(), vgeo.squeeze(), ua.squeeze(), va.squeeze())
Traceback (most recent call last):
  File "/glade/work/kshourd/testing123.py", line 53, in <module>
    isentropic = mpcalc.isentropic_interpolation_as_dataset(isen_lev.squeeze(), temp.squeeze(), ugeo.squeeze(), vgeo.squeeze(), ua.squeeze(), va.squeeze())
  File "/glade/work/kshourd/conda-envs/wrfplot/lib/python3.9/site-packages/metpy/calc/thermo.py", line 2765, in isentropic_interpolation_as_dataset
    ret = isentropic_interpolation(
  File "/glade/work/kshourd/conda-envs/wrfplot/lib/python3.9/site-packages/metpy/xarray.py", line 1544, in wrapper
    return func(*bound_args.args, **bound_args.kwargs)
  File "/glade/work/kshourd/conda-envs/wrfplot/lib/python3.9/site-packages/metpy/xarray.py", line 1328, in wrapper
    result = func(*bound_args.args, **bound_args.kwargs)
  File "/glade/work/kshourd/conda-envs/wrfplot/lib/python3.9/site-packages/metpy/units.py", line 324, in wrapper
    _check_units_inner_helper(func, sig, defaults, dims, *args, **kwargs)
  File "/glade/work/kshourd/conda-envs/wrfplot/lib/python3.9/site-packages/metpy/units.py", line 311, in _check_units_inner_helper
    raise ValueError(msg)
ValueError: This function changed in 1.0--double check that the function is being called properly.
`isentropic_interpolation` given arguments with incorrect units: `pressure` requires "[pressure]" but given "dimensionless"
@spacekace spacekace added the Type: Bug Something is not working like it should label Dec 12, 2023
@kgoebber
Copy link
Collaborator

What is the output if you print the variable pressure?

The quantify() may not be behaving as expected.

@spacekace
Copy link
Author

Thanks for the quick response. The output of printing the pressure variable is as follows:

<xarray.DataArray 'air_pressure' (Time: 1, z: 50, y: 1015, x: 1367)>
<Quantity([[[[101182.47   101184.734  101181.24   ... 101257.836  101258.016
    101261.34  ]
   ...
   [  5270.194    5270.175    5270.171  ...   5253.6226   5253.9478
      5254.26  ]]]], 'pascal')>
Coordinates:
    XLONG    (y, x) float32 -122.6 -122.5 -122.5 -122.5 ... -57.17 -57.12 -57.07
  * y        (y) float64 -2.028e+06 -2.024e+06 -2.02e+06 ... 2.024e+06 2.028e+06
  * x        (x) float64 -2.732e+06 -2.728e+06 ... 2.728e+06 2.732e+06
    XLAT     (y, x) float32 17.65 17.66 17.67 17.68 ... 51.74 51.73 51.71 51.69
  * Time     (Time) datetime64[ns] 2012-06-29T12:00:00
    XTIME    (Time) timedelta64[ns] 11960 days 12:00:00
  * z        (z) float32 0.9969 0.9901 0.9821 0.973 ... 0.0161 0.009015 0.0028
Attributes:
    standard_name:  air_pressure
    grid_mapping:   wrf_projection

@dopplershift
Copy link
Member

dopplershift commented Dec 13, 2023

Ok, the subtle problem here is that you manually are calling .quantify(). MetPy's calculation will do that as needed, so you should never need to do that for MetPy calculations--though it can be needed at times for your own manual calculations, or for passing to other routines. In this case, doing it manually causes a subtle problem, so removing it should fix the problem.

The long answer is that because pressure is accessed as an xarray coordinate, the pint Quantity is stripped down to a numpy array. (See #3260 for more info.) Therefore, the only way for us to pass around unit-information on the coordinates is with the unit attribute metadata, which quantify() replaces by creating a Quantity instance. Messy I know, but unfortunately is the world we live in at the moment. 😞

So I would write your code above as:

import metpy
import metpy.calc as mpcalc
from metpy.units import units
import xarray as xr
import xwrf
import numpy as np

#------------------------------------------------------------------------------
#--- Open the NetCDF file -----------------------------------------------------
filename = "/glade/work/kshourd/29_June_2012/wrfout_d01_2012-06-29_12:00:00"
#--- xWRF postprocess & grid destaggering
ds = xr.open_dataset(filename).xwrf.postprocess()
ds = ds.xwrf.destagger()

lat = ds["XLAT"].drop_vars("CLAT")
lon = ds["XLONG"].drop_vars("CLAT")

#--- Extract variables
z = ds['geopotential_height'].drop_vars("CLAT")
print(z)

ua = ds['wind_east'].drop_vars("CLAT")
va = ds['wind_north'].drop_vars("CLAT")

pressure = ds["air_pressure"].drop_vars("CLAT")

theta = ds["air_potential_temperature"].drop_vars("CLAT")
temp  = mpcalc.temperature_from_potential_temperature(pressure, theta)

# Compute the geostrophic wind
print("Geostrophic wind calculation...")
ugeo, vgeo = mpcalc.geostrophic_wind(z)

#--- Isentropic interpolation w/ MetPy
thlv     = np.arange(298,350,1)
isen_lev = thlv*units.kelvin

isentropic = mpcalc.isentropic_interpolation_as_dataset(isen_lev.squeeze(), temp.squeeze(), ugeo.squeeze(), vgeo.squeeze(), ua.squeeze(), va.squeeze())

@dopplershift dopplershift added Area: Units Pertains to unit information Area: Xarray Pertains to xarray integration labels Dec 13, 2023
@spacekace
Copy link
Author

Thanks for the tip! Unfortunately, I am still receiving the same error:

/glade/work/kshourd/metpy_testing.py:49: UserWarning: More than one time coordinate present for variable .
  isentropic = mpcalc.isentropic_interpolation_as_dataset(isen_lev.squeeze(), temp.squeeze(), ugeo.squeeze(), vgeo.squeeze(), ua.squeeze(), va.squeeze())
Traceback (most recent call last):
  File "/glade/work/kshourd/metpy_testing.py", line 49, in <module>
    isentropic = mpcalc.isentropic_interpolation_as_dataset(isen_lev.squeeze(), temp.squeeze(), ugeo.squeeze(), vgeo.squeeze(), ua.squeeze(), va.squeeze())
  File "/glade/work/kshourd/conda-envs/wrfplot/lib/python3.9/site-packages/metpy/calc/thermo.py", line 2765, in isentropic_interpolation_as_dataset
    ret = isentropic_interpolation(
  File "/glade/work/kshourd/conda-envs/wrfplot/lib/python3.9/site-packages/metpy/xarray.py", line 1544, in wrapper
    return func(*bound_args.args, **bound_args.kwargs)
  File "/glade/work/kshourd/conda-envs/wrfplot/lib/python3.9/site-packages/metpy/xarray.py", line 1328, in wrapper
    result = func(*bound_args.args, **bound_args.kwargs)
  File "/glade/work/kshourd/conda-envs/wrfplot/lib/python3.9/site-packages/metpy/units.py", line 324, in wrapper
    _check_units_inner_helper(func, sig, defaults, dims, *args, **kwargs)
  File "/glade/work/kshourd/conda-envs/wrfplot/lib/python3.9/site-packages/metpy/units.py", line 311, in _check_units_inner_helper
    raise ValueError(msg)
ValueError: This function changed in 1.0--double check that the function is being called properly.
`isentropic_interpolation` given arguments with incorrect units: `pressure` requires "[pressure]" but given "dimensionless"

@dopplershift
Copy link
Member

Hrmmm. What does this show?

print(temp.metpy.vertical)

Also, any chance you could share the data file? That would make it easier to figure out what exactly is going on.

@spacekace
Copy link
Author

spacekace commented Dec 14, 2023

Output from

print(temp.metpy.vertical)

is as follows:

<xarray.DataArray 'z' (z: 50)>
array([0.996905, 0.99012 , 0.982145, 0.973005, 0.96273 , 0.951345, 0.9383  ,
       0.92306 , 0.905715, 0.88637 , 0.86513 , 0.84158 , 0.815355, 0.786675,
       0.75578 , 0.72245 , 0.686535, 0.648435, 0.60856 , 0.5689  , 0.53119 ,
       0.495345, 0.461295, 0.42897 , 0.398295, 0.369195, 0.34161 , 0.315475,
       0.290725, 0.267305, 0.245155, 0.22422 , 0.204445, 0.18578 , 0.168175,
       0.151575, 0.13594 , 0.12123 , 0.1074  , 0.094405, 0.082205, 0.070765,
       0.06005 , 0.05002 , 0.04064 , 0.03188 , 0.02371 , 0.0161  , 0.009015,
       0.0028  ], dtype=float32)
Coordinates:
  * z        (z) float32 0.9969 0.9901 0.9821 0.973 ... 0.0161 0.009015 0.0028
Attributes:
    FieldType:      104
    MemoryOrder:    Z
    description:    eta values on half (mass) levels
    units:
    stagger:
    cell_methods:   Time: mean
    axis:           Z
    standard_name:  atmosphere_hybrid_sigma_pressure_coordinate
    _metpy_axis:    vertical

As far as the file, it is quite large (2.54 GB) and I do not currently have it downloaded. I am accessing the file using NCAR GLADE. The files are publicly available here: https://rda.ucar.edu/datasets/ds559.0/dataaccess/ and I am using domain 01 from 29 June 2012 at 12UTC. If you would still like me to provide a file, please let me know and I can do so.

@jthielen
Copy link
Collaborator

I was helping with this over on xarray-contrib/xwrf#152, so I had the file downloaded. Here (should be) a quick dropbox link: https://www.dropbox.com/scl/fi/bs8bh22pj7nal0diwjlfg/wrfout_d01_2012-06-29_12-00-00?rlkey=bsjb8osu73heutbhdia7kj5gp&dl=0

@dopplershift dopplershift added this to the December 2023 milestone Dec 14, 2023
@dopplershift
Copy link
Member

dopplershift commented Dec 14, 2023

@spacekace So the root problem here is the fact that the WRF output you're working with is on sigma vertical coordinates. isentropic_interpolation_as_dataset is assuming that the vertical coordinate is pressure; it incorrectly ends up passing the sigma coordinates as pressure, which triggers the error about pressure being dimensionless. Really, we should be checking for this and producing an error because there's nothing we can really do in this case in isentropic_interpolation_as_dataset.

Now, in theory you should be able to use isentropic_interpolation directly, passing in your 3-D pressure field, and get back a handful of DataArrays, like:

isen_press, isen_ug, isen_vg, isen_ua, isen_va = mpcalc.isentropic_interpolation(isen_lev.squeeze(), pressure.squeeze(), temp.squeeze(),
                                                                                 ugeo.squeeze(), vgeo.squeeze(), ua.squeeze(), va.squeeze())

Unfortunately, there's currently a bug due to some assumptions we make that prevent that from working. I'm hoping we can quickly fix that for inclusion in the 1.6 release that should be out "any day now".

EDIT: A work-around in the meanwhile is to first interpolate to a fixed set of isobaric levels, though I'm not sure how good those results would be in comparison with interpolation on the native sigma levels.

@dopplershift
Copy link
Member

dopplershift commented Dec 14, 2023

I have a PR ready to go to fix all this up, including adding a pressure argument to isentropic_interpolation_as_dataset to allow this to "just work" with the data on sigma coordinates. Just waiting for #3323 to merge since they will conflict.

@spacekace
Copy link
Author

Thanks, all! Really appreciate the help here. I have also been trying to accomplish this with wrf-python to no avail. Looking forward to the 1.6 release!

dopplershift added a commit to dopplershift/MetPy that referenced this issue Dec 14, 2023
Internally we were already using 3D pressure, so just avoid needless
broadcasting outside the 1D pressure case. This paves the way for direct
interpolation from e.g. sigma coords. See Unidata#3315.
dopplershift added a commit to dopplershift/MetPy that referenced this issue Dec 14, 2023
isentropic_interpolation_as_dataset was blindly assuming that the
vertical coordinate was pressure, resulting in a confusing unit error
when calling isentropic_interpolation. Check the dimensionality and
raise a clearer ValueError when this is not the case. Also allow users
to manually pass (as a keyword) an appropriate pressure array to use instead.
dopplershift added a commit to dopplershift/MetPy that referenced this issue Dec 14, 2023
isentropic_interpolation_as_dataset was blindly assuming that the
vertical coordinate was pressure, resulting in a confusing unit error
when calling isentropic_interpolation. Check the dimensionality and
raise a clearer ValueError when this is not the case. Also allow users
to manually pass (as a keyword) an appropriate pressure array to use instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Units Pertains to unit information Area: Xarray Pertains to xarray integration Type: Bug Something is not working like it should
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants