Fix internally draining gaps and reduce storage footprint

[dtarb]

The following questions came up in the testing of RWD.

1. Is there a bounding domain to use?
2. Is there a way to reduce the storage footprint?

I made a shapefile giving the boundary of all the subwatersheds in gwgrid.shp with the intent of sending this as the bounding domain. However when I looked at this I discovered there were a number of holes in internally draining regions in flat areas of the country. This was particularly evident around Amarillo Texas as illustrated below. If a user was to click in one of these then they would get a “Point located outside the watershed” exception. NHDPlus flow directions do exist for these areas, so watersheds can be delineated, it is just that they were too small to be mapped using the threshold we used for partitioning into subwatersheds. To fix this I have added a check into RWD that if the area is outside of one of our preprocessed subwatersheds, it checks if it is in one of the NHDPlus grid processing regions and if so extracts flow directions from the corresponding flow direction grid and uses that to delineate the watershed. This requires some additional files to be placed on the server and for the new code posted to github to be deployed. Detailed instructions are below.

I also identified that if we do not want to compute watershed attributes beyond area then the files with TauDEM suffixes plen, tlen, ord and ad8 can be deleted from each subwatershed folder. This reduces the size of Subwatershed_ALL folder from 540 GB to 274 GB (including an additional 28 GB of region flow direction files). I deleted these on my system using the ReduceUnneeded.py script that I have provided, and changed the code to not report these attributes. I also changed the code to compute area from the resulting polygon rather than the ad8 file used formerly. This has the advantage that it is calculated for all watersheds, not only those where there is no crossing of NHDPlus grid processing units.

These fixes are in the reduce-storage branch and pull request.

The data needed is in Google Drive.

• Regions.zip. Unzip these files into Subwatershed_ALL folder retaining the folders Region1, Region2 etc.
• Main_Watershed.zip. Unzip these files into Main_Watershed folder. Only the regions.tif file is used by the code. domainfromregions.shp (and associated files) could be used for a bounding domain if you want.

Amarillo Area Figure. If a user had clicked in one of these yellow areas that are internally draining according to NHDPlus flow directions they would have received a point located outsite the watershed exception.

[kdeloach]

For future reference, I was able to confirm this issue on staging. Clicking on any of these locations near "Rick Husband Amarillo International Airport" East of Amarillo, TX yields a "Point located outside the watershed" error.

[kdeloach]

Thanks for the detailed write up and explanation. Very helpful during testing.