Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] r.fill.dir Segfault/stack overflow with larger datasets #3936

Open
byteit101 opened this issue Jun 28, 2024 · 2 comments
Open

[Bug] r.fill.dir Segfault/stack overflow with larger datasets #3936

byteit101 opened this issue Jun 28, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@byteit101
Copy link

Describe the bug

Large datasets seem to cause a segfault. Investigating the resulting coredump, the stack trace was 100k lines long:

(gdb) bt
#0  0x0000555c25ff41a6 in recurse_list (flag=flag@entry=117042, cells=cells@entry=0x7fa04fa00010, sz=sz@entry=4252788, start=<optimized out>) at ./raster/r.fill.dir/dopolys.c:26
#1  0x0000555c25ff41b5 in recurse_list (flag=flag@entry=117042, cells=cells@entry=0x7fa04fa00010, sz=sz@entry=4252788, start=<optimized out>) at ./raster/r.fill.dir/dopolys.c:26
#2  0x0000555c25ff41b5 in recurse_list (flag=flag@entry=117042, cells=cells@entry=0x7fa04fa00010, sz=sz@entry=4252788, start=<optimized out>) at ./raster/r.fill.dir/dopolys.c:26
...snip...
#104756 0x0000555c25ff41b5 in recurse_list (flag=flag@entry=117042, cells=cells@entry=0x7fa04fa00010, sz=sz@entry=4252788, start=<optimized out>) at ./raster/r.fill.dir/dopolys.c:26
#104757 0x0000555c25ff41b5 in recurse_list (flag=flag@entry=117042, cells=cells@entry=0x7fa04fa00010, sz=sz@entry=4252788, start=<optimized out>) at ./raster/r.fill.dir/dopolys.c:26
#104758 0x0000555c25ff41b5 in recurse_list (flag=flag@entry=117042, cells=cells@entry=0x7fa04fa00010, sz=sz@entry=4252788, start=<optimized out>) at ./raster/r.fill.dir/dopolys.c:26
#104759 0x0000555c25ff43d9 in dopolys (fd=fd@entry=6, fm=fm@entry=7, nl=nl@entry=11011, ns=ns@entry=14056) at ./raster/r.fill.dir/dopolys.c:81
#104760 0x0000555c25ff399c in main (argc=<optimized out>, argv=<optimized out>) at ./raster/r.fill.dir/main.c:210

Seems that https://trac.osgeo.org/grass/ticket/2742 wasn't actually fixed.

To reproduce

  1. Download a large chunk of DEM data. I used https://maps.vcgi.vermont.gov/opendata/clipandzip.html?InputLayerName=IMG_VCGI_LIDARDEM_SP_v3&InputFtype=raster with a chunk about 3/4 the size of the towns in the bottom right. Mine was about 600MB.
  2. Extract and import
  3. Run r.fill.dir with it
  4. See error

I had sufficient memory, I have about 16GB free, which wasn't used when this was running

Expected behavior

No Crash

Screenshots

Importing raster map <rast_667ee69b044a211>...
0..3..6..9..12..15..18..21..24..27..30..33..36..39..42..45..48..51..54..57..60..63..66..69..72..75..78..81..84..87..90..93..96..99..100
Reading input elevation raster map...
0..3..6..9..12..15..18..21..24..27..30..33..36..39..42..45..48..51..54..57..60..63..66..69..72..75..78..81..84..87..90..93..96..99..100
Filling sinks...
Determining flow directions for ambiguous cases...
Segmentation fault (core dumped)

System description

  • Operating System: Debian GNU/Linux 12
  • GRASS GIS version: 8.4.1 (Debian)
QGIS version 3.22.16-Białowieża - QGIS code branch - Release 3.22
Qt version 5.15.8
Python version 3.11.1
GDAL/OGR version 3.6.2
PROJ version 9.1.1
EPSG Registry database version v10.076 (2022-08-31)
GEOS version 3.11.1-CAPI-1.17.1
SQLite version 3.40.1
PostgreSQL client version 15.1 (Debian 15.1-1+b1)
SpatiaLite version 5.0.1
QWT version 6.1.4
QScintilla2 version 2.13.3
OS version Debian GNU/Linux 12 (bookworm)
   
Active Python plugins
quick_map_services 0.19.29
DEMto3D 3.6
MetaSearch 0.3.5
db_manager 0.1.20
grassprovider 2.12.99
processing 2.12.99
sagaprovider 2.12.99
@byteit101 byteit101 added the bug Something isn't working label Jun 28, 2024
@marisn
Copy link
Contributor

marisn commented Jun 30, 2024

As there is no link from the Trac to an actual commit, I'd guess it didn't got fixed. Please provide your computational region parameters (g.region -p) to determine how large is large.

@byteit101
Copy link
Author

Hmm, I am unsure where or how to use that command, but QGIS's layer properties says:

Total size: 528.95 MB
Width: 14056
Height: 11011
Data type: Float32 - Thirty two bit floating point
GDAL Driver Description: HFA
GDAL Driver Metadata: Erdas Imagine Images (.img)
Dimensions: X: 14056 Y: 11011 Bands: 1
Pixel Size: 0.6999999999484479707,-0.6999999999484491919
Name: NAD83 / Vermont
Units: meters
Method: Transverse Mercator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants