ENHANCEMENT: Allow -gt option to dramatically improve performance

[GoogleCodeExporter]

This is not an issue, but an enhancement request.

Using "import QGIS/GDAL layer" from qspatialite you can't specify the ogr/gdal 
option of -gt which should improve performance.

The backend, ogr2ogr uses -gt 200 which is very low for importing data to 
spatialite. Increasing that value increases performance dramatically.

The only way to use that option is to put it into the "Data Source" creation 
options in the layer "Save As..." window. But this way you can only do one 
layer at a time, whereas in qspatialite I can do them in batch, but because I 
can't specify a custom -gt (like -gt 1024) it is slow to import shapefiles from 
qspatialite.

This is a much needed improvement to this utility.

Original issue reported on code.google.com by [email protected] on 8 Jan 2013 at 10:45

[GoogleCodeExporter]

See "Performance Hints" section at the bottom of 
http://www.gdal.org/ogr/drv_sqlite.html

From the site:
"SQLite is a Transactional DBMS; while many INSERT statements are executed in 
close sequence, BEGIN TRANSACTION and COMMIT TRANSACTION statements have to be 
invoked appropriately in order to get optimal performance. The default OGR 
behavior is to COMMIT a transaction every 200 inserted rows. This value is 
surely too low for SQLite; and closing too much frequently the current 
transaction causes severe performance degradation. The -gt argument allows to 
explicitly set the number of rows for each transaction. Explicitly defining -gt 
1024 usually ensures a noticeable performance boost; defining an even bigger 
-gt 65536 ensures optimal performance while populating some table containing 
many hundredth thousand or million rows."

Original comment by [email protected] on 8 Jan 2013 at 10:47

[GoogleCodeExporter]

QSpatialite doesn't use OGR to load data.
QSpatiaLite load data in QGIS memory, and use QGIS API to read and insert data 
into the DB. THis allows QSpatiaLite to deal with many problems such as special 
characters, very long column names, and optimising MAPINFO compatibility.

But still, a quick import option, with OGR, should be a good option. I'll be 
working on this.

Original comment by [email protected] on 8 Feb 2013 at 7:49

• Changed state: Accepted

[GoogleCodeExporter]

Thanks for considering this, it will be a highly appreciated improvement =)

Original comment by [email protected] on 8 Feb 2013 at 8:21