Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2023 03 interlis additional models new #108

Merged
merged 143 commits into from
Dec 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
143 commits
Select commit Hold shift + click to select a range
dd5348d
Initial ili2pg-4.5.0dist ili2pg-4.6.1.jar
sjib Mar 13, 2023
13f3fb8
qegpdss/export_py added
sjib Mar 13, 2023
58790f7
qgepdss/mapping.py added
sjib Mar 13, 2023
16c9d0e
qgepdss/model_abwasser.py added
sjib Mar 13, 2023
4695cea
qgepdss/model_qgep.py added
sjib Mar 13, 2023
8bbd933
qgepdss/import_.py added
sjib Mar 13, 2023
ced6371
ili2pg-4.6.1-bindist added - to do remove 4.5.0?
sjib Mar 13, 2023
8e3c95c
ili2db.py added def get_xtf_model(xtf_file):
sjib Mar 13, 2023
e2669ac
adaption to gui_export.py / added new gui import / export for testing
sjib Mar 13, 2023
e10164d
corrected gui_export.py and gui/_init_.py export added
sjib Mar 13, 2023
780544d
_init.py action_do_export updated
sjib Mar 13, 2023
b5cfc7c
updated gui/_init_.py and config.py
sjib Mar 16, 2023
9464716
Adapt gui/gui_export.py
sjib Mar 16, 2023
24917c8
Adapt gui_export.ui - testing with gui_export3.ui
sjib Mar 16, 2023
dfb32fb
add emodel dependency also for labels
sjib Mar 18, 2023
8d167f7
adjusted def gep_export(selection=None, labels_file=None):
sjib Mar 18, 2023
e92a6d6
added empty line
sjib Mar 18, 2023
81090a6
updated gui/_init_.py - duplicate call
sjib Mar 22, 2023
330749a
adapted config and _init_.py for dss and sia405_abwasser only
sjib Mar 22, 2023
b42257c
updated combo selection box / took out try as it seems not to work / …
sjib Mar 22, 2023
54a8f07
export_xtf_data dss - config.ABWASSER_DSS_SCHEMA,
sjib Mar 22, 2023
0bdd877
Added --export_dss parser argument
sjib Mar 24, 2023
ce7d206
return instead of continue if invalid xtf on export
sjib Mar 24, 2023
442929b
Adapted success messages
sjib Mar 24, 2023
9039d1f
back to continue in try
sjib Mar 24, 2023
2f10b85
adjusted show success locations
sjib Mar 24, 2023
0e6050c
Adjust error messages
sjib Mar 24, 2023
2542a09
imodel in import
sjib Mar 24, 2023
27328c7
adding model dependencies on import log_handler
sjib Mar 24, 2023
7b324c0
adjust indention
sjib Mar 24, 2023
5b2a12d
get_qgep_model taken out in gui_import.py as it is model specific
sjib Mar 24, 2023
3a473b2
ili2db improve logger message get_xtf_model
sjib Mar 24, 2023
fbc850f
adjusted NOTES, added _init_.py for qgepdss (empty file)
sjib Mar 24, 2023
9d4d647
Update gui_import.py
sjib Mar 24, 2023
2af24ea
trying to debug crash with editor - taken out model select - to redo …
sjib Mar 25, 2023
ed1c09b
correction config reference
sjib Mar 25, 2023
1f4f4c2
take out comment
sjib Mar 27, 2023
ceed5bb
cleaned up model dependency import
sjib Mar 27, 2023
b5b9a0a
else instead of except
sjib Mar 27, 2023
3a81020
discuss else instead of except
sjib Mar 27, 2023
825ceea
added qgep_import qgepsia405_import
sjib Mar 28, 2023
da74b98
adjusted path
sjib Mar 28, 2023
fde59a7
add folder qgepsia405 - copy from qgep
sjib Mar 28, 2023
71a697a
updated notes
sjib Mar 28, 2023
42d92ad
Add logger info if new organisation is created
sjib Mar 28, 2023
187e047
qgepsia405_export added - to be discussed
sjib Mar 28, 2023
86b5951
gui - _init_.py add logger info about used model
sjib Mar 29, 2023
62b1afb
adaptions sia405 - take out kek part
sjib Mar 29, 2023
a2941bc
add sia405 to init, eliminate kek classes in qgepsia405/export.py
sjib Mar 29, 2023
4064d99
sia405_abwasser import.py - take out kek
sjib Mar 29, 2023
76e2fcd
add additional DEFERRABLE
sjib Mar 29, 2023
15be013
qgepdss moved Importing ABWASSER.erhaltungsereignis_abwasserbauwerkas…
sjib Mar 29, 2023
0f8a91c
took out DEFERRABLE for re_maintenance_event as data should now be av…
sjib Mar 29, 2023
42ea35b
added additional logger.info
sjib Mar 29, 2023
690bc54
datenherr / datenlieferant export obj_id instead of name (that does n…
sjib Mar 31, 2023
4636621
added def check_organisation_subclass_data():
sjib Mar 31, 2023
cbd16e7
gui with orientation
sjib Apr 3, 2023
ab57e15
add orientation as variable to qgep_export
sjib Apr 3, 2023
be607f2
qgepdss export.py add labels export
sjib Apr 3, 2023
1d674d4
qgepdss add configuration for layer catchment_area
sjib Apr 3, 2023
2292641
add catchment_area
sjib Apr 3, 2023
2e80245
adjust extractlabels_interlis for catchment_area
sjib Apr 3, 2023
466c25b
Einrückung korrigiert
sjib Apr 3, 2023
d728be7
Einrückung 2
sjib Apr 3, 2023
7d6aab4
add INPUT_CATCHMENT_AREA_VIEW_LAYER
sjib Apr 3, 2023
49361ca
= added
sjib Apr 3, 2023
52d8838
add and check parser_qgep --labels_orientation
sjib Apr 4, 2023
d3370e1
add additional models qgepdss_export and qgepsia405_export
sjib Apr 4, 2023
3e50a90
info args.label_orientation on import
sjib Apr 4, 2023
3d5f44e
additional models qgep_import qgepsia405 and dss
sjib Apr 5, 2023
6c35848
corrections
sjib Apr 5, 2023
37a39f4
adding new tests for additional models case_f and g
sjib Apr 5, 2023
0d51dfd
changed handling of labels_orientation
sjib Apr 5, 2023
c361300
add labels_orientation in tests
sjib Apr 5, 2023
5752945
--labels_orientation instead of labels_orientation to make it optiona…
sjib Apr 5, 2023
ee79996
updated call in qgep_test for --labels_orientation
sjib Apr 5, 2023
50874fb
correct labels_orientation
sjib Apr 5, 2023
c543cf6
take out help in --labels_orientation
sjib Apr 5, 2023
ae7c549
Change check if args.labels_orientation exists
sjib Apr 5, 2023
02e83b7
change2 check if args.labels_selection exists
sjib Apr 5, 2023
40a9fc0
Change 3
sjib Apr 5, 2023
d54d2bc
test without choices
sjib Apr 5, 2023
f4129a9
take out labels_orientation if
sjib Apr 5, 2023
f51708b
take out args.label_orientation if 2
sjib Apr 5, 2023
ba8096c
add print xtf_file
sjib Apr 5, 2023
cdce9c4
correct imodel to impmodel
sjib Apr 5, 2023
120daa3
check for none in orientation
sjib Apr 5, 2023
aca49ad
change configuration of SCHEMA, ILIMODEL / correct call orientation
sjib Apr 5, 2023
b2d22b7
else missing :
sjib Apr 5, 2023
5ef9a32
add print strline in ili2db
sjib Apr 5, 2023
bf4c7d4
print ("qgep_import: " + SCHEMA + "/" + ILI_MODEL)
sjib Apr 5, 2023
1fa0977
also for other models
sjib Apr 5, 2023
f3b43f7
adjust header without empty lines and commented out lines
sjib Apr 5, 2023
b381ec0
wrong configuration qgepsia and qgepdss import
sjib Apr 5, 2023
a68d503
ili2db change if not line to line2
sjib Apr 5, 2023
ce1291a
wrong export configuration
sjib Apr 5, 2023
497b813
adapt checking labels_orientation
sjib Apr 5, 2023
01a3079
adapt print ili2db get_xtf_model
sjib Apr 5, 2023
3f83e20
test_case_h_export_dss_complete_orientation_90_qgep_to_xtf --export_d…
sjib Apr 5, 2023
36ceedd
Move comment <!-- Vorschlag
sjib Apr 5, 2023
893b968
rewrite get_xtf_model algorithm
sjib Apr 5, 2023
2976d18
add result.strip
sjib Apr 5, 2023
3c37c30
corrections
sjib Apr 5, 2023
32533b2
wrong indent
sjib Apr 5, 2023
34f43d2
model_list.append instead of extend / strmodel instead of mystr
sjib Apr 5, 2023
5dec886
trying to print strmodel / line2
sjib Apr 5, 2023
f6c5288
add print line/line2
sjib Apr 5, 2023
9ec9f76
add print line2 logger
sjib Apr 5, 2023
6da86f5
add more logger info for debugging, change comparison a
sjib Apr 6, 2023
60d0cfb
missing () with stript
sjib Apr 6, 2023
380b1b2
correct stip '"'
sjib Apr 6, 2023
0cffb1c
add get_xtf_model2 - read directly xml data instead of line by line
sjib Apr 11, 2023
6677b62
take out additional DEFERRABLE
sjib Apr 11, 2023
13ae833
Enable comboBox_orientation only if label export is checked
sjib Apr 11, 2023
2279afa
gui - unnecessary label_3
sjib Apr 11, 2023
4599bb4
adjusted WI scale
sjib Apr 11, 2023
8ab493b
Change test database creation to latest models of qgep 1.6.0 and qwat…
sjib Apr 12, 2023
9fc7bc7
added missing sql
sjib Apr 12, 2023
b0364bc
add missing subclass entries for class organisation / add fk_dataowne…
sjib Apr 12, 2023
af9e853
add additional integrity checks - subclass wws and identifier isNull
sjib Apr 12, 2023
19c06b1
set missing fk_owner / fk_operator
sjib Apr 12, 2023
3233ba2
def show_hint / update notsubclasses for identifier check
sjib Apr 12, 2023
a29d04b
identifier check just show hint, but continue
sjib Apr 12, 2023
4b633fe
temporary take out check wastewater_structures - put in again
sjib Apr 12, 2023
9e644e7
update demodata with fk_dataowner / fk_provider in wastewater_structure
sjib Apr 12, 2023
d54209a
temporarly take out test_case_g and h
sjib Apr 14, 2023
810e2ee
added float so cmd line version works also
sjib Apr 14, 2023
063b2fe
delete copies
sjib Apr 17, 2023
0c6806b
add class erhaltungsereignis to qgepdss export - define as regular cl…
sjib Apr 17, 2023
730dad7
adaptions qgepdss export.py add truncate model diffences maintenance …
sjib Apr 18, 2023
a9ca432
adaptions qgep(kek) export.py add truncate model diffences untersuchu…
sjib Apr 18, 2023
53f5b3c
adaption qgep(kek) export.py datei.objekt - truncate 16
sjib Apr 18, 2023
eccbfc8
added logger info for file.object if truncated
sjib Apr 18, 2023
0996024
added logger info for maintenance_event.* if truncated, truncate for …
sjib Apr 18, 2023
a999bf3
add missing commas, move; logger info code
sjib Apr 18, 2023
d93d2e2
missing comma
sjib Apr 18, 2023
20b3ad9
no additional logger.info needed - already in truncate function
sjib Apr 18, 2023
b948cdb
take out stacklevel argument as it seems not to be supported in logge…
sjib Apr 19, 2023
f3b059d
qgepdss import.py add missing class maintenance_event
sjib Apr 19, 2023
8a7e14e
preparation adding gui_import_config to set additional configuration
sjib Apr 19, 2023
6651aaf
Tests adding gui_importc instead of gui_import_config
sjib May 8, 2023
1fb3ce0
adapted so that gui_import and gui_importc are separated
sjib Jul 26, 2023
110c8ef
Change TextVali Bottom instead of Top
sjib Aug 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 124 additions & 11 deletions qgepqwat2ili/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,14 @@

from . import config, utils
from .qgep.export import qgep_export
# 4.4.2023
from .qgepsia405.export import qgep_export as qgepsia405_export
from .qgepdss.export import qgep_export as qgepdss_export
from .qgep.import_ import qgep_import
# 5.4.2023
from .qgepsia405.import_ import qgep_import as qgepsia405_import
from .qgepdss.import_ import qgep_import as qgepdss_import

from .qgep.mapping import get_qgep_mapping
from .qgep.model_abwasser import Base as BaseAbwasser
from .qgep.model_qgep import Base as BaseQgep
Expand Down Expand Up @@ -37,11 +44,27 @@ def main(args):
"--selection",
help="if provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)",
)

# TODO: this only makes sense for export
parser_qgep.add_argument(
"--labels_file",
help="if provided, includes the label positions in the export (the file should be the results of the provided `qgep:extractlabels_interlis` QGIS algorithm as geojson)",
)

#4.4.2023 / 5.4.2023 replaced with choices
#parser_qgep.add_argument(
# "--labels_orientation",
# help="parameter to adjust orientation of label text to fit other default values for north direction. If provided, turns the orientation by the given value [90, -90, 0] degree)",
#)
#parser_tpl.add_argument("model", choices=["qgep", "qwat"])
# back to argument to make it optional
#parser_qgep.add_argument("labels_orientation", choices=["90.0", "0.0","-90.0"])
parser_qgep.add_argument(
"--labels_orientation",
#choices=["90.0", "0.0","-90.0"],
#help="parameter to adjust orientation of label text to fit other default values for north direction. If provided, turns the orientation by the given value [90, -90, 0] degree)",
)

parser_qgep.add_argument(
"--recreate_schema", action="store_true", help="drops schema and reruns ili2pg importschema"
)
Expand All @@ -66,6 +89,12 @@ def main(args):
action="store_true",
help="export the model SIA405_ABWASSER_2015_LV95 (instead of default VSA_KEK_2019_LV95)",
)
# 24.3.2023
parser_qgep.add_argument(
"--export_dss",
action="store_true",
help="export the model DSS_2015_LV95 (instead of default VSA_KEK_2019_LV95)",
)

parser_qwat = subparsers.add_parser(
"qwat",
Expand Down Expand Up @@ -129,24 +158,64 @@ def main(args):
file_handler.setLevel(INFO)
file_handler.setFormatter(Formatter("%(levelname)-8s %(message)s"))
utils.various.logger.addHandler(file_handler)


# to do maybe move orientation_list to CONFIG ?
orientation_list = [0.0, 90.0, -90.0]
#if args.labels_orientation is None:
# args.labels_orientation = "0.0"
# if args.labels_orientation in orientation_list or args.labels_orientation is None :
# check if args.selection exists, if not set to "0.0"
# if args.labels_orientation:
# print ("args.labels_orientation {args.labels_orientation}")
# else:
# #set args.selection to "0.0"
# args.labels_orientation = "0.0"

#if args.labels_orientation in orientation_list :
if args.parser == "qgep":
config.PGSERVICE = args.pgservice
SCHEMA = config.ABWASSER_SCHEMA
ILI_MODEL = config.ABWASSER_ILI_MODEL
#SCHEMA = config.ABWASSER_SCHEMA
#ILI_MODEL = config.ABWASSER_ILI_MODEL
if args.export_sia405:
SCHEMA = config.ABWASSER_SIA405_SCHEMA
ILI_MODEL = config.ABWASSER_SIA405_ILI_MODEL
ILI_MODEL_NAME = config.ABWASSER_ILI_MODEL_NAME_SIA405
ILI_EXPORT_MODEL_NAME = config.ABWASSER_ILI_MODEL_NAME_SIA405
#ABWASSER_SIA405_SCHEMA = config.ABWASSER_SIA405_SCHEMA
#ABWASSER_SIA405_ILI_MODEL = config.ABWASSER_SIA405_ILI_MODEL
# 24.3.2023 added dss export
elif args.export_dss:
SCHEMA = config.ABWASSER_DSS_SCHEMA
ILI_MODEL = config.ABWASSER_SIA405_ILI_MODEL
ILI_MODEL_NAME = config.ABWASSER_DSS_ILI_MODEL
ILI_EXPORT_MODEL_NAME = None
#ABWASSER_DSS_SCHEMA = config.ABWASSER_DSS_SCHEMA
#ABWASSER_DSS_ILI_MODEL = config.ABWASSER_DSS_ILI_MODEL
else:
SCHEMA = config.ABWASSER_SCHEMA
ILI_MODEL = config.ABWASSER_ILI_MODEL
ILI_MODEL_NAME = config.ABWASSER_ILI_MODEL_NAME
ILI_EXPORT_MODEL_NAME = None

if args.direction == "export":
utils.ili2db.create_ili_schema(
SCHEMA, ILI_MODEL, make_log_path(log_path, "ilicreate"), recreate_schema=args.recreate_schema
)
qgep_export(selection=args.selection.split(",") if args.selection else None, labels_file=args.labels_file)

#add model dependency
if args.export_sia405:
# SIA405_ABWASSER_2015_LV95
qgepsia405_export(selection=args.selection.split(",") if args.selection else None, labels_file=args.labels_file, orientation=args.labels_orientation)
elif args.export_dss:
# DSS_2015_LV95 expor5t
qgepdss_export(selection=args.selection.split(",") if args.selection else None, labels_file=args.labels_file, orientation=args.labels_orientation)
else:
# VSA_KEK_2019_LV95 export

#qgep_export(selection=args.selection.split(",") if args.selection else None, labels_file=args.labels_file)
# 4.4.2023 with labels_orientation
#qgep_export(selection=args.selection.split(",") if args.selection else None, labels_file=args.labels_file, orientation=args.labels_orientation if args.labels_orientation else 0)
qgep_export(selection=args.selection.split(",") if args.selection else None, labels_file=args.labels_file, orientation=args.labels_orientation)

utils.ili2db.export_xtf_data(
SCHEMA, ILI_MODEL_NAME, ILI_EXPORT_MODEL_NAME, args.path, make_log_path(log_path, "iliexport")
)
Expand All @@ -162,17 +231,57 @@ def main(args):
if args.selection:
print("Selection is only supported on export")
exit(1)
if args.labels_orientation:
print("Labels_orientation is only supported on export")
exit(1)
if not args.skip_validation:
try:
utils.ili2db.validate_xtf_data(args.path, make_log_path(log_path, "ilivalidate"))
except utils.various.CmdException:
print("Ilivalidator doesn't recognize input as valid ! Run with --skip_validation to ignore")
exit(1)
utils.ili2db.create_ili_schema(
SCHEMA, ILI_MODEL, make_log_path(log_path, "ilicreate"), recreate_schema=args.recreate_schema
)
utils.ili2db.import_xtf_data(SCHEMA, args.path, make_log_path(log_path, "iliimport"))
qgep_import()

#add model dependency, as in __init_.py
impmodel = "nothing"
#impmodel = utils.ili2db.get_xtf_model(args.path)
impmodel = utils.ili2db.get_xtf_model2(args.path)

if impmodel == "VSA_KEK_2019_LV95":
SCHEMA = config.ABWASSER_SCHEMA
ILI_MODEL = config.ABWASSER_ILI_MODEL
utils.ili2db.create_ili_schema(
SCHEMA, ILI_MODEL, make_log_path(log_path, "ilicreate"), recreate_schema=args.recreate_schema
)
utils.ili2db.import_xtf_data(SCHEMA, args.path, make_log_path(log_path, "iliimport"))
print ("qgep_import: " + SCHEMA + "/" + ILI_MODEL)
qgep_import()
elif impmodel == "SIA405_ABWASSER_2015_LV95":
ABWASSER_SIA405_SCHEMA = config.ABWASSER_SIA405_SCHEMA
ABWASSER_SIA405_ILI_MODEL = config.ABWASSER_SIA405_ILI_MODEL
utils.ili2db.create_ili_schema(
ABWASSER_SIA405_SCHEMA, ABWASSER_SIA405_ILI_MODEL, make_log_path(log_path, "ilicreate"), recreate_schema=args.recreate_schema
)
utils.ili2db.import_xtf_data(
ABWASSER_SIA405_SCHEMA, args.path, make_log_path(log_path, "iliimport")
)
print ("qgepsia405_import: " + ABWASSER_SIA405_SCHEMA + "/" + ABWASSER_SIA405_ILI_MODEL)
qgepsia405_import()

elif impmodel == "DSS_2015_LV95":
ABWASSER_DSS_SCHEMA = config.ABWASSER_DSS_SCHEMA
ABWASSER_DSS_ILI_MODEL = config.ABWASSER_DSS_ILI_MODEL
utils.ili2db.create_ili_schema(
ABWASSER_DSS_SCHEMA, ABWASSER_DSS_ILI_MODEL, make_log_path(log_path, "ilicreate"), recreate_schema=args.recreate_schema
)
utils.ili2db.import_xtf_data(
ABWASSER_DSS_SCHEMA, args.path, make_log_path(log_path, "iliimport"))
print ("qgepdss_import: " + ABWASSER_DSS_SCHEMA + "/" + ABWASSER_DSS_ILI_MODEL)
qgepdss_import()

else:
print("MODEL " + impmodel + " schema creation failed: Not yet supported for INTERLIS import - no configuration available in config.py / _init_.py")



elif args.parser == "qwat":
config.PGSERVICE = args.pgservice
Expand Down Expand Up @@ -218,6 +327,7 @@ def main(args):
if args.model == "qgep":
if config.PGSERVICE is None:
config.PGSERVICE = config.QGEP_DEFAULT_PGSERVICE
#to do add model dependency
utils.ili2db.create_ili_schema(config.ABWASSER_SCHEMA, config.ABWASSER_ILI_MODEL, recreate_schema=True)
QGEPMAPPING = get_qgep_mapping()
utils.templates.generate_template("qgep", "abwasser", BaseQgep, BaseAbwasser, QGEPMAPPING)
Expand All @@ -239,5 +349,8 @@ def main(args):
else:
print("Unknown operation")
exit(1)

# else:
# # to do maybe read message from orientation_list
# print("No valid value for labels_orientation: [0.0, 90.0, -90.0]")
# exit(1)
print("Operation completed sucessfully !")
30 changes: 30 additions & 0 deletions qgepqwat2ili/bin/ili2pg-4.5.0-bindist/docs/CHANGELOG.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,36 @@ ideas/open issues/questions
LATEST
---------------------------

ili2db 4.6.1 (2021-11-10)
------------------------------
- Enum values are inserted twice when generating a script (#426)
- export only required MODELs in HEADERSECTION (#413)
- export by --topics with no baskets should create an empty transfer file (#432)
- --sqlColsAsText erzeugt leere Geometrien (#423)
- iox-ili-1.21.8

ili2db 4.6.0 (2021-09-10)
------------------------------
- replaced the legacy GUI with a message box
- fix "illegal value null for attrname" (#401)
- ili2fgdb: fix NullPointerException with --strokeArcs (#399)
- fix export of struct attr in VIEW (#397)
- added Iox2jsonUtility
- support --skipReferenceErrors on export
- fix missing model if --exportModels uses a translated model in another ili-file (#405)
- fix mapping of struct at model level if --exportModels used (#405)
- remove duplicate ReduceToBaseModel (duplicate in iox-ili) (#405)
- ili-cache: set ILI_CACHE_FILENAME=MD5 to use short filenames in $HOME/.ilicache (#420)
- new option --sqlExtRefCols to map EXTERNAL references as t_ili_tid FK (#283)
- new option --sqlColsAsText to map simple type attributes to text columns (#402)
- new option --exportBatchSize to set JDBC batchSize (#414)
- new option --importFetchSize to set JDBC fetchSize (#334)
- ili24: implement MULTICOORD (#408)
- support --option=booleanValue (#393)
- use metaattribute @ili2db.dispName with --createEnumTxtCol (#349)
- iox-ili-1.21.7
- ili2c-5.2.3

ili2db 4.5.0 (2021-03-12)
------------------------------
- ili2h2gis: add schema support (#383)
Expand Down
Loading
Loading