Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qgepdss/export.py - rohrprofil_geometry etc. add if filtered #144

Merged
merged 115 commits into from
Jul 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
cff0534
qgepdss/export.py - rohrprofil_geometry add if filtered
sjib May 14, 2024
534c8b5
Adapt if filtered QGEP.pipe_profile, QGEP.reach
sjib May 15, 2024
fe217e9
Add filter for Subclasses of Anschlussobjekt
sjib May 16, 2024
c730463
adjust filter criteria reach_point, take out reach_point_to statement
sjib May 16, 2024
b0b4998
filter for hydr_geometry and hydr_geomrelation
sjib May 16, 2024
d828c0e
filter for param_ca_general and param_ca_mouse1
sjib May 16, 2024
dc7f826
filters for catchment areas - 4 different fk
sjib May 16, 2024
e4f8984
Add filter control_center
sjib Jun 3, 2024
48c8bc5
Add filter mechanical_pretreatment
sjib Jun 3, 2024
d4d310b
Add filter retention_body
sjib Jun 3, 2024
7821289
add filter hazard_source
sjib Jun 3, 2024
3ced51b
add filter accident
sjib Jun 3, 2024
0caafce
add filter substance
sjib Jun 3, 2024
bc7b075
add filter measuring_point
sjib Jun 3, 2024
14e5c6e
add filter measuring_device and measurement_series
sjib Jun 3, 2024
015122c
add filter measurement_result
sjib Jun 3, 2024
25a7a5f
add filter throttle_shut_off_unit
sjib Jun 3, 2024
5e3fc29
add filter prank_weir
sjib Jun 3, 2024
1e73b72
add filter pump
sjib Jun 3, 2024
26a9d75
add filter leapingweir
sjib Jun 3, 2024
dc4698d
add filter hydraulic_char_data
sjib Jun 3, 2024
b1b7594
comment on filter hydraulic_char_data and backflow_prevention
sjib Jun 3, 2024
2fe230b
comment on filter tank_cleaning
sjib Jun 3, 2024
c3ae924
moved comment to tank_emptying
sjib Jun 3, 2024
08430e8
add filter re_maintenance_event_wastewater_structure
sjib Jun 3, 2024
e17a468
add filter maintenance_event
sjib Jun 3, 2024
a0c45a1
adjust intend
sjib Jun 3, 2024
f27356a
correct syntax
sjib Jun 3, 2024
f81d641
correct syntax
sjib Jun 3, 2024
7aca4b8
correct syntax
sjib Jun 3, 2024
4dbf1cf
correct intend
sjib Jun 3, 2024
6952f69
correct join if also _or
sjib Jun 4, 2024
de245be
debugging if filtered statements
sjib Jun 4, 2024
821926e
added query.union to measuring_point
sjib Jun 5, 2024
5f7c77c
cleanup qgep export
sjib Jun 12, 2024
c8b7ca9
cleanup qgepsia405
sjib Jun 12, 2024
4650fa6
cleanup gepdss export
sjib Jun 12, 2024
14401aa
adding check_fk_in_subsetid
sjib Jun 12, 2024
0fc75cb
correct check_fk_in_subsetid qgepdss
sjib Jun 14, 2024
1c13752
change description of check_fk_in_subsetid
sjib Jun 14, 2024
63fec54
add check_fk_in_subsetid to qgep and qgepsia405
sjib Jun 14, 2024
4c8f993
add first check for None,
sjib Jun 14, 2024
d8ee3a6
catchment_area - changed from get_tid to check_fk_in_subsetid
sjib Jun 17, 2024
15f6434
add ALL DEFERRED
sjib Jun 17, 2024
a1f3bf8
same to qgepdss and qgepsia405
sjib Jun 17, 2024
3e37f9a
debug add logger info
sjib Jun 18, 2024
e39f30f
Merge branch 'master' into 2024-05-14-add-missing-filters-interlis-ex…
sjib Jun 18, 2024
1c54073
add extra logger.info to check the passed on selection
sjib Jun 18, 2024
f186ebd
turn on flag_test to allow subclass test for wastewater_structures
sjib Jun 19, 2024
78043f7
from sqlalchemy.sql import text
sjib Jun 19, 2024
7efbcde
backporting symbology update queries
sjib Jun 21, 2024
4f4cef3
Merge pull request #151 from QGEP/2024-06-21-add-symbology-queries-qgep
ponceta Jun 25, 2024
9416466
Merge pull request #148 from QGEP/2024-06-17-abwasser_session-add-ALL…
ponceta Jun 25, 2024
fa57414
corrections qgepdss export.py
sjib Jun 26, 2024
9d58cf1
adapted filter hydraulic_char_data
sjib Jun 27, 2024
b068658
comment query3 - will not be needed in release 2020 anymore
sjib Jun 28, 2024
8c55b19
qgepdss/export.py - rohrprofil_geometry add if filtered
sjib May 14, 2024
9f102a8
Adapt if filtered QGEP.pipe_profile, QGEP.reach
sjib May 15, 2024
4260659
Add filter for Subclasses of Anschlussobjekt
sjib May 16, 2024
8797998
adjust filter criteria reach_point, take out reach_point_to statement
sjib May 16, 2024
4ab6761
filter for hydr_geometry and hydr_geomrelation
sjib May 16, 2024
4baefea
filter for param_ca_general and param_ca_mouse1
sjib May 16, 2024
273214a
filters for catchment areas - 4 different fk
sjib May 16, 2024
1c417b1
Add filter control_center
sjib Jun 3, 2024
94840a8
Add filter mechanical_pretreatment
sjib Jun 3, 2024
0fbb99c
Add filter retention_body
sjib Jun 3, 2024
1d76fb9
add filter hazard_source
sjib Jun 3, 2024
2e5d777
add filter accident
sjib Jun 3, 2024
0c1df8d
add filter substance
sjib Jun 3, 2024
4ed00f8
add filter measuring_point
sjib Jun 3, 2024
a394545
add filter measuring_device and measurement_series
sjib Jun 3, 2024
462e669
add filter measurement_result
sjib Jun 3, 2024
8e8ea80
add filter throttle_shut_off_unit
sjib Jun 3, 2024
297476e
add filter prank_weir
sjib Jun 3, 2024
0acb137
add filter pump
sjib Jun 3, 2024
25d4c56
add filter leapingweir
sjib Jun 3, 2024
9635bbb
add filter hydraulic_char_data
sjib Jun 3, 2024
8112726
comment on filter hydraulic_char_data and backflow_prevention
sjib Jun 3, 2024
fc06f40
comment on filter tank_cleaning
sjib Jun 3, 2024
3de6c3f
moved comment to tank_emptying
sjib Jun 3, 2024
7c4a61b
add filter re_maintenance_event_wastewater_structure
sjib Jun 3, 2024
bad737c
add filter maintenance_event
sjib Jun 3, 2024
010604c
adjust intend
sjib Jun 3, 2024
11502c6
correct syntax
sjib Jun 3, 2024
9175239
correct syntax
sjib Jun 3, 2024
e1e1ea6
correct syntax
sjib Jun 3, 2024
d3472ea
correct intend
sjib Jun 3, 2024
5d2d8fc
correct join if also _or
sjib Jun 4, 2024
8206d92
debugging if filtered statements
sjib Jun 4, 2024
65790d8
added query.union to measuring_point
sjib Jun 5, 2024
904943e
cleanup qgep export
sjib Jun 12, 2024
60c6588
cleanup qgepsia405
sjib Jun 12, 2024
c20ffd7
cleanup gepdss export
sjib Jun 12, 2024
a22ef11
adding check_fk_in_subsetid
sjib Jun 12, 2024
aa250fa
correct check_fk_in_subsetid qgepdss
sjib Jun 14, 2024
3b7cbf7
change description of check_fk_in_subsetid
sjib Jun 14, 2024
9e85940
add check_fk_in_subsetid to qgep and qgepsia405
sjib Jun 14, 2024
88db2be
add first check for None,
sjib Jun 14, 2024
9b66fd7
catchment_area - changed from get_tid to check_fk_in_subsetid
sjib Jun 17, 2024
3d5d28f
debug add logger info
sjib Jun 18, 2024
c271b12
add extra logger.info to check the passed on selection
sjib Jun 18, 2024
d2676c3
turn on flag_test to allow subclass test for wastewater_structures
sjib Jun 19, 2024
2527a39
corrections qgepdss export.py
sjib Jun 26, 2024
7c6d9bb
adapted filter hydraulic_char_data
sjib Jun 27, 2024
789fd95
comment query3 - will not be needed in release 2020 anymore
sjib Jun 28, 2024
1bedd55
clean up
sjib Jul 17, 2024
b81165c
Merge branch '2024-05-14-add-missing-filters-interlis-export' of http…
sjib Jul 17, 2024
cd3e1f6
correct intends
sjib Jul 17, 2024
17e893f
correct , and )
sjib Jul 17, 2024
7580bdc
add filter hq_relation
sjib Jul 17, 2024
d5342ce
filter overlow_char
sjib Jul 17, 2024
831e916
checked filter hydraulic_char_data
sjib Jul 17, 2024
c9a9410
filter mutation
sjib Jul 17, 2024
a33819a
always export all aquifier
sjib Jul 17, 2024
5d99389
adding comment if no filter is set
sjib Jul 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion qgepqwat2ili/gui/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -440,7 +440,8 @@ def action_do_export():
return

# 2. check wastewater_structure for all data models
flag_test = False
# flag_test - set to true if this test should be applied also
flag_test = True
if flag_test:
check_wastewater_structure = False
check_wastewater_structure = check_wastewater_structure_subclass_data()
Expand Down Expand Up @@ -599,6 +600,10 @@ def action_do_export():
#18.3.2023
# 22.3.2023 added try - seems not to work
# try:

# add logger info to check what selection is used
logger.info(f"Start Exporting with selection {str(export_dialog.selected_ids)}")

if emodel == "VSA_KEK_2019_LV95":
logger.info("Start Exporting VSA_KEK_2019_LV95 - qgep_export")
#qgep_export(selection=export_dialog.selected_ids, labels_file=labels_file_path)
Expand Down
11 changes: 11 additions & 0 deletions qgepqwat2ili/postimport.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,17 @@ def qgep_postimport():
post_session.execute("SELECT qgep_od.wastewater_structure_update_fk_main_wastewater_node('', True);"
)

# add symbology update queries - backporting from tww https://github.com/teksi/wastewater/pull/263
logger.info("update_wastewater_node_symbology for all datasets - please be patient")
post_session.execute("SELECT qgep_od.update_wastewater_node_symbology(NULL, True);")
logger.info("update_wastewater_node_symbology for all datasets - please be patient")
logger.info("update_wastewater_structure_label for all datasets - please be patient")
post_session.execute("SELECT qgep_od.update_wastewater_structure_label(NULL, True);")
logger.info("update_wastewater_node_symbology for all datasets - please be patient")
logger.info("update_wn_symbology_by_overflow for all datasets - please be patient")
post_session.execute("SELECT qgep_od.update_wn_symbology_by_overflow(NULL, True);")


logger.info("Refresh materialized views")
post_session.execute("SELECT qgep_network.refresh_network_simple();")

Expand Down
38 changes: 34 additions & 4 deletions qgepqwat2ili/qgep/export.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,14 @@
from geoalchemy2.functions import ST_Force2D, ST_GeomFromGeoJSON
from sqlalchemy import or_
from sqlalchemy.orm import Session
from sqlalchemy.sql import text

from .. import utils
from ..utils.various import logger
from .model_abwasser import get_abwasser_model
from .model_qgep import get_qgep_model

#def qgep_export(selection=None, labels_file=None):

def qgep_export(selection=None, labels_file=None, orientation=None):
"""
Export data from the QGEP model into the ili2pg model.
Expand All @@ -28,6 +29,9 @@ def qgep_export(selection=None, labels_file=None, orientation=None):
abwasser_session = Session(utils.sqlalchemy.create_engine(), autocommit=False, autoflush=False)
tid_maker = utils.ili2db.TidMaker(id_attribute="obj_id")

# backport from tww https://github.com/teksi/wastewater/blob/3acfba249866d299f8a22e249d9f1e475fe7b88d/plugin/teksi_wastewater/interlis/interlis_model_mapping/interlis_exporter_to_intermediate_schema.py#L83
abwasser_session.execute(text("SET CONSTRAINTS ALL DEFERRED;"))

# Filtering
filtered = selection is not None
subset_ids = selection if selection is not None else []
Expand Down Expand Up @@ -96,7 +100,7 @@ def modulo_angle(val):
if val is None:
return None

# 5.4.2023 add orientation
# add orientation
val = val + float(labelorientation)

val = val % 360.0
Expand All @@ -108,6 +112,27 @@ def modulo_angle(val):

return val

def check_fk_in_subsetid (subset, relation):
"""
checks, whether foreignkey is in the subset_ids - if yes it return the tid of the foreignkey, if no it will return None
"""
# first check for None, as is get_tid
if relation is None:
return None

logger.info(f"check_fk_in_subsetid - Subset ID's '{subset}'")
# get the value of the fk_ attribute as str out of the relation to be able to check whether it is in the subset
fremdschluesselstr = getattr(relation, "obj_id")
logger.info(f"check_fk_in_subsetid - fremdschluesselstr '{fremdschluesselstr}'")

if fremdschluesselstr in subset:
logger.info(f"check_fk_in_subsetid - '{fremdschluesselstr}' is in subset ")
logger.info(f"check_fk_in_subsetid - tid = '{tid_maker.tid_for_row(relation)}' ")
return tid_maker.tid_for_row(relation)
else:
logger.info(f"check_fk_in_subsetid - '{fremdschluesselstr}' is not in subset - replaced with None instead!")
return None

def create_metaattributes(row):
metaattribute = ABWASSER.metaattribute(
# FIELDS TO MAP TO ABWASSER.metaattribute
Expand Down Expand Up @@ -487,7 +512,9 @@ def textpos_common(row, t_type, geojson_crs_def):
QGEP.reach_point.obj_id == QGEP.reach.fk_reach_point_from,
QGEP.reach_point.obj_id == QGEP.reach.fk_reach_point_to,
),
).filter(QGEP.wastewater_networkelement.obj_id.in_(subset_ids))
).filter(
QGEP.wastewater_networkelement.obj_id.in_(subset_ids)
)
for row in query:

# AVAILABLE FIELDS IN QGEP.reach_point
Expand All @@ -507,7 +534,10 @@ def textpos_common(row, t_type, geojson_crs_def):
# --- sia405_baseclass ---
**base_common(row, "haltungspunkt"),
# --- haltungspunkt ---
abwassernetzelementref=get_tid(row.fk_wastewater_networkelement__REL),

# changed call from get_tid to check_fk_in_subsetid so it does not wirte foreignkeys on elements that do not exist
#abwassernetzelementref=get_tid(row.fk_wastewater_networkelement__REL),
abwassernetzelementref=check_fk_in_subsetid(subset_ids, row.fk_wastewater_networkelement__REL),
auslaufform=get_vl(row.outlet_shape__REL),
bemerkung=truncate(emptystr_to_null(row.remark), 80),
bezeichnung=null_to_emptystr(row.identifier),
Expand Down
Loading
Loading