Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lisätty logiikka raw-taulujen luomiseen #74

Merged
merged 1 commit into from
Dec 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions dbt/macros/helpers/create_raw_tables.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
{% macro create_raw_tables() -%}

{% set query %}
SELECT raw_taulu from {{ ref('raw_taulut') }}
{% endset %}

{% set tables = run_query(query) %}

{%set tables_list = tables.columns[0].values() %}

{% if execute %}
{% for table in tables_list %}
{% set create_statement = 'create table if not exists raw.' + table +' (
"data" json NULL,
dw_metadata_source_timestamp_at timestamptz NULL,
dw_metadata_dbt_copied_at timestamptz NULL,
dw_metadata_filename varchar NULL,
dw_metadata_file_row_number int4 NULL
);
CREATE INDEX if not exists ix_'+ table + ' ON raw.' + table+ ' USING btree (dw_metadata_dbt_copied_at);'
%}
{{ print ( create_statement )}}
{% do run_query(create_statement) %}
{% endfor %}

{% endif %}
{%- endmacro %}
11 changes: 11 additions & 0 deletions dbt/seeds/_seeds.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
version: 2

seeds:
- name: raw_taulut
config:
schema: raw
description: |
Tässä tiedostossa on lista kaikista raw-taulusta joita järjestelmässä on. Lisätääksesi uuden raw-taulun
1) Lisää taulun nimi seeds/raw_taulut.csv-tiedostoon omalle riville
2) aja # dbt seed --target=prod
3) aja # dbt run-operation create_raw_tables --target=prod
41 changes: 41 additions & 0 deletions dbt/seeds/raw_taulut.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
raw_taulu
ataru_hakemus
ataru_lomake
hakukohderyhmapalvelu_ryhma
koodisto_koodi
koodisto_relaatio
kouta_ammattinimike
kouta_asiasana
kouta_haku
kouta_hakukohde
kouta_koulutus
kouta_oppilaitoksetjaosat
kouta_pistehistoria
kouta_sorakuvaus
kouta_toteutus
kouta_valintaperuste
ohjausparametrit_parameter
onr_henkilo
onr_yhteystieto
organisaatio_organisaatio
organisaatio_organisaatiosuhde
organisaatio_osoite
organisaatio_ryhma
sure_arvosana
sure_ensikertalainen
sure_harkinnanvaraisuus
sure_opiskelija
sure_opiskeluoikeus
sure_proxysuoritus
sure_suoritus
valintalaskenta_valintakoe_osallistuminen
valintalaskenta_valintalaskennan_tulos
valintaperusteet_hakukohde
valintapiste_service_pistetieto
valintarekisteri_hyvaksyttyjulkaistuhakutoive
valintarekisteri_ilmoittautuminen
valintarekisteri_jonosija
valintarekisteri_lukuvuosimaksu
valintarekisteri_valinnantulos
valintarekisteri_valintatapajono
valintarekisteri_vastaanotto
Loading