You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Sep 23, 2024. It is now read-only.
Describe the bug
Changing the schema for a stream in the catalog file has no affect, since it's alway overwritten with the discovered stream in refresh_streams_schema.
To Reproduce
Steps to reproduce the behavior:
Create a test postgres database with a couple of tables:
CREATE TABLE a (a integer PRIMARY KEY, data jsonb);
INSERT INTO a VALUES (1, '{}');
Create config files for the tap and target, for example:
target_postgres=# SELECT pg_typeof("data") FROM a;
pg_typeof
-----------
jsonb
(1 row)
Expected behavior
If a catalog file is provided, its schema should take precedence over the discovered schema for that stream. The data type in the target should be character varying.
Screenshots
N/A
Your environment
Version of tap: [e.g. 1.8.1]
Version of python [e.g. 3.9.7]
Additional context
I discovered this while using meltano.
The text was updated successfully, but these errors were encountered:
Fixestransferwise#128
- Moves the bulk of the code out of the context manager
(open_connection) since it isn't needed.
- Creates private functions for merging existing metadata and schema to
dicts with those in `new_discovery`.
- Aliases `metadata` (from singer) to `metadata_util`. I kept using
`metadata` as a local var when developing by accident and breaking
things.
This issue prevents us from upgrading this tap as it broke behaviour in an older version - we have a requirement to mutate the catalog with out current ingestion workflow.
Describe the bug
Changing the schema for a stream in the catalog file has no affect, since it's alway overwritten with the discovered stream in
refresh_streams_schema
.To Reproduce
Steps to reproduce the behavior:
Create a test postgres database with a couple of tables:
Create config files for the tap and target, for example:
tap_config.json
target_config.json
Install the tap and create
catalog.json
Install the target
Run the pipeline
Check the table created in the target
Expected behavior
If a catalog file is provided, its schema should take precedence over the discovered schema for that stream. The data type in the target should be
character varying
.Screenshots
N/A
Your environment
Additional context
I discovered this while using
meltano
.The text was updated successfully, but these errors were encountered: