Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OMOP test case: generalize-tsv #98

Closed
wants to merge 16 commits into from
Closed

OMOP test case: generalize-tsv #98

wants to merge 16 commits into from

Conversation

matentzn
Copy link
Contributor

@matentzn matentzn commented Oct 2, 2022 •
edited
Loading

@cmungall
Copy link
Member

can we close this?

@matentzn
Copy link
Contributor Author

I forgot to make a fork, sorry. If you see this PR on the 1st January 2023 just close it. I still hope to do some work here and GitHub is my extended memory. :P

@matentzn matentzn mentioned this pull request Nov 20, 2022
Open
@matentzn matentzn changed the title DO NOT MERGE: just for some personal tracking OMOP test case: generalize-tsv Nov 20, 2022
matentzn
matentzn commented Nov 20, 2022
View reviewed changes
pyproject.toml
@@ -11,7 +11,9 @@ packages = [
]

[tool.poetry.dependencies]
python = "^3.9"
python = ">=3.9,<4.0.0"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The <4.0.0 seems to be necessary for poetry for some reason.

@matentzn
Copy link
Contributor Author

matentzn commented Nov 20, 2022 •
edited
Loading

Environment:

poetry run pip list | grep linkml
The currently activated Python version 3.8.3 is not supported by the project (>=3.9,<4.0.0).
Trying to find and use a compatible version. 
linkml                        1.3.14
linkml-dataops                0.1.0
linkml-runtime                1.3.7

Given this slot:

concept_list:
    description: Container slot for CONCEPT.
    from_schema: https://w3id.org/omop_vocabulary
    multivalued: true
    range: CONCEPT

and this class:

CONCEPT_CONTAINER:
    from_schema: https://w3id.org/omop_vocabulary
    slots:
    - concept_list

I get this error:

Using python3.9 (3.9.13)
Loading from tests/resources/tsvs/CONCEPT.csv
Traceback (most recent call last):
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/bin/linkml-convert", line 8, in <module>
    sys.exit(cli())
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/lib/python3.9/site-packages/linkml/utils/converter.py", line 140, in cli
    obj = loader.load(source=input, target_class=py_target_class, **inargs)
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/lib/python3.9/site-packages/linkml_runtime/loaders/csv_loader.py", line 44, in load
    return JSONLoader().loads(json.dumps({index_slot: objs}), target_class=target_class)
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/lib/python3.9/site-packages/linkml_runtime/loaders/loader_root.py", line 126, in loads
    return self.load(source, target_class, metadata=metadata)
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/lib/python3.9/site-packages/linkml_runtime/loaders/loader_root.py", line 85, in load
    results = self.load_any(*args, **kwargs)
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/lib/python3.9/site-packages/linkml_runtime/loaders/json_loader.py", line 28, in load_any
    return self.load_source(source, loader, target_class,
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/lib/python3.9/site-packages/linkml_runtime/loaders/loader_root.py", line 66, in load_source
    return target_class(**data_as_dict)
  File "<string>", line 13, in __init__
  File "test", line 292, in __post_init__
  File "/Users/matentzn/Library/Caches/pypoetry/virtualenvs/schema-automator-iL9MyQRf-py3.9/lib/python3.9/site-packages/linkml_runtime/utils/yamlutils.py", line 48, in __post_init__
    raise ValueError('\n'.join(messages))
ValueError:  Unknown argument: concept_list = [{'concept_id': 1146945, 'concept_name':
make[1]: *** [tests/outputs/omop_data.ttl] Error 1
make: *** [test-omop] Error 2

When running:

poetry run linkml-convert -t rdf -s tests/outputs/omop.yaml \
		-m tests/outputs/omop/omop.py \
		-S concept_list \
		-C CONCEPT tests/resources/tsvs/CONCEPT.csv \
		--output tests/outputs/omop_data.ttl

@matentzn
Copy link
Contributor Author

matentzn commented Dec 7, 2022

From @sierra-moxon on slack (not to lose)

likely concept_list needs an inlined_as_list: true addition, and your container class CONCEPT_CONTAINER needs a tree_root: true addition. I added both of those in the generated yaml itself (to try and test this), and I still get the error (these are likely changes for schema_automator/linkml-runtime itself). But I think that's because linkml-convert needs to somehow know that your CONCEPT.csv is the content of concept_list - that is where I am in debugging.

I'm finding this difficult to debug as its being called from a Makefile and involves several software packages (linkml, linkml-runtime) upstream -- I started a test here: https://github.com/linkml/linkml/blob/omop/tests/test_utils/test_converter.py

@cmungall cmungall closed this Feb 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

TsvDataGeneralizer should have option to make a container class
2 participants
@matentzn @cmungall