Problems using pheval #347

seoanezonjic · 2024-07-15T15:24:23Z

Hi Pheval authors
Recently I've seen your paper in Researchgate and I'm very interested so I gave a try. Following the instructions here https://monarch-initiative.github.io/pheval/exomiser_pipeline (I use a previous exomiser version installed in my cluster) I've have the following problems:

Tutorial saids in "5. Set PhEval Config YAML File" that exomiser directory program key is "exomiser" but it must be "exomiser-USED VERSION". If not set, it breaks with a dict key error.
The dependency to j2cli to make j2 command works is not defined in poetry files so I have a j2 command not installed error. Maybe the snap version that you use contains jinja2 and j2 cli but I suggest to add "j2cli" to your depencies so all it's handled by python dependency manager
pheval-utils create-spiked-vcfs gives the following error: "Error: No such option: --template-vcf-path (Possible options: --hg19-template-vcf, --hg38-template-vcf)". Downgrading pheval to 0.3.1 fix this error
When I execute "make pheval run" the directory configurations/exomiser-13.1.0-default is created but later breaks because the file configurations/exomiser-13.1.0-default/config.yaml not exists. I think that this is counterintuitive, if the folder is created by pheval, I cannot create the config file in a folder that not exists. The same occurs for the preset-exome-analysis.yml file
The pheval system make symlinks to exomiser configuration files but there is a stage that the file "application.properties" is accessed in write mode. As I work in a cluster and it's no sense install exomiser twice, I referenced the cluster installation that it's write protected so I have a Permission Error. If you simlink this file to write in it (that is not a good practice as the program is also used in other contexts), why not copy it? If you execute exomiser from configurations directory, it will access the copied file instead the original one, so the symlink is not necessary. In fact, I fixed this problem removing the symlink and copying the original "application.properties" file to the exomiser folder within configurations directory.
Exomiser gives me file errors when with configurations/exomiser-13.1.0-default/2302_hg19/2302_hg19_transcripts_ensembl.ser and configurations/exomiser-13.1.0-default/2302_hg38/2302_hg38_transcripts_ensembl.ser. Pheval do NOT simlink these files or give any error about not finding them when simlinking. Simlink manually 2302_hg19 and 2302_hg38 folders from original exomiser installation fixed the problem.
File errors with phenotype files. Pheval simlinks each file of phenotype folder instead simlink the folder. Simlink manually the phenotype folder within exomiser folder in configuration directory fixed the problem.
Due to previous errors I've had several results folders with corrupted exomiser executions. If I execute "make pheval run", make execute and exits from pheval folder instead of resume exomiser execution. To do so, I have to remove manually the results folder. I think that the corresponding rule has to check if exomiser has generated the results files and if not, launch exomiser.

Once I've succesfully executed the pipeline, what is the next step to obtain the performance figures shown in the paper?
Thank you very much in advance
Pedro Seoane

matentzn · 2024-07-22T14:48:15Z

@seoanezonjic Thank you for your message, the whole team is OOO at the moment, but someone will come back to you when they are back!

seoanezonjic · 2024-07-25T10:02:15Z

Thank you @matentzn and @yaseminbridges , I''ll wait to the team and in the meantime I'm playing with the platform. In fact, another problem that I faced is that I'm executing in my university cluster but the computing nodes has not access to internet and the script create_noisy_phenopackets.py fails in the following line:
resource = OntologyResource(slug="hp.obo", local=False)
I think that the script should have an option to set "local" attribute to True and used a cached version of the ontology files in this computing environments (and I think that it will speeds up the execution avoiding repeated queries to servers)
Thank you in advance
Pedro Seoane

yaseminbridges · 2024-07-29T10:44:55Z

Hi, @seoanezonjic!

Thank you for documenting these problems in detail!

I think a lot of these issues may be due to the MakeFile pipeline needing to be updated to what we use in the paper. @souzadevinicius can you confirm?

souzadevinicius · 2024-07-30T08:38:01Z

Hello, @seoanezonjic,

Thank you for your message. I acknowledge that the PhEval pipeline documentation is outdated. I am currently working on updating it.

In the meantime, you can utilize a pipeline described in this repository: https://github.com/monarch-initiative/monarch_pheval. The repository contains different types of experiments, which can be edited by modifying the PhEval Config YAML File.

To execute all experiments outlined in the config file, please follow two steps provided in the pipeline documentation:

Run bash ./resources/generatemakefile.sh
Execute make pheval

I apologize for any inconvenience caused by the outdated documentation. If you happen to have any other issues, please feel free to reach out to me.

Thank you.

julesjacobsen · 2024-08-22T13:06:11Z

@souzadevinicius is the pheval Makefile needed any more if the monarch-pheval repo exists?

souzadevinicius · 2024-08-27T04:44:22Z

@souzadevinicius is the pheval Makefile needed any more if the monarch-pheval repo exists?

No, it's not, @julesjacobsen. I'm updating the PhEval pipeline documentation and removing these unnecessary files.

yaseminbridges · 2024-09-30T10:24:24Z

Hi, @seoanezonjic! Just a brief update:

I have implemented an option to specify a local cached ontology file to the scramble-phenopackets command, allowing you to run in the computing environment you are using.
There is now a docs page on how to execute a benchmark, this is a guide for how to produce performance figures shown in the paper.

These changes are implemented in v0.4.0

seoanezonjic · 2024-10-08T13:13:28Z

Hi @souzadevinicius and @yaseminbridges
Thank you by your suggestions and code changes. First, I was toying with https://github.com/monarch-initiative/monarch_pheval but then Yasemin came around with her message and then I followed the tutorial that she point to me. So, using this tutorial I have the following problems:

Again jinja2 problems: ./resources/generatemakefile.sh: línea 37: j2: command not found. I solved this with pip install j2cli whithin the poetry environment but again I strongly suggest to add this dependency to the project.
Another error related with the yaml listed in the tutorial:

File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval_monarch/pheval/resources/Makefile.j2", line 72, in top-level template code  configurations/{{ config.id }}/config.yaml:
           jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'id'

The line that makes reference: Makefile.j2", line 72 -> configurations/{{ config.id }}/config.yaml: . And config.id was not defined in yaml. I set it to id: exomiser-13.2.0

Finally, the makefile has some problem because this error: Makefile:149 *** target pattern contains no '%'. Stop.. The mentioned makefile line contains:

prepare-corpora: corpora/small_test/scrambled-0.5/corpus.ymlcorpora/small_test/scrambled-0.7/corpus.yml: corpora/small_test/default/corpus.yml $(ROOT_DIR)/testdata/template_vcf/template_exome_hg19.vcf.gz
        test -d $(ROOT_DIR)/corpora/small_test/scrambled-0.7/ || mkdir -p $(ROOT_DIR)/corpora/small_test/scrambled-0.7/
        test -L $(ROOT_DIR)/corpora/small_test/scrambled-0.7/template_exome_hg19.vcf.gz || ln -s $(ROOT_DIR)/testdata/template_vcf/template_exome_hg19.vcf.gz $(ROOT_DIR)/corpora/small_test/scrambled-0.7/template_exome_hg19.vcf.gz

If you could give some clue to fix this error I would thank you a lot.
Pedro Seoane

yaseminbridges · 2024-10-08T14:11:25Z

Hi @seoanezonjic

Apologies, just to clarify, did you face these errors when executing the pipeline here: https://github.com/monarch-initiative/monarch_pheval?

For the time being, you should use the instructions outlined in that repo and tailor it to the experiments that you want to run and ignore the docs regarding the MakeFile pipeline in this repo.

@souzadevinicius is OOO but he will be able to advise you further on your issues with the MakeFile once he returns

seoanezonjic · 2024-10-08T16:23:20Z

Hi @yaseminbridges
I started in that repo BUT when you answered me with your message about pheval 0.4.0 and the new features I removed all files. From zero, I followed the instructions in https://monarch-initiative.github.io/pheval/executing_a_benchmark/ and I ended with the errors enumerated in my previous post and I expected a successful run. I do NOT mixed instructions or files from both repos.
Thank you in advance
Pedro Seoane

yaseminbridges · 2024-10-14T11:55:02Z

Hi @seoanezonjic
Thank you for your patience on this and apologies for the misunderstanding.
The instructions here: https://monarch-initiative.github.io/pheval/executing_a_benchmark/ are separate to the PhEval MakeFile pipeline. In order to execute a benchmark you must already have successfully run and have a directory or several directories of PhEval standardised TSV results. For example, after a single executed pheval run the output directory may look like so:

.
├── pheval_disease_results
│   ├── patient_1-pheval_disease_result.tsv
├── pheval_gene_results
│   ├── patient_1-pheval_gene_result.tsv
├── pheval_variant_results
│   ├── patient_1-pheval_variant_result.tsv
├── raw_results
│   ├── patient_1.json
├── results.yml
└── tool_input_commands
    └── tool_input_commands.txt

The MakeFile pipeline will automatically implement this command in its pipeline so you do not need to do so manually if you are planning to run the MakeFile.

seoanezonjic · 2024-10-15T09:58:03Z

Hi @yaseminbridges
Now I'm understand and I apologize for my misunderstanding, I've a lot in my head the last weeks.
I'll explain my steps with the aim of get a operational pheval in my cluster to benchmark my prioritizer. First, I need to get a working example with exomiser al least. My steps were:

As you tell me that the pheval v0.4.0 has the option to use a local cached ontology file, I'm started from zero. I deleted all my work with https://github.com/monarch-initiative/monarch_pheval and I followed this guide: https://monarch-initiative.github.io/pheval/exomiser_pipeline/
The errors listed in Problems using pheval #347 (comment) are from this fresh install and execution.
I can't execute the instructions on https://monarch-initiative.github.io/pheval/executing_a_benchmark/ guide because the problems I've got with the Makefile

I hope that now it's clear my problem and I'm very sorry for the confusion
Thank you in advance
Pedro Seoane

souzadevinicius · 2024-11-05T10:15:26Z

Hi, @seoanezonjic.

I hope you have successfully run the PhEval pipeline. In any case, the https://monarch-initiative.github.io/pheval/executing_a_benchmark/ has been updated.

If you want to explore the pipeline process further, we have a dedicated repository available at https://github.com/monarch-initiative/monarch_pheval.
This pipeline includes ready-to-use "recipes" structured with Makefile and YAML configuration files, designed to simplify each step.
There is a branch specifically for PhEval paper experiments (the pheval-paper branch). We are actively updating the documentation to enhance usability, and your feedback is essential to this process.

Thank you!

seoanezonjic · 2024-11-05T12:16:14Z

Hi @souzadevinicius
Thank you for the update but I've waiting a recommendation or fix for my problems listed in #347 (comment) when I followed this guide: https://monarch-initiative.github.io/pheval/exomiser_pipeline. I'm stuck with this and I can't continue to the benchmark.
If you want, you can point me to a guide for a Exomiser execution with pheval and another to make the benchmark to start from zero again. In this way, we all stay in the same page an we could debug together the bug origin.
Thank you in advance
Pedro Seoane

souzadevinicius · 2024-11-06T14:21:16Z

Hi @souzadevinicius Thank you for the update but I've waiting a recommendation or fix for my problems listed in #347 (comment) when I followed this guide: https://monarch-initiative.github.io/pheval/exomiser_pipeline. I'm stuck with this and I can't continue to the benchmark. If you want, you can point me to a guide for a Exomiser execution with pheval and another to make the benchmark to start from zero again. In this way, we all stay in the same page an we could debug together the bug origin. Thank you in advance Pedro Seoane

Hello, Pedro.

You're completely right about your suggestion of adding j2cli as a dependency and we did it.
However, as I mentioned, we moved all pipeline code to a new repository to decouple this core PhEval repository from the pipeline code.

Additionally, in the updated documentation we don't mention j2 anymore because everything is encapsulated in Monarch PhEval repository. The Makefile generation is also fixed.
If you have any other questions or issues, let us know.

seoanezonjic · 2024-11-06T14:41:55Z

Hi @souzadevinicius
I've solved the j2cli and the yaml errors but the following it's blocking me (I write the text from the previous message):

Finally, the makefile has some problem because this error: Makefile:149 *** target pattern contains no '%'. Stop.. The mentioned makefile line contains:

prepare-corpora: corpora/small_test/scrambled-0.5/corpus.ymlcorpora/small_test/scrambled-0.7/corpus.yml: corpora/small_test/default/corpus.yml $(ROOT_DIR)/testdata/template_vcf/template_exome_hg19.vcf.gz
        test -d $(ROOT_DIR)/corpora/small_test/scrambled-0.7/ || mkdir -p $(ROOT_DIR)/corpora/small_test/scrambled-0.7/
        test -L $(ROOT_DIR)/corpora/small_test/scrambled-0.7/template_exome_hg19.vcf.gz || ln -s $(ROOT_DIR)/testdata/template_vcf/template_exome_hg19.vcf.gz $(ROOT_DIR)/corpora/small_test/scrambled-0.7/template_exome_hg19.vcf.gz

This error I'm unable to resolve it. Any suggestion?
Thank you in advance
Pedro Seoane

souzadevinicius · 2024-11-12T15:52:08Z

Hi @souzadevinicius I've solved the j2cli and the yaml errors but the following it's blocking me (I write the text from the previous message):

Finally, the makefile has some problem because this error: Makefile:149 *** target pattern contains no '%'. Stop.. The mentioned makefile line contains:
prepare-corpora: corpora/small_test/scrambled-0.5/corpus.ymlcorpora/small_test/scrambled-0.7/corpus.yml: corpora/small_test/default/corpus.yml $(ROOT_DIR)/testdata/template_vcf/template_exome_hg19.vcf.gz
        test -d $(ROOT_DIR)/corpora/small_test/scrambled-0.7/ || mkdir -p $(ROOT_DIR)/corpora/small_test/scrambled-0.7/
        test -L $(ROOT_DIR)/corpora/small_test/scrambled-0.7/template_exome_hg19.vcf.gz || ln -s $(ROOT_DIR)/testdata/template_vcf/template_exome_hg19.vcf.gz $(ROOT_DIR)/corpora/small_test/scrambled-0.7/template_exome_hg19.vcf.gz
This error I'm unable to resolve it. Any suggestion? Thank you in advance Pedro Seoane

Hello, @seoanezonjic.

I recommend generating a new Makefile.

You can do this by cloning the monarch_pheval repository and regenerating the Makefile. Here are the steps to follow:

Run the command: git clone -b new-paper-2025 https://github.com/monarch-initiative/monarch_pheval.git
Access the created directory: cd monarch_pheval
Fill in the ./resources/pheval-config.yaml file with your experiment descriptions.
Execute the script by running: ./resources/generatemakefile.sh
Finally, type make pheval to compile.

Let me know if you have any questions!

seoanezonjic · 2024-11-13T15:54:28Z

Hi @souzadevinicius
My problem is with the step 3, I have not clear documentation so to execute a exomiser run I've modified ./resources/pheval-config.yaml to:

directories:
  data: data
  tmp: data/tmp

corpora:
  - id: lirical
    custom_variants:
      - id: no_phenotype

configs:
  - tool: exomiser
    id: exomiser-14.0.0-2402_phenotype
    version: 14.0.0

runs:
  - tool: exomiser
    corpus: lirical
    corpusvariant: default
    version: 14.0.0
    phenotype: 2402
    configuration: exomiser-14.0.0-2402_phenotype

I've got this error:

mkdir -p /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/results/exomiser-14.0.0-2402_phenotype
pheval run \
 --input-dir /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/configurations/exomiser-14.0.0-2402_phenotype \
 --testdata-dir /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/corpora/lirical/default \
 --runner exomiserphevalrunner \
 --tmp-dir data/tmp/ \
 --version 14.0.0 \
 --output-dir /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/results/exomiser-14.0.0-2402_phenotype
Traceback (most recent call last):
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/bin/pheval", line 8, in <module>
    sys.exit(pheval())
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/pheval/cli_pheval.py", line 87, in run
    runner_class = get_implementation_resolver().lookup(runner)
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/class_resolver/api.py", line 182, in lookup
    return get_cls(
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/class_resolver/api.py", line 424, in get_cls
    raise KeyError(
KeyError: "Invalid PhEvalRunner name: exomiserphevalrunner (normalized to: exomiserphevalrunner). Valid choices are: ['defaultphevalrunner', 'templatephevalrunner']"
make[1]: *** [Makefile:69: /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/results/exomiser-14.0.0-2402_phenotype/results.yml] Error 1
make[1]: Leaving directory '/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval'
make: *** [Makefile:140: pheval] Error 2

A documentation explaining how works the file ./resources/pheval-config.yaml and how must be used would be very appreciated (not only examples but each section and attribute described to launch exomiser, lirical, gado and phen2gene programs I think that each one needs specific configuration).
Thank you in advance
Pedro Seoane

souzadevinicius · 2024-11-15T13:42:29Z

Hi @souzadevinicius My problem is with the step 3, I have not clear documentation so to execute a exomiser run I've modified ./resources/pheval-config.yaml to:

directories:
  data: data
  tmp: data/tmp

corpora:
  - id: lirical
    custom_variants:
      - id: no_phenotype

configs:
  - tool: exomiser
    id: exomiser-14.0.0-2402_phenotype
    version: 14.0.0

runs:
  - tool: exomiser
    corpus: lirical
    corpusvariant: default
    version: 14.0.0
    phenotype: 2402
    configuration: exomiser-14.0.0-2402_phenotype

I've got this error:

mkdir -p /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/results/exomiser-14.0.0-2402_phenotype
pheval run \
 --input-dir /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/configurations/exomiser-14.0.0-2402_phenotype \
 --testdata-dir /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/corpora/lirical/default \
 --runner exomiserphevalrunner \
 --tmp-dir data/tmp/ \
 --version 14.0.0 \
 --output-dir /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/results/exomiser-14.0.0-2402_phenotype
Traceback (most recent call last):
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/bin/pheval", line 8, in <module>
    sys.exit(pheval())
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/pheval/cli_pheval.py", line 87, in run
    runner_class = get_implementation_resolver().lookup(runner)
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/class_resolver/api.py", line 182, in lookup
    return get_cls(
  File "/localscratch/users/pedro/pypoetry/virtualenvs/monarch-pheval-3lvLeukC-py3.10/lib/python3.10/site-packages/class_resolver/api.py", line 424, in get_cls
    raise KeyError(
KeyError: "Invalid PhEvalRunner name: exomiserphevalrunner (normalized to: exomiserphevalrunner). Valid choices are: ['defaultphevalrunner', 'templatephevalrunner']"
make[1]: *** [Makefile:69: /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/results/exomiser-14.0.0-2402_phenotype/results.yml] Error 1
make[1]: Leaving directory '/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval'
make: *** [Makefile:140: pheval] Error 2

A documentation explaining how works the file ./resources/pheval-config.yaml and how must be used would be very appreciated (not only examples but each section and attribute described to launch exomiser, lirical, gado and phen2gene programs I think that each one needs specific configuration). Thank you in advance Pedro Seoane

Hello, @seoanezonjic.

We are almost there. Your Makefile is correct.
Now, you need the pheval.exomiser plugin, just installing it using poetry:

poetry add pheval.exomiser
or pip:
pip install pheval.exomiser

I'm adding to the documentation to make it clearer

seoanezonjic · 2024-11-21T16:18:32Z

Hi @souzadevinicius
I've installed pheval.exomiser and then I've obtained the following errors:

	pheval run \
 --input-dir /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/configurations/exomiser-14.0.0-2402_phenotype \
 --testdata-dir /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/corpora/lirical/default \
 --runner exomiserphevalrunner \
 --tmp-dir data/tmp/ \
 --version 14.0.0 \
 --output-dir /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/results/exomiser-14.0.0-2402_phenotype
preparing
Traceback (most recent call last):
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/bin/pheval", line 8, in <module>
    sys.exit(pheval())
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/pheval/cli_pheval.py", line 90, in run
    runner_instance.prepare()
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/pheval_exomiser/runner.py", line 30, in prepare
    configurations=ExomiserConfigurations.parse_obj(
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/pydantic/main.py", line 1162, in parse_obj
    return cls.model_validate(obj)
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/pydantic/main.py", line 596, in model_validate
    return cls.__pydantic_validator__.validate_python(
pydantic_core._pydantic_core.ValidationError: 3 validation errors for ExomiserConfigurations
application_properties.hg19_data_version
  Input should be a valid string [type=string_type, input_value=2402, input_type=int]
    For further information visit https://errors.pydantic.dev/2.9/v/string_type
application_properties.hg38_data_version
  Input should be a valid string [type=string_type, input_value=2402, input_type=int]
    For further information visit https://errors.pydantic.dev/2.9/v/string_type
application_properties.phenotype_data_version
  Input should be a valid string [type=string_type, input_value=2402, input_type=int]
    For further information visit https://errors.pydantic.dev/2.9/v/string_type
make: *** [Makefile:69: /mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/results/exomiser-14.0.0-2402_phenotype/results.yml] Error 1

This I 've fixed editing runners/configurations/exomiser-14.0.0-2402_phenotype.config.yaml and change 2402 to "2402" but then the following error arises:

Traceback (most recent call last):
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/bin/pheval", line 8, in <module>
    sys.exit(pheval())
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/pheval/cli_pheval.py", line 91, in run
    runner_instance.run()
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/pheval_exomiser/runner.py", line 49, in run
    run_exomiser(
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/pheval_exomiser/run/run.py", line 208, in run_exomiser
    run_exomiser_local(
  File "/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/.venv/lib/python3.10/site-packages/pheval_exomiser/run/run.py", line 120, in run_exomiser_local
    exomiser_jar_file = [
IndexError: list index out of range

I've debugged line 117 from pheval_exomiser/run/run.py and a simple print as following:

    print(f"_{config.exomiser_software_directory}_", file=sys.stderr)
    print(f"_{input_dir.joinpath(config.exomiser_software_directory)}_", file=sys.stderr)

Shows this:

_._
_/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/configurations/exomiser-14.0.0-2402_phenotype_

This is a path error because exomiser is installed in the runners folder as I can see in my local installation of the monarch-pheval. Maybe is a misconfiguration in my yaml file?
Thank you in advance
Pedro Seoane

souzadevinicius · 2024-11-26T10:34:44Z

Yes maybe it is, please share your config.yaml file.

seoanezonjic · 2024-11-26T10:39:44Z

Hi @souzadevinicius
Here it is:

directories:
  data: data
  tmp: data/tmp

corpora:
  - id: lirical
    custom_variants:
      - id: no_phenotype

configs:
  - tool: exomiser
    id: exomiser-14.0.0-2402_phenotype
    version: 14.0.0

runs:
  - tool: exomiser
    corpus: lirical
    corpusvariant: default
    version: 14.0.0
    phenotype: "2402"
    configuration: exomiser-14.0.0-2402_phenotype

souzadevinicius · 2024-12-02T12:21:12Z

The problem is in config.yaml file inside the exomiser configuration folder.

_/mnt/home/users/pab_001_uma/pedro/proyectos/pheval/PIPELINE/monarch_pheval/configurations/exomiser-14.0.0-2402_phenotype_/config.yaml

You need to specify where the exomiser is located by this

exomiser_software_directory:

I have the exomiser installation inside the config folder in my local example.
If you don't follow the same structure, specify where your exomiser installation is.

My current local structure in the configurations folder:

📦configurations
┗ 📂exomiser-14.0.0
┃ ┣ 📂2402_hg19
┃ ┣ 📂2402_hg38
┃ ┣ 📂2402_phenotype
┃ ┣ 📂examples
┃ ┣ 📂lib
┃ ┣ 📂results
┃ ┣ 📜application.properties
┃ ┣ 📜config.yaml
┃ ┣ 📜exomiser-cli-14.0.0.jar
┃ ┗ 📜preset-exome-analysis.yml

The Exomiser content is located in the configurations folder, so the exomiser_software_directory: in the config.yaml file is set to ".", indicating that Exomiser is in the current directory.

seoanezonjic · 2024-12-02T13:48:15Z

Hi @souzadevinicius
I'll try your suggestion but I expect that if the Makefile download and install the software in its own (in this case, it installed exomiser into de runners folder at the same level that configurations folder)), the system should be able to take into account the installation path that was used in configuration steps. Now that I know the exomiser_software_directory:
directive even I could use previous system installations.

I suggest you to think and check that monarch_pheval repository will make a download, installation and execution steps in one Makefile at least with exomiser but with no user configuration (all automatized). Once the user is able to check that the system works in his/her environment, he/she could toy with the configuration to adapt to his/her needs.
Currently its a bit frustrating to work with the workflow.
Thank you in advance
Pedro Seoane

matentzn assigned yaseminbridges Jul 22, 2024

yaseminbridges assigned souzadevinicius Jul 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Problems using pheval #347

Problems using pheval #347

seoanezonjic commented Jul 15, 2024

matentzn commented Jul 22, 2024

seoanezonjic commented Jul 25, 2024

yaseminbridges commented Jul 29, 2024

souzadevinicius commented Jul 30, 2024

julesjacobsen commented Aug 22, 2024

souzadevinicius commented Aug 27, 2024

yaseminbridges commented Sep 30, 2024 •

edited

Loading

seoanezonjic commented Oct 8, 2024

yaseminbridges commented Oct 8, 2024

seoanezonjic commented Oct 8, 2024

yaseminbridges commented Oct 14, 2024

seoanezonjic commented Oct 15, 2024

souzadevinicius commented Nov 5, 2024 •

edited

Loading

seoanezonjic commented Nov 5, 2024

souzadevinicius commented Nov 6, 2024

seoanezonjic commented Nov 6, 2024

souzadevinicius commented Nov 12, 2024 •

edited

Loading

seoanezonjic commented Nov 13, 2024

souzadevinicius commented Nov 15, 2024

seoanezonjic commented Nov 21, 2024

souzadevinicius commented Nov 26, 2024

seoanezonjic commented Nov 26, 2024 •

edited

Loading

souzadevinicius commented Dec 2, 2024

seoanezonjic commented Dec 2, 2024

Problems using pheval #347

Problems using pheval #347

Comments

seoanezonjic commented Jul 15, 2024

matentzn commented Jul 22, 2024

seoanezonjic commented Jul 25, 2024

yaseminbridges commented Jul 29, 2024

souzadevinicius commented Jul 30, 2024

julesjacobsen commented Aug 22, 2024

souzadevinicius commented Aug 27, 2024

yaseminbridges commented Sep 30, 2024 • edited Loading

seoanezonjic commented Oct 8, 2024

yaseminbridges commented Oct 8, 2024

seoanezonjic commented Oct 8, 2024

yaseminbridges commented Oct 14, 2024

seoanezonjic commented Oct 15, 2024

souzadevinicius commented Nov 5, 2024 • edited Loading

seoanezonjic commented Nov 5, 2024

souzadevinicius commented Nov 6, 2024

seoanezonjic commented Nov 6, 2024

souzadevinicius commented Nov 12, 2024 • edited Loading

seoanezonjic commented Nov 13, 2024

souzadevinicius commented Nov 15, 2024

seoanezonjic commented Nov 21, 2024

souzadevinicius commented Nov 26, 2024

seoanezonjic commented Nov 26, 2024 • edited Loading

souzadevinicius commented Dec 2, 2024

seoanezonjic commented Dec 2, 2024

yaseminbridges commented Sep 30, 2024 •

edited

Loading

souzadevinicius commented Nov 5, 2024 •

edited

Loading

souzadevinicius commented Nov 12, 2024 •

edited

Loading

seoanezonjic commented Nov 26, 2024 •

edited

Loading