Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reopen - Formula Validation #238

Open
f-allian opened this issue Aug 31, 2023 · 1 comment
Open

Reopen - Formula Validation #238

f-allian opened this issue Aug 31, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@f-allian
Copy link
Contributor

Reopening an issue for the formula validation for the following PR: #235.

Problem:

The formula validation doesn't work with direct effects, and returns errors for adjustment sets when they should work in practice (externally validated by Dagitty). This also raises a LinAlgError if the data quality is poor.

Temporary fix:

Roll back formula validation commits until we can thoroughly understand the statistical details.

@f-allian f-allian added the bug Something isn't working label Aug 31, 2023
@jmafoster1
Copy link
Contributor

jmafoster1 commented Nov 28, 2023

Steps to reproduce

  1. Download and extract the mwe.zip.
  2. cd into the directory.
  3. Activate a conda environment with CTF v5.3.0.
  4. Execute python run_causal_tests.py --json_path causal_tests.json --dag_path dag.dot --data_path data.csv
  5. Observe error Formula covariates do not satisfy the constructive back-door criterion.

The problem lies in the specified linear regression formula num_shapes_unit ~ num_lines_abs + I(num_shapes_abs/(width*height)) on line 98 of causal_tests.json. This has treatment variable num_lines_abs, outcome variable num_shapes_unit, and three covariates {num_shapes_abs, width, height}. This fails validation, despite being a perfectly valid adjustment set on Dagitty.
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants