Neuro-Symbolic Prompting

A repository for Neuro-Symbolic Prompting experiments on visual reasoning tasks such as CLEVR and HWF.

---

Setup

Start by creating a Python virtual environment and installing dependencies:

make create_environment
source .venv/bin/activate
make requirements

Ensure your python command points to a Python 3.10+ interpreter.

Download Datasets

First create the data/ directory.

HWF

Download from here and place under data/HWF.

CLEVR

Download from here and place under data/CLEVR_v1.0.

Running the Main Experiments

The main experiments were conducted using the scripts/llm_symbolic_eval.py script. After setting up the environment as described above, you can run the experiments locally. For example, to run the CLEVR few-shot experiment with logging:

python scripts/llm_symbolic_eval.py \
  --dataset clevr \
  --log \
  --few_shot \
  --model gemini-2.0-flash \
  --image_before \
  --single_turn

For a full list of options, see:

python scripts/llm_symbolic_eval.py --help