templates-db is a repository for template collections. The provided templates are used to generate SAST benchmarks for mutation-based generation. This repository allows you to check your templates for correctness (in formal language terms) and perform some functional diagnostics.
- Language structural equality
- Invalid imports
- Undefined macro usage
- Unused locally defined macro
- Invalid extensions
- Dangling hole reference
For diagnostic usage, follow steps:
- Install Java (version >= 17)
- Install Python dependencies
pip install -r requirments.txt - Generate parser
./antlr.sh --generate
Repeat step 3 on any grammar (.g4 extension) file change.
- Run diagnostics
python3 src/main.py
If you add a new template file, you should add a similar file to all other languages. In template file case, it means a file with the same relative path and the same(by name) templates inside. Otherwise, it requires only the same relative path.
Sometimes, implemented features may only apply to some languages. In that case, you should add a file with the same relative path and .unsupported extension.
Also, you can use .todo extension to delay implementation. In both cases, you may add
some additional information in the file content.
After making new templates, run diagnostics and fix corresponding issues.
If you add a new diagnostic, consider adding corresponding tests.
After getting a new diagnostic implemented:
- Ensure flake8 hasn't issues
flake8 src/*.py - Ensure all test passes
pytest --no-header -v