A small benchmark for testin the Joana capabilities for taint analysis.
In the very first moment, execute the ./setup.sh script to download the Joana
library and configure it as a local maven dependency.
Run an specific test: mvn test -Dtest="br.unb.cic.joana.securibench.v1.micro.suite.SpecificTest"
Currently, you might run the 'securibench' benchmark using JUnit test cases. Such as:
- Aliasing test suite:
mvn test -Dtest="br.unb.cic.joana.securibench.v1.micro.suite.AliasingTestSuite" - Array test suite:
mvn test -Dtest="br.unb.cic.joana.securibench.v1.micro.suite.ArrayTestSuite" - Basic test suite:
mvn test -Dtest="br.unb.cic.joana.securibench.v1.micro.suite.BasicTestSuite" - Collection test suite:
mvn test -Dtest="br.unb.cic.joana.securibench.v1.micro.suite.CollectionTestSuite" - Datastructure test suite:
mvn test -Dtest="br.unb.cic.joana.securibench.v1.micro.suite.DatastructureTestSuite" - Factory test suite:
mvn test -Dtest="br.unb.cic.joana.securibench.v1.micro.suite.FactoryTestSuite" - Inter test suite:
mvn test -Dtest="br.unb.cic.joana.securibench.v1.micro.suite.InterTestSuite" - Session test suite:
mvn test -Dtest="br.unb.cic.joana.securibench.v1.micro.suite.SessionTestSuite" - StrongUpdate test suite:
mvn test -Dtest="br.unb.cic.joana.securibench.v1.micro.suite.StrongUpdateTestSuite" - All test suite:
mvn test -Dtest="br.unb.cic.joana.securibench.v1.micro.suite.AllTestSuite"
| Test | TP | FP | FN |
|---|---|---|---|
| Aliasing | 2/11 | 2 | 7 |
| Array | 0/0 | 0 | 0 |
| Basic | 0/0 | 0 | 0 |
| Collection | 0/0 | 0 | 0 |
| DataStructure | 0/0 | 0 | 0 |
| Factory | 0/0 | 0 | 0 |
| Inter | 0/0 | 0 | 0 |
| Session | 0/0 | 0 | 0 |
| StrongUpdate | 0/0 | 0 | 0 |
| TOTAL | 0/0 | 0 | 0 |
- Violations: 123/138
- Precision: 0.82
- Recall: 0.72
- F-score: 0.76
- AliasingTest - failed: 4, passed: 2 of 6 tests.
(33.33%)
| Test | Expected | Actual | Status | TP | FP | FN | Precision | Recall | F-score | Comments |
|---|---|---|---|---|---|---|---|---|---|---|
| Aliasing1 | 1 | 1 | ✅ | 1 | 0 | 0 | - | - | - | - |
| Aliasing2 | 0 | 1 | ❌ | 0 | 1 | 0 | - | - | - | - |
| Aliasing3 | 0 | 1 | ❌ | 0 | 1 | 0 | - | - | - | - |
| Aliasing4 | 2 | 1 | ❌ | 0 | 0 | 1 | - | - | - | - |
| Aliasing5 | 1 | 1 | ✅ | 1 | 0 | 0 | - | - | - | - |
| Aliasing6 | 7 | 1 | ❌ | 0 | 0 | 6 | - | - | - | - |
| TOTAL | 11 | 6 | 2/6 | 2 | 2 | 7 | - |
- Violations:
6/11 - Precision:
50% - Recall:
22% - fScore:
31% - Pass Rate:
33.33%