Code

Skip unsupported regression tests#615

Code

Merged

schuessf merged 42 commits intodevultimate-pa/ultimate:devfrom wip/dk/skip-unsupported2ultimate-pa/ultimate:wip/dk/skip-unsupported2Copy head branch name to clipboardSep 9, 2024

+463-35

Commits

Commits on Sep 2, 2024

• add support for ignoring tests after running them

  

  maul-esel

  authored and

  schuessf

  committed
  8b4ea8f

• preliminary support for test result deciders to skip tests

  

  maul-esel

  authored and

  schuessf

  committed
  12536f9

• First attempt to check if a test case is ignored

  schuessf

  committed
  f3ad4f9

• Add possibility to ignore test failures to SafetyCheckTestResultDecider

  schuessf

  committed
  8fae4cd

• Don't overwrite IGNORE result

  schuessf

  committed
  a03eee8

• Skip overflow tests as well

  schuessf

  committed
  2547848

• Ignore safety tests in various folders

  schuessf

  committed
  8cbef81

• Ignore some more tests and move to the correct location

  schuessf

  committed
  688f032

• Ignore another test

  schuessf

  committed
  da2b1a0

• Improve parsing of ignore file

  schuessf

  committed
  1f787f0

• Restructure ignore files, add comments with reason

  schuessf

  committed
  522e294

• Add expected verdict to ignored.txt, only skip with that verdict

  schuessf

  committed
  ed8ccb9

• Fix some expected verdicts

  schuessf

  committed
  dbd795a

• Catch case where String cannot be converted to enum

  schuessf

  committed
  bcbff56

• Minor: Restructure method

  schuessf

  committed
  adde04c

• Fix documentation

  schuessf

  committed
  6362e6a

• Rename ignored.txt to .skip

  schuessf

  committed
  66815b3

• Refactoring, allow .skip file in toolchain and settings dir

  schuessf

  committed
  4de4c3f

• Skip tests with overapproximation

  schuessf

  committed
  75786af

• Fix build in new regression tests

  schuessf

  committed
  fcfd79a

• Throw exception when testsuite does not support skipping tests

  schuessf

  committed
  fa6fc5f

• Update .skip files

  schuessf

  committed
  b03532f

• Skip data race tests

  schuessf

  committed
  9189631

• Fix updating wrong .skip file

  schuessf

  committed
  2467d4a

• Use regex to check if test should be marked as skipped

  schuessf

  committed
  c17b485

• Fix typo

  schuessf

  committed
  989a4f2

• Remove unnecessary .skip file

  schuessf

  committed
  b72f878

• Use YAML for skip files

  schuessf

  committed
  b4ea028

• Add missing space

  schuessf

  committed
  ebef1df

• Use proper YAML syntax for skip files

  schuessf

  committed
  ab8bd81

• Only allow skip files next to the tests itself

  schuessf

  committed
  362adb7

• Use .testignore as filename

  schuessf

  committed
  a782a21

Commits on Sep 3, 2024

• Split .testignore properly for subfolders

  schuessf

  committed
  594d115

• Allow skipping termination tests

  schuessf

  committed
  f845e42

• Add missing import

  schuessf

  committed
  261c3fa

• Add newline

  schuessf

  committed
  10f0efb

• Add checks for no overriden result

  schuessf

  committed
  1fbf05f

• Add newline

  schuessf

  committed
  9a97c39

Commits on Sep 4, 2024

• Avoid parentheses in regex

  schuessf

  committed
  1f5cc6a

• Add timeout

  schuessf

  committed
  c9d2c0b

• Skip termination tests properly

  schuessf

  committed
  95ffcc8

Commits on Sep 9, 2024

• Add separate method to override verdict

  schuessf

  committed
  ecb5fa7