Failed unit test within Docker image

[stschaef]

I'm able to use Marabou locally on my M1 macbook, but I'm seeking to use it in a Docker image. I want to have a portable experiment that I can run on things like AWS or a cluster without having to manually load dependencies etc.

In my current attempt at making this image, Compilation completes 100% but there is a problem when testing. Marabou builds just fine, except a single unit test (Test_SumOfInfeasibilitiesManager) fails. This doesn't happen for me locally, even on the same commit.

I am unsure if this is a configuration issue on my end or some weird bug underlying the lib. Here is my Dockerfile,

FROM nvidia/cuda:11.3.1-cudnn8-devel-ubuntu20.04

FROM continuumio/miniconda3

SHELL ["/bin/bash", "--login", "-c"]

COPY conda_environment.yml .
RUN conda env create -f conda_environment.yml

SHELL ["conda", "run", "-n", "vertical-cas-env", "/bin/bash", "-c"]
RUN echo "conda activate vertical-cas-env" > ~/.bashrc

RUN apt-get -y update && apt-get install -y git gcc g++ cmake

RUN pip install --upgrade pip cvxpy pyinterval

# Can thin this out to lessen loading times
# Marabou dir is in .dockerignore
COPY . .

# Marabou is breaking here, fails 1 out of 69 unit tests...weird
# This takes ~ 10 minutes to build
RUN git clone https://github.com/NeuralNetworkVerification/Marabou.git \
    && mv Marabou GenerateNetworks/Marabou \
    && cd GenerateNetworks/Marabou && mkdir build && cd build \
    && cmake .. -DBUILD_PYTHON=ON -DCMAKE_BUILD_TYPE=Debug \
    && cmake --build .

ENV PYTHONPATH=$PYTHONPATH:$(pwd)/GenerateNetworks/Marabou


CMD echo "running test" && python3 GenerateNetworks/trainVertCAS.py

The above is at Marabou HEAD, but I also tried reverting to a previous release 1271f92 with no luck

and the conda environment for good measure,

name: vertical-cas-env
channels:
  - defaults
dependencies:
      - matplotlib
      - numpy
      - onnxruntime
      - tensorflow
prefix: /opt/homebrew/Caskroom/miniconda/base/envs/vertical-cas-env

and the actual test output,

732.8 24/69 Test #24: Test_SumOfInfeasibilitiesManager ...***Failed    0.99 sec
732.8 Running 8 tests...
732.8 In SumOfInfeasibilitiesManagerTestSuite::test_propose_phase_pattern_update_randomly:
732.8 /VerticalCAS/GenerateNetworks/Marabou/src/engine/tests/Test_SumOfInfeasibilitiesManager.h:339: Error: Expected (*soiManager-> getConstraintsUpdatedInLastProposal().begin() == plConstraints[1]), found ({ C0 28 60 8A FF FF 00 00  } != { 40 2A 60 8A FF FF 00 00  })
732.8 /VerticalCAS/GenerateNetworks/Marabou/src/engine/tests/Test_SumOfInfeasERROR conda.cli.main_run:execute(47): `conda run /bin/bash -c git clone https://github.com/NeuralNetworkVerification/Marabou.git     && mv Marabou GenerateNetworks/Marabou     && cd GenerateNetworks/Marabou && mkdir build && cd build     && cmake .. -DBUILD_PYTHON=ON -DCMAKE_BUILD_TYPE=Debug     && cmake --build .` failed. (See above for error)
732.8 ibilitiesManager.h:343: Error: Expected (cost1 == soiManager->getCurrentSoIPhasePattern()), found ({ 50 22 60 F9 FF FF 00 00 ... } != { B0 22 60 F9 FF FF 00 00 ... })
732.8 /VerticalCAS/GenerateNetworks/Marabou/src/engine/tests/Test_SumOfInfeasibilitiesManager.h:347: Error: Expected (mock->randWasCalled == 3u), found (2 != 3)
732.8 /VerticalCAS/GenerateNetworks/Marabou/src/engine/tests/Test_SumOfInfeasibilitiesManager.h:368: Error: Expected (cost2 == soiManager->getCurrentSoIPhasePattern()), found ({ 50 22 60 F9 FF FF 00 00 ... } != { B0 22 60 F9 FF FF 00 00 ... })
732.8 /VerticalCAS/GenerateNetworks/Marabou/src/engine/tests/Test_SumOfInfeasibilitiesManager.h:372: Error: Expected (*soiManager-> getConstraintsUpdatedInLastProposal().begin() == plConstraints[3]), found ({ C0 2B 60 8A FF FF 00 00  } != { 80 28 C0 82 FF FF 00 00  })
732.8 /VerticalCAS/GenerateNetworks/Marabou/src/engine/tests/Test_SumOfInfeasibilitiesManager.h:378: Error: Expected (cost2 == soiManager->getLastAcceptedSoIPhasePattern()), found ({ 50 22 60 F9 FF FF 00 00 ... } != { B0 22 60 F9 FF FF 00 00 ... })
732.8 /VerticalCAS/GenerateNetworks/Marabou/src/engine/tests/Test_SumOfInfeasibilitiesManager.h:379: Error: Expected (cost2 == soiManager->getCurrentSoIPhasePattern()), found ({ 50 22 60 F9 FF FF 00 00 ... } != { B0 22 60 F9 FF FF 00 00 ... })

[stschaef]

https://hub.docker.com/r/dlshriver/dnnv

This page suggests maybe a licensing issue with the Gurobi solver, although this seems unlikely because all of the other tests pass, meaning the tool was running correctly in those instances

[wu-haoze]

Same issue as #721 , the problem is with the test in this case.