Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upstream breakage in master #78

Closed
bennahugo opened this issue Jan 18, 2021 · 3 comments
Closed

Upstream breakage in master #78

bennahugo opened this issue Jan 18, 2021 · 3 comments
Labels
bug Something isn't working

Comments

@bennahugo
Copy link
Collaborator

Caught by Jenkins. No further log is available. Perhaps the verbosity of the tests needs to be increased?

[0mERROR
tricolour/tests/test_acceptance.py::test_max_chisq[360-10000.0] ERROR
tricolour/tests/test_acceptance.py::test_flag_count[360-0.65] ERROR
tricolour/tests/test_acceptance.py::test_bandwidth_flagged[360-0.4] ERROR
tricolour/tests/test_flagging.py::FLAKE8 PASSED
tricolour/tests/test_flagging.py::TestAsbool::test_uint8 PASSED
tricolour/tests/test_flagging.py::TestAsbool::test_uint16 PASSED
tricolour/tests/test_flagging.py::TestAsbool::test_bool PASSED
tricolour/tests/test_flagging.py::TestAverageFreq::test_one PASSED
tricolour/tests/test_flagging.py::TestAverageFreq::test_divides PASSED
tricolour/tests/test_flagging.py::TestAverageFreq::test_uneven PASSED
tricolour/tests/test_flagging.py::test_time_median PASSED
tricolour/tests/test_flagging.py::TestMedianAbs::test PASSED
tricolour/tests/test_flagging.py::TestMedianAbs::test_all_flagged PASSED
tricolour/tests/test_flagging.py::TestMedianAbs::test_axis0 PASSED
tricolour/tests/test_flagging.py::TestMedianAbs::test_axis0_all_flagged PASSED
tricolour/tests/test_flagging.py::TestLinearlyInterpolateNans::test_basic PASSED
tricolour/tests/test_flagging.py::TestLinearlyInterpolateNans::test_no_nans PASSED
tricolour/tests/test_flagging.py::TestLinearlyInterpolateNans::test_all_nans PASSED
tricolour/tests/test_flagging.py::TestLinearlyInterpolateNans::test_float32 PASSED
tricolour/tests/test_flagging.py::TestLinearlyInterpolateNans::test_2d PASSED
tricolour/tests/test_flagging.py::TestBoxGaussianFilter::test_one_pass PASSED
tricolour/tests/test_flagging.py::TestBoxGaussianFilter::test_width PASSED
tricolour/tests/test_flagging.py::TestBoxGaussianFilter::test_bad_sigma_dim PASSED
tricolour/tests/test_flagging.py::TestBoxGaussianFilter::test_2d PASSED
tricolour/tests/test_flagging.py::TestBoxGaussianFilter::test_axes PASSED
tricolour/tests/test_flagging.py::TestBoxGaussianFilter::test_edge PASSED
tricolour/tests/test_flagging.py::TestMaskedGaussianFilter::test_basic PASSED
tricolour/tests/test_flagging.py::TestMaskedGaussianFilter::test_nan PASSED
tricolour/tests/test_flagging.py::TestGetBackground2D::test_no_flags PASSED
tricolour/tests/test_flagging.py::TestGetBackground2D::test_all_flagged PASSED
tricolour/tests/test_flagging.py::TestGetBackground2D::test_in_flags PASSED
tricolour/tests/test_flagging.py::TestGetBackground2D::test_interpolate PASSED
tricolour/tests/test_flagging.py::TestGetBackground2D::test_iterations PASSED
tricolour/tests/test_flagging.py::TestSumThreshold::test_sum_threshold_all_flagged PASSED
tricolour/tests/test_flagging.py::TestSumThreshold::test_sum_threshold_time PASSED
tricolour/tests/test_flagging.py::TestSumThreshold::test_sum_threshold_frequency PASSED
tricolour/tests/test_flagging.py::TestSumThreshold::test_sum_threshold_existing PASSED
tricolour/tests/test_flagging.py::TestSumThresholdFlagger::test_get_flags PASSED
tricolour/tests/test_flagging.py::TestSumThresholdFlagger::test_get_flags_single_chunk PASSED
tricolour/tests/test_flagging.py::TestSumThresholdFlagger::test_get_flags_many_chunks PASSED
tricolour/tests/test_flagging.py::TestSumThresholdFlagger::test_get_flags_average_freq PASSED
tricolour/tests/test_flagging.py::TestSumThresholdFlagger::test_get_flags_iterations SKIPPED
tricolour/tests/test_flagging.py::TestSumThresholdFlagger::test_get_flags_all_flagged PASSED
tricolour/tests/test_flagging.py::TestSumThresholdFlagger::test_get_flags_all_flagged_average_freq PASSED
tricolour/tests/test_flagging.py::TestSumThresholdFlagger::test_variable_noise PASSED
tricolour/tests/test_flagging_additional.py::FLAKE8 PASSED
tricolour/tests/test_flagging_additional.py::test_flag_nans_and_zeros PASSED
tricolour/tests/test_flagging_additional.py::test_flag_autos PASSED
tricolour/tests/test_flagging_additional.py::test_apply_static_mask PASSED
tricolour/tests/test_packing.py::FLAKE8 PASSED
tricolour/tests/test_packing.py::test_vis_and_flag_packing[numpy] PASSED
tricolour/tests/test_packing.py::test_vis_and_flag_packing[zarr-disk] PASSED
tricolour/tests/test_stokes.py::FLAKE8 PASSED
tricolour/tests/test_stokes.py::test_unpolarised_intensity[stokes0] PASSED
tricolour/tests/test_stokes.py::test_unpolarised_intensity[stokes1] PASSED
tricolour/tests/test_stokes.py::test_unpolarised_intensity[stokes2] PASSED
tricolour/tests/test_stokes.py::test_unpolarised_intensity[stokes3] PASSED
tricolour/tests/test_stokes.py::test_polarised_intensity[stokes0] PASSED
tricolour/tests/test_stokes.py::test_polarised_intensity[stokes1] PASSED
tricolour/tests/test_stokes.py::test_polarised_intensity[stokes2] PASSED
tricolour/tests/test_stokes.py::test_polarised_intensity[stokes3] PASSED
tricolour/tests/test_window_statistics.py::FLAKE8 PASSED
tricolour/tests/test_window_statistics.py::test_window_statistics[ddids0-field_names0-scan_nrs0] PASSED

==================================== ERRORS ====================================
________________ ERROR at setup of test_mean_chisq[360-1000.0] _________________

request = <SubRequest 'flagged_ms' for <Function test_mean_chisq[360-1000.0]>>
tmp_path_factory = TempPathFactory(_given_basetemp=None, _trace=<pluggy._tracing.TagTracerSub object at 0x7ff66f4f0c18>, _basetemp=PosixPath('/tmp/pytest-of-root/pytest-0'))

    @pytest.fixture(params=[360], scope="module")
    def flagged_ms(request, tmp_path_factory):
        """
        fixture yielding an MS flagged by tricolour
        """
        try:
            tarred_ms_filename = os.environ["TRICOLOUR_TEST_MS"]
        except KeyError:
            tar_dir = tmp_path_factory.mktemp("tar-download")
            tarred_ms_filename = os.path.join(tar_dir, "test_data.tar.gz")
    
            _download_file_from_google_drive(_GOOGLE_FILE_ID, tarred_ms_filename)
    
        tmp_path = str(tmp_path_factory.mktemp('data'))
    
        # Open and extract tarred ms
        tarred_ms = tarfile.open(tarred_ms_filename)
        tarred_ms.extractall(tmp_path)
    
        # Set up our paths
        ms_filename = pjoin(tmp_path, _MS_FILENAME)
        test_directory = os.path.dirname(__file__)
    
        args = ['tricolour',
                '-fs', 'total_power',
                '-c', os.path.join(test_directory, 'custom.yaml'),
                ms_filename]
    
        # Flag the MS, waiting for timeout period to expre
        p = subprocess.Popen(args, env=os.environ.copy())
        delay = 1.0
        timeout = int(request.param / delay)
    
        while p.poll() is None and timeout > 0:
            time.sleep(delay)
            timeout -= delay
    
        # timeout reached, kill process if it is still rolling
        ret = p.poll()
    
        if ret is None:
            p.kill()
            ret = 99
    
        if ret == 99:
            raise RuntimeError("Test timeout reached. Killed flagger")
        elif ret != 0:
>           raise RuntimeError("Tricolour exited with non-zero return code")
E           RuntimeError: Tricolour exited with non-zero return code

tricolour/tests/test_acceptance.py:104: RuntimeError
________________ ERROR at setup of test_max_chisq[360-10000.0] _________________

request = <SubRequest 'flagged_ms' for <Function test_mean_chisq[360-1000.0]>>
tmp_path_factory = TempPathFactory(_given_basetemp=None, _trace=<pluggy._tracing.TagTracerSub object at 0x7ff66f4f0c18>, _basetemp=PosixPath('/tmp/pytest-of-root/pytest-0'))

    @pytest.fixture(params=[360], scope="module")
    def flagged_ms(request, tmp_path_factory):
        """
        fixture yielding an MS flagged by tricolour
        """
        try:
            tarred_ms_filename = os.environ["TRICOLOUR_TEST_MS"]
        except KeyError:
            tar_dir = tmp_path_factory.mktemp("tar-download")
            tarred_ms_filename = os.path.join(tar_dir, "test_data.tar.gz")
    
            _download_file_from_google_drive(_GOOGLE_FILE_ID, tarred_ms_filename)
    
        tmp_path = str(tmp_path_factory.mktemp('data'))
    
        # Open and extract tarred ms
        tarred_ms = tarfile.open(tarred_ms_filename)
        tarred_ms.extractall(tmp_path)
    
        # Set up our paths
        ms_filename = pjoin(tmp_path, _MS_FILENAME)
        test_directory = os.path.dirname(__file__)
    
        args = ['tricolour',
                '-fs', 'total_power',
                '-c', os.path.join(test_directory, 'custom.yaml'),
                ms_filename]
    
        # Flag the MS, waiting for timeout period to expre
        p = subprocess.Popen(args, env=os.environ.copy())
        delay = 1.0
        timeout = int(request.param / delay)
    
        while p.poll() is None and timeout > 0:
            time.sleep(delay)
            timeout -= delay
    
        # timeout reached, kill process if it is still rolling
        ret = p.poll()
    
        if ret is None:
            p.kill()
            ret = 99
    
        if ret == 99:
            raise RuntimeError("Test timeout reached. Killed flagger")
        elif ret != 0:
>           raise RuntimeError("Tricolour exited with non-zero return code")
E           RuntimeError: Tricolour exited with non-zero return code

tricolour/tests/test_acceptance.py:104: RuntimeError
_________________ ERROR at setup of test_flag_count[360-0.65] __________________

request = <SubRequest 'flagged_ms' for <Function test_mean_chisq[360-1000.0]>>
tmp_path_factory = TempPathFactory(_given_basetemp=None, _trace=<pluggy._tracing.TagTracerSub object at 0x7ff66f4f0c18>, _basetemp=PosixPath('/tmp/pytest-of-root/pytest-0'))

    @pytest.fixture(params=[360], scope="module")
    def flagged_ms(request, tmp_path_factory):
        """
        fixture yielding an MS flagged by tricolour
        """
        try:
            tarred_ms_filename = os.environ["TRICOLOUR_TEST_MS"]
        except KeyError:
            tar_dir = tmp_path_factory.mktemp("tar-download")
            tarred_ms_filename = os.path.join(tar_dir, "test_data.tar.gz")
    
            _download_file_from_google_drive(_GOOGLE_FILE_ID, tarred_ms_filename)
    
        tmp_path = str(tmp_path_factory.mktemp('data'))
    
        # Open and extract tarred ms
        tarred_ms = tarfile.open(tarred_ms_filename)
        tarred_ms.extractall(tmp_path)
    
        # Set up our paths
        ms_filename = pjoin(tmp_path, _MS_FILENAME)
        test_directory = os.path.dirname(__file__)
    
        args = ['tricolour',
                '-fs', 'total_power',
                '-c', os.path.join(test_directory, 'custom.yaml'),
                ms_filename]
    
        # Flag the MS, waiting for timeout period to expre
        p = subprocess.Popen(args, env=os.environ.copy())
        delay = 1.0
        timeout = int(request.param / delay)
    
        while p.poll() is None and timeout > 0:
            time.sleep(delay)
            timeout -= delay
    
        # timeout reached, kill process if it is still rolling
        ret = p.poll()
    
        if ret is None:
            p.kill()
            ret = 99
    
        if ret == 99:
            raise RuntimeError("Test timeout reached. Killed flagger")
        elif ret != 0:
>           raise RuntimeError("Tricolour exited with non-zero return code")
E           RuntimeError: Tricolour exited with non-zero return code

tricolour/tests/test_acceptance.py:104: RuntimeError
______________ ERROR at setup of test_bandwidth_flagged[360-0.4] _______________

request = <SubRequest 'flagged_ms' for <Function test_mean_chisq[360-1000.0]>>
tmp_path_factory = TempPathFactory(_given_basetemp=None, _trace=<pluggy._tracing.TagTracerSub object at 0x7ff66f4f0c18>, _basetemp=PosixPath('/tmp/pytest-of-root/pytest-0'))

    @pytest.fixture(params=[360], scope="module")
    def flagged_ms(request, tmp_path_factory):
        """
        fixture yielding an MS flagged by tricolour
        """
        try:
            tarred_ms_filename = os.environ["TRICOLOUR_TEST_MS"]
        except KeyError:
            tar_dir = tmp_path_factory.mktemp("tar-download")
            tarred_ms_filename = os.path.join(tar_dir, "test_data.tar.gz")
    
            _download_file_from_google_drive(_GOOGLE_FILE_ID, tarred_ms_filename)
    
        tmp_path = str(tmp_path_factory.mktemp('data'))
    
        # Open and extract tarred ms
        tarred_ms = tarfile.open(tarred_ms_filename)
        tarred_ms.extractall(tmp_path)
    
        # Set up our paths
        ms_filename = pjoin(tmp_path, _MS_FILENAME)
        test_directory = os.path.dirname(__file__)
    
        args = ['tricolour',
                '-fs', 'total_power',
                '-c', os.path.join(test_directory, 'custom.yaml'),
                ms_filename]
    
        # Flag the MS, waiting for timeout period to expre
        p = subprocess.Popen(args, env=os.environ.copy())
        delay = 1.0
        timeout = int(request.param / delay)
    
        while p.poll() is None and timeout > 0:
            time.sleep(delay)
            timeout -= delay
    
        # timeout reached, kill process if it is still rolling
        ret = p.poll()
    
        if ret is None:
            p.kill()
            ret = 99
    
        if ret == 99:
            raise RuntimeError("Test timeout reached. Killed flagger")
        elif ret != 0:
>           raise RuntimeError("Tricolour exited with non-zero return code")
E           RuntimeError: Tricolour exited with non-zero return code

tricolour/tests/test_acceptance.py:104: RuntimeError
=============================== warnings summary ===============================
../usr/local/lib/python3.6/dist-packages/_pytest/config/__init__.py:1233
  /usr/local/lib/python3.6/dist-packages/_pytest/config/__init__.py:1233: PytestConfigWarning: Unknown config option: collect_ignore
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
ERROR tricolour/tests/test_acceptance.py::test_mean_chisq[360-1000.0] - Runti...
ERROR tricolour/tests/test_acceptance.py::test_max_chisq[360-10000.0] - Runti...
ERROR tricolour/tests/test_acceptance.py::test_flag_count[360-0.65] - Runtime...
ERROR tricolour/tests/test_acceptance.py::test_bandwidth_flagged[360-0.4] - R...
======== 75 passed, 1 skipped, 1 warning, 4 errors in 91.13s (0:01:31) =========
The command '/bin/sh -c py.test --flake8 -s -vvv .' returned a non-zero code: 1
Build step 'Execute shell' marked build as failure
Sending e-mails to: [email protected]
Finished: FAILURE
@bennahugo bennahugo added the bug Something isn't working label Jan 18, 2021
@smasoka
Copy link
Contributor

smasoka commented Jan 18, 2021

I think this is related to dask update on computing meta.
On the branch #76 @sjperkins fixed it with this commit

@sjperkins
Copy link
Member

I think this is related to dask update on computing meta.
On the branch #76 @sjperkins fixed it with this commit

Yes, this is probably it. It'd require a dask-ms release to fix, which I plan todo before the end of the month.

@sjperkins sjperkins mentioned this issue Feb 24, 2021
Closed
@sjperkins
Copy link
Member

Closed by #79

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sjperkins @bennahugo @smasoka