v0.11.0

What's Changed since v0.10.5

This is a major upgrade due to the removal of some deprecated and internally unused modules and functions.
We also changed the default of CameraCalibrator.apply_waveform_timeshift and invalid configuration options will now raise an error,
so some configs might need updates.

One major change is that the event sources now will adapt the subarray and trigger info if allowed_tels is used.

Breaking changes

• Change default of CameraCalibrator.apply_waveform_timeshift to False (#1692) @maxnoe
• Consistent use of allowed_tels (#1697) @LukasNickel
• Raise error in CameraReadout.from_name if no corresponding file can be found (#1703) @maxnoe
• Remove unused io modules, fixes #1527 (#1693) @maxnoe
• Change definition of concentration_cog, fixes #1571 (#1572) @maxnoe
• Transform warning into error for wrong config (#1658) @maxnoe
• Remove charge resolution tools, fixes #1634 (#1646) @maxnoe

New features

• Store package versions in provenance (#1685) @maxnoe
• Store used simtel R1 calibration in event.monitoring (#1691) @maxnoe
• Add start, stop, step to read_table (#1684) @maxnoe
• Support http(s):// and dataset:// for path traitlet (#1638) @maxnoe
• Add brightest_island function (#1587) @maxnoe
• Allow specifying extractors per telescope, fixes #1660 (#1661) @maxnoe
• Use new random API (#1708) @maxnoe
• Pin sphinx to 3.5 for now (#1709) @maxnoe
• Allow appending of rows to existing tables in HDF5TableWriter, fixes #1655 (#1657) @maxnoe

Bug fixes / maintenance

• Fix bugs in TwoPassWindowSum and define better unit-test (#1698) @HealthyPear
• Update documentation Makefile (#1705) @HealthyPear
• Fix hillas intersection for "missing" telescope ids, fixes #1635 (#1702) @maxnoe
• Update mailmap (#1700) @maxnoe
• Fix two numpy deprecation warnings (#1704) @maxnoe
• Fix pixel picking in mpl display, add test (#1701) @maxnoe
• Fix dl1source (#1696) @LukasNickel
• Make sure hdf5 file is closed when necessary in read_table, fixes #1694 (#1695) @maxnoe
• Treat sphinx warnings as erros, fix deploy workflow name (#1686) @maxnoe
• Do not use numba cache for lts_linear_fit (#1680) @maxnoe
• Fix syntax typo in call of TwoPassWindowSum with 2ndPass disabled (#1681) @HealthyPear
• Remove DL1Writer.setup, just properly initialize in init (#1667) @maxnoe
• Fix codecov missing coverage for cli tools (#1652) @maxnoe
• Use tqdm.auto (#1659) @maxnoe
• Require at least astropy 4.0.5, drop restriction to h5py 2.x (#1676) @maxnoe
• Fix some documentation warnings (#1677) @HealthyPear
• Create cache dir only when needed (#1670) @maxnoe
• Use conda-forge channel in README and add some notes (#1671) @maxnoe
• Update dl1writer.py docstring (#1666) @vuillaut
• Fix time_delta cleaning (#1656) @LukasNickel
• Update badge to 0.10.5 (#1650) @maxnoe
• Generalize URL for downloading test data (#1651) @HealthyPear
• Update README.rst (#1647) @orelgueta

Contributors

@HealthyPear, @LukasNickel, @maxnoe, @orelgueta and @vuillaut

v0.10.5

What's Changed since v0.10.4

• fix a bug in twopass extractor where the timing slope was wrong (#1639) @kosack
• Add missing f-string in dl1-merge to see broken filenames, fixes #1643 (#1644) @Hckjs
• Make sure each test gets a copy of event/subarray (#1637) @maxnoe

Contributors

@Hckjs, @kosack and @maxnoe

v0.10.4

This is a bugfix release for ctapipe 0.10 fixing several issues with creating and reading merged DL1 files as well as a coordinate bug in the HillasIntersection together with several minor fixes.

What's Changed since v0.10.3

• Show installed io plugins in ctapipe info, fixes #1540 (#1630) @maxnoe
• Mixed up positions in hillas_intersection direction assignment (#1620) @ParsonsRD
• List logfile and quiet help once (#1633) @nbiederbeck
• Try fixing docs version (#1628) @maxnoe
• Fix dl1 merge tool not writing correct metadata / stage1 not being able to read merged files (#1614) @maxnoe
• Remove travis-sphinx from docs dependencies (#1622) @HealthyPear
• use simpler algorithm to group telescope id ranges in info() (#1609) @kosack
• Tilted groundframe conversion (#1606) @nbiederbeck

Contributors

@HealthyPear, @ParsonsRD, @kosack, @maxnoe and @nbiederbeck

v0.10.3

This is a hotfix release for the 0.10 series. It fixes applying the stage1 tool and reading its output.
We also updated the example configuration to contain better cleaning levels. These are however still only roughly optimized.

What's Changed since v0.10.2

• fix read_table (#1605) @kosack
• Update stage1 example config with new cleaning levels (#1603) @maxnoe
• Fix error when my true image does not meet quality query (#1604) @maxnoe

Contributors

@kosack and @maxnoe

v0.10.2

What's Changed since v0.10.1

• Match data types of default values and calculated parameters (#1601) @LukasNickel
• Refactor conversions, implement possibility to transform images and peak times to int (#1586) @maxnoe
• Add corsikaio, fixes #1497 (#1602) @maxnoe
• Traverse classes attribute in classes_with_traits to enable more complete --help-all (#1580) @maxnoe
• Fix tutorial text (#1592) @LukasNickel
• Add black code formatter to environment. (#1600) @nbiederbeck
• Fix numpy deprecations warnings for numpy 1.20 (#1593) @maxnoe
• signal extractor SlidingWindowMaxSum (#1568) @jsitarek
• Remove shower max estimator, fixes #1595 (#1596) @maxnoe
• Set default of event.simulation to None to accomodate observed data (#1570) @maxnoe
• Cache files by full url/path (#1589) @maxnoe
• Mailmap update (#1581) @maxnoe
• Change DL1EventSource.is_compatible() check to meta DATA LEVEL, fixes #1578 (#1579) @Hckjs
• fix bug in CameraDisplay.overlay_moments (#1590) @kosack
• Add test for tailcuts with threshold arrays (#1582) @maxnoe
• Faster muon reconstruction using numba (#1583) @RuneDominik
• bump version of ctapipe-extra directory to v0.3.2 (#1588) @kosack
• Include more submodules in docs, fix some doc warnings (#1569) @maxnoe
• to support eventio 1.5 update dtype of true_image (#1585) @kosack
• Fix 1566, telescope parameter cli (#1567) @maxnoe

Contributors

@Hckjs, @LukasNickel, @RuneDominik, @jsitarek, @kosack, @maxnoe and @nbiederbeck

v0.10.1

This is a bug-fix and performance improvement release for v0.10.0

What's Changed since v0.10.0

Bug Fixes

• Fix handling of telescope parameters from cli for string values, fixes #1559 (#1560) @maxnoe
• minor fix to print each line of subarray info (#1561) @kosack
• correctly print subarray info for prod5 (#1557) @kosack
• Fix DL1 EventSource (#1546) @LukasNickel
• Fix dl1 event source for use in tools (#1542) @maxnoe
• Do not use hash in eq (#1549) @maxnoe
• Update MergerTool examples, adding allowed_tels and tests. Fixes #1533 and #1534 (#1551) @Hckjs
• Compare cam geom with allclose instead of exact equality (#1543) @maxnoe
• Fix missing traceback in colored formatter, fixes #1528 (#1541) @maxnoe

Refactoring and Performance Improvements

• Only convert to angle if not already (#1539) @maxnoe
• Avoid deepcopying defaults for containers in stage1 (#1538) @maxnoe
• Add cache=True to numba functions (#1537) @maxnoe
• Speed up trigger time creation in SimTelEventSource (#1536) @maxnoe

Packaging and Testing

• Move from travis to github actions (#1544) @maxnoe
• Fix badge url (#1547) @maxnoe
• Fix CI status linkes (#1550) @maxnoe
• Fix minuit version to 1.x (#1553) @maxnoe
• increase minimum astropy requirement to 4, fixes #1554 (#1555) @kosack
• Small change to test actions on fork (#1548) @maxnoe

Contributors

@Hckjs, @LukasNickel, @kosack and @maxnoe

v0.10.0.post1

v0.10.0.post1

Add v0.10.0 doi to README, test for automatic pypi deployment.

v0.10.0

What's Changed since v0.9.1

DL1 file support is much improved with DL1Writer and DL1EventSource components, which together allow one to write and read DL1 data trivially. Any code that used EventSource to read data will automatically be able to read DL1 data with no change needed. R0-DL0 data can be analyzed and written to DL1 using the Tool ctapipe-stage1 and merged together using ctapipe-merge

ctapipe-extra is no longer needed (but still supported). Test files are downloaded automatically from the data server.

The analysis and calibration has been improved, with time shift corrections now applied when available, and uncertainties on Hillas parameters.

The event_source helper function was removed and you don't need to use EventSource.from_url or EventSource.from_config anymore, using EventSource('/path/to/file') will "just work" and return the compatible event source if one is available.

New Features

• Dl1 event source (#1357) @LukasNickel
• Dl1 merge tool (#1447) @Hckjs
• Implement time_shift dl1 correction (#1463) @maxnoe
• Implementation of Hillas Parameters uncertainties (#1443) @andres-baquero
• Automatically Download test files (#1498) @maxnoe
• Make sure get_dataset_path returns Path, fix event source test, fixes #1517 (#1519) @maxnoe
• Apply time calib simtel (#1465) @maxnoe
• dump CameraReadout files in ctapipe-dump-instrument (#1451) @kosack
• Add --quiet/-q option and --logfile (#1310) @nbiederbeck
• support user table in OpticsDescription.from_name (#1358) @kosack
• Use traits for TelescopeParameter (#1473) @maxnoe
• Make data volume reducer configurable for CameraCalibrator (#1392) @maxnoe
• Create a DL1Writer and ImageProcessor to simplify stage1 (#1499) @kosack

API Changes

• Remove redundant run_array_direction from SimulationCnfig container (#1521) @maxnoe
• Remove need for from_url / from_config. Remove now unneeded event_source (#1515) @maxnoe
• Use enum for pixel shapes (#1491) @maxnoe
• Refactor simulation event info (#1495) @kosack
• Remove tels_with_data, fixes #852 (#1489) @maxnoe
• Remove need for add_subcomponent include subcomponents in Component.get_current_config (#1444) @maxnoe

Packaging and Docs

• add DOI link for v0.9.1 in readme (#1520) @kosack
• Use setuptools_scm for versioning, fixes #1333 (#1334) @maxnoe
• use 2020 resolver for pip installs on travis (#1501) @kosack
• Update maintainer info, fixes #1466 (#1485) @maxnoe
• Use eventio from conda-forge (#1486) @maxnoe
• Require python >= 3.7 (#1467) @maxnoe
• Bump traitlets to version 5 (#1472) @maxnoe

Bug Fixes

• Add removed exclude back. (#1529) @maxnoe
• Fix EventSource.__new__ already calling __init__ of source (#1523) @maxnoe
• Fix exclusion filter for pointing table (#1525) @maxnoe
• Fix parent handling in EventSource.from_config (#1514) @maxnoe
• Fix dl1 writer for observed data (#1511) @maxnoe
• reduce erroneous warnings in code style checkers (#1509) @kosack
• Fix naming of image -> true_image for simulated containers. (#1507) @nbiederbeck
• Run tools in tmpdir during tests, do not write provlog if only help was run (#1475) @maxnoe
• Make time shift corrections optional (#1492) @maxnoe
• Enable setting input_url through config, fixes #1502 (#1503) @maxnoe
• Add camera frame in TelescopeDescription.from_name (#1496) @maxnoe
• Fix bug in h5_table_to_astropy and HDF5TableWriter (#1471) @kosack
• DOC: Add black pre-commit hook, fix #1410. (#1488) @nbiederbeck
• Do not calculate errors on time fit parameters, fixes #1420 (#1487) @maxnoe
• Fix Path.validate for None (#1481) @maxnoe
• Fix enum deserialization in HDF table writer (#1483) @maxnoe
• (Re)move trigger related fields from R0/R1/DL0 Container (#1477) @maxnoe
• Do not write tels with trigger in pointing table (#1479) @maxnoe
• Add correct frame to CameraGeometry in SubArrayDescription.from_hdf, fixes #1469 (#1476) @maxnoe
• Correct pedestal before extracting the pulse, fixes #1460 (#1461) @maxnoe
• remove no-longer-used EnergyRegressor and EventClassifier (#1464) @kosack
• Fix missing config=True, add test so this never happens again (#1462) @maxnoe
• Simplify pixel log-likelihood. (#1403) @nbiederbeck

Contributors

@Hckjs, @LukasNickel, @andres-baquero, @kosack, @maxnoe and @nbiederbeck

v0.9.1

This is a primarily a bugfix release, other than one new feature: ctapipe.io.astropy_helpers.h5_table_to_astropy()

What's Changed since v0.9.0

Bug Fixes

• Fix loss of performance in TwoPassWindowSum (#1442) @HealthyPear
• Fix rewrite subarray (#1446) @maxnoe
• Return nan for non-invertible matrices in ctapipe.fitting.lts_fit (#1439) @maxnoe
• Add fits option to meta.to_dict, silences warnings (#1433) @maxnoe

Features

• add function to get astropy table from ctapipe hdf5 file (#1427) @kosack

other

• replace Release Drafter App with Workflow (#1454) @kosack

Contributors

@HealthyPear, @kosack and @maxnoe

v0.9.0

This release should be compatible with v0.8.0, but with some bug fixes, speed improvements.

Notable bug fix:

• the DL1 format generated by v0.8.0 was missing the telescope trigger time, due to a bug which was fixed in this version.

API changes:

• EventSeeker (which provides random access to event sources) now relies on the get_event_index() and get_event_id() methods to move between events, rather than an implicit __get__ function. Code that used it should change as follows:

  previously:

  event = seeker[12]  # move to event at index 12
  event = seeker["1233"] # move to event id 1233

  now:

  event = seeker.get_event_index(12)
  event = seeker.get_event_id(1233)

What's Changed since v0.8.0

• Use lts regression instead of siegel slopes for timing parameters (#1419) @maxnoe
• add column description to attrs (#1437) @kosack
• Calculate log likelihood analytically in muon intensity fit (#1390) @maxnoe
• Fix reading of optics table, write version number, fixes #1416, fixes #1359 (#1417) @maxnoe
• Remove unused code (#1378) @maxnoe
• Make reading subarray name optional, fixes reading of 0.8.0 dl1 files (#1428) @maxnoe
• Add config options to disable integration correction, fixes #1418, fixes #1376 (#1429) @maxnoe
• TailCutsDataVolumeReducer now instantiates TailCutsImageCleaner, fixes #1399 (#1404) @Hckjs
• Fix writing of camera description in subarray.to_hdf, fixes #1422 (#1423) @maxnoe
• Add missing factor of 0.5 in muon efficiency fit, fixes #1377 (#1379) @maxnoe
• cosmetic change to CameraDisplay (#1409) @kosack
• Read and write subarray to hdf methods (#1405) @maxnoe
• Fix CameraGeometry coordinate conversion for spherical frames (#1407) @kosack
• Update EventSeeker (#1401) @watsonjj
• Add support to read multiple containers from one table at once (#1374) @LukasNickel
• Fix README formatting. (#1402) @nbiederbeck
• Use sparse matrix approach for neighbor window sum (#1398) @maxnoe
• Override from_name for TelescopeComponent to be able to provide subarray (#1393) @maxnoe
• Mask for muons, fixes #1373 (#1389) @moralejo
• Make gain_selector a trait of SimTelEventSource, fixes eventsource plugins (#1385) @maxnoe
• Add missing is_simulation guards to stage1 tool (#1386) @maxnoe
• Fix missing units of MuonEfficiencyContainer defaults (#1380) @maxnoe
• Make sure event source provides R1 or DL0, fix datalevels of SimTelEventSource (#1387) @maxnoe
• Update README (DOI + install) (#1366) @vuillaut
• Improve clarity of FixedWindowSum (#1370) @watsonjj
• Add support to read from prefixed columns (#1364) @LukasNickel
• fix exclusion pattern that accidentally removed tels_with_trigger (#1369) @kosack
• Fix FACT focal length in guessing key (#1365) @maxnoe
• improve TwoPassWindowSum extractor (#1356) @kosack
• numba_of_islands (#1361) @maxnoe
• Compare .value of quantity to 0 in cocentration (#1363) @maxnoe

Contributors

@Bultako, @Hckjs, @LukasNickel, @maxnoe, @kosack, @moralejo, @nbiederbeck, @vuillaut and @watsonjj