Common neutrino telescope alert schema #236

blaufuss · 2025-01-22T03:43:39Z

Description

This PR adds a common GCN alert schema for alerts from neutrino telescopes.

It also reimplements the IceCube Gold/Bronze track alerts using this common schema, with example.

KM3Net will also implement their alerts using this common base schema.

We are happy to move the location of the shared base schema as determined by GCN dev team.

…Bronze track alerts

…example

0 - initial 1 - update 1 2 - update 2...

gcn/notices/neutrino_telescope/NeutrinoAlert.schema.json

blaufuss · 2025-01-30T15:56:04Z

Please advise on next steps to move this forward, thanks

Vidushi-GitHub · 2025-02-10T05:11:14Z

Hi @blaufuss, we decided to create a template folder alongside the core schema to host generic schema examples.

blaufuss · 2025-02-10T13:02:40Z

thanks, Is this already in main?

Vidushi-GitHub · 2025-02-12T01:41:21Z

thanks, Is this already in main?

Soon will be, see PR: #243

blaufuss · 2025-02-13T03:06:44Z

thanks, Is this already in main?

Soon will be, see PR: #243

I guess template means something. this is just a common unified schema. But I'm ok if that's where it should go. Please advise when this is merged to main and I will update my PR

Vidushi-GitHub · 2025-02-13T22:21:56Z

thanks, Is this already in main?

Soon will be, see PR: #243

I guess template means something. this is just a common unified schema. But I'm ok if that's where it should go. Please advise when this is merged to main and I will update my PR

Sure, I will let you know.

Vidushi-GitHub · 2025-02-18T20:57:41Z

Hi @blaufuss , we discussed this in the GCN meeting.
Please add a neutrino directory parallel to IceCube for the generic neutrino schema, and add it as gcn.notices.neutrino.alert.schema.json.

blaufuss · 2025-02-19T22:37:07Z

Hi @blaufuss , we discussed this in the GCN meeting. Please add a neutrino directory parallel to IceCube for the generic neutrino schema, and add it as gcn.notices.neutrino.alert.schema.json.

This is done

gcn/notices/icecube/gold_bronze_track_alerts.schema.json

gcn/notices/neutrino/Alert.schema.json

Vidushi-GitHub · 2025-03-07T18:43:40Z

gcn/notices/icecube/gold_bronze_track_alerts.example.json

-  "alert_type": "update",
+  "alert_type": "initial",
+  "alert_tense": "current",
+  "analysis_pipeline": "IceCube Bronze Track alert",


Hi @blaufuss , we were checking this schema with our upcoming base schema suggestion.
Everything looks consistent, kindly add:
(a) record_number, as, initial, update are already in use
(b) Instead of pipeline, you may add:
"notice_type": {
"enum": ["field_1", "field_2", "field_3"],
"description": "Defines the category or type of notice and serves as its title. It is recommended when a single schema is used to produce multiple notice types.
}

The enum could be ["gold","bronze"]. This could be in addition to the pipeline, if it makes sense to have both.

A few comments

(b) "analysis pipeline" is intended to be a name of the analysis that generated the alert. And is coming
from the base schema. The gcn kafka labels should be the determinate?
And to migrate this to a enum means changing this enum for each new alert type in any experiment if I understand correctly?

(a) I don't understand the comment. What record number are you referring to?

Hi @jracusin @Vidushi-GitHub I would advocate in favor of not using enum in "analysis _pipeline" since the pipelines are specific to each experiments (the purpose is similar to the Ligo-Virgo-Kagra "Pipeline" field).

About a "notice_type" field, it is interesting, but having it, with enum type, in a Neutrino.Alert base schema may not be relevant because here also each experiment can have their own notice_type. However I see an interest of having this field free in one of the core shema.

For the record_number, you mean the one of the core Reporter.schema.json? In what sense do you suggest to use it?

record_number - this is an integer that should be incremented if more than one record might be sent for a particular neutrino. For example, an initial alert, then an update or a retraction. The idea is that the user will know which is the most recent one, in case they end up ingested out of order. If there will never be an update, then it's not needed.

notice_type - we didn't put this in the core schema, but are adding it to the documentation (nasa-gcn/gcn.nasa.gov#2909), because it's custom to each observatory. This could go in the IceCube and KM3Net-specific schema, not in this base neutrino schema.

in response to this guidance, I've made several changes:

added the "record_number" to shared neutrino alert class

removed "analysis_pipeline" from shared neutrino alert class

added "notice_type" to IceCube specific alert class with an enum including the Gold Track, Bronze Track and Cascade Alert as enum members (first set of alerts we'd re-implement in GCN Kafka).
** per guidance these would be in the same kafka topic I believe with different notice types.

Given the icecube alert class a more generic name (old one was Gold/Bronze specific: single_neutrino_alerts

Please let us know if there's anything else before this can be approved.

Thanks, that's all much improved. We just posted a few more minor comments. Once those are resolved, we'll get this merged, and tag a new minor update version of the schema.

Remove common "analysis_pipeline" Add notice_type to IceCube specific format Add record_number for ordered message tracking to Alert schema

…lert schema that can be used by all alert streams in IceCube

jracusin · 2025-03-14T18:31:29Z

gcn/notices/icecube/single_neutrino_alerts.example.json

@@ -13,6 +20,6 @@
  "healpix_url": "https://roc.icecube.wisc.edu/public/",


Do you intend to have this link to an individual healpix map associated with this event, or just this general link? The actual map would be more useful.

Updating to include a full example that's available online

gcn/notices/icecube/single_neutrino_alerts.example.json

record_num starts at 1 and full path to good example FITS

… alerts

blaufuss · 2025-03-28T16:19:20Z

@jracusin I think I've addressed the open issues, please let me know if I missed anything.

Vidushi-GitHub · 2025-03-14T18:25:05Z

gcn/notices/icecube/single_neutrino_alerts.schema.json

+    },
+    "notice_type": {
+      "enum": ["Gold Track Alert", "Bronze Track Alert", "Cascade Alert"],
+      "description": "Type of IceCube Alert",


Could you add more description or paper reference in the description?

My expectation was to add this to the GCN website documentation. The enum description doesn't seem
like the correct place to describe the selections behind each enumeration. Please let me know if I'm misreading what you're talking about...

A brief description of the basis of Gold, bronze, and Cascade alerts, "Type of IceCube Alerts," looks vague.
Sure, add the detailed information on the GCN website as well.

I expanded the description of this. Each selection pipeline does have it's own unique backing publication, but do feel this is best in the details documentation rather than trying to fit them into the schema. Does this work for you?

Otherwise, maybe you have an example of what more you want? Each pipeline will have a unique reference and detailed selection

gcn/notices/icecube/single_neutrino_alerts.schema.json

gcn/notices/neutrino/Alert.schema.json

codecov · 2025-03-31T17:43:48Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (main@4bfa70c). Learn more about missing BASE report.

Additional details and impacted files

@@          Coverage Diff          @@
##             main   #236   +/-   ##
=====================================
  Coverage        ?      0           
=====================================
  Files           ?      0           
  Lines           ?      0           
  Branches        ?      0           
=====================================
  Hits            ?      0           
  Misses          ?      0           
  Partials        ?      0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

lpsinger · 2025-03-31T17:44:13Z

gcn/notices/neutrino/Alert.schema.json

+    "alert_topology": {
+      "description": "Event topology of the alert",
+      "enum": ["Track", "Shower", "Multiplet"]
+    }


Is this really called topology? It sounds like morphology.

It is the standard terminology in the field. e.g.
https://gcn.gsfc.nasa.gov/doc/High_Energy_Neutrino_Cascade_Alerts.pdf
https://icecube.wisc.edu/news/research/2020/02/new-insights-into-astrophysical-neutrino-flux/
https://arxiv.org/abs/1903.04175

Weird. Thanks for checking.

blaufuss added 5 commits January 8, 2025 16:02

Add base neutrino telescope alert class, and use it for IceCube Gold/…

e309b7d

…Bronze track alerts

Use correct name for n_events

915bb11

Move Neutrino_energy to icecube specific, include "tense" in icecube …

30560e4

…example

Remove unused test item

523b61d

Add unique ID matching revision number to IceCube specific ID...

3701584

0 - initial 1 - update 1 2 - update 2...

jracusin reviewed Jan 23, 2025

View reviewed changes

gcn/notices/neutrino_telescope/NeutrinoAlert.schema.json Outdated Show resolved Hide resolved

jracusin reviewed Jan 23, 2025

View reviewed changes

gcn/notices/neutrino_telescope/NeutrinoAlert.schema.json Outdated Show resolved Hide resolved

jracusin reviewed Jan 23, 2025

View reviewed changes

gcn/notices/neutrino_telescope/NeutrinoAlert.schema.json Outdated Show resolved Hide resolved

Remove explicit list of items in description fields

a454a00

Vidushi-GitHub self-requested a review February 10, 2025 05:11

Vidushi-GitHub mentioned this pull request Feb 12, 2025

Add templates folder and sample schema #243

Closed

Move location of GCN schema for common neutrino alert

aea811f

Vidushi-GitHub reviewed Feb 20, 2025

View reviewed changes

gcn/notices/icecube/gold_bronze_track_alerts.schema.json Outdated Show resolved Hide resolved

Vidushi-GitHub requested a review from lpsinger February 20, 2025 23:51

jracusin reviewed Mar 7, 2025

View reviewed changes

gcn/notices/neutrino/Alert.schema.json Outdated Show resolved Hide resolved

Update base schema description field as request, it's not a topic

a018ccb

Vidushi-GitHub reviewed Mar 7, 2025

View reviewed changes

jracusin requested a review from VincentLuminy March 7, 2025 19:00

VincentLuminy approved these changes Mar 13, 2025

View reviewed changes

blaufuss added 2 commits March 13, 2025 10:59

Update based on GCN team review:

7e38800

Remove common "analysis_pipeline" Add notice_type to IceCube specific format Add record_number for ordered message tracking to Alert schema

Rename the "gold_bronze" alerts to a more generic "single neutrino" a…

67caa98

…lert schema that can be used by all alert streams in IceCube

blaufuss requested a review from jracusin March 13, 2025 15:12

blaufuss requested review from VincentLuminy and Vidushi-GitHub March 13, 2025 15:12

VincentLuminy approved these changes Mar 14, 2025

View reviewed changes

VincentLuminy mentioned this pull request Mar 14, 2025

KM3NeT notice schema #253

Open

jracusin reviewed Mar 14, 2025

View reviewed changes

gcn/notices/icecube/single_neutrino_alerts.example.json Outdated Show resolved Hide resolved

jracusin mentioned this pull request Mar 14, 2025

IceCube single_neutrino_alerts nasa-gcn/gcn.nasa.gov#2941

Open

6 tasks

blaufuss added 2 commits March 28, 2025 12:16

Use "pipeline" name over notice_type as enum.

d57ebb6

record_num starts at 1 and full path to good example FITS

improve title and description to better fit the generic nature of the…

8c02678

… alerts

blaufuss requested a review from jracusin March 28, 2025 16:20

Vidushi-GitHub requested changes Mar 28, 2025

View reviewed changes

blaufuss added 2 commits March 28, 2025 14:11

Add some spacing in the name...no need to be stingy

8de3684

Don't skimp on whitespace

64ecff8

blaufuss requested a review from Vidushi-GitHub March 28, 2025 18:22

Expand the description of the pipeline entry

dddcf62

Vidushi-GitHub approved these changes Mar 28, 2025

View reviewed changes

jracusin approved these changes Mar 31, 2025

View reviewed changes

Vidushi-GitHub merged commit a15ff21 into nasa-gcn:main Mar 31, 2025
11 checks passed

lpsinger reviewed Mar 31, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Common neutrino telescope alert schema #236

Common neutrino telescope alert schema #236

blaufuss commented Jan 22, 2025

blaufuss commented Jan 30, 2025

Vidushi-GitHub commented Feb 10, 2025

blaufuss commented Feb 10, 2025

Vidushi-GitHub commented Feb 12, 2025

blaufuss commented Feb 13, 2025

Vidushi-GitHub commented Feb 13, 2025 •

edited

Loading

Vidushi-GitHub commented Feb 18, 2025 •

edited

Loading

blaufuss commented Feb 19, 2025

Vidushi-GitHub Mar 7, 2025

jracusin Mar 7, 2025

blaufuss Mar 7, 2025

VincentLuminy Mar 11, 2025

jracusin Mar 11, 2025

blaufuss Mar 13, 2025

jracusin Mar 14, 2025

jracusin Mar 14, 2025

blaufuss Mar 28, 2025

blaufuss commented Mar 28, 2025

Vidushi-GitHub Mar 14, 2025

blaufuss Mar 28, 2025

Vidushi-GitHub Mar 28, 2025

blaufuss Mar 28, 2025

codecov bot commented Mar 31, 2025

lpsinger Mar 31, 2025

jracusin Mar 31, 2025

lpsinger Mar 31, 2025

		@@ -13,6 +20,6 @@
		"healpix_url": "https://roc.icecube.wisc.edu/public/",

Common neutrino telescope alert schema #236

Common neutrino telescope alert schema #236

Conversation

blaufuss commented Jan 22, 2025

Description

blaufuss commented Jan 30, 2025

Vidushi-GitHub commented Feb 10, 2025

blaufuss commented Feb 10, 2025

Vidushi-GitHub commented Feb 12, 2025

blaufuss commented Feb 13, 2025

Vidushi-GitHub commented Feb 13, 2025 • edited Loading

Vidushi-GitHub commented Feb 18, 2025 • edited Loading

blaufuss commented Feb 19, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

blaufuss commented Mar 28, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Mar 31, 2025

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vidushi-GitHub commented Feb 13, 2025 •

edited

Loading

Vidushi-GitHub commented Feb 18, 2025 •

edited

Loading