Version 2.0.2:

- read a CSV file as an event catalog - An option flag to specify the ISC bulletin has been added to obspyDMT - Update tutorial
kasra-hosseini · Jun 1, 2017 · 2be289e · 2be289e
1 parent b618a8f
commit 2be289e
Show file tree

Hide file tree

Showing 6 changed files with 179 additions and 35 deletions.
diff --git a/README.md b/README.md
@@ -132,6 +132,30 @@ The results of some basic statistics (magnitude and depth histograms) are also g
 Note the rendering of coloured beach balls in the map inset (deepest seismicity in the foreground).
 The global map also contains beach balls rather than just simple black dots, but they do not become apparent at this zoom level.
 
+**ISC catalog**
+
+The International Seismological Centre (ISC) provides two catalogs:
+
+ * **COMPREHENSIVE bulletin** contains all events collected by the ISC, 
+   including most recent events, which are awaiting review.
+ * **REVIEWED bulletin** includes all events that have been 
+   reviewed and relocated by an ISC analyst.
+
+``--isc_catalog`` option specifies the ISC bulletin type (default: COMPREHENSIVE). Example:
+
+```bash
+obspyDMT --datapath test_isc_comprehensive --min_date 2013-01-01 --max_date 2015-01-01 --min_mag 6.5 --event_catalog ISC --isc_catalog COMPREHENSIVE --event_info --plot_seismicity
+```
+
+<p align="center">
+<img src="figures/isc_catalog_comprehensive.png" width="100%" align="middle">
+</p>
+
+Retrieval result for the same request using ``--isc_catalog REVIEWED``:
+
+<p align="center">
+<img src="figures/isc_catalog_reviewed.png" width="100%" align="middle">
+</p>
 
 ## Event-based mode
 
@@ -359,6 +383,35 @@ Print supported earthquake catalogs that can be passed as arguments to ``--event
 obspyDMT --print_event_catalogs
 ```
 
+**Read an existing local event catalog**
+
+``--read_catalog <PATH>`` option flag reads in an existing event catalog located at ``<PATH>`` and proceeds. 
+Currently supported catalog metadata formats: "CSV", "QUAKEML", "NDK", "ZMAP" (Refer to obspy documentation for details on QuakeML, NDK and ZMAP formats).
+
+**CSV format:** obspyDMT can read a CSV file as an event catalog. 
+This must be a list of comma-separated values containing some or all of the fields below, 
+one event per line:
+
+```bash
+event_number,event_id,datetime,latitude,longitude,depth,magnitude,magnitude_type,author,flynn_region,mrr,mtt,mpp,mrt,mrp,mtp,stf_func,stf_duration
+```
+
+File ``catalog.txt``, generated by obspyDMT in ``EVENTS-INFO`` subdirectory provides an example of such a file.
+
+Example:
+
+```bash
+#number,event_id,datetime,latitude,longitude,depth,magnitude,magnitude_type,author,flynn_region,mrr,mtt,mpp,mrt,mrp,mtp,stf_func,stf_duration
+1,20110311_054623.a,2011-03-11T05:46:23.200000Z,38.2963,142.498,19.7,9.1,MW,None,NAN,None,None,None,None,None,None,triangle,164.914
+```
+
+datetime, latitude, longitude, depth and magnitude are mandatory. Optional fields may be set to ```None```, as in the following example where only datetime, latitude, longitude, depth and magnitude are set:
+
+```bash
+#number,event_id,datetime,latitude,longitude,depth,magnitude,magnitude_type,author,flynn_region,mrr,mtt,mpp,mrt,mrp,mtp,stf_func,stf_duration
+1,None,2011-03-11T05:46:23.200000Z,38.2963,142.498,19.7,9.1,None,None,None,None,None,None,None,None,None,None,None
+```
+
 ## Directory structure
 
 For each request, obspyDMT creates the depicted directory tree inside the user-specified directory `datapath/`, and arranges the retrieved data either in different event directories (for event-based requests) or in chronologically named directories (for continuous requests). It also creates a directory in which a catalog of all requested events/time spans is stored. Raw waveforms, StationXML/response files and corrected waveforms are collected in sub-directories. While retrieving the data, obspyDMT creates metadata files such as station/event location files, stored in the `info/` directory of each event.

diff --git a/figures/isc_catalog_comprehensive.png b/figures/isc_catalog_comprehensive.png
diff --git a/figures/isc_catalog_reviewed.png b/figures/isc_catalog_reviewed.png
diff --git a/obspyDMT/utils/event_handler.py b/obspyDMT/utils/event_handler.py
@@ -67,15 +67,30 @@ def get_time_window(input_dics, request):
     events = []
     events_qml = Catalog(events=[])
     try:
-        events_local = read_info(input_dics)
-        if input_dics['event_catalog'].lower() == 'local':
-            if events_local == 'no_local':
-                print("[WARNING] no local event was found!")
-                if request == 'event_based':
-                    print("[WARNING] use IRIS catalog instead!")
-                    input_dics['event_catalog'] = 'IRIS'
+        if not input_dics['read_catalog']:
+            events_local = read_info(input_dics)
+            if input_dics['event_catalog'].lower() == 'local':
+                if events_local == 'no_local':
+                    print("[WARNING] no local event was found!")
+                    if request == 'event_based':
+                        print("[WARNING] use IRIS catalog instead!")
+                        input_dics['event_catalog'] = 'IRIS'
+                else:
+                    events = copy.deepcopy(events_local)
+        else:
+            events_QML, ev_already_list = \
+                read_local_events(input_dics['read_catalog'])
+            if not ev_already_list:
+                # no matter if list was passed or requested, sort catalogue,
+                # plot events and proceed
+                events_QML = sort_catalogue(events_QML)
+                events = qml_to_event_list(events_QML)
             else:
-                events = copy.deepcopy(events_local)
+                events = copy.deepcopy(events_QML)
+                events_QML = False
+            for i in range(len(events)):
+                events[i]['t1'] = events[i]['datetime'] - input_dics['preset']
+                events[i]['t2'] = events[i]['datetime'] + input_dics['offset']
 
         if events == 'no_local' or events == []:
             if request.lower() == 'event_based':
@@ -254,7 +269,7 @@ def event_info(input_dics):
                             max_mag=input_dics['max_mag'],
                             mag_type=input_dics['mag_type'],
                             req_mag_agcy='Any',
-                            rev_comp=input_dics['isc_rev_comp'])
+                            rev_comp=input_dics['isc_catalog'])
 
         elif event_switch == 'local':
             events_QML = readEvents(input_dics['read_catalog'])
@@ -296,6 +311,51 @@ def event_info(input_dics):
 
     return events, events_QML
 
+# ##################### read_local_events #####################################
+
+
+def read_local_events(catalog_add):
+    """
+    read local events
+    :param catalog_add:
+    :return:
+    """
+    try:
+        events_QML = readEvents(catalog_add)
+        return events_QML, False
+    except Exception as error:
+        try:
+            ev_csv = np.loadtxt(catalog_add, delimiter=',',
+                                comments='#', ndmin=2, dtype='object')
+            events = []
+            for i in range(np.shape(ev_csv)[0]):
+                events.append(OrderedDict(
+                    [('number', i+1),
+                     ('latitude', eval(ev_csv[i][3])),
+                     ('longitude', eval(ev_csv[i][4])),
+                     ('depth', eval(ev_csv[i][5])),
+                     ('datetime', UTCDateTime(ev_csv[i][2])),
+                     ('magnitude', eval(ev_csv[i][6])),
+                     ('magnitude_type', ev_csv[i][7]),
+                     ('author', ev_csv[i][8]),
+                     ('event_id', ev_csv[i][1]),
+                     ('origin_id', -12345),
+                     ('focal_mechanism', [eval(ev_csv[i][10]),
+                                          eval(ev_csv[i][11]),
+                                          eval(ev_csv[i][12]),
+                                          eval(ev_csv[i][13]),
+                                          eval(ev_csv[i][14]),
+                                          eval(ev_csv[i][15])
+                                          ]),
+                     ('source_duration', [ev_csv[i][16], eval(ev_csv[i][17])]),
+                     ('flynn_region', ev_csv[i][9])]))
+            return events, True
+        except Exception as error:
+            print(60*'-')
+            print('[WARNING] %s' % error)
+            print(60*'-')
+            return Catalog(events=[]), False
+
 # ##################### qml_to_event_list #####################################
 
 
@@ -315,6 +375,21 @@ def qml_to_event_list(events_QML):
             event_time_hour = '%02i' % int(event_time.hour)
             event_time_minute = '%02i' % int(event_time.minute)
             event_time_second = '%02i' % int(event_time.second)
+
+            if not hasattr(events_QML.events[i], 'preferred_mag'):
+                events_QML.events[i].preferred_mag = \
+                    events_QML.events[i].magnitudes[0].mag
+                events_QML.events[i].preferred_mag_type = \
+                    events_QML.events[i].magnitudes[0].magnitude_type
+                events_QML.events[i].preferred_author = 'None'
+            else:
+                if not hasattr(events_QML.events[i], 'preferred_author'):
+                    if events_QML.events[i].preferred_magnitude().creation_info:
+                        events_QML.events[i].preferred_author = \
+                            events_QML.events[i].preferred_magnitude().creation_info.author
+                    elif events_QML.events[i].magnitudes[0].creation_info:
+                        events_QML.events[i].preferred_author = \
+                            events_QML.events[i].magnitudes[0].creation_info.author
         except Exception as error:
             print(error)
             continue
@@ -852,22 +927,20 @@ def isc_catalog(bot_lat=-90, top_lat=90,
                                req_mag_agcy=req_mag_agcy,
                                rev_comp=rev_comp)
 
-    print("URL:\n%s" % base_url)
+    print("\nURL:\n%s\n" % base_url)
 
-    try_url = 5
-    while try_url > 0:
-        print("Try: %s" % try_url)
+    try_url = 1
+    isc_events = Catalog(events=[]) 
+    while try_url < 6:
+        print("---> Send event request (Try: %s)" % try_url)
         try:
             isc_req = urlopen(base_url)
             isc_contents = isc_req.read()
             isc_events = readEvents(isc_contents)
-            try_url = 0
+            try_url = 100
         except Exception as e:
             print("requested content from ISC:\n%s" % e)
-        try_url -= 1
-
-    isc_events = isc_events.filter("magnitude >= %s" % min_mag,
-                                   "magnitude <= %s" % max_mag)
+        try_url += 1
 
     remove_index = []
     for i in range(len(isc_events)):
@@ -881,7 +954,8 @@ def isc_catalog(bot_lat=-90, top_lat=90,
                     isc_events.events[i].magnitudes[j].magnitude_type
                 isc_events.events[i].preferred_author = \
                     isc_events.events[i].magnitudes[j].creation_info.author
-                found_mag_type = True
+                if (min_mag <= isc_events.events[i].preferred_mag <= max_mag):
+                    found_mag_type = True
                 break
         if not found_mag_type:
             remove_index.append(i)
@@ -926,7 +1000,7 @@ def isc_url_builder(search_domain='rectangular', bot_lat=-90, top_lat=90,
     """
     base_url = 'http://www.isc.ac.uk/cgi-bin/web-db-v4?'
     base_url += 'request=%s' % rev_comp
-    base_url += '&out_format=CATQuakeML'
+    base_url += '&out_format=QuakeML'
     if search_domain == 'rectangular':
         base_url += '&searchshape=RECT'
         base_url += '&bot_lat=%s' % bot_lat
@@ -957,13 +1031,17 @@ def isc_url_builder(search_domain='rectangular', bot_lat=-90, top_lat=90,
                                             end_time.second)
     base_url += '&min_dep=%s' % min_dep
     base_url += '&max_dep=%s' % max_dep
-    base_url += '&null_dep=off'
     base_url += '&min_mag=%s' % min_mag
     base_url += '&max_mag=%s' % max_mag
-    base_url += '&null_mag=off'
     base_url += '&req_mag_type=%s' % mag_type
     base_url += '&req_mag_agcy=%s' % req_mag_agcy
+
+    base_url += '&min_def='
+    base_url += '&max_def='
     base_url += '&include_links=on'
+    base_url += '&include_magnitudes=on'
+    base_url += '&include_headers=on'
+    base_url += '&include_comments=on'
     return base_url
 
 # ##################### output_shell_event ####################################

diff --git a/obspyDMT/utils/input_handler.py b/obspyDMT/utils/input_handler.py
@@ -327,6 +327,10 @@ def command_parse():
     helpmsg = "Event catalog, currently supports LOCAL, NEIC_USGS, " \
               "GCMT_COMBO, IRIS, NCEDC, USGS, " \
               "INGV, ISC, NERIES (default: LOCAL). " \
+              "'--event_catalog LOCAL' searches for an existing " \
+              "event catalog on the user's local machine, " \
+              "in the EVENTS-INFO subdirectory of --datapath <PATH>. " \
+              "This is usually a previously retrieved catalog. " \
               "Example: IRIS"
     group_ev_based.add_option("--event_catalog", action="store",
                               dest="event_catalog", help=helpmsg)
@@ -337,8 +341,12 @@ def command_parse():
                               dest="event_info", help=helpmsg)
 
     helpmsg = "Read in an existing local event catalog and proceed. " \
-              "Currently supported catalogue metadata formats: " \
-              "'QUAKEML', 'NDK, 'ZMAP'." \
+              "Currently supported catalog metadata formats: " \
+              "'CSV', 'QUAKEML', 'NDK', 'ZMAP'. " \
+              "Format of the plain text CSV (comma-separated values) is " \
+              "explained in the obspyDMT tutorial. " \
+              "Refer to obspy documentation for details on " \
+              "QuakeML, NDK and ZMAP formats. " \
               "Example: /path/to/file.ml"
     group_ev_based.add_option("--read_catalog", action="store",
                               dest="read_catalog", help=helpmsg)
@@ -388,13 +396,15 @@ def command_parse():
     group_ev_based.add_option("--event_circle", action="store",
                               dest="event_circle", help=helpmsg)
 
-    # XXXXXX
-    helpmsg = "searches the ISC Bulletin (COMPREHENSIVE) or " \
-              "searches the Reviewed ISC Bulletin (REVIEWED) - " \
-              "a subset of the ISC Bulletin, " \
-              "which is reviewed by ISC analysts."
-    group_ev_based.add_option("--isc_rev_comp", action="store",
-                              dest="isc_rev_comp", help=helpmsg)
+    helpmsg = "Search either the COMPREHENSIVE or the REVIEWED bulletin " \
+              "of the International Seismological Centre (ISC). " \
+              "COMPREHENSIVE: all events collected by the ISC, " \
+              "including most recent events that are awaiting review. " \
+              "REVIEWED: includes only events that have been " \
+              "relocated by ISC analysts. " \
+              "(default: COMPREHENSIVE). Example: 'REVIEWED'"
+    group_ev_based.add_option("--isc_catalog", action="store",
+                              dest="isc_catalog", help=helpmsg)
     parser.add_option_group(group_ev_based)
 
     # --------------- continuous time series mode
@@ -715,7 +725,7 @@ def read_input_command(parser, **kwargs):
                   'evlonmin': None, 'evlonmax': None,
                   'evlat': None, 'evlon': None,
                   'evradmin': None, 'evradmax': None,
-                  'isc_rev_comp': "COMPREHENSIVE",
+                  'isc_catalog': "COMPREHENSIVE",
                   'syngine_bg_model': 'iasp91_2s',
                   'syngine_geocentric_lat': True,
 
@@ -827,7 +837,7 @@ def read_input_command(parser, **kwargs):
     if options.version:
         print('\n\t\t' + '*********************************')
         print('\t\t' + '*        obspyDMT version:      *')
-        print('\t\t' + '*\t' + 5*' ' + '2.0.1' + '\t\t*')
+        print('\t\t' + '*\t' + 5*' ' + '2.0.2' + '\t\t*')
         print('\t\t' + '*********************************')
         print('\n')
         sys.exit(2)
@@ -1150,7 +1160,10 @@ def read_input_command(parser, **kwargs):
     input_dics['evradmax'] = options.evradmax
     input_dics['evradmin'] = options.evradmin
 
-    input_dics['isc_rev_comp'] = options.isc_rev_comp
+    if 'rev' in options.isc_catalog.lower():
+        input_dics['isc_catalog'] = 'REVIEWED'
+    else:
+        input_dics['isc_catalog'] = 'COMPREHENSIVE'
     input_dics['syngine'] = options.syngine
     input_dics['syngine_bg_model'] = options.syngine_bg_model
     if str(options.syngine_geocentric_lat).lower() in ['false']:

diff --git a/setup.py b/setup.py
@@ -5,7 +5,7 @@
 
 setup(
     name="obspyDMT",
-    version="2.0.1",
+    version="2.0.2",
     keywords=["obspyDMT", "obspy", "seismology", "geophysics"],
     requires=['matplotlib', 'numpy'],
     description="obspyDMT: A Python Toolbox for Retrieving, Processing and "