Merge pull request #6 from iris-liu0312/master

Editable Sidebar

Author: iris-liu0312

.gitignore

@@ -5,4 +5,4 @@ venv
 env
 dist
 temp.txt
-notes.py
+scratch.py

README.md

@@ -37,16 +37,21 @@ The tool mainly uses 3 resources:
 
 ## Usage
 
-Currently the tool has 4 sections, namely Home, Viewer, Compare, Github. GitHub links to this repository.
+Currently only the **Viewer** is available for use. **Home** is a landing page and **GitHub** links to this repository.
 
 ### Viewer
-* An **Upload Schema** button that opens a modal to upload file from the local file system.
-* A **Canvas** where the schema is represented. The canvas allows one to zoom-in and zoom-out, move one node at a time and move the entire structure. There is a **reload** icon on the top right corner of the canvas that resets the graph to its original position. Nodes can be clicked to show subtrees of children and participants.
-* A **JSON viewer** that gives the json view of the schema. This viewer allows editing of the schema which in turn updates the graph. The viewer allows 4 main funtionalities.
-  * **Copy**: A "Copy-to-Clipboard" icon shows up on every object of the json structure which allows one to copy the entire value. For object, number and string, it copies the value assigned to the property. For arrays, it copies all the entries in the list.
-  * **Add**: A "+" icon signifies adding an entry. Within objects, it expects a key and initialize it with "NULL" which can then be edited to the required value. In arrays, it expects a value in the form of an object or a string.
-  * **Edit**: Selecting "notepad-with-pen" icon allows editing the value of the respective key. This option is missing in arrays.
-  * **Delete**: Clicking on "X" icon will delete that entry in the object. Using it in array will remove the entire object and reduce the length of the list by 1.
-  
-  For an interactive [demo](https://mac-s-g.github.io/react-json-view/demo/dist/).
-* A **SideBar** opens up on the left side of the canvas giving information about the selected node. This window opens only when a node is right-clicked. It gives the details about the nodes like name, id, description, comments, explanation from TA1, importance etc. These information will only be visible if they are mentioned in the schema.
+- **Upload Schema** to upload a JSON file from your local file system, and **download** it when you are done with curation.
+- **Canvas** shows a graphical representation of the uploaded JSON file. The canvas allows for zoom and node drag-and-drop. **Reloading** the canvas is supported with a reload icon on the top right corner of the canvas. Nodes can be clicked to show subtrees of children nodes and participants.
+- **JSON Editor**[[1]](https://github.com/josdejong/jsoneditor/tree/master) showing the uploaded JSON. Editing the JSON will update the graph in the canvas. Functionalities include:
+  - **⚠ A bug crashes the website when a field with an empty value is instantiated, so *please duplicate existing fields* instead of adding, for example, empty strings when adding new information.**
+  - **Expand and Collapse All** lists and dictionaries of the JSON.
+  - **Sort contents**
+  - **History** for accidents.
+  - **Search Bar** to locate events more easily.
+  - **Drag** to move fields and their values between lists and dictionaries.
+  - **Templates** of events, children, participants, etc. for easier curation.
+  - **Duplication**
+  - **Deletion**
+- A **Sidebar** is available on the left side of the canvas giving information about the selected node. This window opens only when a node is right-clicked. It gives the details about the node, such as its name, id, description, comments, explanation from TA1, importance, etc. All information is directly taken from the JSON.
+---
+[[1]](https://github.com/josdejong/jsoneditor/tree/master) A web-based tool to view, edit, format, and validate JSON by Jos de Jong

app.py

@@ -11,14 +11,11 @@
 
 nodes = {}
 edges = []
-
-# TODO: handle multiple root nodes
+schemaJson = {}
 
 # SDF version 1.3
 schema_key_dict = {
-    'root': ['@id', 'name', 'comment', 'description', 'aka', 'qnode', 'qlabel', 'minDuration',
-             'maxDuration', 'goal', 'ta1explanation', 'importance', 'children_gate'],
-             # TODO: handle xor children_gates
+    'root': ['@id', 'name', 'comment', 'description', 'aka', 'qnode', 'qlabel', 'minDuration', 'maxDuration', 'goal', 'ta1explanation', 'importance', 'children_gate'],
     'participant': ['@id', 'roleName', 'entity'],
     'child': ['child', 'comment', 'optional', 'importance', 'outlinks', 'outlink_gate'],
     'privateData': ['@type', 'template', 'repeatable', 'importance']
@@ -100,6 +97,7 @@ def get_nodes_and_edges(schema):
     nodes = {}
     edges = []
     containers = []
+    parentless_xor = []
 
     for scheme in schema:
         # create event node
@@ -133,21 +131,25 @@ def get_nodes_and_edges(schema):
             nodes[scheme_id]['data']['_type'] = 'child'
             nodes[scheme_id]['data']['_shape'] = 'ellipse'
         # handle repeatable
-        if nodes[scheme_id]['data']['repeatable']:
+        if 'repeatable' in nodes[scheme_id]['data'] and nodes[scheme_id]['data']['repeatable']:
             edges.append(create_edge(scheme_id, scheme_id, _edge_type='child_outlink'))
 
         # participants
         if 'participants' in scheme:
             for participant in scheme['participants']:
-                _label = participant['roleName'].split('/')[-1].replace('_', '')
+                _label = participant['roleName'].split('/')[-1]
                 nodes[participant['@id']] = extend_node(create_node(participant['@id'], _label, 'participant', 'square'), participant)
 
                 edges.append(create_edge(scheme_id, participant['@id'], _edge_type='step_participant'))
 
         # children
         if 'children' in scheme:
+            gate = 'or'
+            if nodes[scheme_id]['data']['children_gate'] == 'xor':
+                gate = 'xor'
+            elif nodes[scheme_id]['data']['children_gate'] == 'and':
+                gate = 'and'
             for child in scheme['children']:
-
                 child_id = child['child']
                 # node already exists
                 if child_id in nodes:
@@ -158,7 +160,19 @@ def get_nodes_and_edges(schema):
                 # new node
                 else:                    
                     nodes[child_id] = extend_node(create_node(child_id, child['comment'], 'child', 'ellipse'), child)
-                edges.append(create_edge(scheme_id, child_id, _edge_type='step_child'))
+
+                # handle xor gate
+                if gate == 'xor':
+                    xor_id = f'{scheme_id}xor'
+                    nodes[xor_id] = create_node(xor_id, 'XOR', 'gate', 'rectangle')
+                    edges.append(create_edge(xor_id, child_id, _edge_type='child_outlink'))
+                    if 'Container' in scheme_id:
+                        edges.append(create_edge(scheme_id, xor_id, _edge_type='child_outlink'))
+                        parentless_xor.append((xor_id, child_id))
+                    else:
+                        edges.append(create_edge(scheme_id, xor_id, _edge_type='step_child'))
+
+                edges.append(create_edge(scheme_id, child_id, _edge_type='child_outlink' if gate == 'and' else 'step_child'))
 
                 # check for outlinks
                 if len(child['outlinks']):
@@ -171,22 +185,37 @@ def get_nodes_and_edges(schema):
         # handle containers, ie. connect previous node to all their successors
         edges_to_remove = []
         for container in containers:
+            edge_type = False
+            source_id = False
             for edge in edges:
                 if 'searched' in edge['data'] and edge['data']['searched']:
                     continue
                 if edge['data']['target'] == container:
                     source_id = edge['data']['source']
-                    nodes[source_id]['data']['children_gate'] = nodes[container]['data']['children_gate']
-                    edges_to_remove.append(edge)
                     edge['data']['searched'] = True
-                if edge['data']['source'] == container:
-                    edges.append(create_edge(source_id, edge['data']['target'], _edge_type='step_child'))
                     edges_to_remove.append(edge)
+                if edge['data']['source'] == container:
+                    edge_type = edge['data']['_edge_type']
                     edge['data']['searched'] = True
+                if source_id and edge_type:
+                    edges_to_remove.append(edge)
+                    edges.append(create_edge(source_id, edge['data']['target'], _edge_type=edge_type))
+                    edge_type = False
 
         for index in edges_to_remove:
             edges.remove(index)
 
+        # give xor nodes a parent node they belong to
+        parent_found = []
+        for xor_id, child_id in parentless_xor:
+            if xor_id in parent_found:
+                continue
+            for edge in edges:
+                if edge['data']['target'] == child_id and edge['data']['_edge_type'] == 'step_child':
+                    edges.append(create_edge(edge['data']['source'], xor_id, _edge_type='step_child'))
+                    parent_found.append(xor_id)
+                    break
+
         # === are these two necessary? / what are these for ===
         # TODO: entities
         # TODO: relations
@@ -204,6 +233,85 @@ def get_nodes_and_edges(schema):
 
     return nodes, edges
 
+def update_json(values):
+    """Updates JSON with values.
+
+    Parameters:
+    values (dict): contains node id, key, and value to change key to.
+    e.g. {id: node_id, key: name, value: Test}
+
+    Returns:
+    schemaJson (dict): new JSON 
+    """
+    global schemaJson
+    new_json = schemaJson
+    node_id = values['id']
+    key = values['key']
+    new_value = values['value']
+    node_type = node_id.split('/')[0].split(':')[-1]
+    array_to_modify = False
+    isRoot = new_json['events'][0]['@id'] == node_id
+
+    # what key is it?
+    # special cases
+    if key in ['@id', 'child']:
+        key = '@id'
+        array_to_modify = 'id'
+    elif key == 'importance':
+        array_to_modify = 'importance'
+    elif key == 'name':
+        array_to_modify = 'name'
+    # look for the key in schema_key_dict
+    if not array_to_modify:
+        if node_type == 'Participants':
+            array_to_modify = 'participant'
+        else:
+            for keys in schema_key_dict:
+                if key in schema_key_dict[keys]:
+                    array_to_modify = keys
+                    break
+
+    # change schemaJson
+    for scheme in new_json['events']:
+        # scheme data
+        if scheme['@id'] == node_id:
+            if array_to_modify in ['root', 'name', 'privateData']:
+                if key in schema_key_dict['privateData']:
+                    scheme['privateData'][key] = new_value
+                    break
+                else:
+                    scheme[key] = new_value
+                    if key != 'name':
+                        break
+                if isRoot:
+                    break    
+            elif array_to_modify == 'importance':
+                if isRoot:
+                    scheme['privateData'][key] = new_value
+                    break
+            elif array_to_modify == 'id':
+                scheme[key] = new_value
+                if isRoot:
+                    break
+        # participant data
+        if array_to_modify in ['participant', 'id'] and 'participants' in scheme:
+            for participant in scheme['participants']:
+                if participant['@id'] == node_id:
+                    scheme[key] = new_value
+        # children data
+        if array_to_modify in ['child', 'id', 'name'] and 'children' in scheme:
+            for child in scheme['children']:
+                if child['child'] == node_id:
+                    if array_to_modify == 'name':
+                        child['comment'] = new_value
+                    elif array_to_modify == 'id':
+                        child['child'] = new_value
+                    else:
+                        child[key] = new_value
+
+    schemaJson = new_json
+    return schemaJson
+
 def get_connected_nodes(selected_node):
     """Constructs graph to be visualized by the viewer.
 
@@ -259,12 +367,14 @@ def homepage():
 
 @app.route('/upload', methods=['POST'])
 def upload():
+    """Uploads JSON and processes it for graph view."""
     file = request.files['file']
     schema_string = file.read().decode("utf-8")
-    schemaJson = json.loads(schema_string)
-    schema = schemaJson['events']
+    global schemaJson
     global nodes
     global edges
+    schemaJson = json.loads(schema_string)
+    schema = schemaJson['events']
     nodes, edges = get_nodes_and_edges(schema)
     schema_name = schema[0]['name']
     parsed_schema = get_connected_nodes('root')
@@ -274,28 +384,39 @@ def upload():
         'schemaJson': schemaJson
     })
 
-@app.route('/node', methods=['GET'])
-def get_subtree():
-    """Gets subtree of the selected node."""
+@app.route('/node', methods=['GET', 'POST'])
+def get_subtree_or_update_node():
     if not (bool(nodes) and bool(edges)):
         return 'Parsing error! Upload the file again.', 400
-    node_id = request.args.get('ID')
-    subtree = get_connected_nodes(node_id)
-    return json.dumps(subtree)
+
+    if request.method == 'GET':        
+        """Gets subtree of the selected node."""
+        node_id = request.args.get('ID')
+        subtree = get_connected_nodes(node_id)
+        return json.dumps(subtree)
+    # it won't work and i don't know why
+    else:
+        """Posts updates to selected node and reloads schema."""
+        values = json.loads(request.data.decode("utf-8"))
+        new_json = update_json(values)
+        return json.dumps(new_json)
+        
+
 
 @app.route('/reload', methods=['POST'])
 def reload_schema():
     """Reloads schema; does the same thing as upload."""
     schema_string = request.data.decode("utf-8")
-    schemaJson = json.loads(schema_string)
-    schema = schemaJson['events']
+    global schemaJson
     global nodes
     global edges
+    schemaJson = json.loads(schema_string)
+    schema = schemaJson['events']
     nodes, edges = get_nodes_and_edges(schema)
     schema_name = schema[0]['name']
     parsed_schema = get_connected_nodes('root')
     return json.dumps({
         'parsedSchema': parsed_schema,
         'name': schema_name,
         'schemaJson': schemaJson
-    })
+    })

documentation/curation.md

@@ -0,0 +1,58 @@
+# Curation
+SDF v1.3 | Notes on curation fields and where to get them. Some descriptions from SDF documentation
+---
+- [Curation](#curation)
+  - [SDF v1.3 | Notes on curation fields and where to get them. Some descriptions from SDF documentation](#sdf-v13--notes-on-curation-fields-and-where-to-get-them-some-descriptions-from-sdf-documentation)
+  - [Events](#events)
+  - [Participants](#participants)
+  - [Children](#children)
+  - [Entities](#entities)
+  - [Relations](#relations)
+
+---
+Generally one event has at least 2 references in the JSON: as an event and as a child of some other event. The event reference gives more specific information about the event (node information), while the child reference simply links the event to its parent and its siblings (edge information).
+
+## Events
+⚠ denotes required fields.
+- ⚠ `@id`: `prefix:Events/<unique-5-digit-number>/<anything>`, The 5-digit number is unique per event and belongs to the Events list.
+    - `resin:Events/10000/resin:Events_disease-outbreak`
+- ⚠ `name`: human-readable label
+- `description` or `comment`: human-readable description.
+  - Generally only the first event uses `description`, all other events seem to use `comment`.
+  - It appears that this is where all container nodes have the comment `container node`.
+- `qnode` and `qlabel`: q-node from Wikidata. `qnode` denotes the QID and `qlabel` denotes the name, eg. [disease outbreak (Q3241045)](https://www.wikidata.org/wiki/Q3241045)
+- `minDuration` and `maxDuration`: minimum and maximum duration of the event. 
+- `goal`: string for TA2 defining when the event achieves a goal
+- (⚠) `ta1explanation`: Explanation of the event. Required for events without children.
+- `privateData`:
+  - `template`: ?
+  - `repeatable`: whether the event can occur multiple times.
+  - `importance`: [0, 1.0] represents importance. Also present in the child reference for the event.
+- (⚠) `participants`: [participants](#participants) of an event. Required when there are no children.
+- `children`: [children](#children) of an event.
+- (⚠) `children_gate`: logical processing of node's children. Can be `and`, `or`, or `xor`. Required when there are children.
+
+## Participants
+- `@id`: `prefix:Participants/<unique-5-digit-number>/event@id_participantName`; the 5-digit number is unique per participant and belongs to the Participants list.
+- `roleName`: taken from the XPO overlay. ask for the JSON.
+- `entity`: @id reference to [entity](#entities)
+
+## Children
+- `child`: @id reference to event
+- `comment`: human-readable description.
+- `optional`: whether an event is optional.
+- `importance`: [0, 1.0] represents importance of the event in the subevent.
+- `outlinks`: list of @ids of other events. Generally the next event in the sequence.
+
+## Entities
+Forms coreference links between events. For example, if two different events have participants with the same `victim` entity, they are referencing the same `victim`.
+- ⚠ `@id`: `prefix:Entities/<unique-5-digit-number>`; the 5-digit number is unique per entity and belongs to the Entities list.
+- ⚠ `name`: human-readable label
+- ⚠ `qnode` and `qlabel`: q-node from Wikidata. `qnode` denotes the QID and `qlabel` denotes the name, eg. [disease outbreak (Q3241045)](https://www.wikidata.org/wiki/Q3241045)
+
+## Relations
+Specifies event-event / entity-entity relations.
+- ⚠ `relationSubject`: @id reference to event / entity
+- ⚠ `relationPredicate`: Wikidata q-node or p-node
+- ⚠ `relationObject`: @id reference to event / entity
+- ⚠ `@id`: `prefix:Relations/<unique-5-digit-number>`; the 5-digit number is unique per entity and belongs to the Relations list.

documentation/expanded.png

@@ -0,0 +1,22 @@
+# For Development
+Where do I even start?!
+---
+- [For Development](#for-development)
+  - [Where do I even start?!](#where-do-i-even-start)
+  - [Top level](#top-level)
+  - [static/src/template](#staticsrctemplate)
+
+---
+## Top level
+- Documentation folder for documentation
+- `app.py` (Flask) for data management to process and send data to front-end
+
+## static/src/template
+- `Canvas.jsx` for the Cytoscape graph
+- `Compare.jsx` for Compare page
+- `Home.jsx` for Home page
+- `JsonEdit.jsx` for Json Editor [[1]](https://github.com/josdejong/jsoneditor/tree/master)
+- `SideBar.jsx` for Sidebar
+- `UploadModal.jsx` for Upload pop-up window
+- `Viewer.jsx` to combine all of the above into the Viewer page
+- `SchemaModal.jsx` for Schema edit pop-up window (not in use)