JSONStream Manual

The JSONStream Manual

The JSONStream framework allows to do communication between GraphStream and Gephi with the idea of "streams of graph events" in JSON protocol. There are two parts in the framework, a sender and a receiver.

Sender

The sender part is responsible for sending events from GraphStream to Gephi. GraphStream works as a client and Gephi works as a sever. Every time the graph in the GraphStream changes, a corresponding event will be send to Gephi. The Gephi handles the event and changes its graph object. In this way, the sender part works as a sink of the GraphStream graph, so it must implement the sink interface. In gs-gephi, the JSONSender class works as the sender part. It implements the sink interface, mainly the following methods: .. class::code-java

void nodeAdded(sourceId, timeId, nodeId);
void nodeRemoved(sourceId, timeId, nodeId);
void edgeAdded(sourceId, timeId, edgeId, fromNodeId, toNodeId, directed);
void edgeRemoved(sourceId, timeId, edgeId);
void graphCleared(sourceId, timeId);
void graphAttributeAdded(sourceId, timeId, attribute, value);
void graphAttributeChanged(sourceId, timeId, attribute, oldValue, newValue);
void graphAttributeRemoved(sourceId, timeId, attribute);
void nodeAttributeAdded(sourceId, timeId,  nodeId, attribute, value);
void nodeAttributeChanged(sourceId, timeId, nodeId, attribute, oldValue, newValue);
void nodeAttributeRemoved(sourceId, timeId, nodeId, attribute);
void edgeAttributeAdded(sourceId, timeId, edgeId, attribute, value);
void edgeAttributeChanged(sourceId, timeId, edgeId, attribute, oldValue, newValue);
void edgeAttributeRemoved(sourceId, timeId, edgeId, attribute);

Each method corresponding to one graph event, which includes graph element event and attribute event. In each method, we first encode the event message into a JSON string, then send to Gephi. In order to send events to Gephi, we connect to Gephi and use "updateGraph" operation. The corresponding URL is "http://host:port/workspace0?operation=updateGraph". The host and port must match with the Gephi sever and the workspace0 is a destination workspace of Gephi, for example an URL can be "http://127.0.0.1:8080/workspace0?operation=updateGraph". The Gephi server and client is built with the Graph Streaming API in the Gephi-plugin. So if we want JSONSender to work rightly, we must run Graph Streaming plugin and start the Graph Streaming Server of the current workspace on Gephi side.

To make the sender part works, we just need to create a JSONSender instance, and plugin it as a sink of the graph in GS. Since then, every change in GS graph will be dynamically sent to Gephi. The following is an example.

Example

import org.graphstream.graph.Graph;
import org.graphstream.graph.implementations.SingleGraph;
import org.graphstream.stream.gephi.JSONSink;

/**
 * a example of using the JSONStream sender
 */
 public class ExampleJSONSender {

   public static void main(String args[]) {

       Graph graph = new SingleGraph("Tutorial 1");

       JSONSender sender = new JSONSender("localhost", 8080, "workspace0");
       graph.addSink(sender);

       //graph.display();

       graph.addNode("A");
       graph.addNode("B");
       graph.addNode("C");
       sleep();
       graph.addEdge("AB", "A", "B");
       graph.addEdge("BC", "B", "C");
       graph.addEdge("CA", "C", "A");
       sleep();

       //graph.clear();
   }

   protected static void sleep() {
       try {
           Thread.sleep(1000);
       } catch (Exception e) {}
   }
}

Attention: to run ExampleJSONSender, the Graph Streaming server in Graph Streaming plugin must be started. The following operations must be done step by step:

1. Run the Graph Streaming plugin project
2. Create an empty workspace by choose menu "File/New Project" (the new workspace must be named "workspace0", if not we must change the workspace name in the example)
3. Go to the Streaming window and click on Server/Start
4. Run ExampleJSONSender

Receiver

The receiver part is responsible for receiving events from Gephi. It listens to Gephi and waits for events. Every time the graph in the Gephi changes, a corresponding event will be send to GraphStream. Then the GraphStream handles the event and changes its graph object. In this way, the receiver part works as a source of the GraphStream graph. In gs-gephi project, JSONReceiver class works as the receiver part. It has a field currentStream, an instance of ThreadProxyPipe, to deal with the events received from Gephi. Every time an event comes, the corresponding method of currentStream with matched event type (ADD, REMOVE or CHANGE) and element type (GRAPH, NODE or EDGE) will be called. Then if we want to view those events in GraphStream graph, we just need to plugin currentStream (access by method "getStream") as a sink of the graph. As for as the listening to Gephi events, we use a URL within "getGraph" operation to connect to Gephi. The corresponding URL is "http://host:port/workspace0?operation=getGraph", where the host and port must match with the Gephi Graph Streaming server, and the workspace0 is the current workspace of the Gephi server. The following is a receiver example.

Example

import org.graphstream.graph.Graph;
import org.graphstream.graph.implementations.MultiGraph;
import org.graphstream.stream.gephi.JSONReceiver;
import org.graphstream.stream.thread.ThreadProxyPipe;

/**
 * a example of using the JSONStream receiver
 */
public class ExampleJSONReceiver {

   public static void main(String[] args) {
       // TODO Auto-generated method stub
       // ----- On the receiver side -----
       //
       // a graph that will display the received events
       Graph g = new MultiGraph("G",false,true);
       g.display();
       // the receiver that waits for events
       JSONReceiver receiver = new JSONReceiver("localhost", 8080,"workspace0");
       ThreadProxyPipe pipe = receiver.getStream();
       // plug the pipe to the sink of the graph
       pipe.addSink(g);
       // The receiver pro-actively checks for events on the ThreadProxyPipe
       while (true) {
           pipe.pump();
       }
   }
}

Attention: to run ExampleJSONReceiver, the following operations must be done step by step:

1. Run the Graph Streaming plugin project
2. Load a file by choose menu "File/Open" (the corresponding workspace should be named "workspace0", if not we must change the workspace name in the example)
3. Go to the Streaming window and click on Server/Start
4. Run ExampleJSONReceiver

Tutorials

We will give two tutorials to show how to do Gephi-GraphStream real-time connection.

GraphSender

This tutorial shows loading a graph in GraphStream and dynamically displays it on a Gephi workspace. This process must be real-time. To do this, we need the following operations.

1. create a graph instance
2. create a JSONSender instance
3. plugin the JSONSender instance as a sink of the graph instance.
4. generate the graph or load the graph from file

import org.graphstream.graph.Graph;
import org.graphstream.graph.implementations.MultiGraph;
import org.graphstream.stream.gephi.JSONSender;

/*
 * A simple example to show loading or generating a graph in GS side and send it to Gephi
 * @author Min WU
 */
public class GraphSender {

   public static void main(String args[]) {
       Graph graph = new MultiGraph("Tutorial 1 GraphSender"); //1

       graph.display();

       JSONSender sender = new JSONSender("localhost", 8080, "workspace0"); //2

       // 3. plug the graph to the sender so that graph events can be sent automatically
       graph.addSink(sender);

       // 4. generate the graph on the client side
       String style = "node{fill-mode:plain;fill-color:#567;size:6px;}";
       graph.addAttribute("stylesheet", style);
       graph.addAttribute("ui.antialias", true);
       graph.addAttribute("layout.stabilization-limit", 0);
       for (int i = 0; i < 500; i++) {
           graph.addNode(i + "");
           if (i > 0) {
               graph.addEdge(i + "-" + (i - 1), i + "", (i - 1) + "");
               graph.addEdge(i + "--" + (i / 2), i + "", (i / 2) + "");
           }
       }
   }
}

Attention: to make this tutorial works, some operations must be done step by step:

1. Run the Graph Streaming plugin project
2. Create an empty workspace by choose menu "File/New Project" (the new workspace must be named "workspace0", if not we must change the workspace name in the tutorial).
3. Go to Layout window, choose Force Atlas layout and run the layout.
4. Go to Streaming window and start the Server.
5. Run GraphSender

LinLog Layout

Lin-Log layout in GraphStream is dedicated to find communities in a graph. But currently Gephi doesn't support this layout. This tutorial will show you doing LinLog layout in GraphStream and sending the layout information to Gephi in real time by using the gs-gephi connection. The detailed user case is that we first load a graph in Gephi, display it and do some algorithm and so on. Then at some point, we send the graph to GraphStream and do Lin-Log layout for the graph on GraphStream side. Meanwhile we need to view the layout process on Gephi side in real time. To achieve the user case, on GraphStream side we need to do the following steps.

1. create a graph instance
2. create a JSONReceiver instance
3. get the ThreadProxyPipe instance and plugin the graph instance as an ElementSink of the pipe instance.
4. do LinLog layout for the graph
5. Build a new thread, create a JSONSender instance and plugin it as a sink of the graph layout.

import java.io.IOException;
import org.graphstream.graph.Graph;
import org.graphstream.graph.implementations.MultiGraph;
import org.graphstream.stream.GraphParseException;
import org.graphstream.ui.swingViewer.Viewer;
import org.graphstream.stream.ProxyPipe;
import org.graphstream.stream.thread.ThreadProxyPipe;
import org.graphstream.ui.layout.springbox.implementations.LinLog;
import org.graphstream.stream.gephi.JSONReceiver;
import org.graphstream.stream.gephi.JSONSender;

/**
* A tutorial to show connecting to Gephi, receive the graph,
* do LinLog layout for the graph in GS side
* and then send layout information to Gephi
* @author Min WU
*/
public class LinLogLayoutReceiver {

   public static void main(String args[]) throws IOException, GraphParseException {

       (new LinLogLayoutReceiver()).findCommunities();
   }

   //graph object
   private Graph graph;
   //view object
   private Viewer viewer;

   //LinLog layout parameter
   private LinLog layout;
   private double a = 0;
   private double r = -1.3;
   private double force = 3;

   //a proxy pipe to do interaction in the GS viewer
   private ProxyPipe fromViewer;

   //receive
   private JSONReceiver receiver;

   public void findCommunities() throws IOException, GraphParseException {

       // 1. create a graph instance
       graph = new MultiGraph("Communities",false,true);
       viewer = graph.display(false);
       fromViewer = viewer.newThreadProxyOnGraphicGraph();

       // 4. do LinLog layout for the graph
       layout = new LinLog(false);
       layout.configure(a, r, true, force);
       layout.addSink(graph);
       graph.addSink(layout);
       fromViewer.addSink(graph);

       graph.addAttribute("ui.antialias");
       graph.addAttribute("ui.stylesheet", styleSheet);

       // ----- On the receiver side -----
       // 2. build the receiver that waits for events
       receiver = new JSONReceiver("localhost", 8080, "workspace0");
       // 3. get the ThreadProxyPipe instance
       ThreadProxyPipe pipe = receiver.getStream();
       // plugin the graph instance as an ElementSink of the pipe instance.
       pipe.addElementSink(graph);

       // ----- On the sender side -----
       new Thread() {
           public void run() {
               // 5. build a sender that sends events(layout information) to Gephi
               JSONSender sender = new JSONSender("localhost", 8080, "workspace0");
               layout.addSink(sender);
           }
       }.start();

       //The receiver pro-actively checks for events on the ThreadProxyPipe
       pipe.pump();
       while(!graph.hasAttribute("ui.viewClosed")) {
           pipe.pump();
           fromViewer.pump();
           layout.compute();
       }
   }

   protected static String styleSheet =
           "node { size: 7px; fill-color: rgb(150,150,150); }" +
           "edge { size: 2px; fill-color: rgb(255,50,50); }";
}

Attention: to make this tutorial works, the following operations must be done step by step:

1. Run the Graph Streaming plugin project
2. Load a file by choose menu "File/Open" (the corresponding workspace should be named "workspace0", if not we must change the workspace name in the tutorial)
3. Go to the Streaming window and click on Server/Start
4. Run LinLogLayoutReceiver

In LinLogLayoutReceiver, we plugin the graph instance as an ElementSink of the pipe instance instead of Sink in ExampleJSONReceiver. This is a trick. Because if we use addSink here, we will come across a problem. In this tutorial, GraphStream will always listen to Gephi graph to receive events from Gephi, while Gephi also always listens to GraphStream to receive layout information from GraphStream. thus falling into an infinite loop. Since the layout events sent by GraphStream are almost attribute events, while graph events sent by Gephi are all element events, we use addElementSink to distinguish them so as to get rid of the infinite loop.

Just as said before, this is just a trick and is only usable for this user case. In a more complicated user case, the Gephi and GraphStream may both send not only element events but also attribute events. To get rid of the infinite loop problem, a synchronization issue must be solved, so that event created in one source will never be sent to itself. On GraphStream side, the combination of event sourceId and timeId is used to solve the synchronization problem, while the current graph streaming plugin doesn't support it. By the way, this synchronization problem may be related with the Gephi core, thus making it difficult to solve just in Gephi graph streaming plugin. But definitely to say, this synchronization must be solved in the future.