-
Notifications
You must be signed in to change notification settings - Fork 0
/
lets-play-gephi-streaming-api-the-hidden-websocket.html
40 lines (36 loc) · 15 KB
/
lets-play-gephi-streaming-api-the-hidden-websocket.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Let's Play Gephi : Streaming API - The 'hidden' Websocket - Toto do stuff</title><meta name="description" content="Hi everyone, I know, I already talked about the Streaming module of Gephi here Gephi and the Streaming module . But I just discovered something…"><meta name="generator" content="Publii Open-Source CMS for Static Site"><link rel="canonical" href="https://totetmatt.github.io/lets-play-gephi-streaming-api-the-hidden-websocket.html"><link rel="alternate" type="application/atom+xml" href="https://totetmatt.github.io/feed.xml"><link rel="alternate" type="application/json" href="https://totetmatt.github.io/feed.json"><meta property="og:title" content="Let's Play Gephi : Streaming API - The 'hidden' Websocket"><meta property="og:site_name" content="Toto do stuff"><meta property="og:description" content="Hi everyone, I know, I already talked about the Streaming module of Gephi here Gephi and the Streaming module . But I just discovered something…"><meta property="og:url" content="https://totetmatt.github.io/lets-play-gephi-streaming-api-the-hidden-websocket.html"><meta property="og:type" content="article"><style>:root{--body-font:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans","Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";--heading-font:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans","Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";--logo-font:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans","Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";--menu-font:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,Cantarell,"Fira Sans","Droid Sans","Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol"}</style><link rel="stylesheet" href="https://totetmatt.github.io/assets/css/style.css?v=825c89ac06c7215b642eda05e8a14751"><script type="application/ld+json">{"@context":"http://schema.org","@type":"Article","mainEntityOfPage":{"@type":"WebPage","@id":"https://totetmatt.github.io/lets-play-gephi-streaming-api-the-hidden-websocket.html"},"headline":"Let's Play Gephi : Streaming API - The 'hidden' Websocket","datePublished":"2014-06-15T09:55","dateModified":"2020-06-20T00:35","description":"Hi everyone, I know, I already talked about the Streaming module of Gephi here Gephi and the Streaming module . But I just discovered something…","author":{"@type":"Person","name":"Totetmatt","url":"https://totetmatt.github.io/authors/totetmatt/"},"publisher":{"@type":"Organization","name":"Totetmatt"}}</script></head><body><div class="site-container"><header class="top" id="js-header"><a class="logo" href="https://totetmatt.github.io/">Toto do stuff</a></header><main><article class="post"><div class="hero"><figure class="hero__image hero__image--overlay"><img src="https://totetmatt.github.io/media/website/bg.jpg" srcset="https://totetmatt.github.io/media/website/responsive/bg-xs.jpg 300w, https://totetmatt.github.io/media/website/responsive/bg-sm.jpg 480w, https://totetmatt.github.io/media/website/responsive/bg-md.jpg 768w, https://totetmatt.github.io/media/website/responsive/bg-lg.jpg 1024w, https://totetmatt.github.io/media/website/responsive/bg-xl.jpg 1360w, https://totetmatt.github.io/media/website/responsive/bg-2xl.jpg 1600w" sizes="(max-width: 1600px) 100vw, 1600px" loading="eager" alt=""></figure><header class="hero__content"><div class="wrapper"><div class="post__meta"><time datetime="2014-06-15T09:55">14/06/15</time></div><h1>Let's Play Gephi : Streaming API - The 'hidden' Websocket</h1><div class="post__meta post__meta--author"><a href="https://totetmatt.github.io/authors/totetmatt/" class="feed__author invert">Totetmatt</a></div></div></header></div><div class="wrapper post__entry"><p>Hi everyone,</p><p>I know, I already talked about the Streaming module of Gephi here <a title="Permalien pour Gephi and the Streaming module" href="http://matthieu-totet.fr/Koumin/2013/09/16/gephi-and-the-streaming-module/" rel="bookmark">Gephi and the Streaming module</a> . But I just discovered something by reading the code and by a little hint from <a class="external text" href="http://gephi.org/about/people/" rel="nofollow">André Panisson</a> on the wiki that talk about <a href="https://wiki.gephi.org/index.php/Graph_Streaming#Using_a_WebSocket_Client">Websocket</a>.</p><p>To be short, <a href="http://en.wikipedia.org/wiki/WebSocket">Websocket </a>is a recent standard protocol over HTTP that allow you to do bi-directonnal communication. The main idea here is to have Browser talking and getting notified ( pushed ) from a server (instead of pull with traditional ajax).</p><h1>Gephi in your Browser</h1><p>Trying to motivate myself to code something for gephi, I read the source code of the Graph Streaming Plug in, because I know it (almost use it everytime) and I wanted to maybe enhance it.</p><p>I was also curious about one thing : Looking at some video & if I use it in the conventional way (not dummy REST call), the master actually have a list of client connected to it, so it means there is some "session" that are created between a client and a master.</p><p>When looking around into the code, I discovered some "Websocket" implementation. Interesting ! Let's try with a dummy code in python (using ws4py) :</p><pre>from ws4py.client import WebSocketBaseClient
from ws4py.manager import WebSocketManager
from ws4py import format_addresses, configure_logger
logger = configure_logger()
m = WebSocketManager()
class EchoClient(WebSocketBaseClient):
def handshake_ok(self):
logger.info("Opening %s" % format_addresses(self))
m.add(self)
def received_message(self, msg):
logger.info(str(msg))
if __name__ == '__main__':
import time
m.start()
client = EchoClient('<strong>ws://localhost:8080/workspace0</strong>')
client.connect()</pre><p>I opened Gephi with a new project and with the Master Streaming enabled. When I ran the script here was the magic :</p><pre>[2014-06-08 18:05:13,441] INFO Using select as epoll is not available
[2014-06-08 18:05:13,482] INFO Opening [Local => 127.0.0.1:12231 | Remote => 127.0.0.1:8080]
[2014-06-08 18:05:13,483] INFO Managing websocket [Local => 127.0.0.1:12231 | Remote => 127.0.0.1:8080]</pre><p>And looking at Gephi Stream :</p><figure class="alignnone size-full wp-image-357"><a href="https://totetmatt.github.io/media/posts/31/StreamingWS.png"><img loading="lazy" src="https://totetmatt.github.io/media/posts/31/StreamingWS.png" sizes="(max-width: 48em) 100vw, 768px" srcset="https://totetmatt.github.io/media/posts/31/responsive/StreamingWS-xs.png 300w, https://totetmatt.github.io/media/posts/31/responsive/StreamingWS-sm.png 480w, https://totetmatt.github.io/media/posts/31/responsive/StreamingWS-md.png 768w, https://totetmatt.github.io/media/posts/31/responsive/StreamingWS-lg.png 1024w, https://totetmatt.github.io/media/posts/31/responsive/StreamingWS-xl.png 1360w, https://totetmatt.github.io/media/posts/31/responsive/StreamingWS-2xl.png 1600w" alt="StreamingWS" width="167" height="143"></a></figure><p></p><p>So sounds like websocket is supported here ! What if I add a node ? Well it reacts in my script !</p><pre>[2014-06-08 18:09:31,710] INFO {"an":{"2":{"g":0.6,"b":0.6,"r":0.6,"z":0,"y":567.9358,"x":-747.53937,"size":10}}}
[2014-06-08 18:09:31,759] INFO {"cn":{"2":{}}}</pre><p>Hell yeah ! And actually if you restart the websocket client, Gephi will send you the current whole graph with the same method at connection time. Basically every change in the graph (add node, change value etc...) will be send to clients that are connected to the Master. Exception for the moment, when you run a Layout (Like force atlas) the graph isn't send and you don't have the new (x;y;z) value for the nodes.</p><p>Filters seems to works, but it actually send a<strong> {"cg":{}}</strong> action when you activate the filter (which means <strong>change graph</strong> I guess) and only send<strong> delete action</strong>. If the filter is disabled, no information is send, so I would think the implementation isn't really complete here.</p><p>And cherry on the cake, if from my python script I send a message, I update the gephi graph !</p><p>I wanted also to test the "Client" part of Gephi Streaming, to check if it works. But it's impossible to enter a websocket url (Only http://blablabla is possible, not ws://lolylol ). I think I'll have to really do a deep dive into the code to change that ;)</p><p>Here is the funny part, remember that websocket is just a standard and was mainly targeted for Web Browser <strong>aka</strong> there is a implementation of <strong>WebSocket</strong> in <strong>Javascript</strong>. So I implemented a quick library that allow you to '<strong>plug</strong>' a Web Application into a Gephi Streaming instance.</p><p><a href="https://github.com/totetmatt/gephi.stream.graph.websocket">https://github.com/totetmatt/gephi.stream.graph.websocket</a></p><h1>And now ?</h1><p>Where to begin ?</p><p>For the moment, it's working pretty well for a local network, but let's imagine you want to do the same but over internet ? 2 choices :</p><ul><li>You can do it by opening a port on your home router and giving the IP:Port to people.</li><li>OR you can implement a simple Python / JS server that will only do proxy between you and the other people without configuring your network at home.</li></ul><p>And with the emerging Wamp.ws / Autobahn.ws / Crossbar.io, it will getting very easy and will allow much more complex architecture of distributed and collaborative application based on Websocket.</p><p>Now, with this JS lib, we could have a Gephi that is calculating a Layout for all the client. But remember : Websocket is agnostic to the language. So as long there is an implementation, you can use it, and Java can do it, Python can do it, C++ can do ti. Which means, we can immagine to have a Gephi made of components fully independant and only talking to each other via a websocket based architecture.</p><figure class="wp-image-384 size-large"><a href="http://matthieu-totet.fr/Koumin/wp-content/uploads/2014/06/Gephi_Comp.jpg"><img loading="lazy" src="https://totetmatt.github.io/media/posts/31/Gephi_Comp-1024x557.jpg" sizes="(max-width: 48em) 100vw, 768px" srcset="https://totetmatt.github.io/media/posts/31/responsive/Gephi_Comp-1024x557-xs.jpg 300w, https://totetmatt.github.io/media/posts/31/responsive/Gephi_Comp-1024x557-sm.jpg 480w, https://totetmatt.github.io/media/posts/31/responsive/Gephi_Comp-1024x557-md.jpg 768w, https://totetmatt.github.io/media/posts/31/responsive/Gephi_Comp-1024x557-lg.jpg 1024w, https://totetmatt.github.io/media/posts/31/responsive/Gephi_Comp-1024x557-xl.jpg 1360w, https://totetmatt.github.io/media/posts/31/responsive/Gephi_Comp-1024x557-2xl.jpg 1600w" alt="Gephi_Comp" width="1024" height="557"></a></figure>Draft idea of decentralized Gephi<p></p><p>And here, the components can be coded in any language and can even be deployed in a dedicated box. Can be interesting ideas for heavy computation processes for example.</p></div><footer class="wrapper post__footer"><p class="post__last-updated">This article was updated on 20/06/20</p><ul class="post__tag"><li><a href="https://totetmatt.github.io/gephi/">Gephi</a></li><li><a href="https://totetmatt.github.io/javascript/">Javascript</a></li><li><a href="https://totetmatt.github.io/python/">Python</a></li><li><a href="https://totetmatt.github.io/stream/">Stream</a></li><li><a href="https://totetmatt.github.io/websocket/">Websocket</a></li></ul><div class="post__share"></div><div class="post__bio bio"><div class="bio__info"><h3 class="bio__name"><a href="https://totetmatt.github.io/authors/totetmatt/" class="invert" rel="author">Totetmatt</a></h3></div></div></footer></article><nav class="post__nav"><div class="post__nav-inner"><div class="post__nav-prev"><svg width="1.041em" height="0.416em" aria-hidden="true"><use xlink:href="https://totetmatt.github.io/assets/svg/svg-map.svg#arrow-prev"/></svg> <a href="https://totetmatt.github.io/tgv-et-ponctualite.html" class="invert post__nav-link" rel="prev"><span>Previous</span> TGV et Ponctualité</a></div><div class="post__nav-next"><a href="https://totetmatt.github.io/eurovision-2015.html" class="invert post__nav-link" rel="next"><span>Next</span> Eurovision 2015 </a><svg width="1.041em" height="0.416em" aria-hidden="true"><use xlink:href="https://totetmatt.github.io/assets/svg/svg-map.svg#arrow-next"/></svg></div></div></nav><div class="post__related related"><div class="wrapper"><h2 class="h5 related__title">You should also read:</h2><article class="related__item"><div class="feed__meta"><time datetime="2016-04-25T20:33" class="feed__date">16/04/25</time></div><h3 class="h1"><a href="https://totetmatt.github.io/twitter-streaming-importer-naoyun-as-a-gephi-plugin.html" class="invert">Twitter Streaming Importer : Naoyun as a Gephi Plugin</a></h3></article><article class="related__item"><div class="feed__meta"><time datetime="2015-11-05T08:55" class="feed__date">15/11/05</time></div><h3 class="h1"><a href="https://totetmatt.github.io/the-final-hack-of-the-gephi-streaming-plugin.html" class="invert">The Final hack of the Gephi Streaming Plugin</a></h3></article><article class="related__item"><div class="feed__meta"><time datetime="2015-09-06T22:47" class="feed__date">15/09/06</time></div><h3 class="h1"><a href="https://totetmatt.github.io/lets-play-gephi-dbpedia-rdf-sparql-and-your-favorite-actors.html" class="invert">Let's Play Gephi : Dbpedia, RDF, Sparql and your favorite Actors</a></h3></article></div></div></main><footer class="footer"><div class="footer__copyright"><p>Powered by <a href="https://getpublii.com" target="_blank" rel="nofollow noopener">Publii Static CMS</a></p></div><button class="footer__bttop js-footer__bttop" aria-label="Back to top"><svg><title>Back to top</title><use xlink:href="https://totetmatt.github.io/assets/svg/svg-map.svg#toparrow"/></svg></button></footer></div><script>window.publiiThemeMenuConfig = {
mobileMenuMode: 'sidebar',
animationSpeed: 300,
submenuWidth: 'auto',
doubleClickTime: 500,
mobileMenuExpandableSubmenus: true,
relatedContainerForOverlayMenuSelector: '.top',
};</script><script defer="defer" src="https://totetmatt.github.io/assets/js/scripts.min.js?v=f4c4d35432d0e17d212f2fae4e0f8247"></script><script>var images = document.querySelectorAll('img[loading]');
for (var i = 0; i < images.length; i++) {
if (images[i].complete) {
images[i].classList.add('is-loaded');
} else {
images[i].addEventListener('load', function () {
this.classList.add('is-loaded');
}, false);
}
}</script></body></html>