index.xml

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Linked Data Benchmark Council</title>
    <link>https://ldbcouncil.org/</link>
    <description>Recent content on Linked Data Benchmark Council</description>
    <generator>Hugo -- gohugo.io</generator>
    <language>en-us</language>
    <lastBuildDate>Fri, 30 Aug 2024 09:00:00 -0800</lastBuildDate><atom:link href="https://ldbcouncil.org/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Eighteenth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/eighteenth-tuc-meeting/</link>
      <pubDate>Fri, 30 Aug 2024 09:00:00 -0800</pubDate>
      
      <guid>https://ldbcouncil.org/event/eighteenth-tuc-meeting/</guid>
      <description>&lt;p&gt;&lt;strong&gt;Organizers:&lt;/strong&gt; Shipeng Qi, Wenyuan Yu (Alibaba Damo), Yan Zhou (CreateLink)&lt;/p&gt;
&lt;p&gt;LDBC is hosting a &lt;strong&gt;two-day&lt;/strong&gt; hybrid workshop, co-located in &lt;strong&gt;Guangzhou&lt;/strong&gt; with &lt;a href=&#34;https://vldb.org/2024/&#34;&gt;VLDB 2024&lt;/a&gt; on &lt;strong&gt;August 30-31 (Friday-Saturday)&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The program consists of 10- and 15-minute talks followed by a Q&amp;amp;A session. The talks will be recorded and made available&lt;br&gt;
online. &lt;strong&gt;If you would like to participate please register using &lt;a href=&#34;https://forms.gle/aVPrrcxXpSwrWPnh6&#34;&gt;our form&lt;/a&gt;.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;LDBC will host a &lt;strong&gt;social event&lt;/strong&gt; on Friday at the &lt;a href=&#34;https://www.marriott.com/en-us/hotels/canwi-the-westin-pazhou/overview/&#34;&gt;3F Xingang Room, the Westin Pazhou,&lt;br&gt;
Guangzhou&lt;/a&gt;. The address is &lt;a href=&#34;https://www.google.com/maps/search/?api=1&amp;amp;query=23.09611%2C113.363612&amp;amp;hl=en-US&#34;&gt;Area C, Canton Fair Complex, 681 Fengpu Zhong Rd., Haizhu District, Guangzhou, Guangdong&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;In addition, Ant Group, Alibaba Damo, and CreateLink will host a &lt;strong&gt;Happy Hour&lt;/strong&gt; on Saturday. Join us by the Pearl River at the Vino Restaurant, located on the &lt;a href=&#34;https://www.google.com/maps/place/Pati+Beer+Cultural+Art+Area/@23.10676,113.3395851,17z/data=!3m1!4b1!4m6!3m5!1s0x3402ff9921d813ff:0x47f2671cfc7a6eec!8m2!3d23.10676!4d113.34216!16s%2Fg%2F11rcyt04t8?hl=en&amp;amp;entry=ttu&#34;&gt;8th Floor, No. 118 Modiesha Street, Area B, Beer Culture Creative Art District, Guangzhou&lt;/a&gt;.&lt;/p&gt;
&lt;h3 id=&#34;program-and-slides&#34;&gt;Program and Slides&lt;/h3&gt;
&lt;p&gt;This event is hosted in Guangzhou. Local time is &lt;strong&gt;CST (China Standard time) on the first column&lt;/strong&gt;. &lt;strong&gt;CEST (Central European Standard time) on the third column&lt;/strong&gt; and &lt;strong&gt;PDT (Pacific Daylight time) on the forth column&lt;/strong&gt; are listed in addition for the participants in European and Pacific timezone.&lt;/p&gt;
&lt;h4 id=&#34;august-30-friday&#34;&gt;August 30, Friday&lt;/h4&gt;
&lt;p&gt;&lt;strong&gt;Location:&lt;/strong&gt; &lt;a href=&#34;https://www.langhamhotels.com/en/the-langham/guangzhou/&#34;&gt;Langham Place&lt;/a&gt;, Guangzhou, &lt;strong&gt;room 1&lt;/strong&gt;,&lt;br&gt;
co-located with VLDB (N0.630-638 Xingang Dong Road, Haizhu District, Guangzhou, China). See the map &lt;a href=&#34;https://maps.app.goo.gl/86jD3Dy9Aa7bwLs36&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;CST Start&lt;/th&gt;
&lt;th&gt;CST End&lt;/th&gt;
&lt;th&gt;CEST Start&lt;/th&gt;
&lt;th&gt;PDT Start&lt;/th&gt;
&lt;th&gt;Speaker&lt;/th&gt;
&lt;th&gt;Topic&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;09:00&lt;/td&gt;
&lt;td&gt;09:20&lt;/td&gt;
&lt;td&gt;03:00&lt;/td&gt;
&lt;td&gt;18:00&lt;/td&gt;
&lt;td&gt;Alastair Green (LDBC)&lt;/td&gt;
&lt;td&gt;State of the Union - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/1.%20LDBC%20_State%20of%20the%20Union_%20--%2018th%20TUC,%20Guangzhou,%202024-08-30%20Alastair%20Green.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/I0R72mAKQMQ&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;09:20&lt;/td&gt;
&lt;td&gt;09:40&lt;/td&gt;
&lt;td&gt;03:20&lt;/td&gt;
&lt;td&gt;18:20&lt;/td&gt;
&lt;td&gt;Lei Zou (Peking University)&lt;/td&gt;
&lt;td&gt;Unified Graph Query Plan Representation and Optimization - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/2.%20Unified%20Graph%20Query%20Plan%20Representation%20and%20Optimization.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/fzsQ--DW49k&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;09:40&lt;/td&gt;
&lt;td&gt;10:00&lt;/td&gt;
&lt;td&gt;03:40&lt;/td&gt;
&lt;td&gt;18:40&lt;/td&gt;
&lt;td&gt;Wei Wang (East China Normal University)&lt;/td&gt;
&lt;td&gt;From OpenRank to OpenPerf: Enhancing Open-Source Ecosystem Insights with Graph-Based Approaches - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/3.%20From%20OpenRank%20to%20OpenPerf.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/XKGJ0VPtadg&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:00&lt;/td&gt;
&lt;td&gt;10:20&lt;/td&gt;
&lt;td&gt;04:00&lt;/td&gt;
&lt;td&gt;19:00&lt;/td&gt;
&lt;td&gt;coffee break&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:20&lt;/td&gt;
&lt;td&gt;10:40&lt;/td&gt;
&lt;td&gt;04:20&lt;/td&gt;
&lt;td&gt;19:20&lt;/td&gt;
&lt;td&gt;Keith W. Hare (JCC / WG3)&lt;/td&gt;
&lt;td&gt;The GQL Standard is Published! Now what? - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/4.%20GQL%20standard%20is%20published%202024-08-30.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/RoebDzkbn9c&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:40&lt;/td&gt;
&lt;td&gt;11:10&lt;/td&gt;
&lt;td&gt;04:40&lt;/td&gt;
&lt;td&gt;19:40&lt;/td&gt;
&lt;td&gt;Michael Burbidge (LDBC), co-present with &lt;br&gt;Alessio Stalla, Federico Tomassetti, Tracey Walsh&lt;/td&gt;
&lt;td&gt;The LDBC GQL Implementation Working Group mission and progress - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/5.%20GQL%20Implementation%20WG.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/WTT6X7zQisg&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:10&lt;/td&gt;
&lt;td&gt;11:30&lt;/td&gt;
&lt;td&gt;05:10&lt;/td&gt;
&lt;td&gt;20:10&lt;/td&gt;
&lt;td&gt;Oskar Van Rest (Oracle)&lt;/td&gt;
&lt;td&gt;The SQL/PGQ Standard: SQL support for property graphs - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/6.%20PGQ%20Standard%20final.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/wvXm0wgkrF8&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:30&lt;/td&gt;
&lt;td&gt;12:00&lt;/td&gt;
&lt;td&gt;05:30&lt;/td&gt;
&lt;td&gt;20:30&lt;/td&gt;
&lt;td&gt;Peter Boncz (CWI / LDBC)&lt;/td&gt;
&lt;td&gt;The state of DuckPGQ - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/7.%20The%20State%20of%20DuckPGQ.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/ZlXu0UM2LUw&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:00&lt;/td&gt;
&lt;td&gt;13:00&lt;/td&gt;
&lt;td&gt;06:00&lt;/td&gt;
&lt;td&gt;21:00&lt;/td&gt;
&lt;td&gt;lunch at Langham hotel&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:00&lt;/td&gt;
&lt;td&gt;13:20&lt;/td&gt;
&lt;td&gt;07:00&lt;/td&gt;
&lt;td&gt;22:00&lt;/td&gt;
&lt;td&gt;Xuntao Cheng (vesoft)&lt;/td&gt;
&lt;td&gt;GQL and its implementation - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/8.%20GQL_talk_Xuntao_Cheng.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/XsRxw06B-jY&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:20&lt;/td&gt;
&lt;td&gt;13:40&lt;/td&gt;
&lt;td&gt;07:20&lt;/td&gt;
&lt;td&gt;22:20&lt;/td&gt;
&lt;td&gt;Alastair Green (JCC / LDBC)&lt;/td&gt;
&lt;td&gt;LEX: The future of Graph Schema - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/9.%20LEX-074%20LEX%20LDBC%2018th%20LDBC%20TUC%20%20%2030%20August%202024.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/txskwX2DwxA&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:40&lt;/td&gt;
&lt;td&gt;14:00&lt;/td&gt;
&lt;td&gt;07:40&lt;/td&gt;
&lt;td&gt;22:40&lt;/td&gt;
&lt;td&gt;Valerio Malenchino (Neo4j)&lt;/td&gt;
&lt;td&gt;OpenCypher, a path to GQL. – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/10.%20openCypher,%20a%20path%20to%20GQL.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/qm7ViOYfwgY&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:00&lt;/td&gt;
&lt;td&gt;14:55&lt;/td&gt;
&lt;td&gt;08:00&lt;/td&gt;
&lt;td&gt;23:00&lt;/td&gt;
&lt;td&gt;Host: Alastair Green (JCC / LDBC), panelists: Peter Boncz (CWI / LDBC), Chuntao Hong (AntGroup), Mingxi Wu (TigerGraph), Xuntao Cheng(vesoft)&lt;/td&gt;
&lt;td&gt;ISO GQL Panel - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/11.%20Panel%20questions.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/qVcQYYbx71g&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:55&lt;/td&gt;
&lt;td&gt;15:15&lt;/td&gt;
&lt;td&gt;08:55&lt;/td&gt;
&lt;td&gt;23:55&lt;/td&gt;
&lt;td&gt;coffee break&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:15&lt;/td&gt;
&lt;td&gt;15:35&lt;/td&gt;
&lt;td&gt;09:15&lt;/td&gt;
&lt;td&gt;00:15&lt;/td&gt;
&lt;td&gt;Shixuan Sun (Shanghai Jiao Tong University)&lt;/td&gt;
&lt;td&gt;FaaSGraph: Enabling Scalable, Efficient, and Cost-Effective Graph Processing with Serverless Computing - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/12.%20FaaSGraph-ShixuanSun-LDBC.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/c-KxJLcyzto&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:35&lt;/td&gt;
&lt;td&gt;15:55&lt;/td&gt;
&lt;td&gt;09:35&lt;/td&gt;
&lt;td&gt;00:35&lt;/td&gt;
&lt;td&gt;Cheng Chen (ByteDance)&lt;/td&gt;
&lt;td&gt;BG3: A Cost Effective and I/O Efficient Graph Database in ByteDance - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/13.%20bg3-ldbc24.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/pFUbg34lvPo&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:55&lt;/td&gt;
&lt;td&gt;16:15&lt;/td&gt;
&lt;td&gt;09:55&lt;/td&gt;
&lt;td&gt;00:55&lt;/td&gt;
&lt;td&gt;Mingxi Wu (TigerGraph)&lt;/td&gt;
&lt;td&gt;TigerGraph: Key Lessons from Running SNB BI Workload on Large-Scale Data Set - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/14.%20LDBC-SNB-108T-2024-Aug.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/9XiOL7fjz0k&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:15&lt;/td&gt;
&lt;td&gt;16:35&lt;/td&gt;
&lt;td&gt;10:15&lt;/td&gt;
&lt;td&gt;01:15&lt;/td&gt;
&lt;td&gt;Gabor Szarnyas (DuckDB / LDBC)&lt;/td&gt;
&lt;td&gt;The LDBC benchmark suite - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/15.%20gabor-szarnyas-the-ldbc-benchmark-suite.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/j6C4G0O-bQM&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:35&lt;/td&gt;
&lt;td&gt;16:55&lt;/td&gt;
&lt;td&gt;10:35&lt;/td&gt;
&lt;td&gt;01:35&lt;/td&gt;
&lt;td&gt;Shipeng Qi (AntGroup / LDBC)&lt;/td&gt;
&lt;td&gt;The LDBC Financial Benchmark - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day1/16.%20The%20LDBC%20FinBench%20v0.2.0.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/hTfK-9Toor4&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:55&lt;/td&gt;
&lt;td&gt;17:40&lt;/td&gt;
&lt;td&gt;10:55&lt;/td&gt;
&lt;td&gt;01:55&lt;/td&gt;
&lt;td&gt;In-person participants&lt;/td&gt;
&lt;td&gt;LDBC Voting Members Meeting&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;18:00&lt;/td&gt;
&lt;td&gt;22:00&lt;/td&gt;
&lt;td&gt;12:00&lt;/td&gt;
&lt;td&gt;03:00&lt;/td&gt;
&lt;td&gt;Dinner at 3/F Xingang Room, the Westin Pazhou, Guangzhou&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h4 id=&#34;august-31-saturday&#34;&gt;August 31, Saturday&lt;/h4&gt;
&lt;p&gt;&lt;strong&gt;Location:&lt;/strong&gt; Alibaba Center, Guangzhou (N0.88 Dingxin Road, Haizhu District, Guangzhou, China), near to VLDB Langham Place. See the map &lt;a href=&#34;https://maps.app.goo.gl/HgEVafZMRmrzUsgW8&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;CST Start&lt;/th&gt;
&lt;th&gt;CST End&lt;/th&gt;
&lt;th&gt;CEST Start&lt;/th&gt;
&lt;th&gt;PDT Start&lt;/th&gt;
&lt;th&gt;Speaker&lt;/th&gt;
&lt;th&gt;Topic&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;09:00&lt;/td&gt;
&lt;td&gt;09:20&lt;/td&gt;
&lt;td&gt;03:00&lt;/td&gt;
&lt;td&gt;18:00&lt;/td&gt;
&lt;td&gt;Ora Lassila (AWS Neptune)&lt;/td&gt;
&lt;td&gt;Aligning RDF and LPGs: A Status Report of the OneGraph Initiative - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/1.%20onegraph-ldbc-tuc18-202408-final.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/to_dWaaX498&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;09:20&lt;/td&gt;
&lt;td&gt;09:40&lt;/td&gt;
&lt;td&gt;03:20&lt;/td&gt;
&lt;td&gt;18:20&lt;/td&gt;
&lt;td&gt;Xiangyu Ke (Zhejiang University)&lt;/td&gt;
&lt;td&gt;View-based explanation for graph neural networks - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/2.%20GVEX-XiangyuKE.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/EgojnYfRuuw&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;09:40&lt;/td&gt;
&lt;td&gt;10:00&lt;/td&gt;
&lt;td&gt;03:40&lt;/td&gt;
&lt;td&gt;18:40&lt;/td&gt;
&lt;td&gt;Chengying Huan (Insitute of software, CAS)&lt;/td&gt;
&lt;td&gt;Hybrid Sampling Algorithm for Dynamic Graph Sampling - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/3.%20Dynamic_sampling.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/sOuz2MYZYoI&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:00&lt;/td&gt;
&lt;td&gt;10:20&lt;/td&gt;
&lt;td&gt;04:00&lt;/td&gt;
&lt;td&gt;19:00&lt;/td&gt;
&lt;td&gt;Chen Zhang (CreateLink)&lt;/td&gt;
&lt;td&gt;Parameterized Algorithm Routine: The Perfect Balance between Performance and Usability - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/4.%20TUC_Galaxybase.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/d4wPsf4pyCE&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:20&lt;/td&gt;
&lt;td&gt;10:50&lt;/td&gt;
&lt;td&gt;04:20&lt;/td&gt;
&lt;td&gt;19:20&lt;/td&gt;
&lt;td&gt;coffee break&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:50&lt;/td&gt;
&lt;td&gt;11:10&lt;/td&gt;
&lt;td&gt;04:50&lt;/td&gt;
&lt;td&gt;19:50&lt;/td&gt;
&lt;td&gt;Long Yuan (Nanjing University of Science and Technology)&lt;/td&gt;
&lt;td&gt;Revisit benchmarking graph analytics - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/5.%20Revisiting%20Graph%20Analytics%20Benchmarks.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/9VOX3th7fV4&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:10&lt;/td&gt;
&lt;td&gt;11:30&lt;/td&gt;
&lt;td&gt;05:10&lt;/td&gt;
&lt;td&gt;20:10&lt;/td&gt;
&lt;td&gt;Longbin Lai (Alibaba Group)&lt;/td&gt;
&lt;td&gt;A Graph-Native Query Optimization Framework - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/6.%20GOPT-TUC-v0.2.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/De39jeVmif8&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:30&lt;/td&gt;
&lt;td&gt;11:50&lt;/td&gt;
&lt;td&gt;05:30&lt;/td&gt;
&lt;td&gt;20:30&lt;/td&gt;
&lt;td&gt;Jingbo Xu (Alibaba Group)&lt;/td&gt;
&lt;td&gt;Apache GraphAr: An Open-Source Standard File Format for Graph Data Storage and Retrieval - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/7.%20GraphAr-TUCv1.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/tGx5MUghofc&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:50&lt;/td&gt;
&lt;td&gt;12:10&lt;/td&gt;
&lt;td&gt;05:50&lt;/td&gt;
&lt;td&gt;20:50&lt;/td&gt;
&lt;td&gt;Sijie Shen (Alibaba Group)&lt;/td&gt;
&lt;td&gt;Introducing GART: Real-Time Online Graph Data Analysis for SQL - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/8.%20GART.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/pMVUc72BdpQ&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:10&lt;/td&gt;
&lt;td&gt;13:30&lt;/td&gt;
&lt;td&gt;06:10&lt;/td&gt;
&lt;td&gt;21:10&lt;/td&gt;
&lt;td&gt;lunch at Alibaba Center&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:30&lt;/td&gt;
&lt;td&gt;13:50&lt;/td&gt;
&lt;td&gt;07:30&lt;/td&gt;
&lt;td&gt;22:30&lt;/td&gt;
&lt;td&gt;Atanas Kiryakov (OntoText)&lt;/td&gt;
&lt;td&gt;Graph RAG Varieties - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/9.%20Graph%20RAG%20Varieties%20-%20LDBC.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/dFyPlllOm1s&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:10&lt;/td&gt;
&lt;td&gt;14:30&lt;/td&gt;
&lt;td&gt;08:10&lt;/td&gt;
&lt;td&gt;23:10&lt;/td&gt;
&lt;td&gt;Boci Peng (AntGroup)&lt;/td&gt;
&lt;td&gt;Subgraph Retrieval Enhanced by Graph-Text Alignment for Commonsense Question Answering - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/11.%20SEPTA_tuc_pbc.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/hZwkbLoOcUA&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:30&lt;/td&gt;
&lt;td&gt;14:50&lt;/td&gt;
&lt;td&gt;08:30&lt;/td&gt;
&lt;td&gt;23:30&lt;/td&gt;
&lt;td&gt;Damien Hilloulin (Oracle Labs)&lt;/td&gt;
&lt;td&gt;Graphs, Graph-RAG, and LLMs: An Introduction - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/12.%202024-08-31-GraphsGraph-RAGsandLLMsAnIntroduction-tuc.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/ENCso_xlNgQ&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:50&lt;/td&gt;
&lt;td&gt;15:20&lt;/td&gt;
&lt;td&gt;08:50&lt;/td&gt;
&lt;td&gt;23:50&lt;/td&gt;
&lt;td&gt;coffee break&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:20&lt;/td&gt;
&lt;td&gt;15:50&lt;/td&gt;
&lt;td&gt;09:20&lt;/td&gt;
&lt;td&gt;00:20&lt;/td&gt;
&lt;td&gt;Xiyang (Kuzu Inc.)&lt;/td&gt;
&lt;td&gt;Progress and Roadmap of the Kuzu Graph DBMS - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/13.%20Kuzu%20TUC%20Meeting%20Aug%202024.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/m7cSJRa__t4&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:50&lt;/td&gt;
&lt;td&gt;16:10&lt;/td&gt;
&lt;td&gt;09:50&lt;/td&gt;
&lt;td&gt;00:50&lt;/td&gt;
&lt;td&gt;Ricky Sun (Ultipa)&lt;/td&gt;
&lt;td&gt;A Graph Analytics Supercharge Case Study of GPU vs. CPU on Performance, Greenness and Cost - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/14.%20A%20Graph%20Analytics%20Supercharge%20Case%20Study%20of%20CPU%20vs.%20GPU%20on%20Performance,%20Cost%20and%20ESG%20v2.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/ay_4SAY_PPo&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:10&lt;/td&gt;
&lt;td&gt;16:30&lt;/td&gt;
&lt;td&gt;10:10&lt;/td&gt;
&lt;td&gt;01:10&lt;/td&gt;
&lt;td&gt;Min Wu (Fabarta)&lt;/td&gt;
&lt;td&gt;ArcNeural: A Multi-Modal Database for the Gen-AI Era - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/15.%20Fabarta.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/opmyhgkIYok&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:30&lt;/td&gt;
&lt;td&gt;16:50&lt;/td&gt;
&lt;td&gt;10:30&lt;/td&gt;
&lt;td&gt;01:30&lt;/td&gt;
&lt;td&gt;Juan Yang (StarGraph)&lt;/td&gt;
&lt;td&gt;AtlasGraph: A High-Performance Graph Database Solution for Complex Data Challenges - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighteenth-tuc-meeting/day2/16.%20StarGraph%202024%20VLDB%202024-08-31.pdf&#34;&gt;slides&lt;/a&gt; -&lt;a href=&#34;https://youtu.be/iF7KFref9Rs&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;18:00&lt;/td&gt;
&lt;td&gt;22:00&lt;/td&gt;
&lt;td&gt;12:00&lt;/td&gt;
&lt;td&gt;03:00&lt;/td&gt;
&lt;td&gt;Happy Hour sponsored by AntGroup, Alibaba Damo, CreateLink&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h4 id=&#34;tuc-event-locations&#34;&gt;TUC event locations&lt;/h4&gt;
&lt;p&gt;A &lt;a href=&#34;https://www.google.com/maps/d/u/0/edit?mid=19_fi4fV-3-PZkNWCCcmhU86ct2EZXbgo&#34;&gt;map of the LDBC TUC events&lt;/a&gt; we hosted so far.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Seventeenth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/seventeenth-tuc-meeting/</link>
      <pubDate>Sun, 09 Jun 2024 09:00:00 -0400</pubDate>
      
      <guid>https://ldbcouncil.org/event/seventeenth-tuc-meeting/</guid>
      <description>&lt;p&gt;&lt;strong&gt;Organizers:&lt;/strong&gt; Renzo Angles, Sebastián Ferrada&lt;/p&gt;
&lt;p&gt;LDBC is hosting a one-day in-person workshop, co-located in &lt;strong&gt;Santiago de Chile&lt;/strong&gt; with &lt;a href=&#34;https://2024.sigmod.org/venue.shtml&#34;&gt;SIGMOD 2024&lt;/a&gt; on &lt;strong&gt;June 9 (Sunday)&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The workshop will be held in the &lt;strong&gt;Hotel Plaza El Bosque Ebro&lt;/strong&gt; (&lt;a href=&#34;https://www.plazaelbosque.cl&#34;&gt;https://www.plazaelbosque.cl&lt;/a&gt;), which is two blocks away from SIGMOD&amp;rsquo;s venue. See the map &lt;a href=&#34;https://maps.app.goo.gl/78oiw3zo2pH3gy5R6&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;h3 id=&#34;program&#34;&gt;Program&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;All times are in Chile time (GMT-4).&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Each speaker will have 20 minutes for exposition plus 5 minutes for questions.&lt;/strong&gt;&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Time&lt;/th&gt;
&lt;th&gt;Speaker&lt;/th&gt;
&lt;th&gt;Title&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;09:00&lt;/td&gt;
&lt;td&gt;Renzo Angles (U. Talca)&lt;/td&gt;
&lt;td&gt;Welcome&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;09:30&lt;/td&gt;
&lt;td&gt;Alastair Green (LDBC Vice-chair)&lt;/td&gt;
&lt;td&gt;Status of the LDBC Extended GQL Schema Working Group&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:00&lt;/td&gt;
&lt;td&gt;Hannes Voigt (Neo4j)&lt;/td&gt;
&lt;td&gt;Inside the Standardization Machine Room: How ISO/IEC 39075:2024 GQL was produced  - &lt;a href=&#34;https://ldbcouncil.org/docs/tuc17th/inside.pdf&#34;&gt;slides&lt;/a&gt; - &lt;a href=&#34;https://youtu.be/EpwHzPSOF3o&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:30&lt;/td&gt;
&lt;td&gt;Calin Iorgulescu (Oracle)&lt;/td&gt;
&lt;td&gt;PGX.D: Distributed graph processing engine&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:00&lt;/td&gt;
&lt;td&gt;Coffee break&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:30&lt;/td&gt;
&lt;td&gt;Ricky Sun (Ultipa, Inc.)&lt;/td&gt;
&lt;td&gt;A Unified Graph Framework with SCC (Storage-Compute Coupled) and HDC (High-Density Computing) Clustering&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:00&lt;/td&gt;
&lt;td&gt;Daan de Graaf (TU Eindhoven)&lt;/td&gt;
&lt;td&gt;Algorithm Support in a Graph Database, Done Right  - &lt;a href=&#34;https://ldbcouncil.org/docs/tuc17th/graphalg.pdf&#34;&gt;slides&lt;/a&gt; - &lt;a href=&#34;https://youtu.be/eys5wJmy2PY&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:30&lt;/td&gt;
&lt;td&gt;Angela Bonifati (Lyon 1 University and IUF, France)&lt;/td&gt;
&lt;td&gt;Transforming Property Graphs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:00&lt;/td&gt;
&lt;td&gt;Brunch&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:00&lt;/td&gt;
&lt;td&gt;Juan Sequeda (data.world)&lt;/td&gt;
&lt;td&gt;A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model&amp;rsquo;s Accuracy for Question Answering on Enterprise SQL Databases - &lt;a href=&#34;https://youtu.be/P0WoWFp1fN8&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:30&lt;/td&gt;
&lt;td&gt;Olaf Hartig (Linköping University)&lt;/td&gt;
&lt;td&gt;FedShop: A Benchmark for Testing the Scalability of SPARQL Federation Engines  - &lt;a href=&#34;https://ldbcouncil.org/docs/tuc17th/fedshop.pdf&#34;&gt;slides&lt;/a&gt; - &lt;a href=&#34;https://youtu.be/J7rAowKmAcE&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:00&lt;/td&gt;
&lt;td&gt;Olaf Hartig (Amazon)&lt;/td&gt;
&lt;td&gt;Datatypes for Lists and Maps in RDF Literals&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:30&lt;/td&gt;
&lt;td&gt;Peter Boncz (CWI and MotherDuck)&lt;/td&gt;
&lt;td&gt;The state of DuckPGQ - &lt;a href=&#34;https://ldbcouncil.org/docs/tuc17th/duckpgq.pdf&#34;&gt;slides&lt;/a&gt; - &lt;a href=&#34;https://youtu.be/VoxV0z_XQPM&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:00&lt;/td&gt;
&lt;td&gt;Coffee break&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:30&lt;/td&gt;
&lt;td&gt;Juan Reutter (IMFD and PUC Chile)&lt;/td&gt;
&lt;td&gt;MillenniumDB: A Persistent, Open-Source, Graph Database&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;17:00&lt;/td&gt;
&lt;td&gt;Carlos Rojas (IMFD)&lt;/td&gt;
&lt;td&gt;WDBench: A Wikidata Graph Query Benchmark&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;17:30&lt;/td&gt;
&lt;td&gt;Sebastián Ferrada (IMFD and Univ. de Chile)&lt;/td&gt;
&lt;td&gt;An algebra for evaluating path queries&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;19:30&lt;/td&gt;
&lt;td&gt;Dinner&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
</description>
    </item>
    
    <item>
      <title>Record-Breaking SNB Interactive Results for GraphScope</title>
      <link>https://ldbcouncil.org/post/record-breaking-snb-interactive-results-for-graphscope/</link>
      <pubDate>Sun, 26 May 2024 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/record-breaking-snb-interactive-results-for-graphscope/</guid>
      <description>&lt;p&gt;We are happy to annonunce new &lt;a href=&#34;https://ldbcouncil.org/benchmarks/snb-interactive/&#34;&gt;audited results for the SNB Interactive workload&lt;/a&gt;, achieved by the open-source &lt;a href=&#34;https://github.com/alibaba/GraphScope&#34;&gt;GraphScope Flex&lt;/a&gt; system.&lt;/p&gt;
&lt;p&gt;The current audit of the system has broken several records:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;It achieved 130.1k ops/s on scale factor 100, compared to the previous record of 48.8k ops/s.&lt;/li&gt;
&lt;li&gt;It achieved 131.3k ops/s on scale factor 300, compared to the previous record of 48.3k ops/s.&lt;/li&gt;
&lt;li&gt;It is the first system to successfully complete the benchmark on scale factor 1000. It achieved a throughput of 127.8k ops/s&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The audit was commissioned by the &lt;a href=&#34;https://www.alibabacloud.com/&#34;&gt;Alibaba Cloud&lt;/a&gt; and was conducted by &lt;a href=&#34;https://www.linkedin.com/in/arnau-prat-a70283bb/&#34;&gt;Dr. Arnau Prat-Pérez&lt;/a&gt;, one of the original authors of the SNB Interactive benchmark. The queries were implemented as C++ stored procedures and the benchmark was executed on the Alibaba Cloud&amp;rsquo;s infrastructure. The &lt;a href=&#34;https://ldbcouncil.org/benchmarks/snb/LDBC_SNB_I_20240514_SF100-300-1000_graphscope-executive_summary.pdf&#34;&gt;executive summary&lt;/a&gt;, &lt;a href=&#34;https://ldbcouncil.org/benchmarks/snb/LDBC_SNB_I_20240514_SF100-300-1000_graphscope.pdf&#34;&gt;full disclosure report&lt;/a&gt;, and &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/audits/LDBC_SNB_I_20240514_SF100-300-1000_graphscope-attachments.tar.gz&#34;&gt;supplementary package&lt;/a&gt; describe the benchmark&amp;rsquo;s steps and include instructions for reproduction.&lt;/p&gt;
&lt;p&gt;LDBC would like to congratulate the GraphScope Flex team on their record-breaking results.&lt;/p&gt;
&lt;div align=&#34;center&#34;&gt;&lt;img src=&#34;https://ldbcouncil.org/images/graphscope.svg&#34; width=&#34;200&#34;&gt;&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Launching open-source language tools for ISO/IEC GQL</title>
      <link>https://ldbcouncil.org/post/ldbc-announces-open-source-gql-tools/</link>
      <pubDate>Thu, 09 May 2024 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/ldbc-announces-open-source-gql-tools/</guid>
      <description>&lt;p&gt;Following the publication of ISO/IEC GQL (graph query language) in April 2024, LDBC today launches open-source language engineering tools to help implementers, and assist in generation of code examples and tests for the GQL language. See this &lt;a href=&#34;https://ldbcouncil.org/pages/opengql-announce&#34;&gt;announcement from Alastair Green, Vice-chair of LDBC&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;These tools are the work of the &lt;strong&gt;LDBC GQL Implementation Working Group&lt;/strong&gt;, headed up by Michael Burbidge. Damian Wileński and Dominik Tomaszuk have worked with Michael to create these artefacts based on his ANTLR grammar for GQL.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Announcing the Official Release of LDBC Financial Benchmark v0.1.0</title>
      <link>https://ldbcouncil.org/post/announcing-the-official-release-of-ldbc-financial-benchmark/</link>
      <pubDate>Tue, 27 Jun 2023 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/announcing-the-official-release-of-ldbc-financial-benchmark/</guid>
      <description>&lt;p&gt;We are delighted to announce the official release of the initial version (v0.1.0) of &lt;a href=&#34;https://ldbcouncil.org/benchmarks/finbench/&#34;&gt;Financial Benchmark (FinBench)&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The Financial Benchmark (FinBench) project defines a graph database benchmark targeting financial scenarios such as anti-fraud and risk control. It is maintained by the &lt;a href=&#34;https://ldbcouncil.org/benchmarks/finbench/ldbc-finbench-work-charter.pdf&#34;&gt;LDBC FinBench Task Force&lt;/a&gt;. The benchmark has one workload currently, &lt;strong&gt;Transaction Workload&lt;/strong&gt;, capturing OLTP scenario with complex read queries that access the neighbourhood of a given node in the graph and write queries that continuously insert or delete data in the graph.&lt;/p&gt;
&lt;p&gt;Compared to LDBC SNB, the FinBench differs in application scenarios, data patterns, and workloads, resulting in different schema characteristics, latency bounds, path filters, etc. For a brief overview, see the &lt;a href=&#34;https://ldbcouncil.org/benchmarks/finbench/finbench-talk-16th-tuc.pdf&#34;&gt;slides&lt;/a&gt; in the 16th TUC. The &lt;a href=&#34;https://arxiv.org/pdf/2306.15975.pdf&#34;&gt;Financial Benchmark&amp;rsquo;s specification&lt;/a&gt; can be found on arXiv.&lt;/p&gt;
&lt;p&gt;The release of FinBench initial version (v0.1.0) was approved by LDBC on June 23, 2023. It is the good beginning of FinBench. In the future, the FinBench Task Force will polish the benchmark continuously.&lt;/p&gt;
&lt;p&gt;If you are interested in joining FinBench Task Force, please reach out at info at ldbcouncil.org or qishipeng.qsp at antgroup.com.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Sixteenth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/sixteenth-tuc-meeting/</link>
      <pubDate>Fri, 23 Jun 2023 09:00:00 -0800</pubDate>
      
      <guid>https://ldbcouncil.org/event/sixteenth-tuc-meeting/</guid>
      <description>&lt;p&gt;&lt;strong&gt;Organizers:&lt;/strong&gt; Oskar van Rest, Alastair Green, Gábor Szárnyas&lt;/p&gt;
&lt;p&gt;LDBC is hosting a &lt;strong&gt;two-day&lt;/strong&gt; hybrid workshop, co-located with &lt;a href=&#34;https://2023.sigmod.org/venue.shtml&#34;&gt;SIGMOD 2023&lt;/a&gt; on &lt;strong&gt;June 23-24 (Friday-Saturday)&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The program consists of 10- and 15-minute talks followed by a Q&amp;amp;A session. The talks will be recorded and made available online. &lt;strong&gt;If you would like to participate please register using &lt;a href=&#34;https://forms.gle/T6bwVHzK9V5FaKyR9&#34;&gt;our form&lt;/a&gt;.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;LDBC will host a &lt;strong&gt;social event&lt;/strong&gt; on Friday at the &lt;a href=&#34;https://www.blackbottleseattle.com/&#34;&gt;Black Bottle gastrotavern&lt;/a&gt; in Belltown: &lt;a href=&#34;https://goo.gl/maps/hQzBRR2nerZEQExw7&#34;&gt;2600 1st Ave (on the corner of Vine), Seattle, WA 98121&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;In addition, AWS will host a &lt;strong&gt;Happy Hour&lt;/strong&gt; (rooftop grill with beverages) on Saturday on the Amazon Nitro South building&amp;rsquo;s 8th floor deck: &lt;a href=&#34;https://goo.gl/maps/md5kWUHaNUGhR9JB7&#34;&gt;2205 8th Ave, Seattle, WA 98121&lt;/a&gt;.&lt;/p&gt;
&lt;h3 id=&#34;program&#34;&gt;Program&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;All times are in PDT.&lt;/strong&gt;&lt;/p&gt;
&lt;h4 id=&#34;friday&#34;&gt;Friday&lt;/h4&gt;
&lt;p&gt;&lt;strong&gt;Location:&lt;/strong&gt; Hyatt Regency Bellevue on Seattle&amp;rsquo;s Eastside, &lt;strong&gt;room Grand K&lt;/strong&gt;, co-located with SIGMOD (900 Bellevue Way NE, Bellevue, WA 98004-4272)&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;start&lt;/th&gt;
&lt;th&gt;finish&lt;/th&gt;
&lt;th&gt;speaker&lt;/th&gt;
&lt;th&gt;title&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;08:30&lt;/td&gt;
&lt;td&gt;08:45&lt;/td&gt;
&lt;td&gt;Oskar van Rest (Oracle)&lt;/td&gt;
&lt;td&gt;LDBC – State of the union – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/oskar-van-rest-ldbc-state-of-the-union.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/Frk7ITssaSY&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;08:50&lt;/td&gt;
&lt;td&gt;09:05&lt;/td&gt;
&lt;td&gt;Keith Hare (JCC / WG3)&lt;/td&gt;
&lt;td&gt;An update on the GQL &amp;amp; SQL/PGQ standards efforts  – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/keith-hare-an-update-on-the-gql-and-sql-pgq-standards-efforts.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/LQYkal_0j6E&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;09:10&lt;/td&gt;
&lt;td&gt;09:25&lt;/td&gt;
&lt;td&gt;Stefan Plantikow (Neo4j / WG3)&lt;/td&gt;
&lt;td&gt;GQL - Introduction to a new query language standard – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/stefan-plantikow-gql-v1.pdf&#34;&gt;slides&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;09:30&lt;/td&gt;
&lt;td&gt;09:45&lt;/td&gt;
&lt;td&gt;Leonid Libkin (University of Edinburgh &amp;amp; RelationalAI)&lt;/td&gt;
&lt;td&gt;Formalizing GQL – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/leonid-libkin-formalizing-gql.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/YZE1a00h1I4&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;09:50&lt;/td&gt;
&lt;td&gt;10:05&lt;/td&gt;
&lt;td&gt;Semen Panenkov (JetBrains Research)&lt;/td&gt;
&lt;td&gt;Mechanizing the GQL semantics in Coq – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/semyon-panenkov-gql-in-coq.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/5xBGohqWCzo&#34;&gt;videos&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:10&lt;/td&gt;
&lt;td&gt;10:25&lt;/td&gt;
&lt;td&gt;Oskar van Rest (Oracle)&lt;/td&gt;
&lt;td&gt;SQL Property Graphs in Oracle Database and Oracle Graph Server (PGX) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/oskar-van-rest-sql-property-graphs-in-oracle-database-and-oracle-graph-server-pgx.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/owM9WiQubpg&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:30&lt;/td&gt;
&lt;td&gt;11:00&lt;/td&gt;
&lt;td&gt;&lt;em&gt;coffee break&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:00&lt;/td&gt;
&lt;td&gt;11:15&lt;/td&gt;
&lt;td&gt;Alastair Green (JCC)&lt;/td&gt;
&lt;td&gt;LDBC&amp;rsquo;s organizational changes and fair use policies – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/alastair-green-ldbc-corporate-restructuring-and-fair-use-policies.pdf&#34;&gt;slides&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:20&lt;/td&gt;
&lt;td&gt;11:35&lt;/td&gt;
&lt;td&gt;Ioana Manolescu (INRIA)&lt;/td&gt;
&lt;td&gt;Integrating Connection Search in Graph Queries – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/ioana-manolescu-integrating-connection-search-in-graph-queries.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/LQPnmcrkUpY&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:40&lt;/td&gt;
&lt;td&gt;11:55&lt;/td&gt;
&lt;td&gt;Maciej Besta (ETH Zurich)&lt;/td&gt;
&lt;td&gt;Neural Graph Databases with Graph Neural Networks – &lt;a href=&#34;https://youtu.be/ce5qNievRNs&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:00&lt;/td&gt;
&lt;td&gt;12:10&lt;/td&gt;
&lt;td&gt;Longbin Lai (Alibaba Damo Academy)&lt;/td&gt;
&lt;td&gt;To Revisit Benchmarking Graph Analytics – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/longbin-lai-benchmark-ldbc.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/s9Vtt-6t_FI&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:15&lt;/td&gt;
&lt;td&gt;13:30&lt;/td&gt;
&lt;td&gt;&lt;em&gt;lunch&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:30&lt;/td&gt;
&lt;td&gt;13:45&lt;/td&gt;
&lt;td&gt;Yuanyuan Tian (Gray Systems Lab, Microsoft)&lt;/td&gt;
&lt;td&gt;The World of Graph Databases from An Industry Perspective – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/yuanyuan-tian-world-of-graph-databases.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/AZuP_b95GPM&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:50&lt;/td&gt;
&lt;td&gt;14:05&lt;/td&gt;
&lt;td&gt;Alin Deutsch (UC San Diego &amp;amp; TigerGraph)&lt;/td&gt;
&lt;td&gt;TigerGraph&amp;rsquo;s Parallel Computation Model – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/alin-deutsch-tigergraphs-computation-model.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/vcxdieJB80Y&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:10&lt;/td&gt;
&lt;td&gt;14:25&lt;/td&gt;
&lt;td&gt;Chen Zhang (CreateLink)&lt;/td&gt;
&lt;td&gt;Applications of a Native Distributed Graph Database in the Financial Industry – &lt;a href=&#34;https://youtu.be/GCCT79Sps9I&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:30&lt;/td&gt;
&lt;td&gt;14:45&lt;/td&gt;
&lt;td&gt;Ricky Sun (Ultipa)&lt;/td&gt;
&lt;td&gt;Design of highly scalable graph database systems – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/ricky-sun-ultipa.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/Sg1F64O4vGM&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:50&lt;/td&gt;
&lt;td&gt;15:30&lt;/td&gt;
&lt;td&gt;&lt;em&gt;coffee break&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:30&lt;/td&gt;
&lt;td&gt;15:45&lt;/td&gt;
&lt;td&gt;Heng Lin (Ant Group)&lt;/td&gt;
&lt;td&gt;The LDBC SNB implementation in TuGraph – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/heng-lin-the-ldbc-snb-implementation-in-tugraph.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/fy8AuVerwnY&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:50&lt;/td&gt;
&lt;td&gt;16:05&lt;/td&gt;
&lt;td&gt;Shipeng Qi (Ant Group)&lt;/td&gt;
&lt;td&gt;FinBench: The new LDBC benchmark targeting financial scenario – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/shipeng-qi-finbench.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/0xLZadDOfZk&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:10&lt;/td&gt;
&lt;td&gt;17:00&lt;/td&gt;
&lt;td&gt;host: Heng Lin (Ant Group), panelists: Longbin Lai (Alibaba Damo Academy), Ricky Sun (Ultipa), Gabor Szarnyas (CWI), Yuanyuan Tian (Gray Systems Lab, Microsoft)&lt;/td&gt;
&lt;td&gt;FinBench panel – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/heng-lin-finbench-panel.pdf&#34;&gt;slides&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;19:00&lt;/td&gt;
&lt;td&gt;22:00&lt;/td&gt;
&lt;td&gt;&lt;em&gt;dinner&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;&lt;em&gt;&lt;a href=&#34;https://www.blackbottleseattle.com/&#34;&gt;Black Bottle gastrotavern&lt;/a&gt; in Belltown: &lt;a href=&#34;https://goo.gl/maps/hQzBRR2nerZEQExw7&#34;&gt;2600 1st Ave (on the corner of Vine), Seattle, WA 98121&lt;/a&gt;&lt;/em&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h4 id=&#34;saturday&#34;&gt;Saturday&lt;/h4&gt;
&lt;p&gt;&lt;strong&gt;Location:&lt;/strong&gt; Amazon Nitro South building, &lt;strong&gt;room 03.204&lt;/strong&gt; (&lt;a href=&#34;https://goo.gl/maps/md5kWUHaNUGhR9JB7&#34;&gt;2205 8th Ave, Seattle, WA 98121&lt;/a&gt;)&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;start&lt;/th&gt;
&lt;th&gt;finish&lt;/th&gt;
&lt;th&gt;speaker&lt;/th&gt;
&lt;th&gt;title&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;09:00&lt;/td&gt;
&lt;td&gt;09:45&lt;/td&gt;
&lt;td&gt;Brad Bebee (AWS)&lt;/td&gt;
&lt;td&gt;Customers don&amp;rsquo;t want a graph database, so why are we still here? – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/brad-bebee-tuc-keynote.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/bJlkpDC--fM&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:00&lt;/td&gt;
&lt;td&gt;10:15&lt;/td&gt;
&lt;td&gt;Muhammad Attahir Jibril (TU Ilmenau)&lt;/td&gt;
&lt;td&gt;Fast and Efficient Update Handling for Graph H2TAP – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/muhammad-attahir-jibril-fast-and-efficient-update-handling-for-graph-h2tap.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/e8ZAszBsXV0&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:20&lt;/td&gt;
&lt;td&gt;11:00&lt;/td&gt;
&lt;td&gt;&lt;em&gt;coffee break&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:00&lt;/td&gt;
&lt;td&gt;11:15&lt;/td&gt;
&lt;td&gt;Gabor Szarnyas (CWI)&lt;/td&gt;
&lt;td&gt;LDBC Social Network Benchmark and Graphalytics – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/gabor-szarnyas-ldbc-social-network-benchmark-and-graphalytics.pdf&#34;&gt;slides&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:20&lt;/td&gt;
&lt;td&gt;11:30&lt;/td&gt;
&lt;td&gt;Atanas Kiryakov and Tomas Kovachev (Ontotext)&lt;/td&gt;
&lt;td&gt;GraphDB – Benchmarking against LDBC SNB &amp;amp; SPB – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/tomas-kovatchev-atanas-kiryakov-benchmarking-graphdb-with-snb-and-spb.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/U6OPpNFOWqg&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:35&lt;/td&gt;
&lt;td&gt;11:50&lt;/td&gt;
&lt;td&gt;Roi Lipman (Redis Labs)&lt;/td&gt;
&lt;td&gt;Delta sparse matrices within RedisGraph – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/roi-lipman-delta-matrix.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/qfKsplV4Ihk&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:55&lt;/td&gt;
&lt;td&gt;12:05&lt;/td&gt;
&lt;td&gt;Rathijit Sen (Microsoft)&lt;/td&gt;
&lt;td&gt;Microarchitectural Analysis of Graph BI Queries on RDBMS – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/rathijit-sen-microarchitectural-analysis.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/55B8CkH09js&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:10&lt;/td&gt;
&lt;td&gt;13:30&lt;/td&gt;
&lt;td&gt;&lt;em&gt;lunch&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;&lt;em&gt;on your own&lt;/em&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:30&lt;/td&gt;
&lt;td&gt;13:45&lt;/td&gt;
&lt;td&gt;Alastair Green (JCC)&lt;/td&gt;
&lt;td&gt;LEX &amp;ndash; LDBC Extended GQL Schema – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/alastair-green-lex.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/DVpeb4Ce9Uw&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:50&lt;/td&gt;
&lt;td&gt;14:05&lt;/td&gt;
&lt;td&gt;Ora Lassila (AWS)&lt;/td&gt;
&lt;td&gt;Why limit yourself to {RDF, LPG} when you can do {RDF, LPG}, too – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/ora-lassila-why-limit-yourself-to-lpg-when-you-can-do-rdf-too.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/7uAInoUwdds&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:10&lt;/td&gt;
&lt;td&gt;14:25&lt;/td&gt;
&lt;td&gt;Jan Hidders (Birkbeck, University of London)&lt;/td&gt;
&lt;td&gt;PG-Schema: a proposal for a schema language for property graphs – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/jan-hidders-pg-schema.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/yQNL8hBTE4M&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:30&lt;/td&gt;
&lt;td&gt;14:45&lt;/td&gt;
&lt;td&gt;Max de Marzi (RageDB and RelationalAI)&lt;/td&gt;
&lt;td&gt;RageDB: Building a Graph Database in Anger – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/max-de-marzi-ragedb-building-a-graph-database-in-anger.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/LBbF8aslYFE&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:50&lt;/td&gt;
&lt;td&gt;15:30&lt;/td&gt;
&lt;td&gt;&lt;em&gt;coffee break&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:30&lt;/td&gt;
&lt;td&gt;15:45&lt;/td&gt;
&lt;td&gt;Umit Catalyurek (AWS)&lt;/td&gt;
&lt;td&gt;HPC Graph Analytics on the OneGraph Model – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/umit-catalyurek-onegraph-hpc.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/64tv5LA6Wr8&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:50&lt;/td&gt;
&lt;td&gt;16:05&lt;/td&gt;
&lt;td&gt;David J. Haglin (Trovares)&lt;/td&gt;
&lt;td&gt;How LDBC impacts Trovares – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/david-haglin-trovares.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:10&lt;/td&gt;
&lt;td&gt;16:25&lt;/td&gt;
&lt;td&gt;Wenyuan Yu (Alibaba Damo Academy)&lt;/td&gt;
&lt;td&gt;GraphScope Flex: A Graph Computing Stack with LEGO-Like Modularity – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/wenyuan-yu-graphscope-flex.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/cRikoyDmMks&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:30&lt;/td&gt;
&lt;td&gt;16:40&lt;/td&gt;
&lt;td&gt;Scott McMillan (Carnegie Mellon University)&lt;/td&gt;
&lt;td&gt;Graph processing using GraphBLAS – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/scott-mcmillan-graph-processing-using-graphblas.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/yb4hGBhUzQQ&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:45&lt;/td&gt;
&lt;td&gt;16:55&lt;/td&gt;
&lt;td&gt;Tim Mattson (Intel)&lt;/td&gt;
&lt;td&gt;Graphs (GraphBLAS) and storage (TileDB) as Sparse Linear algebra – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixteenth-tuc-meeting/attachments/tim-mattson-graphblas-and-tiledb.pdf&#34;&gt;slides&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;17:00&lt;/td&gt;
&lt;td&gt;20:00&lt;/td&gt;
&lt;td&gt;&lt;em&gt;happy hour (rooftop grill with beverages)&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;&lt;em&gt;on the Nitro South building&amp;rsquo;s 8th floor deck&lt;/em&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h4 id=&#34;tuc-event-locations&#34;&gt;TUC event locations&lt;/h4&gt;
&lt;p&gt;A &lt;a href=&#34;https://www.google.com/maps/d/u/0/edit?mid=19_fi4fV-3-PZkNWCCcmhU86ct2EZXbgo&#34;&gt;map of the LDBC TUC events&lt;/a&gt; we hosted so far.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>LDBC SNB – Early 2023 updates</title>
      <link>https://ldbcouncil.org/post/ldbc-snb-early-2023-updates/</link>
      <pubDate>Wed, 15 Feb 2023 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/ldbc-snb-early-2023-updates/</guid>
      <description>&lt;p&gt;2023 has been an eventful year for us so far. Here is a summary of our recent activities.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Our paper &lt;a href=&#34;https://ldbcouncil.org/docs/papers/ldbc-snb-bi-vldb-2022.pdf&#34;&gt;The LDBC Social Network Benchmark: Business Intelligence Workload&lt;/a&gt; was published in PVLDB.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;David Püroja just completed his MSc thesis on creating a design towards &lt;a href=&#34;https://ldbcouncil.org/docs/papers/msc-thesis-david-puroja-snb-interactive-v2-2023.pdf&#34;&gt;SNB Interactive v2&lt;/a&gt; at CWI&amp;rsquo;s Database Architectures group. David and I gave a deep-dive talk at the FOSDEM conference&amp;rsquo;s graph developer room titled &lt;a href=&#34;https://fosdem.org/2023/schedule/event/graph_ldbc/&#34;&gt;The LDBC Social Network Benchmark&lt;/a&gt; (&lt;a href=&#34;https://www.youtube.com/watch?v=YNF6z6gtXY4&#34;&gt;YouTube mirror&lt;/a&gt;).&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;I gave a lightning talk at FOSDEM&amp;rsquo;s HPC developer room titled &lt;a href=&#34;https://www.youtube.com/watch?v=q26DHnQFw54&#34;&gt;The LDBC Benchmark Suite&lt;/a&gt; (&lt;a href=&#34;https://www.youtube.com/watch?v=q26DHnQFw54&#34;&gt;YouTube mirror&lt;/a&gt;).&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Our auditors have successfully benchmark a number of systems:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;SPB with the Ontotext GraphDB systems for the SF3 and SF5 data sets (auditor: Pjotr Scholtze)&lt;/li&gt;
&lt;li&gt;SNB Interactive with the Ontotext GraphDB system for the SF30 data set (auditor: David Püroja)&lt;/li&gt;
&lt;li&gt;SNB Interactive with the TuGraph system running in the Aliyun cloud for the SF30, SF100, and SF300 data sets (auditor: Márton Búr)&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The results and the full disclosure reports are available under the &lt;a href=&#34;https://ldbcouncil.org/benchmarks/spb/&#34;&gt;SPB&lt;/a&gt; and &lt;a href=&#34;https://ldbcouncil.org/benchmarks/snb/&#34;&gt;SNB benchmark pages&lt;/a&gt;.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>LDBC SNB Datagen – The winding path to SF100K</title>
      <link>https://ldbcouncil.org/post/ldbc-snb-datagen-the-winding-path-to-sf100k/</link>
      <pubDate>Tue, 13 Sep 2022 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/ldbc-snb-datagen-the-winding-path-to-sf100k/</guid>
      <description>&lt;p&gt;LDBC SNB provides a data generator, which produces synthetic datasets, mimicking a social network’s activity during a period of time. Datagen is defined by the charasteristics of realism, scalability, determinism and usability. More than two years have elapsed since my &lt;a href=&#34;https://ldbcouncil.org/post/speeding-up-ldbc-snb-datagen/&#34;&gt;last technical update&lt;/a&gt; on LDBC SNB Datagen, in which I discussed the reasons for moving the code to Apache Spark from the MapReduce-based Apache Hadoop implementation and the challenges I faced during the migration. Since then, we reached several goals such as we refactored the serializers to use Spark&amp;rsquo;s high-level writers to support the popular Parquet data format and to enable running on spot nodes; brought back factor generation; implemented support for the novel BI benchmark; and optimized the runtime to generate SF30K on 20 i3.4xlarge machines on AWS.&lt;/p&gt;
&lt;h1 id=&#34;moving-to-sparksql&#34;&gt;Moving to SparkSQL&lt;/h1&gt;
&lt;p&gt;We planned to move parts of the code to SparkSQL, an optimized runtime framework for tabular data. We hypothesized that this would benefit us on multiple fronts: SparkSQL offers an efficient batch analytics runtime, with higher level abstractions that are simpler to understand and work with, and we could easily add support for serializing to Parquet based on SparkSQL&amp;rsquo;s capabilites.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Spark SQL is a Spark module for structured data processing. It provides a programming abstraction called DataFrames and can also act as a distributed SQL query engine. Spark SQL includes a cost-based optimizer, columnar storage, and code generation to make queries fast.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Dealing with the dataset generator proved quite tricky, because it samples from various hand-written distributions and dictionaries, and contains complex domain logic, for which SparkSQL unsuitable. We assessed that the best thing we could do is wrap entire entity generation procedures in UDFs (user defined SQL functions). However, several of these generators return entity trees&lt;sup id=&#34;fnref:1&#34;&gt;&lt;a href=&#34;#fn:1&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;1&lt;/a&gt;&lt;/sup&gt;, which are spread across multiple tables by the serializer, and these would have needed to be split up. Further complicating matters, we would have also had to find a way to coordinate the inner random generators&amp;rsquo; state between the UDFs to ensure deterministic execution. Weighing these and that we could not find much benefit in SparkSQL, we ultimately decided to leave entity generation as it is. We limited the SparkSQL refactor to the following areas:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;table manipulations related to shaping the output into the supported layouts and data types as set forth in the specification;&lt;/li&gt;
&lt;li&gt;deriving the Interactive and BI datasets;&lt;/li&gt;
&lt;li&gt;and generating the factor tables, which contain analytic information, such as population per country, number of friendships between city pairs, number of messages per day, etc., used by the substitution parameter generator to ensure predictable query runtimes.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;We refer to points (1.) and (2.) collectively as dataset transformation, while (3.) as factor generation. Initially, these had been part of the generator, extracted as part of this refactor, which resulted in cleaner, more maintainable design.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;datagen_df_0.png&#34; alt=&#34;Datagen stages&#34;&gt;&lt;/p&gt;
&lt;p&gt;The diagram above shows the components on a high level. The generator outputs a dataset called IR (intermediate representation), which is immediately written to disk. Then, the IR is input to the dataset transformation and factor generation stages, which respectively generate the final dataset and the factor tables. We are aware that spitting out the IR adds considerable runtime overhead and doubles the disk requirements in the worst-case scenario, however, we found that there&amp;rsquo;s no simple way to avoid&lt;br&gt;
it, as the generator produces entity trees, which are incompatible with the flat, tabular, column oriented layout of SparkSQL. On the positive side, this design enables us to reuse the generator output for multiple transformations and add new factor tables without regenerating the data.&lt;/p&gt;
&lt;p&gt;I&amp;rsquo;ll skip describing the social network graph dataset generator (i.e. stage 1) in any more detail, apart from its serializer, as that was the only part involved in the current refactor. If you are interested in more details, you may look up the &lt;a href=&#34;https://ldbcouncil.org/post/speeding-up-ldbc-snb-datagen/&#34;&gt;previous blogpost in the series&lt;/a&gt; or the &lt;a href=&#34;https://arxiv.org/abs/2001.02299&#34;&gt;Interactive benchmark specification&lt;/a&gt;.&lt;/p&gt;
&lt;h1 id=&#34;transformation-pipeline&#34;&gt;Transformation pipeline&lt;/h1&gt;
&lt;p&gt;The dataset transformation stage sets off where generation finished, and applies an array of pluggable transformations:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;explodes edges and / or attributes into separate tables,&lt;/li&gt;
&lt;li&gt;subsets the snapshot part and creates insert / delete batches for the BI workload,&lt;/li&gt;
&lt;li&gt;subsets the snapshot part for the Interactive workload,&lt;/li&gt;
&lt;li&gt;applies formatting related options such as date time representation,&lt;/li&gt;
&lt;li&gt;serializes the data to a Spark supported format (CSV, Parquet),&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We utilize a flexible data pipeline that operates on the graph.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-scala&#34; data-lang=&#34;scala&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;trait&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Transform&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;M1&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;&amp;lt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Mode&lt;/span&gt;, &lt;span style=&#34;color:#66d9ef&#34;&gt;M2&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;&amp;lt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Mode&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;extends&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;Graph&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;M1&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;=&amp;gt;&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Graph&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;M2&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;])&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;In&lt;/span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Graph&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;M1&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Out&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Graph&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;M2&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; transform&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;input&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;In&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Out&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;override&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; apply&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;v&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Graph&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;M1&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;])&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Graph&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;M2&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; transform&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;v&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The &lt;code&gt;Transform&lt;/code&gt; trait encodes a pure (side effect-free) function polymorphic over graphs, so that transformation pipelines can be expressed with ordinary function composition in a type safe manner. Let&amp;rsquo;s see some of the transformations we have.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-scala&#34; data-lang=&#34;scala&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;case&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;class&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;RawToBiTransform&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;mode&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;BI&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; simulationStart&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Long&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; simulationEnd&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Long&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; keepImplicitDeletes&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Boolean&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;extends&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Transform&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Raw.&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt;, &lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.BI&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;override&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; transform&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;input&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;In&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Out&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;???&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;case&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;class&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;RawToInteractiveTransform&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;mode&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Interactive&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; simulationStart&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Long&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; simulationEnd&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Long&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;extends&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Transform&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Raw.&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt;, &lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Interactive&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;override&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; transform&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;input&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;In&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Out&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;???&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;object&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;ExplodeEdges&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;extends&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Transform&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Raw.&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt;, &lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Raw.&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;override&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; transform&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;input&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;In&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Out&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;???&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;object&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;ExplodeAttrs&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;extends&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Transform&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Raw.&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt;, &lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Raw.&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;override&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; transform&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;input&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;In&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Out&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;???&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Therefore, a transformation pipeline may look like this:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-scala&#34; data-lang=&#34;scala&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; transform &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;ExplodeAttrs&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;andThen&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;ExplodeEdges&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;andThen&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;RawToInteractiveTransform&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;params&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; start&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; end&lt;span style=&#34;color:#f92672&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; outputGraph &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; transform&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;inputGraph&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The &lt;code&gt;Graph&lt;/code&gt; record has a &lt;code&gt;definition&lt;/code&gt; field containing graph-global metadata, whereas &lt;code&gt;entities&lt;/code&gt; holds the datasets keyed by their entity type. There are 3 graph &lt;em&gt;modes&lt;/em&gt; currently: &lt;code&gt;Raw&lt;/code&gt;, &lt;code&gt;Interactive&lt;/code&gt; and &lt;code&gt;BI&lt;/code&gt;. The BI dataset has different layout than the rest, as it contains incremental inserts and deletes for the entities additionally to the bulk snapshot. This is captured in the &lt;code&gt;Layout&lt;/code&gt; dependent type, over which the entities are polymorphic.&lt;/p&gt;
&lt;p&gt;It&amp;rsquo;s important to understand that &lt;code&gt;Graph&lt;/code&gt; holds &lt;code&gt;DataFrame&lt;/code&gt;s, and these are lazily computed by Spark. So, &lt;code&gt;Graph&lt;/code&gt; is merely a description of transformations used to derive the comprising datasets, which makes them subject to all the SparkSQL fanciness such as query optimization, whole stage code generation, and so on. Processing is delayed until an action (such as a disk write) forces it.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-scala&#34; data-lang=&#34;scala&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;case&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;class&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;GraphDef&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;+M&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;&amp;lt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Mode&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;](&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    isAttrExploded&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Boolean&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    isEdgesExploded&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Boolean&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    useTimestamp&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Boolean&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    mode&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;M&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    entities&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Map&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;EntityType&lt;/span&gt;, &lt;span style=&#34;color:#66d9ef&#34;&gt;Option&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;String&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;case&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;class&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Graph&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;+M&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;&amp;lt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Mode&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;](&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    definition&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;GraphDef&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;M&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    entities&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Map&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;EntityType&lt;/span&gt;, &lt;span style=&#34;color:#66d9ef&#34;&gt;M&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;#&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;Layout&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;sealed&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;trait&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Mode&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Layout&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#75715e&#34;&gt;/* ... */&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;object&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Mode&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;final&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;case&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;object&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Raw&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;extends&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Mode&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Layout&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;DataFrame&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#75715e&#34;&gt;/* ... */&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;final&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;case&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;class&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Interactive&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;bulkLoadPortion&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Double&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;extends&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Mode&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Layout&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;DataFrame&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#75715e&#34;&gt;/* ... */&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;final&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;case&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;class&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;BI&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;bulkloadPortion&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Double&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; batchPeriod&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;String&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;extends&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Mode&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Layout&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;BatchedEntity&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#75715e&#34;&gt;/* ... */&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;You may notice that &lt;code&gt;Transform&lt;/code&gt; is statically typed w.r.t. &lt;code&gt;Mode&lt;/code&gt;, however other properties, like &lt;code&gt;isAttrExploded&lt;/code&gt;, or &lt;code&gt;isEdgesExploded&lt;/code&gt; are not captured in the type, and remain merely dynamic. This makes some nonsensical transformation pipelines (i.e. that explodes edges twice in a row) syntactically valid. This trade-off in compile-time safety was made to prevent overcomplicating the types.&lt;/p&gt;
&lt;p&gt;As we already mentioned, &lt;code&gt;Graph&lt;/code&gt; is essentially a persistent container of &lt;code&gt;EntityType -&amp;gt; DataFrame&lt;/code&gt; mappings. &lt;code&gt;EntityType&lt;/code&gt; can be &lt;code&gt;Node&lt;/code&gt;, &lt;code&gt;Edge&lt;/code&gt; and &lt;code&gt;Attr&lt;/code&gt;, and is used to identify the entity and embellish with static metadata, such a descriptive name and primary key, whether it is static or dynamic (as per the specification), and in case of edges, the source and destination type and cardinality. This makes it very simple to create transformation rules on static entity properties with pattern matching.&lt;/p&gt;
&lt;p&gt;Usually, a graph transformation involves matching entities based on their &lt;code&gt;EntityType&lt;/code&gt;, and modifying the mapping (and if required, other metadata). Take, for example, the &lt;code&gt;ExplodeAttrs&lt;/code&gt; transformation, which explodes into separate tables the values of two columns of &lt;code&gt;Person&lt;/code&gt; stored as arrays:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-scala&#34; data-lang=&#34;scala&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;object&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;ExplodeAttrs&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;extends&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Transform&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Raw.&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt;, &lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Raw.&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;override&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; transform&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;input&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;In&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Out&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;if&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;input&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;definition&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;isAttrExploded&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;                              &lt;span style=&#34;color:#75715e&#34;&gt;// assert at runtime that the transformation hasn&amp;#39;t been applied yet
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;      &lt;span style=&#34;color:#66d9ef&#34;&gt;throw&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;new&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;AssertionError&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Attributes already exploded in the input graph&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; explodedAttr&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;attr&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Attr&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; node&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;DataFrame&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; column&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Column&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;      attr &lt;span style=&#34;color:#f92672&#34;&gt;-&amp;gt;&lt;/span&gt; node&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;select&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;withRawColumns&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;attr&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;id&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;as&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;s&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;${&lt;/span&gt;attr&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;parent&lt;span style=&#34;color:#e6db74&#34;&gt;}&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;Id&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt; explode&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;split&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;column&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;;&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)).&lt;/span&gt;as&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;s&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;${&lt;/span&gt;attr&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;attribute&lt;span style=&#34;color:#e6db74&#34;&gt;}&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;Id&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; modifiedEntities &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; input&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;entities
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;      &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;collect &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;case&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;k &lt;span style=&#34;color:#66d9ef&#34;&gt;@&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Node&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Person&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;false&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt; df&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;=&amp;gt;&lt;/span&gt;                &lt;span style=&#34;color:#75715e&#34;&gt;// match the Person node. This is the only one ExplodeAttrs should modify
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;        &lt;span style=&#34;color:#a6e22e&#34;&gt;Map&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;          explodedAttr&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;Attr&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Email&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; k&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;EmailAddress&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt; df&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;email&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt; &lt;span style=&#34;color:#75715e&#34;&gt;// add a new &amp;#34;PersonEmailEmailAddress&amp;#34; entity derived by exploding the email column of Person
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;          explodedAttr&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;Attr&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Speaks&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; k&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Language&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt; df&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;language&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt; &lt;span style=&#34;color:#75715e&#34;&gt;// add a new &amp;#34;PersonSpeaksLanguage&amp;#34; entity derived by exploding the language column of Person
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;          k &lt;span style=&#34;color:#f92672&#34;&gt;-&amp;gt;&lt;/span&gt; df&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;drop&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;email&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;language&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;                             &lt;span style=&#34;color:#75715e&#34;&gt;// drop the exploded columns from person
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;        &lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;      &lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; updatedEntities &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; modifiedEntities
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;      &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;foldLeft&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;input&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;entities&lt;span style=&#34;color:#f92672&#34;&gt;)(&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;_&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;++&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;_&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;                                 &lt;span style=&#34;color:#75715e&#34;&gt;// merge-replace the modified entities in the graph
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; updatedEntityDefinitions &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; modifiedEntities
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;      &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;foldLeft&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;input&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;definition&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;entities&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;e&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; v&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;=&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;        e &lt;span style=&#34;color:#f92672&#34;&gt;++&lt;/span&gt; v&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;map&lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;case&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;k&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; v&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;=&amp;gt;&lt;/span&gt; k &lt;span style=&#34;color:#f92672&#34;&gt;-&amp;gt;&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Some&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;v&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;schema&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;toDDL&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;          &lt;span style=&#34;color:#75715e&#34;&gt;// update the entity definition schema to reflect the modifications
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;      &lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; l &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; lens&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;In&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt;                                                    &lt;span style=&#34;color:#75715e&#34;&gt;// lenses provide a terse syntax for modifying nested fields
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;l&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;definition&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;isAttrExploded &lt;span style=&#34;color:#f92672&#34;&gt;~&lt;/span&gt; l&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;definition&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;entities &lt;span style=&#34;color:#f92672&#34;&gt;~&lt;/span&gt; l&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;entities&lt;span style=&#34;color:#f92672&#34;&gt;).&lt;/span&gt;set&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;input&lt;span style=&#34;color:#f92672&#34;&gt;)((&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;true&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; updatedEntityDefinitions&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; updatedEntities&lt;span style=&#34;color:#f92672&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Note that &lt;code&gt;EntityType&lt;/code&gt; does not hold the dataset&amp;rsquo;s full SQL schema currently, as it&amp;rsquo;s not useful for pattern matching, but can be accessed directly from &lt;code&gt;DataFrame&lt;/code&gt; if needed.&lt;/p&gt;
&lt;h1 id=&#34;inputoutput&#34;&gt;Input/output&lt;/h1&gt;
&lt;p&gt;The &lt;code&gt;Reader&lt;/code&gt; and &lt;code&gt;Writer&lt;/code&gt; typeclasses are used to read from a &lt;code&gt;Source&lt;/code&gt; and write to a &lt;code&gt;Sink&lt;/code&gt; respectively, terminating a graph transformation pipeline&lt;br&gt;
on both ends.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-scala&#34; data-lang=&#34;scala&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;trait&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Reader&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;T&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Ret&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; read&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;self&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;T&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Ret&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; exists&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;self&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;T&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Boolean&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;trait&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Writer&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;S&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Data&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;def&lt;/span&gt; write&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;self&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Data&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; sink&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;S&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;:&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;Unit&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;There are implementations under &lt;code&gt;ldbc.datagen.io.instances&lt;/code&gt; that read a graph from a &lt;code&gt;GraphSource&lt;/code&gt; and write to a &lt;code&gt;GraphSink&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-scala&#34; data-lang=&#34;scala&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;import&lt;/span&gt; ldbc.snb.datagen.model
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;import&lt;/span&gt; ldbc.snb.datagen.model.Mode
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;import&lt;/span&gt; ldbc.snb.datagen.io.graphs.&lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;GraphSource&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;GraphSink&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;import&lt;/span&gt; ldbc.snb.datagen.io.instances._
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// read
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; inputPath &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;path/to/input/graph&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; inputFormat &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;parquet&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; source &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;GraphSource&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;model&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;graphs&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;Raw&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;graphDef&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; inputPath&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; inputFormat&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; graph &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Reader&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;GraphSource&lt;/span&gt;, &lt;span style=&#34;color:#66d9ef&#34;&gt;Graph&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Raw.&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]].&lt;/span&gt;read&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;source&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// transform
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; transform &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;ExplodeAttrs&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;andThen&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;ExplodeEdges&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; transformedGraph &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; transform&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;graph&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// write
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; outputPath &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;path/to/output/graph&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; outputFormat &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;csv&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; sink &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;GraphSink&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;outputPath&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; outputFormat&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;Writer&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;GraphSink&lt;/span&gt;, &lt;span style=&#34;color:#66d9ef&#34;&gt;Graph&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;Mode.Raw.&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;type&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;]].&lt;/span&gt;write&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;transformedGraph&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; sink&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We provide &lt;a href=&#34;https://github.com/typelevel/simulacrum&#34;&gt;Ops syntax&lt;/a&gt; to make it shorter:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-scala&#34; data-lang=&#34;scala&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;import&lt;/span&gt; ldbc.snb.datagen.model
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;import&lt;/span&gt; ldbc.snb.datagen.model.Mode
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;import&lt;/span&gt; ldbc.snb.datagen.io.graphs.&lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;GraphSource&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;GraphSink&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;import&lt;/span&gt; ldbc.snb.datagen.io.instances._
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;import&lt;/span&gt; ldbc.snb.datagen.io.Reader.ops._
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;import&lt;/span&gt; ldbc.snb.datagen.io.Writer.ops._
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// read
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; inputPath &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;path/to/input/graph&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; inputFormat &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;parquet&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; graph &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;GraphSource&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;model&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;graphs&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;Raw&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;graphDef&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; inputPath&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; inputFormat&lt;span style=&#34;color:#f92672&#34;&gt;).&lt;/span&gt;read
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// transform
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; transformedGraph &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;???&lt;/span&gt; &lt;span style=&#34;color:#75715e&#34;&gt;/* ... */&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// write
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; outputPath &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;path/to/output/graph&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; outputFormat &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;csv&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;transformedGraph&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;write&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;GraphSink&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;outputPath&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; outputFormat&lt;span style=&#34;color:#f92672&#34;&gt;))&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The reader/writer architecture is layered, the graph reader/writer uses dataframe readers/writers for each of its entities. One interesting aspect of implementing the reader was dealing with the input schema. Parquet is self-describing, however as we also support the CSV format, we had to provide a way for correct schema detection and column parsing.&lt;/p&gt;
&lt;p&gt;Spark has a facility to derive SparkSQL schema from case classes automatically&lt;sup id=&#34;fnref:2&#34;&gt;&lt;a href=&#34;#fn:2&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;2&lt;/a&gt;&lt;/sup&gt;. We created case classes for each entity in the &lt;code&gt;Raw&lt;/code&gt; dataset.  We also created a typeclass &lt;code&gt;EntityTraits&lt;/code&gt; associating these classes with their &lt;code&gt;EntityType&lt;/code&gt;, so we can summon them (and consequently their SparkSQL schema) in the reader.&lt;/p&gt;
&lt;p&gt;The case classes are used during the serialization of the generated dataset too, but more about that later.&lt;/p&gt;
&lt;h1 id=&#34;factor-generation&#34;&gt;Factor generation&lt;/h1&gt;
&lt;p&gt;As we already mentioned, factor generation was originally part of the data generator, i.e. factor tables were calculated on the fly and emitted as side outputs. This design had some problems. Auxiliary data structures had to be maintained and interleaved with generation, which violated separation of concerns, consequently hurting readability and maintainability. Also, anything more complicated than entity local aggregates where impossible to express in the original MapReduce framework. To keep the preceding Spark rewrite at a managable scope, the original factor generation code had been removed.&lt;/p&gt;
&lt;p&gt;We decided it&amp;rsquo;s best to reintroduce factor generation as a post-processing step that operates on the generated data. This makes it possible to express more complex analytical queries, requires no prior knowledge about the generator, can be done in SparkSQL (making it much simpler), and removes the impact on the generator&amp;rsquo;s performance, so that we can optimize them separately. Since this refactor, we almost tripled the number factor tables (up to 31 to cover both SNB workloads, BI and Interactive). The queries computing of certain factor tables even use &lt;a href=&#34;https://spark.apache.org/graphx/&#34;&gt;GraphX&lt;/a&gt;, which was unimaginable with the previous design.&lt;/p&gt;
&lt;p&gt;Factor tables are added by extending a map with a &lt;code&gt;name -&amp;gt; Factor&lt;/code&gt; pair. &lt;code&gt;Factor&lt;/code&gt; declares is input entities, and accepts a function that receives input &lt;code&gt;DataFrames&lt;/code&gt;, and returns a single &lt;code&gt;DataFrame&lt;/code&gt; as output.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-scala&#34; data-lang=&#34;scala&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; factors &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Map&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;personDisjointEmployerPairs&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;-&amp;gt;&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Factor&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;PersonType&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;PersonKnowsPersonType&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;OrganisationType&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;PersonWorkAtCompanyType&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;    &lt;span style=&#34;color:#66d9ef&#34;&gt;case&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;Seq&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;person&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; personKnowsPerson&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; organisation&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; workAt&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;=&amp;gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;      &lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; knows &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; undirectedKnows&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;personKnowsPerson&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;      &lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; company &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; organisation&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;where&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;$&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Type&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;===&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Company&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;).&lt;/span&gt;cache&lt;span style=&#34;color:#f92672&#34;&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;      &lt;span style=&#34;color:#66d9ef&#34;&gt;val&lt;/span&gt; personSample &lt;span style=&#34;color:#66d9ef&#34;&gt;=&lt;/span&gt; person
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;        &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;orderBy&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;$&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;id&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;        &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;limit&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;20&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;      personSample
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;        &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;as&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Person2&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;        &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;join&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;knows&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;as&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;knows&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt; $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;knows.person2Id&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;===&lt;/span&gt; $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Person2.id&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;        &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;join&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;workAt&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;as&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;workAt&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt; $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;workAt.PersonId&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;===&lt;/span&gt; $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;knows.Person1id&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;        &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;join&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;company&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;as&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Company&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt; $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Company.id&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#f92672&#34;&gt;===&lt;/span&gt; $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;workAt.CompanyId&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;        &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;select&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;          $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Person2.id&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;alias&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;person2id&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;          $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Company.name&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;alias&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;companyName&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;          $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Company.id&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;alias&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;companyId&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;          $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Person2.creationDate&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;alias&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;person2creationDate&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;),&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;          $&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Person2.deletionDate&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;alias&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;person2deletionDate&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;        &lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;        &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;distinct&lt;span style=&#34;color:#f92672&#34;&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#75715e&#34;&gt;/* more factors */&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;As you can see, it&amp;rsquo;s not much complicated than using plain SQL, with the added benefit of being able to extract recurring subqueries to functions (e.g. &lt;code&gt;undirectedKnows&lt;/code&gt;). Currently, there&amp;rsquo;s no parallelization between different factor tables (although each of them is parallelized internally by Spark). The Factor table writer uses the same componentized architecture as the graph writer, i.e. it uses the dataframe writer under the hood.&lt;/p&gt;
&lt;h1 id=&#34;revamping-the-data-generators-serializer&#34;&gt;Revamping the data generator&amp;rsquo;s serializer&lt;/h1&gt;
&lt;p&gt;At this point, both the transformation pipeline and factor generator was ready, however the data generator was still chugging with the old serializer, emitting the IR in CSV. We wanted to move this to Parquet to improve performance and reduce its size, but there was a problem: due to the generator&amp;rsquo;s custom data representation, SparkSQL (and its DataSource API) was off-limits. So we&amp;rsquo;ve bitten the bullet, and rewritten the existing serializer to emit Parquet.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;a href=&#34;https://parquet.apache.org/&#34;&gt;Parquet&lt;/a&gt; is an open source data format that evolved to be the de facto standard for Big Data batch pipelines. It offers a column-oriented, compressed, schemaful representation that is space-efficient and suited for analytic queries. The file format leverages a record shredding and assembly model, which originated at Google. This results in a file that is optimized for query performance and minimizing I/O.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The new serialization framework is heavily influenced by the design of Java &lt;code&gt;OutputStreams&lt;/code&gt;, in the sense that stateful objects are composed to form a pipeline. For example, in case of &lt;em&gt;activities&lt;/em&gt;, the input is an activity tree, and the output is a set of rows in multiple files (eg. forum, forumHasTag, post, postHasTag, etc.). The components that take part in activity serialization are shown on the diagram below. The activity tree is iterated (1st component) and the corresponding entity serializer is called (2nd component), which is fed into a component that splits the records (3rd one) among several output streams writing individual files (last).&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;activity.png&#34; alt=&#34;Activity serialization pipeline&#34;&gt;&lt;/p&gt;
&lt;p&gt;The benefit of this architecture is that only the last component needs to change when we add support for a new output format.&lt;/p&gt;
&lt;p&gt;To support Parquet, we made use of row-level serializers available in Hadoop&amp;rsquo;s Parquet library (bundled with SparkSQL), and internal classes in SparkSQL to derive Parquet schema for our entities. Remember how we used case classes for the &lt;code&gt;Raw&lt;/code&gt; entities to derive the input schema in the graph reader during dataset transformation? Here we use the same classes (e.g. &lt;code&gt;Forum&lt;/code&gt;) and Spark&amp;rsquo;s &lt;code&gt;Encoder&lt;/code&gt; framework to encode the entities in Parquet, which means that the generated output remains consistent with &lt;code&gt;DataFrame&lt;/code&gt;-based reader, and we spare a lot of code duplication.&lt;/p&gt;
&lt;h1 id=&#34;optimizations&#34;&gt;Optimizations&lt;/h1&gt;
&lt;p&gt;After these refactors, we were able to generate the BI dataset with scale factor 10K on 300 i3.4xlarge machines in one hour. Decreasing the number of machines resulted in out of memory errors in the generator. We realized partition sizes (and thus the number of partitions) should be determined based on available memory. Our experiments showed that a machine with 128GB of memory is capable of generating SF3K (scale factor 3000) reliably with 3 blocks&lt;sup id=&#34;fnref:3&#34;&gt;&lt;a href=&#34;#fn:3&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;3&lt;/a&gt;&lt;/sup&gt; per partition given ample disk size to allow for spills (tested with 3.8TB); while less partitions (subsequently, larger block/partition ratio) would introduce OOM errors. Furthermore, we split the data generator output after a certain number of rows written, to fend against the skew between different kinds of entities possibly causing problems during transformation&lt;sup id=&#34;fnref:4&#34;&gt;&lt;a href=&#34;#fn:4&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;4&lt;/a&gt;&lt;/sup&gt;. These optimizations enabled us to run SF10K reliably on 4 i3.4xlarge machines in 11 hours (which is still more than 6x reduction in cost). We weren&amp;rsquo;t able to run SF30K run on 10 machines (1 machine / SF3K), even 15 ran out of disk. This non-linear disk use should be investigated further as it complicates calculating cluster sizes for larger scale factors.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;./tools/emr/submit_datagen_job.py sf3k_bi &lt;span style=&#34;color:#ae81ff&#34;&gt;3000&lt;/span&gt; parquet bi &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  --sf-per-executor &lt;span style=&#34;color:#ae81ff&#34;&gt;3000&lt;/span&gt; &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  --partitions &lt;span style=&#34;color:#ae81ff&#34;&gt;330&lt;/span&gt; &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  --jar $JAR_NAME &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  --instance-type i3.4xlarge &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  --bucket $BUCKET_NAME &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  -- --explode-edges --explode-attrs
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;./tools/emr/submit_datagen_job.py sf10k_bi &lt;span style=&#34;color:#ae81ff&#34;&gt;10000&lt;/span&gt; parquet bi &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  --sf-per-executor &lt;span style=&#34;color:#ae81ff&#34;&gt;3000&lt;/span&gt; &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  --partitions &lt;span style=&#34;color:#ae81ff&#34;&gt;1000&lt;/span&gt; &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  --jar $JAR_NAME &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  --instance-type i3.4xlarge &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  --bucket $BUCKET_NAME &lt;span style=&#34;color:#ae81ff&#34;&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;&lt;/span&gt;  -- --explode-edges --explode-attrs
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The above examples working configurations for generating the 3K and 10K BI datasets. The &lt;code&gt;--sf-per-executor&lt;/code&gt; option controls the number of worker nodes allocated, in this case 1 node per every 3000 SF, i.e. 1 and 4 nodes correspondingly. The &lt;code&gt;--partitions&lt;/code&gt; option controls the total number of partitions, and was calculated based on the number of persons using the formula &lt;code&gt;partitions = ceil(number_of_persons / block_size / 3)&lt;/code&gt; to get a maximum of 3 blocks per partition.&lt;/p&gt;
&lt;h1 id=&#34;conclusion&#34;&gt;Conclusion&lt;/h1&gt;
&lt;p&gt;These improvements made LDBC SNB datagen more modular, maintainable and efficient, costing under a cent per scale factor to generate the BI dataset, which enables us to generate datasets beyond SF 100K.&lt;/p&gt;
&lt;h1 id=&#34;footnotes&#34;&gt;Footnotes&lt;/h1&gt;
&lt;div class=&#34;footnotes&#34; role=&#34;doc-endnotes&#34;&gt;
&lt;hr&gt;
&lt;ol&gt;
&lt;li id=&#34;fn:1&#34;&gt;
&lt;p&gt;The generator produces hierarchies, such as forum wall with a random number of posts, that have comments, etc. This tree is iterated, and different entities are written to separate files.&amp;#160;&lt;a href=&#34;#fnref:1&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:2&#34;&gt;
&lt;p&gt;Shameless plug: You can learn more on this from &lt;a href=&#34;https://www.dataversity.net/case-study-deriving-spark-encoders-and-schemas-using-implicits/&#34;&gt;another blogpost of mine&lt;/a&gt;.&amp;#160;&lt;a href=&#34;#fnref:2&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:3&#34;&gt;
&lt;p&gt;The datagenerator produces blocks of 10,000 persons and their related entities. Entities from different blocks are unrelated (isolated).&amp;#160;&lt;a href=&#34;#fnref:3&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:4&#34;&gt;
&lt;p&gt;The maximum row count per file is currently 10M, however, this can be modified with a command line option. We also had an alternative design in mind where this number would have been determined based on the average row size of each entity, however, we stayed with the first version for simplicity.&amp;#160;&lt;a href=&#34;#fnref:4&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Fifteenth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/fifteenth-tuc-meeting/</link>
      <pubDate>Fri, 17 Jun 2022 09:20:00 -0500</pubDate>
      
      <guid>https://ldbcouncil.org/event/fifteenth-tuc-meeting/</guid>
      <description>&lt;p&gt;&lt;strong&gt;Organizers:&lt;/strong&gt; Gábor Szárnyas, Jack Waudby, Peter Boncz, Alastair Green&lt;/p&gt;
&lt;p&gt;LDBC is hosting a &lt;strong&gt;two-day&lt;/strong&gt; hybrid workshop, co-located with &lt;a href=&#34;https://2022.sigmod.org/venue.shtml&#34;&gt;SIGMOD 2022&lt;/a&gt; on &lt;strong&gt;June 17-18 (Friday-Saturday)&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The program consists of 10-15 minute talks followed by a Q&amp;amp;A session. The talks will be recorded and made available online.&lt;br&gt;
The tenative program is the following. &lt;strong&gt;All times are in EDT.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;We will have a social event on Friday at 17:30 at &lt;a href=&#34;https://elvezrestaurant.com/&#34;&gt;El Vez&lt;/a&gt; (&lt;a href=&#34;https://g.page/ElVezPhilly&#34;&gt;Google Maps&lt;/a&gt;).&lt;/p&gt;
&lt;h4 id=&#34;friday-pennsylvania-convention-centerhttpswwwpaconventioncom-room-204bhttps2022sigmodorgprogramshtml&#34;&gt;Friday (&lt;a href=&#34;https://www.paconvention.com/&#34;&gt;Pennsylvania Convention Center&lt;/a&gt;, &lt;a href=&#34;https://2022.sigmod.org/program.shtml&#34;&gt;room 204B&lt;/a&gt;)&lt;/h4&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;start&lt;/th&gt;
&lt;th&gt;finish&lt;/th&gt;
&lt;th&gt;speaker&lt;/th&gt;
&lt;th&gt;title&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;09:20&lt;/td&gt;
&lt;td&gt;09:30&lt;/td&gt;
&lt;td&gt;Peter Boncz (LDBC/CWI)&lt;/td&gt;
&lt;td&gt;State of the union – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/peter-boncz-state-of-the-union.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/39BoOIGk9Is&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;09:30&lt;/td&gt;
&lt;td&gt;09:45&lt;/td&gt;
&lt;td&gt;Alastair Green (LDBC/Birkbeck)&lt;/td&gt;
&lt;td&gt;LDBC&amp;rsquo;s fair use policies – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/alastair-green-fair-use-of-the-ldbc-trademark.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/7zmCysN4Rpg&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;09:50&lt;/td&gt;
&lt;td&gt;10:05&lt;/td&gt;
&lt;td&gt;Gábor Szárnyas (LDBC/CWI), Jack Waudby (Newcastle University)&lt;/td&gt;
&lt;td&gt;LDBC Social Network Benchmark: Business Intelligence workload v1.0 – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/gabor-szarnyas-the-ldbc-social-network-benchmark-business-intelligence-workload.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/AJ96M8_njxE&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:10&lt;/td&gt;
&lt;td&gt;10:25&lt;/td&gt;
&lt;td&gt;Heng Lin (Ant Group)&lt;/td&gt;
&lt;td&gt;LDBC Financial Benchmark introduction – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/heng-lin-ldbc-financial-benchmark-introduction.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/iBhud_YjafY&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:30&lt;/td&gt;
&lt;td&gt;11:00&lt;/td&gt;
&lt;td&gt;&lt;em&gt;coffee break&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:00&lt;/td&gt;
&lt;td&gt;11:15&lt;/td&gt;
&lt;td&gt;Chen Zhang (CreateLink)&lt;/td&gt;
&lt;td&gt;New LDBC SNB benchmark record by Galaxybase: More than 6 times faster and 70% higher throughput – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/chen-zhang-new-ldbc-snb-benchmark-record-by-galaxybase-more-than-6-times-faster-and-70-percent-higher-throughput.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/sMzTsb8iw_Y&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:20&lt;/td&gt;
&lt;td&gt;11:35&lt;/td&gt;
&lt;td&gt;James Clarkson (Neo4j)&lt;/td&gt;
&lt;td&gt;LDBC benchmarks: Promoting good science and industrial consumption – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/james-clarkson-ldbc-benchmarks-promoting-good-science-and-industrial-consumption.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/VYG1mzcl9qQ&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:40&lt;/td&gt;
&lt;td&gt;11:55&lt;/td&gt;
&lt;td&gt;Oskar van Rest (Oracle)&lt;/td&gt;
&lt;td&gt;Creating and querying property graphs in Oracle, on-premise and in the cloud – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/oskar-van-rest-creating-and-querying-property-graphs-in-oracle-on-premise-and-in-the-cloud.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/2HX2Vixf2gs&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:00&lt;/td&gt;
&lt;td&gt;12:15&lt;/td&gt;
&lt;td&gt;Mingxi Wu (TigerGraph)&lt;/td&gt;
&lt;td&gt;Conquering LDBC SNB BI at SF-10k – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/mingxi-wu-conquering-ldbc-snb-bi-at-sf10k.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/oJbqzQ_t3G8&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:20&lt;/td&gt;
&lt;td&gt;13:20&lt;/td&gt;
&lt;td&gt;&lt;em&gt;lunch (on your own)&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:20&lt;/td&gt;
&lt;td&gt;13:35&lt;/td&gt;
&lt;td&gt;Altan Birler (Technische Universität München)&lt;/td&gt;
&lt;td&gt;Relational databases can handle graphs too! Experiences with optimizing the Umbra RDBMS for LDBC SNB BI – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/altan-birler-relational-databases-can-handle-graphs-too.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/cRgbdY3I2i4&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:40&lt;/td&gt;
&lt;td&gt;13:55&lt;/td&gt;
&lt;td&gt;David Püroja (CWI)&lt;/td&gt;
&lt;td&gt;LDBC Social Network Benchmark: Interactive workload v2.0 – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/david-puroja-ldbc-snb-interactive-workload-v2.0.pdf&#34;&gt;slides&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:00&lt;/td&gt;
&lt;td&gt;14:15&lt;/td&gt;
&lt;td&gt;Angela Bonifati (Lyon 1 University)&lt;/td&gt;
&lt;td&gt;The quest for schemas in graph databases – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/angela-bonifati-the-quest-for-schemas-in-graph-databases.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/VT7cx3Jp7V8&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:20&lt;/td&gt;
&lt;td&gt;14:35&lt;/td&gt;
&lt;td&gt;Matteo Lissandrini (Aalborg University)&lt;/td&gt;
&lt;td&gt;Understanding graph data representations in triplestores – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/matteo-lissandrini-understanding-graph-data-representations-in-triplestores.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/xqVMJZfh_JU&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:40&lt;/td&gt;
&lt;td&gt;14:55&lt;/td&gt;
&lt;td&gt;Wim Martens (University of Bayreuth)&lt;/td&gt;
&lt;td&gt;Path representations – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/wim-martens-path-representations.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/Ma-E5dwgf-E&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:00&lt;/td&gt;
&lt;td&gt;15:20&lt;/td&gt;
&lt;td&gt;Audrey Cheng	(UC Berkeley)&lt;/td&gt;
&lt;td&gt;TAOBench: An end-to-end benchmark for social network workloads – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/audrey-cheng-taobench.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/1p8AStxS3es&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h4 id=&#34;saturday-philadelphia-marriott-downtownhttpswwwmarriottcomen-ushotelsphldt-philadelphia-marriott-downtown-room-401-402-4th-floor&#34;&gt;Saturday (&lt;a href=&#34;https://www.marriott.com/en-us/hotels/phldt-philadelphia-marriott-downtown/&#34;&gt;Philadelphia Marriott Downtown&lt;/a&gt;, room 401-402, 4th floor)&lt;/h4&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;start&lt;/th&gt;
&lt;th&gt;finish&lt;/th&gt;
&lt;th&gt;speaker&lt;/th&gt;
&lt;th&gt;title&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;10:00&lt;/td&gt;
&lt;td&gt;10:15&lt;/td&gt;
&lt;td&gt;Keith Hare (WG3)&lt;/td&gt;
&lt;td&gt;An update on the GQL &amp;amp; SQL/PGQ standards efforts – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/keith-hare-property-graph-standards-process-and-timing.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/xFVD3LWnKlc&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:20&lt;/td&gt;
&lt;td&gt;10:35&lt;/td&gt;
&lt;td&gt;Leonid Libkin (ENS Paris)&lt;/td&gt;
&lt;td&gt;Pattern matching in GQL and SQL/PGQ  – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/leonid-libkin-pattern-matching-in-gql-and-sql-pgq.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/OvGsa0qLANE&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:40&lt;/td&gt;
&lt;td&gt;10:55&lt;/td&gt;
&lt;td&gt;Petra Selmer (Neo4j/WG3)&lt;/td&gt;
&lt;td&gt;An overview of GQL – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/petra-selmer-towards-gql-v1-a-property-graph-query-language-standard.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/tncf2FgyIyo&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:00&lt;/td&gt;
&lt;td&gt;11:15&lt;/td&gt;
&lt;td&gt;Alastair Green (LDBC/WG3)&lt;/td&gt;
&lt;td&gt;GQL 2.0: A technical manifesto – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/alastair-green-gql-2.0-a-technical-manifesto.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/upIvpYy8C2g&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:20&lt;/td&gt;
&lt;td&gt;11:35&lt;/td&gt;
&lt;td&gt;George Fletcher (TU Eindhoven)&lt;/td&gt;
&lt;td&gt;PG-Keys (LDBC Property Graph Schema Working Group) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/george-fletcher-pg-keys-keys-for-property-graphs.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/_W8-jOtcObc&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:40&lt;/td&gt;
&lt;td&gt;11:55&lt;/td&gt;
&lt;td&gt;Arvind Shyamsundar (Microsoft)&lt;/td&gt;
&lt;td&gt;Graph capabilities in Microsoft SQL Server and Azure SQL Database – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/arvind-shyamsundar-graph-capabilities-in-microsoft-sql-server-and-azure-database.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/xxV2BfZupGw&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:00&lt;/td&gt;
&lt;td&gt;13:30&lt;/td&gt;
&lt;td&gt;&lt;em&gt;lunch (on your own)&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:30&lt;/td&gt;
&lt;td&gt;13:45&lt;/td&gt;
&lt;td&gt;Daniël ten Wolde (CWI)&lt;/td&gt;
&lt;td&gt;Implementing SQL/PGQ in DuckDB – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/daniel-ten-wolde-implementing-sql-pgq-in-duckdb.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/JmSfU0BTH5w&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:50&lt;/td&gt;
&lt;td&gt;14:05&lt;/td&gt;
&lt;td&gt;Oszkár Semeráth, Kristóf Marussy (TU Budapest)&lt;/td&gt;
&lt;td&gt;Generation techniques for consistent, realistic, diverse, and scalable graphs – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/oszkar-semerath-generation-techniques-for-consistent-realistic-diverse-and-scalable-graphs.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/hB6j6mvh-vA&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:10&lt;/td&gt;
&lt;td&gt;14:25&lt;/td&gt;
&lt;td&gt;Molham Aref (RelationalAI)&lt;/td&gt;
&lt;td&gt;Graph Normal Form – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/molham-aref-graph-normal-form.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/-kP4Raqr5KA&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:30&lt;/td&gt;
&lt;td&gt;14:45&lt;/td&gt;
&lt;td&gt;Naomi Arnold (Queen Mary University of London)&lt;/td&gt;
&lt;td&gt;Temporal graph analysis of the far-right social network Gab – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/naomi-arnold-temporal-graph-analysis-of-the-far-right-social-network-gab.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/ugSkFlif4PE&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:50&lt;/td&gt;
&lt;td&gt;15:05&lt;/td&gt;
&lt;td&gt;Domagoj Vrgoč (PUC Chile)&lt;/td&gt;
&lt;td&gt;Evaluating path queries in MillenniumDB – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/domagoj-vrgoc-regular-path-queries-in-millenniumdb.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/_OzJ6vI7GNU&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:10&lt;/td&gt;
&lt;td&gt;15:25&lt;/td&gt;
&lt;td&gt;Pavel Klinov, Evren Sirin (Stardog)&lt;/td&gt;
&lt;td&gt;Stardog&amp;rsquo;s experience with LDBC – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifteenth-tuc-meeting/attachments/evren-sirin-stardog-experience-with-ldbc.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/CBrEeOTqGKM&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
</description>
    </item>
    
    <item>
      <title>Announcing the LDBC Financial Benchmark Task Force</title>
      <link>https://ldbcouncil.org/post/announcing-the-ldbc-financial-benchmark-task-force/</link>
      <pubDate>Thu, 26 May 2022 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/announcing-the-ldbc-financial-benchmark-task-force/</guid>
      <description>&lt;p&gt;We are delighted to announce the set up of the &lt;a href=&#34;https://ldbcouncil.org/benchmarks/finbench/&#34;&gt;Financial Benchmark (FinBench) task force&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The Financial Benchmark (FinBench) project aims to define a graph database evaluating benchmark and develop a data generation process and a query driver to make the evaluation of the graph database representative, reliable and comparable, especially in financial scenarios, such as anti-fraud and risk control. The FinBench is scheduled to be released in the end of 2022.&lt;/p&gt;
&lt;p&gt;Compared to LDBC SNB, the FinBench will differ in application scenarios, data patterns, and workloads, resulting in different schema characteristics, latency bounds, path filters, etc. FinBench is going to redesign the data pattern and workloads, including the data generation, the query driver, and also some other facilities referred to LDBC SNB.&lt;/p&gt;
&lt;p&gt;The FinBench Task Force was approved by LDBC on May 16, 2022. The FinBench Task Force is led by Ant Group, and the initial members also include Pometry, Create Link, StarGraph, Ultipa, Katana, Intel, Memgraph (observer) and Koji Annoura (individual member). See the &lt;a href=&#34;https://ldbcouncil.org/benchmarks/finbench/ldbc-finbench-work-charter.pdf&#34;&gt;Work Charter for FinBench&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;If you are interested in joining FinBench Task Force, please reach out at info at ldbcouncil.org or guozhihui.gzh at antgroup.com.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Fourteenth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/fourteenth-tuc-meeting/</link>
      <pubDate>Mon, 16 Aug 2021 16:00:00 +0200</pubDate>
      
      <guid>https://ldbcouncil.org/event/fourteenth-tuc-meeting/</guid>
      <description>&lt;p&gt;LDBC was hosting a one-day hybrid workshop, co-located with &lt;a href=&#34;https://vldb.org/2021/&#34;&gt;VLDB 2021&lt;/a&gt; on &lt;strong&gt;August 16 (Monday) between 16:00–20:00 CEST&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The physical part of the workshop was held in room Akvariet 2 of the &lt;a href=&#34;https://www.tivolihotel.com/&#34;&gt;Tivoli Hotel&lt;/a&gt; (Copenhagen), while the virtual part was hosted on Zoom. Our programme consisted of talks that provide an overview of LDBC&amp;rsquo;s recent efforts. Moreover, we have invited industry practitioners and academic researchers to present their latest results.&lt;/p&gt;
&lt;p&gt;Talks were scheduled to be 10 minutes with a short Q&amp;amp;A session. We had three sessions. Their schedules are shown below.&lt;/p&gt;
&lt;h4 id=&#34;16001725-cest-ldbc-updates-benchmarks-query-languages&#34;&gt;[16:00–17:25 CEST] LDBC updates, benchmarks, query languages&lt;/h4&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;start&lt;/th&gt;
&lt;th&gt;speaker&lt;/th&gt;
&lt;th&gt;title&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;16:00&lt;/td&gt;
&lt;td&gt;Peter Boncz (CWI)&lt;/td&gt;
&lt;td&gt;State of the union – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/peter-boncz-state-of-the-union.pdf&#34;&gt;slides&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:05&lt;/td&gt;
&lt;td&gt;Gábor Szárnyas (CWI)&lt;/td&gt;
&lt;td&gt;Overview of LDBC benchmarks – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/gabor-szarnyas-ldbc-benchmarks.pdf&#34;&gt;slides&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:12&lt;/td&gt;
&lt;td&gt;Mingxi Wu (TigerGraph)&lt;/td&gt;
&lt;td&gt;LDBC Social Network Benchmark results with TigerGraph – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/mingxi-wu-tigergraph-snb-preliminary-results.pdf&#34;&gt;slides&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:24&lt;/td&gt;
&lt;td&gt;Xiaowei Zhu (Ant Group)&lt;/td&gt;
&lt;td&gt;Financial Benchmark proposal – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/xiaowei-zhu-financial-benchmark.pdf&#34;&gt;slides&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:36&lt;/td&gt;
&lt;td&gt;Petra Selmer (Neo4j)&lt;/td&gt;
&lt;td&gt;Status report from the Existing Languages Working Group (ELWG) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/petra-selmer-elwg.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/I5A8VuFDhsA&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:48&lt;/td&gt;
&lt;td&gt;Jan Hidders (Birkbeck)&lt;/td&gt;
&lt;td&gt;Status report from the Property Graph Schema Working Group (PGSWG) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/jan-hidders-pgswg.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/iEbVi9T-HVk&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;17:00&lt;/td&gt;
&lt;td&gt;Keith Hare (JCC Consulting)&lt;/td&gt;
&lt;td&gt;Database Language Standards Structure and Process, SQL/PGQ – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/keith-hare-database-language-standards-structure-and-process-sql-pgq.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/ZgFCuzods4g&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;17:12&lt;/td&gt;
&lt;td&gt;Stefan Plantikow (GQL Editor)&lt;/td&gt;
&lt;td&gt;Report on the GQL standard – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/stefan-plantikow-gql.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/z0pN5NwKsgc&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;em&gt;coffee break (10 minutes)&lt;/em&gt;&lt;/p&gt;
&lt;h4 id=&#34;17351845-cest-systems-and-data-structures&#34;&gt;[17:35–18:45 CEST] Systems and data structures&lt;/h4&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;start&lt;/th&gt;
&lt;th&gt;speaker&lt;/th&gt;
&lt;th&gt;title&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;17:35&lt;/td&gt;
&lt;td&gt;Vasileios Trigonakis (Oracle Labs)&lt;/td&gt;
&lt;td&gt;PGX.D aDFS: An Almost Depth-First-Search Distributed Graph-Querying System  – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/vasileios-trigonakis-pgxd-adfs.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/cv2ZfWRBOek&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;17:47&lt;/td&gt;
&lt;td&gt;Matthias Hauck (SAP)&lt;/td&gt;
&lt;td&gt;JSON, Spatial, Graph – Multi-model Workloads with SAP HANA Cloud  – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/matthias-hauck-json-spatial-graph-sap-hana-cloud.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/dgpMJFho6Q8&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;17:59&lt;/td&gt;
&lt;td&gt;Nikolay Yakovets (Eindhoven University of Technology)&lt;/td&gt;
&lt;td&gt;AvantGraph  – &lt;a href=&#34;https://youtu.be/z0pN5NwKsgcttachments/nikolay-yakovets-avantgraph.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/9M9FOycovTw&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;18:11&lt;/td&gt;
&lt;td&gt;Semih Salihoglu (University of Waterloo)&lt;/td&gt;
&lt;td&gt;GRainDB: Making RDBMSs Efficient on Graph Workloads Through Predefined Joins  – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/semih-salihoglu-graindb.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/FFK3y6vPHJs&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;18:23&lt;/td&gt;
&lt;td&gt;Semyon Grigorev (Saint Petersburg University)&lt;/td&gt;
&lt;td&gt;Context-free path querying: Obstacles on the way to adoption  – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/semyon-grigorev-cfpq.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/pha1xIpEL3I&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;18:35&lt;/td&gt;
&lt;td&gt;Per Fuchs (Technical University of Munich)&lt;/td&gt;
&lt;td&gt;Sortledton: A universal, transactional graph data structure  – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/per-fuchs-sortledton.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/33ZjsNN0hhU&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;em&gt;coffee break (10 minutes)&lt;/em&gt;&lt;/p&gt;
&lt;h4 id=&#34;1855-2000-cest-high-level-approaches-and-benchmarks&#34;&gt;[18:55-20:00 CEST] High-level approaches and benchmarks&lt;/h4&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;start&lt;/th&gt;
&lt;th&gt;speaker&lt;/th&gt;
&lt;th&gt;title&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;18:55&lt;/td&gt;
&lt;td&gt;Angelos-Christos Anadiotis (Ecole Polytechnique and Institut Polytechnique de Paris)&lt;/td&gt;
&lt;td&gt;Empowering Investigative Journalism with Graph-based Heterogeneous Data Management – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/angelos-christos-anadiotis-investigative-journalism-graph-data-management.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/a1VYjyec8dg&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;19:07&lt;/td&gt;
&lt;td&gt;Vasia Kalavri (Boston University)&lt;/td&gt;
&lt;td&gt;Learning to partition unbounded graph streams – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/vasia-kalavri-learning-to-partition-unbounded-graph-streams.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/PTlUABKWniA&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;19:19&lt;/td&gt;
&lt;td&gt;Muhammad Attahir Jibril (TU Ilmenau)&lt;/td&gt;
&lt;td&gt;Towards a Hybrid OLTP-OLAP Graph Benchmark – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/muhammad-attahir-jibril-hybrid-oltp-olap-benchmark.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/tMBVszTSJXc&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;19:31&lt;/td&gt;
&lt;td&gt;Riccardo Tommasini (University of Tartu)&lt;/td&gt;
&lt;td&gt;An outlook on Benchmarks for Graph Stream Processing – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/riccardo-tommasini-graph-stream-processing-benchmarks.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/HabvJvPXsLc&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;19:43&lt;/td&gt;
&lt;td&gt;Mohamed Ragab (University of Tartu)&lt;/td&gt;
&lt;td&gt;Benchranking: Towards prescriptive analysis of big graph processing: the case of SparkSQL – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourteenth-tuc-meeting/attachments/mohamed-ragab-benchranking.pdf&#34;&gt;slides&lt;/a&gt;, &lt;a href=&#34;https://youtu.be/mZ8LhGUq7Wg&#34;&gt;video&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
</description>
    </item>
    
    <item>
      <title>Thirteenth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/thirteenth-tuc-meeting/</link>
      <pubDate>Tue, 30 Jun 2020 14:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/event/thirteenth-tuc-meeting/</guid>
      <description>&lt;p&gt;LDBC is pleased to announce its Thirteenth Technical User Community (TUC) meeting.&lt;/p&gt;
&lt;p&gt;LDBC Technical User Community meetings serve to (1) learn about progress in the LDBC task forces on graph benchmarks and graph standards, (2) to give feedback on these, and (3) hear about user experiences with graph data management technologies or (4) learn about new graph technologies from researchers or industry – LDBC counts Oracle, IBM, Intel, Neo4j, TigerGraph and Huawei among its members.&lt;/p&gt;
&lt;p&gt;This TUC meeting will be a two-day event hosted online. We welcome all users of RDF and Graph technologies to attend. If you are interested to attend the event, please, contact Gabor Szarnyas (BME) to register.&lt;/p&gt;
&lt;h3 id=&#34;snb-task-force&#34;&gt;SNB Task Force&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;Progress report
&lt;ul&gt;
&lt;li&gt;ACID compliance test suite&lt;/li&gt;
&lt;li&gt;Integrating deletions to Datagen&lt;/li&gt;
&lt;li&gt;Migrating Datagen to Spark&lt;/li&gt;
&lt;li&gt;Redesign of BI read queries&lt;/li&gt;
&lt;li&gt;Extensions to the driver&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;Ongoing work
&lt;ul&gt;
&lt;li&gt;Datagen: tuning the distribution of deletes&lt;/li&gt;
&lt;li&gt;Interactive 2.0 workload&lt;/li&gt;
&lt;li&gt;BI 1.0 workload&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Zoom links will be sent through email.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Speeding Up LDBC SNB Datagen</title>
      <link>https://ldbcouncil.org/post/speeding-up-ldbc-snb-datagen/</link>
      <pubDate>Fri, 12 Jun 2020 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/speeding-up-ldbc-snb-datagen/</guid>
      <description>&lt;p&gt;LDBC&amp;rsquo;s &lt;a href=&#34;#references&#34;&gt;Social Network Benchmark [4]&lt;/a&gt; (LDBC SNB) is an industrial and academic initiative, formed by principal actors in the field of graph-like data management. Its goal is to define a framework where different graph-based technologies can be fairly tested and compared, that can drive the identification of systems&amp;rsquo; bottlenecks and required functionalities, and can help researchers open new frontiers in high-performance graph data management.&lt;/p&gt;
&lt;p&gt;LDBC SNB provides &lt;a href=&#34;https://github.com/ldbc/ldbc_snb_datagen&#34;&gt;Datagen&lt;/a&gt; (Data Generator), which produces synthetic datasets, mimicking a social network&amp;rsquo;s activity during a period of time. Datagen is defined by the charasteristics of realism, scalability, determinism and usability. To address scalability in particular, Datagen has been implemented on the MapReduce computation model to enable scaling out across a distributed cluster. However, since its inception in the early 2010s there has been a tremendous amount of development in the big data landscape, both in the sophistication of distributed processing platforms, as well as public cloud IaaS offerings. In the light of this, we should reevaluate this implementation, and in particular, investigate if Apache Spark would be a more cost-effective solution for generating datasets on the scale of tens of terabytes, on public clouds such as Amazon Web Services (AWS).&lt;/p&gt;
&lt;h2 id=&#34;overview&#34;&gt;Overview&lt;/h2&gt;
&lt;p&gt;The benchmark&amp;rsquo;s specification describes a social network &lt;a href=&#34;https://github.com/ldbc/ldbc_snb_docs/blob/9253abbde94ec7eaccd366c5d4c15cca30752e36/figures/schema-comfortable.pdf&#34;&gt;data model&lt;/a&gt; which divides its components into two broad categories: static and dynamic. The dynamic element consists of an evolving network where people make friends, post in forums, comment or like each others posts, etc. In contrast, the static component contains related attributes such as countries, universities and organizations and are fixed values. For the detailed specifications of the benchmark and the Datagen component, see &lt;a href=&#34;#references&#34;&gt;References&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Datasets are generated in a multi-stage process captured as a sequence of MapReduce steps (shown in the diagram below).&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;datagen_flow.png&#34; alt=&#34;&#34;&gt; \ &lt;em&gt;Figure 1. LDBC SNB Datagen Process on Hadoop&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;In the initialization phase dictionaries are populated and distributions are initialized. In the first generation phase persons are synthesized, then relationships are wired between them along 3 dimensions (university, interest and random). After merging the graph of person relationships, the resulting dataset is output. Following this, activities such as forum posts, comments, likes and photos are generated and output. Finally, the static components are output.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Note: The diagram shows the call sequence as implemented. All steps are sequential &amp;ndash; including the relationship generation &amp;ndash;, even in cases when the data dependencies would allow for parallelization.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Entities are generated by procedural Java code and are represented as POJOs in memory and as sequence files on disk. Most entities follow a shallow representation, i.e foreign keys (in relational terms) are mapped to integer ids, which makes serialization straightforward.&lt;sup id=&#34;fnref:1&#34;&gt;&lt;a href=&#34;#fn:1&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;1&lt;/a&gt;&lt;/sup&gt; A notable exception is the Knows edge which contains only the target vertex, and is used as a navigation property on the source Person. The target Person is replaced with only the foreign key augmented with some additional information in order to keep the structure free of cycles. Needless to say, this &lt;em&gt;edge as property&lt;/em&gt; representation makes the data harder to handle in SQL than it would be with a flat join table.&lt;/p&gt;
&lt;p&gt;Entity generation amounts to roughly one fifth of the main codebase. It generates properties drawn from several random distributions using mutable pRNGs. Determinism is achieved by initializing the pRNGs to seeds that are fully defined by the configuration with constants, and otherwise having no external state in the logic.&lt;sup id=&#34;fnref:2&#34;&gt;&lt;a href=&#34;#fn:2&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;
&lt;p&gt;Serialization is done by hand-written serializers for the supported output formats (e.g. CSV) and comprises just a bit less than one third of the main codebase. Most of the output is created by directly interacting with low-level HDFS file streams. Ideally, this code should be migrated to higher-level writers that handle faults and give consistent results when the task has to be restarted.&lt;/p&gt;
&lt;h2 id=&#34;motivations-for-the-migration&#34;&gt;Motivations for the migration&lt;/h2&gt;
&lt;p&gt;The application is written using Hadoop MapReduce, which is now largely superseded by more modern distributed batch processing platforms, notably Apache Spark. For this reason, it was proposed to migrate Datagen to Spark. The migration provides the following benefits:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Better memory utilization:&lt;/strong&gt; MapReduce is disk-oriented, i.e. it writes the output to disk after each reduce stage which is then read by the next MapReduce job. As public clouds provide virtual machines with sufficient RAM to encapsulate any generated dataset, time and money are wasted by the overhead this unnecessary disk I/O incurs. Instead, the intermediate results should be cached in memory where possible. The lack of support for this is a well-known limitation of MapReduce.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Smaller codebase:&lt;/strong&gt; The Hadoop MapReduce library is fairly ceremonial and boilerplatey. Spark provides a higher-level abstraction that is simpler to work with, while still providing enough control on the lower-level details required for this workload.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Small entry cost:&lt;/strong&gt; Spark and MapReduce are very close conceptually, they both utilise HDFS under the hood, and run on the JVM. This means that a large chunk of the existing code can be reused, and migration to Spark can, therefore, be completed with relatively small effort. Additionally, MapReduce and Spark jobs can be run on AWS EMR using basically the same HW/SW configuration, which facilitates straightforward performance comparisons.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Incremental improvements:&lt;/strong&gt; Spark exposes multiple APIs for different workloads and operating on different levels of abstraction. Datagen may initially utilise the lower-level, Java-oriented RDDs (which offer the clearest 1 to 1 mapping when coming from MapReduce) and gradually move towards DataFrames to support Parquet output in the serializers and maybe unlock some SQL optimization capabilities in the generators later down the road.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;OSS, commodity:&lt;/strong&gt; Spark is one of the most widely used open-source big data platforms. Every major public cloud provides a managed offering for Spark. Together these mean that the migration increases the approachability and portability of the code.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;first-steps&#34;&gt;First steps&lt;/h2&gt;
&lt;p&gt;The first milestone is a successful run of LDBC Datagen on Spark while making the minimum necessary amount of code alterations. This entails the migration of the Hadoop wrappers around the generators and serializers. The following bullet-points summarize the key notions that cropped up during the process.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Use your memory:&lt;/strong&gt; A strong focus was placed on keeping the call sequence intact, so that the migrated code evaluates the same steps in the same order, but with data passed as RDDs. It was hypothesised that the required data could be either cached in memory entirely at all times, or if not, regenerating them would still be faster than involving the disk I/O loop (e.g. by using &lt;code&gt;MEMORY_AND_DISK&lt;/code&gt;). In short, the default caching strategy was used everywhere.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Regression tests:&lt;/strong&gt; Lacking tests apart from an id uniqueness check, meant there were no means to detect bugs introduced by the migration. Designing and implementing a comprehensive test suite was out of scope, so instead, regression testing was utilised, with the MapReduce output as the baseline. The original output mostly consists of Hadoop sequence files which can be read into Spark, allowing comparisons to be drawn with the output from the RDD produced by the migrated code.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Thread-safety concerns:&lt;/strong&gt; Soon after migrating the first generator and running the regression tests, there were clear discrepancies in the output. These only surfaced when the parallelization level was set greater than 1. This indicated the presence of potential race conditions. Thread-safety wasn&amp;rsquo;t a concern in the original implementation due to the fact that MapReduce doesn&amp;rsquo;t use thread-based parallelization for mappers and reducers.&lt;sup id=&#34;fnref:3&#34;&gt;&lt;a href=&#34;#fn:3&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;3&lt;/a&gt;&lt;/sup&gt; In Spark however, tasks are executed by parallel threads in the same JVM application, so the code is required to be thread-safe. After some debugging, a bug was discovered originating from the shared use of java.text.SimpleDateFormat (notoriously known to be not thread-safe) in the serializers. This was resolved simply by changing to java.time.format.DateTimeFormatter. There were multiple instances of some static field on an object being mutated concurrently. In some cases this was a temporary buffer and was easily resolved by making it an instance variable. In another case a shared context variable was used, which was resolved by passing dedicated instances as function arguments. Sadly, the Java language has the same syntax for accessing locals, fields and statics, &lt;sup id=&#34;fnref:4&#34;&gt;&lt;a href=&#34;#fn:4&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;4&lt;/a&gt;&lt;/sup&gt; which makes it somewhat harder to find potential unguarded shared variables.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;case-study-person-ranking&#34;&gt;Case study: Person ranking&lt;/h2&gt;
&lt;p&gt;Migrating was rather straightforward, however, the so-called person ranking step required some thought. The goal of this step is to organize persons so that similar ones appear close to each other in a deterministic order. This provides a scalable way to cluster persons according to a similarity metric, as introduced in the &lt;a href=&#34;#references&#34;&gt;S3G2 paper [3]&lt;/a&gt;.&lt;/p&gt;
&lt;h3 id=&#34;the-original-mapreduce-version&#34;&gt;The original MapReduce version&lt;/h3&gt;
&lt;p&gt;&lt;img src=&#34;person_ranking.svg&#34; alt=&#34;&#34;&gt; \ &lt;em&gt;Figure 2. Diagram of the MapReduce code for ranking persons&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The implementation, shown in pseudocode above, works as follows:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;The equivalence keys are mapped to each person and fed into TotalOrderPartitioner which maintains an order sensitive partitioning while trying to emit more or less equal sized groups to keep the data skew low.&lt;/li&gt;
&lt;li&gt;The reducer keys the partitions with its own task id and a counter variable which has been initialized to zero and incremented on each person, establishing a local ranking inside the group. The final state of the counter (which is the total number of persons in that group) is saved to a separate &amp;ldquo;side-channel&amp;rdquo; file upon the completion of a reduce task.&lt;/li&gt;
&lt;li&gt;In a consecutive reduce-only stage, the global order is established by reading all of these previously emitted count files in the order of their partition number in each reducer, then creating an ordered map from each partition number to the corresponding cumulative count of persons found in all preceding ones. This is done in the setup phase. In the reduce function, the respective count is incremented and assigned to each person.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Once this ranking is done, the whole range is sliced up into equally sized blocks, which are processed independently. For example, when wiring relationships between persons, only those appearing in the same block are considered.&lt;/p&gt;
&lt;h3 id=&#34;the-migrated-version&#34;&gt;The migrated version&lt;/h3&gt;
&lt;p&gt;Spark provides a sortBy function which takes care of the first step above in a single line. The gist of the problem remains collecting the partition sizes and making them available in a later step. While the MapReduce version uses a side output, in Spark the partition sizes are collected in a separate job and passed into the next phase using a broadcast variable. The resulting code size is a fraction of the original one.&lt;/p&gt;
&lt;h2 id=&#34;benchmarks&#34;&gt;Benchmarks&lt;/h2&gt;
&lt;p&gt;Benchmarks were carried out on AWS &lt;a href=&#34;https://aws.amazon.com/emr/&#34;&gt;EMR&lt;/a&gt;, originally utilising &lt;a href=&#34;https://aws.amazon.com/ec2/instance-types/i3/&#34;&gt;i3.xlarge&lt;/a&gt; instances because of their fast NVMe SSD storage and ample amount of RAM.&lt;/p&gt;
&lt;p&gt;The application parameter hadoop.numThreads controls the number of reduce threads in each Hadoop job for the MapReduce version and the number of partitions in the serialization jobs in the Spark one. For MapReduce, this was set to n_nodes, i.e. the number of machines; experimentation yield slowdowns for higher values. The Spark version on the other hand, performed better with this parameter set to n_nodes * v_cpu. The scale factor (SF) parameter determines the output size. It is defined so that one SF unit generates around 1 GB of data. That is, SF10 generates around 10 GB, SF30 around 30 GB, etc. It should be noted however, that incidentally the output was only 60% of this in these experiments, stemming from two reasons. One, update stream serialization was not migrated to Spark, due to problems in the original implementation. Of course, for the purpose of faithful comparison the corresponding code was removed from the MapReduce version as well before executing the benchmarks. This explains a 10% reduction from the expected size. The rest can be attributed to incorrectly tuned parameters.&lt;sup id=&#34;fnref:5&#34;&gt;&lt;a href=&#34;#fn:5&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;5&lt;/a&gt;&lt;/sup&gt; The MapReduce results were as follows:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;SF&lt;/th&gt;
&lt;th&gt;workers&lt;/th&gt;
&lt;th&gt;Platform&lt;/th&gt;
&lt;th&gt;Instance Type&lt;/th&gt;
&lt;th&gt;runtime (min)&lt;/th&gt;
&lt;th&gt;runtime * worker/SF (min)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;MapReduce&lt;/td&gt;
&lt;td&gt;i3.xlarge&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;1.60&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;30&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;MapReduce&lt;/td&gt;
&lt;td&gt;i3.xlarge&lt;/td&gt;
&lt;td&gt;34&lt;/td&gt;
&lt;td&gt;1.13&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;100&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;MapReduce&lt;/td&gt;
&lt;td&gt;i3.xlarge&lt;/td&gt;
&lt;td&gt;40&lt;/td&gt;
&lt;td&gt;1.20&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;300&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;MapReduce&lt;/td&gt;
&lt;td&gt;i3.xlarge&lt;/td&gt;
&lt;td&gt;44&lt;/td&gt;
&lt;td&gt;1.32&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;It can be observed that the runtime per scale factor only increases slowly, which is good. The metric charts show an underutilized, bursty CPU. The bursts are supposedly interrupted by the disk I/O parts when the node is writing the results of a completed job. It can also be seen that the memory only starts to get consumed after 10 minutes of the run have  assed.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;mr_sf100_cpu_load.png&#34; alt=&#34;&#34;&gt; &lt;br&gt;
&lt;em&gt;Figure 3. CPU Load for the Map Reduce cluster is bursty and less than&lt;br&gt;
50% on average (SF100, 2nd graph shows master)&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;mr_sf100_mem_free.png&#34; alt=&#34;&#34;&gt; &lt;br&gt;
&lt;em&gt;Figure 4. The job only starts to consume memory when already 10 minutes&lt;br&gt;
into the run (SF100, 2nd graph shows master)&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s see how Spark fares.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;SF&lt;/th&gt;
&lt;th&gt;workers&lt;/th&gt;
&lt;th&gt;Platform&lt;/th&gt;
&lt;th&gt;Instance Type&lt;/th&gt;
&lt;th&gt;runtime (min)&lt;/th&gt;
&lt;th&gt;runtime * worker/SF (min)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;Spark&lt;/td&gt;
&lt;td&gt;i3.xlarge&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;1.00&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;30&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;Spark&lt;/td&gt;
&lt;td&gt;i3.xlarge&lt;/td&gt;
&lt;td&gt;21&lt;/td&gt;
&lt;td&gt;0.70&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;100&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;Spark&lt;/td&gt;
&lt;td&gt;i3.xlarge&lt;/td&gt;
&lt;td&gt;27&lt;/td&gt;
&lt;td&gt;0.81&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;300&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;Spark&lt;/td&gt;
&lt;td&gt;i3.xlarge&lt;/td&gt;
&lt;td&gt;36&lt;/td&gt;
&lt;td&gt;1.08&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1000&lt;/td&gt;
&lt;td&gt;30&lt;/td&gt;
&lt;td&gt;Spark&lt;/td&gt;
&lt;td&gt;i3.xlarge&lt;/td&gt;
&lt;td&gt;47&lt;/td&gt;
&lt;td&gt;1.41&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3000&lt;/td&gt;
&lt;td&gt;90&lt;/td&gt;
&lt;td&gt;Spark&lt;/td&gt;
&lt;td&gt;i3.xlarge&lt;/td&gt;
&lt;td&gt;47&lt;/td&gt;
&lt;td&gt;1.41&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;A similar trend here, however the run times are around 70% of the MapReduce version. It can be seen that the larger scale factors (SF1000 and SF3000) yielded a long runtime than expected. On the metric charts of SF100 the CPU shows full utilization, except at the end, when the results are serialized in one go and the CPU is basically idle (the snapshot of the diagram doesn&amp;rsquo;t include this part unfortunately). Spark can be seen to have used up all memory pretty fast even in case of SF100. In case of SF1000 and SF3000, the nodes are running so low on memory that most probably some of the RDDs have to be calculated multiple times (no disk level serialization was used here), which seem to be the most plausible explanation for the slowdowns experienced. In fact, the OOM errors encountered when running SF3000 supports this hypothesis even further. It was thus proposed to scale up the RAM in the instances. The CPU utilization hints that adding some extra vCPUs as well can further yield speedup.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;spark_sf100_cpu_load.png&#34; alt=&#34;&#34;&gt; &lt;br&gt;
&lt;em&gt;Figure 5. Full CPU utilization for Spark (SF100, last graph shows&lt;br&gt;
master)&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;spark_sf100_mem_free.png&#34; alt=&#34;&#34;&gt; &lt;br&gt;
&lt;em&gt;Figure 6. Spark eats up memory fast (SF100, 2nd graph shows master)&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;i3.2xlarge would have been the most straightforward option for scaling up the instances, however the humongous 1.9 TB disk of this image is completely unnecessary for the job. Instead the cheaper r5d.2xlarge instance was utilised, largely identical to i3.2xlarge, except it &lt;em&gt;only&lt;/em&gt; has a 300 GB SSD.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;SF&lt;/th&gt;
&lt;th&gt;workers&lt;/th&gt;
&lt;th&gt;Platform&lt;/th&gt;
&lt;th&gt;Instance Type&lt;/th&gt;
&lt;th&gt;runtime (min)&lt;/th&gt;
&lt;th&gt;runtime * worker/SF (min)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;100&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;Spark&lt;/td&gt;
&lt;td&gt;r5d.2xlarge&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;0.48&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;300&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;Spark&lt;/td&gt;
&lt;td&gt;r5d.2xlarge&lt;/td&gt;
&lt;td&gt;21&lt;/td&gt;
&lt;td&gt;0.63&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1000&lt;/td&gt;
&lt;td&gt;30&lt;/td&gt;
&lt;td&gt;Spark&lt;/td&gt;
&lt;td&gt;r5d.2xlarge&lt;/td&gt;
&lt;td&gt;26&lt;/td&gt;
&lt;td&gt;0.78&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3000&lt;/td&gt;
&lt;td&gt;90&lt;/td&gt;
&lt;td&gt;Spark&lt;/td&gt;
&lt;td&gt;r5d.2xlarge&lt;/td&gt;
&lt;td&gt;25&lt;/td&gt;
&lt;td&gt;0.75&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10000&lt;/td&gt;
&lt;td&gt;303&lt;/td&gt;
&lt;td&gt;Spark&lt;/td&gt;
&lt;td&gt;r5d.2xlarge&lt;/td&gt;
&lt;td&gt;25&lt;/td&gt;
&lt;td&gt;0.75&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The last column clearly demonstrates our ability to keep the cost per scale factor unit constant.&lt;/p&gt;
&lt;h2 id=&#34;next-steps&#34;&gt;Next steps&lt;/h2&gt;
&lt;p&gt;The next improvement is refactoring the serializers so they use Spark&amp;rsquo;s high-level writer facilities. The most compelling benefit is that it will make the jobs fault-tolerant, as Spark maintains the integrity of the output files in case the task that writes it fails. This makes Datagen more resilient and opens up the possibility to run on less reliable hardware configuration (e.g. EC2 spot nodes on AWS) for additional cost savings. They will supposedly also yield some speedup on the same cluster configuration.&lt;/p&gt;
&lt;p&gt;As already mentioned, the migration of the update stream serialization was ignored due to problems with the original code. Ideally, they should be implemented with the new serializers.&lt;/p&gt;
&lt;p&gt;The Spark migration also serves as an important building block for the next generation of LDBC benchmarks. As part of extending the SNB benchmark suite, the SNB task force has recently extended Datagen with support for &lt;a href=&#34;#references&#34;&gt;generating delete operations [1]&lt;/a&gt;. The next step for the task force is to fine-tune the temporal distributions of these deletion operations to ensure that the emerging sequence of events is realistic, i.e. the emerging distribution resembles what a database system would experience when serving a real social network.&lt;/p&gt;
&lt;h2 id=&#34;acknowledgements&#34;&gt;Acknowledgements&lt;/h2&gt;
&lt;p&gt;This work is based upon the work of Arnau Prat, Gábor Szárnyas, Ben Steer, Jack Waudby and other LDBC contributors. Thanks for your help and feedback!&lt;/p&gt;
&lt;h2 id=&#34;references&#34;&gt;References&lt;/h2&gt;
&lt;p&gt;[1] &lt;a href=&#34;https://ldbcouncil.org/docs/papers/datagen-deletes-grades-nda-2020.pdf&#34;&gt;Supporting Dynamic Graphs and Temporal Entity Deletions in the LDBC Social Network Benchmark&amp;rsquo;s Data Generator&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;[2] &lt;a href=&#34;https://www.youtube.com/watch?v=ZQOLuCOOpSI&#34;&gt;9th TUC Meeting &amp;ndash; LDBC SNB Datagen Update &amp;ndash; Arnau Prat (UPC)&lt;/a&gt; - &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75431942.pdf&#34;&gt;slides&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;[3] &lt;a href=&#34;https://research.vu.nl/en/publications/s3g2-a-scalable-structure-correlated-social-graph-generator&#34;&gt;S3G2: a Scalable Structure-correlated Social Graph Generator&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;[4] &lt;a href=&#34;https://arxiv.org/abs/2001.02299&#34;&gt;The LDBC Social Network Benchmark&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;[5] &lt;a href=&#34;https://ldbcouncil.org/&#34;&gt;LDBC&lt;/a&gt; - &lt;a href=&#34;https://github.com/ldbc&#34;&gt;LDBC GitHub organization&lt;/a&gt;&lt;/p&gt;
&lt;div class=&#34;footnotes&#34; role=&#34;doc-endnotes&#34;&gt;
&lt;hr&gt;
&lt;ol&gt;
&lt;li id=&#34;fn:1&#34;&gt;
&lt;p&gt;Also makes it easier to map to a tabular format thus it is a SQL friendly representation.&amp;#160;&lt;a href=&#34;#fnref:1&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:2&#34;&gt;
&lt;p&gt;It&amp;rsquo;s hard to imagine this done declaratively in SQL.&amp;#160;&lt;a href=&#34;#fnref:2&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:3&#34;&gt;
&lt;p&gt;Instead, multiple YARN containers have to be used if you want to parallelize on the same machine.&amp;#160;&lt;a href=&#34;#fnref:3&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:4&#34;&gt;
&lt;p&gt;Although editors usually render these using different font styles.&amp;#160;&lt;a href=&#34;#fnref:4&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:5&#34;&gt;
&lt;p&gt;With the addition of deletes, entities often get inserted and deleted during the simulation (which is normal in a social network). During serialization, we check for such entities and omit them. However, we forgot to calculate this when determining the output size, which we will amend when tuning the distributions.&amp;#160;&lt;a href=&#34;#fnref:5&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Twelfth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/twelfth-tuc-meeting/</link>
      <pubDate>Fri, 05 Jul 2019 08:30:00 +0100</pubDate>
      
      <guid>https://ldbcouncil.org/event/twelfth-tuc-meeting/</guid>
      <description>&lt;p&gt;LDBC is pleased to announce its Twelfth Technical User Community (TUC) meeting.&lt;/p&gt;
&lt;p&gt;LDBC Technical User Community meetings serve to (1) learn about progress in the LDBC task forces on graph benchmarks and graph standards, (2) to give feedback on these, and (3) hear about user experiences with graph data management technologies or (4) learn about new graph technologies from researchers or industry &amp;ndash; LDBC counts Oracle, IBM, Intel, Neo4j, TigerGraph and Huawei among its members.&lt;/p&gt;
&lt;p&gt;This TUC meeting will be a one-day event on the last Friday of &lt;strong&gt;&lt;a href=&#34;https://sigmod2019.org/&#34;&gt;SIGMOD/PODS 2019&lt;/a&gt;&lt;/strong&gt; in Amsterdam, The Netherlands, in the conference venue of &lt;strong&gt;&lt;a href=&#34;http://sigmod2019.org/conf_venue&#34;&gt;Beurs van Berlage&lt;/a&gt;&lt;/strong&gt;. The room is the Mendes da Silva kamer. Please check its tips for &lt;strong&gt;&lt;a href=&#34;http://sigmod2019.org/accommodation&#34;&gt;accommodation in Amsterdam&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Note also that at SIGMOD/PODS in Amsterdam on Sunday, June 30, there is a research workshop on graph data management technology called GRADES-NDA 2019, that may be of interest to our audience (this generally holds for the whole SIGMOD/PODS program, of course).&lt;/p&gt;
&lt;p&gt;We welcome all users of RDF and Graph technologies to attend. If you are interested to attend the event, please, contact Damaris Coll (UPC) at &lt;a href=&#34;mailto:damaris@ac.upc.edu&#34;&gt;damaris@ac.upc.edu&lt;/a&gt; to register.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;=&amp;gt; registration is free, but required &amp;lt;=&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;You need to be registered in order to get into the SIGMOD/PODS venue. Friday, July 5, is the final, workshop, day of SIGMOD/PODS, and the LDBC TUC meeting joins the other workshops for coffee and lunch.&lt;/p&gt;
&lt;p&gt;In the agenda, there will be talks given by LDBC members and LDBC activities, but there will also be room for a number of short 20-minute talks by other participants. We are specifically interested in learning about new challenges in graph data management (where benchmarking would become useful) and on hearing about actual user stories and scenarios that could inspire benchmarks. Further, talks that provide feedback on existing benchmark (proposals) are very relevant. But nothing is excluded a priori if it is related to graph data management.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Talk proposals can be sent to Peter Boncz&lt;/strong&gt;, who is also the local organizer. &lt;strong&gt;Please also send your slides to this email for archiving on this site.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Further, we call on you if you or your colleagues would happen to have contacts with companies that deal with graph data management scenarios to also attend and possibly present. LDBC is always looking to expand its circle of participants in TUCs meeting, its graph technology users contacts but also eventually its membership base.&lt;/p&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;In the TUC meeting, there will be:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;updates on progress with LDBC benchmarks, specifically the Social Network Benchmark (SNB) and its Interactive, Business Intelligence and Graphalytics workloads.&lt;/li&gt;
&lt;li&gt;talks by data management practitioners highlighting graph data management challenges and products&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The morning slot (08:30-10:30) is reserved for an LDBC Board Meeting, to which in principle only LDBC directors are invited (that meeting will be held in the same room).&lt;/p&gt;
&lt;p&gt;The TUC meeting will start on Friday morning after the morning coffee break of SIGMOD/PODS 2019 (&lt;strong&gt;room: Mendes da Silva kamer&lt;/strong&gt;):&lt;/p&gt;
&lt;p&gt;08:30-10:30 LDBC Board Meeting (non-public)&lt;/p&gt;
&lt;p&gt;10:30-11:00 Coffee&lt;/p&gt;
&lt;p&gt;11:00-12:45 Session 1: Graph Benchmarks&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;11:00-11:05 Welcome &amp;amp; introduction&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/112230404.pdf&#34;&gt;11:05-11:45 Gabor Szarnyas (BME), Benjamin Steer (QMUL), Jack Waudby (Newcastle University): Business Intelligence workload: Progress report and roadmap&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706117.pdf&#34;&gt;11:45-12:00 Frank McSherry (Materialize): Experiences implementing LDBC queries in a dataflow system&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706118.pdf&#34;&gt;12:00-12:25 Vasileios Trigonakis (Oracle): Evaluating a new distributed graph query engine with LDBC: Experiences and limitations&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706130.pdf&#34;&gt;12:25-12:45 Ahmed Musaafir (VU Amsterdam): LDBC Graphalytics&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;12:45-14:00 Lunch&lt;/p&gt;
&lt;p&gt;14:00-16:05 Session 2: Graph Query Languages&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706120.pdf&#34;&gt;14:00-14:25 Juan Sequeda (Capsenta): Property Graph Schema Working Group: A progress report&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706121.pdf&#34;&gt;14:25-14:50 Stefan Plantikow (Neo4j): GQL: Scope and features&lt;/a&gt;, &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706122.pdf&#34;&gt;report&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706119.pdf&#34;&gt;14:50-15:15 Vasileios Trigonakis (Oracle): Property graph extensions for the SQL standard&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706129.pdf&#34;&gt;15:15-15:40 Alin Deutsch (TigerGraph): Modern graph analytics support in GSQL, TigerGraph&amp;rsquo;s query language&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/112230401.pdf&#34;&gt;15:40-16:05 Jan Posiadała (Nodes and Edges, Poland): Executable semantics of graph query language&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;16:05-16:30 Coffee&lt;/p&gt;
&lt;p&gt;16:30-17:50 Session 3: Graph System Performance&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111968258.pdf&#34;&gt;16:30-16:50 Per Fuchs (CWI): Fast, scalable WCOJ graph-pattern matching on in-memory graphs in Spark&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706124.pdf&#34;&gt;16:50-17:10 Semih Salihoglu (University of Waterloo): Optimizing subgraph queries with a mix of tradition and modernity&lt;/a&gt; &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706116.pptx&#34;&gt;pptx&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706128.pdf&#34;&gt;17:10-17:30 Roi Lipman (RedisGraph): Evaluating Cypher queries and procedures as algebraic operations within RedisGraph&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/twelfth-tuc-meeting/attachments/106233859/111706133.pdf&#34;&gt;17:30-17:50 Alexandru Uta (VU Amsterdam): Low-latency Spark queries on updatable data&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If there is interest, we will organize a social dinner on Friday evening for LDBC attendees.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Eleventh TUC Meeting</title>
      <link>https://ldbcouncil.org/event/eleventh-tuc-meeting/</link>
      <pubDate>Fri, 08 Jun 2018 08:30:00 -0500</pubDate>
      
      <guid>https://ldbcouncil.org/event/eleventh-tuc-meeting/</guid>
      <description>&lt;p&gt;LDBC Technical User Community meetings serve to (1) learn about progress in the LDBC task forces on graph benchmark development, (2) to give feedback on these, and (3) hear about user experiences with graph data management technologies or (4) learn about new graph technologies from researchers or industry &amp;ndash; LDBC counts Oracle, IBM, Intel, Neo4j and Huawei among its members.&lt;/p&gt;
&lt;p&gt;This TUC meeting will be a one-day event preceding the &lt;a href=&#34;https://sigmod2018.org/&#34;&gt;SIGMOD/PODS 2018&lt;/a&gt; conference in Houston, Texas (not too far away, the whole next week). Note also that at SIGMOD/PODS in Houston on Sunday 10, there is a research workshop on graph data management technology called GRADES-NDA 2018 as well, so you might combine travel.&lt;/p&gt;
&lt;p&gt;We welcome all users of RDF and Graph technologies to attend. If you are interested to attend the event, please, contact Damaris Coll (UPC) at &lt;a href=&#34;mailto:damaris@ac.upc.edu&#34;&gt;damaris@ac.upc.edu&lt;/a&gt; to register.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;=&amp;gt; registration is free, but required &amp;lt;=&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;In the agenda, there will be talks given by LDBC members and LDBC activities, but there will also be room for a number of short 20-minute talks by other participants. We are specifically interested in learning about new challenges in graph data management (where benchmarking would become useful) and on hearing about actual user stories and scenarios that could inspire benchmarks. Further, talks that provide feedback on existing benchmark (proposals) are very relevant. But nothing is excluded a priori if it is related to graph data management. Talk proposals are handled by Peter Boncz g band Larri. Local organizer is Juan Sequeda.&lt;/p&gt;
&lt;p&gt;Further, we call on you if you or your colleagues would happen to have contacts with companies that deal with graph data management scenarios to also attend and possibly present. LDBC is always looking to expand its circle of participants in TUCs meeting, its graph technology users contacts but also eventually its membership base.&lt;/p&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;In the TUC meeting there will be:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;updates on progress with LDBC benchmarks, specifically the Social Network Benchmark (SNB) and its interactive, business analytics and graphalytics workloads.&lt;/li&gt;
&lt;li&gt;talks by data management practitioners highlighting graph data management challenges and products&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The meeting will start on Friday morning, with a program from 10:30-17:00:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;10:30-10:35 Peter Boncz (CWI) - introduction to the LDBC TUC meeting&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eleventh-tuc-meeting/attachments/91422722/99090478.pdf&#34;&gt;10:35-11:00 Juan Sequeda (Capsenta) - Announcing: gra.fo&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;11:00-11:30 coffee break&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eleventh-tuc-meeting/attachments/91422722/99090466.pdf&#34;&gt;11:30-11:55 Gabor Szarnyas (BME) - LDBC benchmarks: three aspects of graph processing&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eleventh-tuc-meeting/attachments/91422722/99090463.pdf&#34;&gt;11:55-12:20 Peter Boncz (CWI) - G-CORE: a composable graph query language by LDBC&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eleventh-tuc-meeting/attachments/91422722/99090472.pdf&#34;&gt;12:20-12:45 Yinglong Xia (Huawei) - Graph Engine for Cloud AI&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;12:45-14:00 lunch&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eleventh-tuc-meeting/attachments/91422722/99090474.pdf&#34;&gt;14:00-14:25 Stefan Plantikow (Neo4j) - Composable Graph Queries and Multiple Named Graphs in Cypher for Apache Spark&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eleventh-tuc-meeting/attachments/91422722/99090481.pdf&#34;&gt;14:25-14:50 Oskar van Rest (Oracle) - Analyzing Stack Exchange data using Property Graph in Oracle&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eleventh-tuc-meeting/attachments/91422722/99090485.pdf&#34;&gt;14:50-15:15 Brad Bebee (Amazon) - Neptune: the AWS graph management service&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;15:15-15:40 coffee break&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eleventh-tuc-meeting/attachments/91422722/99811329.pdf&#34;&gt;15:40-16:05 Bryon Jacob (data.world): Broadening the Semantic Web&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eleventh-tuc-meeting/attachments/91422722/99287041.pdf&#34;&gt;16:05-16:30 Jason Plurad (IBM) - Graph Computing with JanusGraph&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eleventh-tuc-meeting/attachments/91422722/99745793.pdf&#34;&gt;16:30-16:55 Arthur Keen (Cambridge Semantics): AnzoGraph&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;a href=&#34;http://relational.ai/&#34;&gt;16:55-17:20 Molham Aref (relational.ai)&lt;/a&gt;) - Introducing.. &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eleventh-tuc-meeting/attachments/91422722/99418113.pdf&#34;&gt;relational.ai&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;18:00 - 20:00 social dinner in Austin (sponsored by Intel Corp.), Coopers BBQ, 217 Congress Ave, Austin, TX 78701&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;location&#34;&gt;Location&lt;/h3&gt;
&lt;p&gt;The TUC will be held at the &lt;a href=&#34;https://www.cs.utexas.edu/&#34;&gt;University of Texas at Austin, Department of Computer Science&lt;/a&gt; in the &lt;a href=&#34;https://www.google.com/maps/place/The+University+of+Texas:+Department+of+Computer+Science/@30.2860955,-97.737582,18z/data=!4m5!3m4!1s0x0:0x12edecc8226b3241!8m2!3d30.2862279!4d-97.7365348&#34;&gt;Gates Dell Complex (GDC): 2317 Speedway, Austin TX, 78712&lt;/a&gt; Room: GDC 6.302&lt;/p&gt;
&lt;p&gt;The GDC building has a North and a South building. GDC 6.302 is in the North building. When you enter the main entrance, the North building is on the left and it is served by a pair of elevators. You can take or the elevator to the 6th floor. Exit the elevator on the 6th floor. Turn left, right, left.&lt;/p&gt;
&lt;h3 id=&#34;from-austin-to-sigmodpods-houston-on-saturday-june-9&#34;&gt;From Austin to SIGMOD/PODS (Houston) on Saturday June 9&lt;/h3&gt;
&lt;p&gt;Many of the attendees will be going to SIGMOD/PODS which will be held in Houston.&lt;/p&gt;
&lt;h4 id=&#34;bus&#34;&gt;Bus&lt;/h4&gt;
&lt;p&gt;One option is to take a &lt;a href=&#34;https://us.megabus.com/journey-planner/journeys?days=1&amp;amp;concessionCount=0&amp;amp;departureDate=2018-06-09&amp;amp;destinationId=318&amp;amp;inboundOtherDisabilityCount=0&amp;amp;inboundPcaCount=0&amp;amp;inboundWheelchairSeated=0&amp;amp;nusCount=0&amp;amp;originId=320&amp;amp;otherDisabilityCount=0&amp;amp;pcaCount=0&amp;amp;totalPassengers=1&amp;amp;wheelchairSeated=0&#34;&gt;MegaBus that departs from downtown Austin and arrives at downtown Houston&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;There is a bus that departs at 12:00PM and arrives at 3:00pm. Cost is $20 (as of April 23).&lt;/p&gt;
&lt;p&gt;If you want to spend the day in Austin, there is a bus that departs at 9:55PM and arrives at 12:50am. Cost is $5 (as of April 23).&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Tenth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/tenth-tuc-meeting/</link>
      <pubDate>Fri, 01 Sep 2017 10:30:00 +0100</pubDate>
      
      <guid>https://ldbcouncil.org/event/tenth-tuc-meeting/</guid>
      <description>&lt;p&gt;This will be a one-day event at the &lt;a href=&#34;http://www.vldb.org/2017&#34;&gt;VLDB 2017&lt;/a&gt; conference in Munich, Germany on September 1, 2017.&lt;/p&gt;
&lt;p&gt;Topics and activities of interest in these TUC meetings are:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Presentation on graph data management usage scenarios.&lt;/li&gt;
&lt;li&gt;Presentation of the benchmarking results for the different benchmarks, as well as the graph query language task force.&lt;/li&gt;
&lt;li&gt;Interaction with the new LDBC Board of Directors and the LDBC organisation officials.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We welcome all users of RDF and Graph technologies to attend. If you are interested to attend the event, please, contact Adrian Diaz (UPC) at &lt;a href=&#34;mailto:adiaz@ac.upc.edu&#34;&gt;adiaz@ac.upc.edu&lt;/a&gt; to register; registration is free, but required.&lt;/p&gt;
&lt;p&gt;In the agenda, there will be talks given by LDBC members and LDBC activities, but there will also be room for a number of short 20-minute talks by other participants. We are specifically interested in learning about new challenges in graph data management (where benchmarking would become useful) and on hearing about actual user stories and scenarios that could inspire benchmarks. Further, talks that provide feedback on existing benchmark (proposals) are very relevant. But nothing is excluded a priori if it is related to graph data management. Talk proposals are handled by Peter Boncz and Larri.&lt;/p&gt;
&lt;p&gt;Further, we call on you if you or your colleagues would happen to have contacts with companies that deal with graph data management scenarios to also attend and possibly present. LDBC is always looking to expand its circle of participants in TUCs meeting, its graph technology users contacts but also eventually its membership base.&lt;/p&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;In the TUC meeting there will be:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;updates on progress with LDBC benchmarks, specifically the Social Network Benchmark (SNB) and its Interactive, Business Intelligence and Graphalytics workloads.&lt;/li&gt;
&lt;li&gt;talks by data management practitioners highlighting graph data management challenges&lt;/li&gt;
&lt;li&gt;selected scientific talks on graph data management technology&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The meeting will start on Friday morning, with a program from 10:30-17:00&lt;/p&gt;
&lt;p&gt;10:30-12:00:  TUC session (public)&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/87588865.pdf&#34;&gt;Peter Boncz (CWI): GraphQL task force update - the G-CORE proposal&lt;/a&gt; (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/86868018.pptx&#34;&gt;pptx&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/86868008.pdf&#34;&gt;Gabor Szarnyas (Budapest University of Technology and Economics Hungarian Academy of Sciences): Updates on the Social Network Benchmark BI Workload&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Alexandru Iosup, Wing Lung Ngai (VU/TU Delft): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/86868014.pdf&#34;&gt;LDBC Graphalytics v0.9&lt;/a&gt;, &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/86868013.pdf&#34;&gt;Graphalytics Global Competition and Graphalytics Custom Benchmark&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;12:00-13:30:  lunch break&lt;/p&gt;
&lt;p&gt;13:30-15:00:  TUC session (public)&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/86868024.pdf&#34;&gt;Arnau Prat (UPC): Datasynth: Democratizing property graph generation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/86868026.pdf&#34;&gt;Marcus Paradies (SAP): SAP HANA GraphScript&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/87031809.pdf&#34;&gt;Yinglong Xia (Huawei): The EYWA Graph Engine in a Cloud AI Platform&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Gaétan Hains (Huawei): Cost semantics for graph queries&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;15:00-15:30:  break&lt;/p&gt;
&lt;p&gt;15:30-17:00:  TUC session (public)&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/87031812.pdf&#34;&gt;Petra Selmer and Stefan Plantikow (Neo4j): openCypher Developments in 2017&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/87195650.pdf&#34;&gt;Markus Kaindl (Springer): SN SciGraph &amp;ndash; Building a Linked Data Knowledge Graph for the Scholarly Publishing Domain&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Irini Fundulaki (FORTH): The HOBBIT Link Discovery and Versioning Benchmarks&lt;/li&gt;
&lt;li&gt;Ghislain Atemezing (Mondeca): Benchmarking Enterprise RDF stores with Publications Office Dataset&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Speakers should aim for a &lt;strong&gt;20-minute talk&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Further:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;on Friday evening (19:00-21:00) there will be a &lt;strong&gt;social dinner&lt;/strong&gt; at Löwenbräukeller, sponsored and arranged by LDBC member Huawei (who have their European Research Center in Munich).&lt;/li&gt;
&lt;li&gt;on Friday morning (8:30-10:30) there will be a meeting of the LDBC board of directors, but this meeting is not public.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;venue&#34;&gt;Venue&lt;/h3&gt;
&lt;p&gt;The Technical University of Munich (TUM) is hosting that week the &lt;a href=&#34;http://www.vldb.org/2017&#34;&gt;VLDB conference&lt;/a&gt;; on the day of the TUC meeting the main conference will have finished, but there will be a number of co-located workshops ongoing, and the TUC participants will blend in with that crowd for the breaks and lunch.&lt;/p&gt;
&lt;p&gt;The TUC meeting will be held in in &lt;strong&gt;Room 2607&lt;/strong&gt; alongside the VLDB workshops that day (MATES, ADMS, DMAH, DBPL and BOSS).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;address: Technische Universität München (TUM), Arcisstraße 21, 80333 München&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://www.google.nl/maps/place/Technische+Universit%C3%A4t+M%C3%BCnchen/@48.14966,11.5656715,17z/data=!3m1!4b1!4m5!3m4!1s0x479e7261336d8c11:0x79a04d44dc5bf19d!8m2!3d48.14966!4d11.5678602?hl=en&#34;&gt;Google Maps&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/81920002.jpg&#34; alt=&#34;&#34;&gt;&lt;br&gt;
&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/tenth-tuc-meeting/attachments/81920005/81920003.jpg&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Ninth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/ninth-tuc-meeting/</link>
      <pubDate>Thu, 09 Feb 2017 15:07:18 -0400</pubDate>
      
      <guid>https://ldbcouncil.org/event/ninth-tuc-meeting/</guid>
      <description>&lt;p&gt;LDBC is pleased to announce its Ninth Technical User Community (TUC) meeting.&lt;/p&gt;
&lt;p&gt;This will be a two-day event at &lt;a href=&#34;https://websmp201.sap-ag.de/~sapidp/011000358700001204882013E.pdf&#34;&gt;SAP Headquarters&lt;/a&gt; in Walldorf, Germany on February 9+10, 2017.&lt;/p&gt;
&lt;p&gt;This will be the third TUC meeting after the finalisation of the LDBC FP7 EC funded project. The event will basically set the following aspects:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Two day event with one day devoted to User&amp;rsquo;s experiences and one day devoted to benchmarking experiences.&lt;/li&gt;
&lt;li&gt;Presentation of the benchmarking results for the different benchmarks.&lt;/li&gt;
&lt;li&gt;Interaction with the new LDBC Board of Directors and the LDBC organisation officials.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We welcome all users of RDF and Graph technologies to attend. If you are interested, please, contact Damaris Coll (UPC) at &lt;a href=&#34;mailto:damaris@ac.upc.edu&#34;&gt;damaris@ac.upc.edu&lt;/a&gt;;&lt;/p&gt;
&lt;p&gt;In the agenda, there will be talks given by LDBC members and LDBC activities, but there will also be room for a number of short 20-minute talks by other participants. We are specifically interested in learning about new challenges in graph data management (where benchmarking would become useful) and on hearing about actual user stories and scenarios that could inspire benchmarks. Further, talks that provide feedback on existing benchmark (proposals) are very relevant. But nothing is excluded a priori if it is related to graph data management. Talk proposals can be forwarded to Damaris as well and will be handled by Peter Boncz and Larri.&lt;/p&gt;
&lt;p&gt;Further, we call on you if you or your colleagues would happen to have contacts with companies that deal with graph data management scenarios to also attend and possibly present. LDBC is always looking to expand its circle of participants in TUCs meeting, its graph technology users contacts but also eventually its membership base.&lt;/p&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;In the TUC meeting there will be&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;updates on progress with LDBC benchmarks, specifically the Social Network Benchmark (SNB) and its Interactive, Business Inalytics and Graphalytics workloads.&lt;/li&gt;
&lt;li&gt;talks by data management practitioners highlighting graph data management challenges&lt;/li&gt;
&lt;li&gt;selected scientific talks on graph data management technology&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The meeting will start on Thursday morning, with a program from 09:00-18:00, interrupted by a lunch break.&lt;/p&gt;
&lt;p&gt;Thursday evening (19:00-21:00) there will be a &lt;strong&gt;social dinner&lt;/strong&gt; in Heidelberg.&lt;/p&gt;
&lt;p&gt;Friday morning the event resumes from 9:00-12:00. In the afternoon, there is a (closed) LDBC Board of Directors meeting (13:00-16:30) at the same venue.&lt;/p&gt;
&lt;h4 id=&#34;social-dinner&#34;&gt;Social Dinner&lt;/h4&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75235334.png&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Address: Hauptstraße 217, 69117 Heidelberg&lt;/strong&gt;&lt;br&gt;
&lt;strong&gt;Time: 19:00 / 7pm&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;(See attachments at the bottom of the page)&lt;/p&gt;
&lt;h5 id=&#34;thursday&#34;&gt;Thursday&lt;/h5&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;start time&lt;/th&gt;
&lt;th&gt;title – speaker&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;9:00&lt;/td&gt;
&lt;td&gt;Welcome and logistics - Marcus Paradies (SAP)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;9:10&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75235329.pdf&#34;&gt;Intro + state of the LDBC - Josep Lluis Larriba Pey&lt;/a&gt; (UPC)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;9:20&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75235338.pdf&#34;&gt;LDBC Graph QL task force&lt;/a&gt; - Hannes Voigt (TU Dresden)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;9:40&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75235335.pdf&#34;&gt;PGQL Status Update and Comparison to LDBC&amp;rsquo;s Graph QL proposals&lt;/a&gt; - Oskar van Rest (Oracle Labs)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:00&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75628546.pdf&#34;&gt;Adding shortest-paths to MonetDB&lt;/a&gt; - Dean de Leo (CWI)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:20&lt;/td&gt;
&lt;td&gt;coffee&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:50&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75431939.pdf&#34;&gt;Evolving Cypher for processing multiple graphs&lt;/a&gt; - Stefan Plantikow (Neo Technology)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:10&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75235346.pdf&#34;&gt;Standardizing Graph Database Functionality - An Invitation to Collaborate&lt;/a&gt; - Jan Michels (ISO/ANSI SQL, Oracle)&amp;quot;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:30&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75235343.pdf&#34;&gt;Dgraph: Graph database for production environment&lt;/a&gt; - Tomasz Zdybal (Dgraph.io)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:00&lt;/td&gt;
&lt;td&gt;lunch&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:00&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75431945.pdf&#34;&gt;LDBC Graphalytics: Current Capabilities, Upcoming Features, and Long-Term Roadmap&lt;/a&gt; - Alexandru Iosup (TU Delft)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:20&lt;/td&gt;
&lt;td&gt;LDBC Graphalytics: Demo of the Live Archive and Competition Features - Tim Hegeman (TU Delft)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:40&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75431942.pdf&#34;&gt;LDBC SNB Datagen Update&lt;/a&gt; - Arnau Prat (UPC)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:00&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75431943.pdf&#34;&gt;LDBC SNB Business Intelligence Workload: Chokepoint Analysis&lt;/a&gt; - Arnau Prat (UPC)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:20&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75431947.pdf&#34;&gt;LDBC Benchmark Cost Specification&lt;/a&gt; (+discussion) - Moritz Kaufmann (TU Munich)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14:40&lt;/td&gt;
&lt;td&gt;coffee break&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:10&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/76316673.pdf&#34;&gt;EYWA: the Distributed Graph Engine in Huawei MIND Platform&lt;/a&gt; (Yinglong Xia)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:30&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75431949.pdf&#34;&gt;Graph Processing in SAP HANA&lt;/a&gt; - Marcus Paradies (SAP)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15:50&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75628563.pdf&#34;&gt;Distributed Graph Analytics with Gradoop&lt;/a&gt; - Martin Junghanns (Univ Leipzig)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:10&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/76152834.pdf&#34;&gt;Distributed graph flows: Cypher on Flink and Gradoop&lt;/a&gt; - Max Kießling (Neo Technology)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16:30&lt;/td&gt;
&lt;td&gt;closing - Peter Boncz&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;17:30&lt;/td&gt;
&lt;td&gt;end&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h5 id=&#34;friday&#34;&gt;Friday&lt;/h5&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;start time&lt;/th&gt;
&lt;th&gt;title – speaker&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;9:00&lt;/td&gt;
&lt;td&gt;welcome - Peter Boncz&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;9:20&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/76152833.pdf&#34;&gt;Graph processing in obi4wan&lt;/a&gt; - Frank Smit (OBI4WAN)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;9:40&lt;/td&gt;
&lt;td&gt;Graph problems in the space domain - Albrecht Schmidt (ESA)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:00&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/75792387.pdf&#34;&gt;Medical Ontologies for Healthcare&lt;/a&gt; - Michael Neumann (SAP)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:20&lt;/td&gt;
&lt;td&gt;coffee&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10:50&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/76447745.pdf&#34;&gt;The Train Benchmark: Cross-Technology Performance Evaluation of Continuous Model Queries&lt;/a&gt; - Gabor Szarnyas (BME)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:10&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/76021761.pdf&#34;&gt;Efficient sparse matrix computations and their generalization to graph computing applications&lt;/a&gt; - Albert-Jan Yzelman (Huawei)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11:30&lt;/td&gt;
&lt;td&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/76152837.pdf&#34;&gt;Experiments on Semantic Publishing Benchmark with large scale real news and LOD data at FactForge&lt;/a&gt; - Atanas Kyriakov (Ontotext)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:00&lt;/td&gt;
&lt;td&gt;lunch&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:00&lt;/td&gt;
&lt;td&gt;LDBC Board of Directors Meeting&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;17:00&lt;/td&gt;
&lt;td&gt;end&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h3 id=&#34;logistics&#34;&gt;Logistics&lt;/h3&gt;
&lt;h5 id=&#34;important-things-to-know&#34;&gt;&lt;strong&gt;Important things to know&lt;/strong&gt;&lt;/h5&gt;
&lt;p&gt;The following PDF guide provides additional information, such as recommended restaurants as well as sightseeing spots: &lt;a href=&#34;https://websmp201.sap-ag.de/~sapidp/011000358700001204882013E.pdf&#34;&gt;link&lt;/a&gt;&lt;/p&gt;
&lt;h5 id=&#34;venue&#34;&gt;&lt;strong&gt;Venue&lt;/strong&gt;&lt;/h5&gt;
&lt;p&gt;The TUC meeting will be held in the &lt;a href=&#34;https://websmp201.sap-ag.de/~sapidp/011000358700001204882013E.pdf&#34;&gt;SAP Headquarters&lt;/a&gt; at the SAP Guesthouse Kalipeh. The address is:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;WDF 44 / SAP Guesthouse Kalipeh&lt;br&gt;
Dietmar-Hopp-Allee 15&lt;br&gt;
69190 Walldorf&lt;br&gt;
Germany&lt;/strong&gt;&lt;/p&gt;
&lt;h6 id=&#34;maps-and-situation&#34;&gt;&lt;strong&gt;Maps and situation&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;&lt;a href=&#34;https://www.google.com/maps/place/SAP+Guesthouse+Kalipeh/@49.2951903,8.6436224,17z/data=!3m1!4b1!4m5!3m4!1s0x4797bea343a566af:0xd70698f3503ab74b!8m2!3d49.2951868!4d8.6458111&#34;&gt;Google Maps link&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/ninth-tuc-meeting/attachments/59277315/69042180.png&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;h4 id=&#34;getting-there&#34;&gt;&lt;strong&gt;Getting there&lt;/strong&gt;&lt;/h4&gt;
&lt;h5 id=&#34;by-plane&#34;&gt;&lt;strong&gt;By plane&lt;/strong&gt;&lt;/h5&gt;
&lt;p&gt;There are two airports close to SAP&amp;rsquo;s headquarter: Frankfurt Airport (FRA) and Stuttgart-Echterdingen Airport (STR). The journey from Frankfurt Airport to SAP headquarters takes about one hour by car, while it takes slightly longer from Stuttgart- Echterdingen Airport. Concerning airfare, flights to Frankfurt are usually somewhat more expensive than to Stuttgart.&lt;/p&gt;
&lt;p&gt;When booking flights to Frankfurt, you should be aware of Frankfurt-Hahn Airport (HHN), which serves low-cost carriers but is not connected to Frankfurt Airport. Frankfurt Hahn is approximately one hour from the Frankfurt main airport by car.&lt;/p&gt;
&lt;p&gt;The journey from Frankfurt Airport to SAP headquarters takes about one hour by car (95 kilometers, or 59 miles).&lt;/p&gt;
&lt;p&gt;Journey time from Stuttgart-Echterdingen Airport to SAP headquarters takes about 1 hour and 15 minutes by car (115 kilometers, or 71 miles).&lt;/p&gt;
&lt;h6 id=&#34;driving-directions&#34;&gt;&lt;strong&gt;Driving directions&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;&lt;strong&gt;Traveling from Frankfurt Airport (FRA) to SAP Headquarters:&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Directions to SAP headquarters:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;When leaving the airport, follow the highway symbol onto &amp;ldquo;A3/Würzburg/A5/Kassel/Basel/Frankfurt.&amp;rdquo;&lt;/li&gt;
&lt;li&gt;Follow the A5 to &amp;ldquo;Basel/Karlsruhe/Heidelberg.&amp;rdquo;&lt;/li&gt;
&lt;li&gt;Take exit 39 &amp;ndash; &amp;ldquo;Walldorf/Wiesloch.&amp;rdquo;&lt;/li&gt;
&lt;li&gt;Turn left onto B291.&lt;/li&gt;
&lt;li&gt;Turn right onto Dietmar-Hopp-Allee.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;(Should you use a navigational system which does not recognize the street name &amp;lsquo;Dietmar-Hopp-Allee&amp;rsquo; please use &amp;lsquo;Neurottstrasse&amp;rsquo; instead.)&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Traveling from Stuttgart-Echterdingen Airport (STR) to SAP Headquarters:&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;To get to SAP headquarters by car, there are two possible routes to take. The first leads you via Heilbronn and the second via Karlsruhe. The route via Karlsruhe is a bit shorter yet may be more congested.&lt;/p&gt;
&lt;p&gt;Directions to SAP headquarters:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;When leaving the airport, follow the highway symbol onto &amp;ldquo;A8/Stuttgart/B27.&amp;rdquo;&lt;/li&gt;
&lt;li&gt;Stay on A8 and follow the sign for &amp;ldquo;Karlsruhe/Heilbronn/Singen/A8.&amp;rdquo;&lt;/li&gt;
&lt;li&gt;Follow A8 to Karlsruhe.&lt;/li&gt;
&lt;li&gt;Take exit 41 &amp;ndash; &amp;ldquo;Dreieck Karlsruhe&amp;rdquo; to merge onto A5 toward &amp;ldquo;Frankfurt/Mannheim/Karlsruhe/Landau (Pfalz).&amp;rdquo;&lt;/li&gt;
&lt;li&gt;Take exit 39 &amp;ndash; &amp;ldquo;Walldorf/Wiesloch.&amp;rdquo;&lt;/li&gt;
&lt;li&gt;Turn left onto B291.&lt;/li&gt;
&lt;li&gt;Turn right onto Dietmar-Hopp-Allee.&lt;/li&gt;
&lt;/ul&gt;
&lt;h6 id=&#34;parking&#34;&gt;&lt;strong&gt;Parking&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;The closest parking lot to the event location is P7 (see figure above).&lt;/p&gt;
&lt;h5 id=&#34;by-train&#34;&gt;&lt;strong&gt;By Train&lt;/strong&gt;&lt;/h5&gt;
&lt;p&gt;As the infrastructure is very well developed in Europe, and in Germany in particular, taking the train is a great and easy way of traveling. Furthermore, the trains usually run on time, so this mode of travel is very convenient, especially for a group of people on longer journeys to major cities.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;From Frankfurt Airport (FRA) to SAP Headquarters&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Directions to SAP headquarters:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Go to Terminal 1, level T (see overview in Appendix).&lt;/li&gt;
&lt;li&gt;Go to the AIRail Terminal &amp;ndash; &amp;ldquo;Fernbahnhof&amp;rdquo; (long-distance trains).&lt;/li&gt;
&lt;li&gt;Choose a connection with the destination train station &amp;ldquo;Wiesloch&amp;ndash;Walldorf&amp;rdquo;.&lt;/li&gt;
&lt;li&gt;From station &amp;ldquo;Wiesloch&amp;ndash;Walldorf,&amp;rdquo; take bus number 707 or 721 toward &amp;ldquo;Industriegebiet Walldorf, SAP.&amp;rdquo; It is a 10-minute ride to reach bus stop &amp;lsquo;SAP headquarters&amp;rsquo;.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;From Stuttgart-Echterdingen Airport (STR) to SAP Headquarters&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Directions to SAP headquarters:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Go to the S-Bahn station in the airport, following the sign (station is called &amp;ldquo;Stuttgart Flughafen/Messe&amp;rdquo;).&lt;/li&gt;
&lt;li&gt;Take train number S2 or S3 to &amp;ldquo;Stuttgart Hauptbahnhof&amp;rdquo; (main station).&lt;/li&gt;
&lt;li&gt;From Stuttgart Hauptbahnhof choose a connection with the destination train station &amp;ldquo;Wiesloch&amp;ndash;Walldorf&amp;rdquo;.&lt;/li&gt;
&lt;li&gt;From station &amp;ldquo;Wiesloch&amp;ndash;Walldorf,&amp;rdquo; take bus number 707 or 721 toward &amp;ldquo;Industriegebiet Walldorf, SAP&amp;rdquo;. It is a 10-minute ride to reach bus stop &amp;lsquo;SAP headquarters&amp;rsquo;.&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>LDBC Is Proud to Announce the New LDBC Graphalytics Benchmark Draft Specification</title>
      <link>https://ldbcouncil.org/post/ldbc-is-proud-to-announce-the-new-ldbc-graphalytics-benchmark-draft-specification/</link>
      <pubDate>Tue, 06 Sep 2016 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/ldbc-is-proud-to-announce-the-new-ldbc-graphalytics-benchmark-draft-specification/</guid>
      <description>&lt;p&gt;LDBC is proud to announce the new LDBC Graphalytics Benchmark draft specification.&lt;/p&gt;
&lt;p&gt;LDBC Graphalytics is the first industry-grade graph data management benchmark for graph analysis platforms such as Giraph. It consists of six core algorithms, standard datasets, synthetic dataset generators, and reference outputs, enabling the objective comparison of graph analysis platforms.  It has strong industry support from Oracle, Intel, Huawei and IBM, and was tested and optimized on the best industrial and open-source systems.&lt;/p&gt;
&lt;p&gt;Tim Hegeman of &lt;a href=&#34;https://www.tudelft.nl&#34;&gt;TU Delft&lt;/a&gt; is today presenting the technical paper describing LDBC Graphalytics at the important &lt;a href=&#34;https://www.vldb.org/conference.html&#34;&gt;VLDB&lt;/a&gt; (Very Large DataBases) conference in New Delhi, where his talk also marks the release by LDBC of Graphalytics as a benchmark draft. Practitioners are invited to read the PVLDB paper, download the software and try running it.&lt;/p&gt;
&lt;p&gt;LDBC is eager to use any feedback for its future adoption of LDBC Graphalytics.&lt;/p&gt;
&lt;p&gt;Learn more: [/ldbc-graphalytics](LDBC Graphalytics)&lt;/p&gt;
&lt;p&gt;GitHub: &lt;a href=&#34;https://github.com/tudelft-atlarge/graphalytics&#34;&gt;https://github.com/tudelft-atlarge/graphalytics&lt;/a&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Eighth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/eighth-tuc-meeting/</link>
      <pubDate>Wed, 22 Jun 2016 14:45:20 -0400</pubDate>
      
      <guid>https://ldbcouncil.org/event/eighth-tuc-meeting/</guid>
      <description>&lt;p&gt;The LDBC consortium is pleased to announce its Eighth Technical User Community (TUC) meeting.&lt;/p&gt;
&lt;p&gt;This will be a two-day event/eighth-tuc-meeting/attachments at &lt;a href=&#34;http://www.oracle.com/technetwork/database/rdb/hqcc-dir-134199.pdf&#34;&gt;Oracle Conference Center&lt;/a&gt; in Redwood Shores facility on &lt;strong&gt;Wednesday and Thursday June 22-23, 2016&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;This will be the second TUC meeting after the finalisation of the LDBC FP7 EC funded project. The event/eighth-tuc-meeting/attachments will basically set the following aspects:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Two day event/eighth-tuc-meeting/attachments with one day devoted to User&amp;rsquo;s experiences and one day devoted to benchmarking experiences.&lt;/li&gt;
&lt;li&gt;Presentation of the benchmarking results for the different benchmarks.&lt;/li&gt;
&lt;li&gt;Interaction with the new LDBC Board of Directors and the LDBC organisation officials.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We welcome all users of RDF and Graph technologies to attend. If you are interested, please, contact Damaris Coll (UPC) at &lt;a href=&#34;mailto:damaris@ac.upc.edu&#34;&gt;damaris@ac.upc.edu&lt;/a&gt;; in order to notify Oracle security in advance, registration requests need to be in by &lt;strong&gt;June 12&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;In the agenda, there will be talks given by LDBC members and LDBC activities, but there will also be room for a number of short 20-minute talks by other participants. We are specifically interested in learning about new challenges in graph data management (where benchmarking would become useful) and on hearing about actual user stories and scenarios that could inspire benchmarks. Further, talks that provide feedback on existing benchmark (proposals) are very relevant. But nothing is excluded a priori if it is graph data management related. Talk proposals can be forwarded to Damaris as well and will be handled by Peter Boncz and Larri.&lt;/p&gt;
&lt;p&gt;Further, we call on you if you or your colleagues would happen to have contacts with companies that deal with graph data management scenarios to also attend and possibly present. LDBC is always looking to expand its circle of participants in TUCs meeting, its graph technology users contacts but also event/eighth-tuc-meeting/attachmentsually its membership base.&lt;/p&gt;
&lt;p&gt;In this page, you&amp;rsquo;ll find information about the following items:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#agenda&#34;&gt;Agenda&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#logistics&#34;&gt;Logistics&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#date&#34;&gt;Date&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#venue&#34;&gt;Venue&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#getting-there&#34;&gt;Getting there&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#accommodation&#34;&gt;Accommodation&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;On Wednesday, lunch is provided for all attendees at 12 pm. The TUC Meeting will start at 1pm.&lt;/p&gt;
&lt;h6 id=&#34;wednesday-22th-of-june-2016-room-203&#34;&gt;&lt;strong&gt;Wednesday, 22th of June 2016 (&lt;strong&gt;Room 203)&lt;/strong&gt;&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;(full morning: LDBC Board of Directors meeting)&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;12:00 - 13:00 Lunch (provided)&lt;/li&gt;
&lt;li&gt;13:00 - 13:30 Hassan Chafi (Oracle) and Josep L. Larriba-Pey (Sparsity) Registration and welcome.&lt;/li&gt;
&lt;li&gt;13:30 - 14:00 Peter Boncz (CWI) &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52133891.pdf&#34;&gt;LDBC introduction and status update&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;14:00 - 15:00 Details on the progress of LDBC Task Forces 1 (chair Josep L. Larriba-Pey)&lt;/li&gt;
&lt;li&gt;14:00 Arnau Prat (DAMA-UPC). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52133902.pdf&#34;&gt;Social Network Benchmark, Interactive workload&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;14:30 Tim Hegeman (TU Delft). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52133893.pdf&#34;&gt;Social Network Benchmark, Analytics workload&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;15:00 - 15:30 Coffee break&lt;/li&gt;
&lt;li&gt;15:30 - 17:00 Applications and use of Graph Technologies (chair Hassan Chafi)
&lt;ul&gt;
&lt;li&gt;15:30 Martin Zand (University of Rochester Clinical and Translational Science Institute). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52133897.pdf&#34;&gt;Graphing Healthcare Networks: Data, Analytics, and Use Cases.&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;16:00 David Meibusch, Nathan Hawes (Oracle Labs Australia). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52133901.pdf&#34;&gt;Frappé: Querying and managing evolving code dependency graphs&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;16:30 Jerven Bolleman (SIB Swiss Institute of Bioinformatics/UniProt consortium). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52133895.pdf&#34;&gt;UniProt: challenges of a public SPARQL endpoint.&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;17:00 - 18:30 Graph Technologies (chair Peter Boncz)
&lt;ul&gt;
&lt;li&gt;17:00 Eugene I. Chong (Oracle USA). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52133904.pdf&#34;&gt;Balancing Act to improve RDF Query Performance in Oracle Database&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;17:30 Lijun Chang (University of New South Wales). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52133906.pdf&#34;&gt;Efficient Subgraph Matching by Postponing Cartesian Products&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;18:00 Weining Qian (East China Normal University). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52133908.pdf&#34;&gt;On Statistical Characteristics of Real-Life Knowledge Graphs&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h6 id=&#34;thursday-23th-of-june-2016-room-203&#34;&gt;&lt;strong&gt;Thursday, 23th of June 2016 (Room 203)&lt;/strong&gt;&lt;/h6&gt;
&lt;ul&gt;
&lt;li&gt;08:00 - 09:00 Breakfast (provided)&lt;/li&gt;
&lt;li&gt;09:00 - 10:00 Details on the progress of LDBC Task Forces 2 (chair Josep L. Larriba-Pey)
&lt;ul&gt;
&lt;li&gt;09:00 Peter Boncz (CWI). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52133896.pdf&#34;&gt;Query Language Task Force status&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;09:45 Marcus Paradies (SAP). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52297729.pdf&#34;&gt;Social Network Benchmark, Business Intelligence workload&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;10:00 - 12:00 Graph Technologies and Benchmarking (chair Oskar van Rest)
&lt;ul&gt;
&lt;li&gt;10:00 Sergey Edunov (Facebook). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52297731.pdf&#34;&gt;Generating realistic trillion-edge graphs&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;10:30 George Fletcher (TU Eindhoven). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52297733.pdf&#34;&gt;An open source framework for schema-driven graph instance and graph query workload generation&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;11:00 Yinglong Xia (Huawei Research America): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52297735.pdf&#34;&gt;An Efficient Big Graph Analytics Platform&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;11:30 Zhe Wu (Oracle USA). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52297737.pdf&#34;&gt;Bridging RDF Graph and Property Graph Data Models&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;12:00 - 13:30 Lunch (provided)&lt;/li&gt;
&lt;li&gt;13:30 - 15:30 Graph Technologies (chair Arnau Prat)
&lt;ul&gt;
&lt;li&gt;13:30 Tobias Lindaaker (Neo Technology). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52297740.pdf&#34;&gt;An open standard for graph queries: the Cypher contribution&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;14:00 Arash Termehchy (Oregon State University). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52297742.pdf&#34;&gt;Toward Representation Independent Graph Querying &amp;amp; Analytics&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;14:30 Jerven Bolleman (SIB Swiss Institute of Bioinformatics/UniProt consortium). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52297745.pdf&#34;&gt;In the service of the federation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;15:00 Nandish Jayaram (Pivotal). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52297747.pdf&#34;&gt;Orion: Enabling Suggestions in a Visual Query Builder for Ultra-Heterogeneous Graphs&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;15:30 - 16:00 Coffee break&lt;/li&gt;
&lt;li&gt;16:00 - 17:15 Applications and use of Graph Technologies (chair Hassan Chafi)
&lt;ul&gt;
&lt;li&gt;16:00 Jans Aasman (Franz Inc.). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52428806.pdf&#34;&gt;Semantic Data Lake for Healthcare&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;16:15 Kevin Madden (Tom Sawyer Software). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52428812.pdf&#34;&gt;Dismantling Criminal Networks with Graph and Spatial Visualization and Analysis&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;16:45 Juan Sequeda (Capsenta). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52428810.pdf&#34;&gt;Using graph representation and semantic technology to virtually integrate and search multiple diverse data sources&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;17:15 Kevin Wilkinson (Hewlett Packard Labs). &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/52428808.pdf&#34;&gt;LDBC SNB extensions&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;17:45 - 18:15 Closing discussion&lt;/li&gt;
&lt;/ul&gt;
&lt;h6 id=&#34;friday-24th-of-june-2016-room-105&#34;&gt;&lt;strong&gt;Friday, 24th of June 2016 (Room 105)&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;At the same venue: the fourth international workshop on Graph Data Management, Experience and Systems (&lt;strong&gt;GRADES16&lt;/strong&gt;).&lt;/p&gt;
&lt;p&gt;18:30 social dinner for GRADES registrants (place to be announced)&lt;/p&gt;
&lt;h3 id=&#34;logistics&#34;&gt;Logistics&lt;/h3&gt;
&lt;h6 id=&#34;date&#34;&gt;&lt;strong&gt;Date&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;22nd and 23rd June 2016&lt;/p&gt;
&lt;h6 id=&#34;venue&#34;&gt;&lt;strong&gt;Venue&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;The TUC meeting will be held in the &lt;a href=&#34;http://www.oracle.com/technetwork/database/rdb/hqcc-dir-134199.pdf&#34;&gt;Oracle Conference Center&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;The address is:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Room 203 (Wed-Thu) &amp;amp; Room 105 (Fri)&lt;/strong&gt;&lt;br&gt;
&lt;strong&gt;Oracle Conference Center&lt;/strong&gt;&lt;br&gt;
&lt;strong&gt;350 Oracle Parkway&lt;/strong&gt;&lt;br&gt;
&lt;strong&gt;Redwood City, CA 94065, USA&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Maps and situation&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://www.google.com/maps/place/Oracle+Conference+Center/@37.5322827,-122.2667034,17z/data=!3m1!4b1!4m2!3m1!1s0x808f98b5450e8ca3:0xdc75e8b1c02bbb91&#34;&gt;Google Maps link&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Oracle Campus map:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/eighth-tuc-meeting/attachments/40927235/40927234.jpg&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;h5 id=&#34;getting-there&#34;&gt;&lt;strong&gt;Getting there&lt;/strong&gt;&lt;/h5&gt;
&lt;h6 id=&#34;driving-directions&#34;&gt;&lt;strong&gt;Driving directions&lt;/strong&gt;&lt;/h6&gt;
&lt;ul&gt;
&lt;li&gt;[Southbound] &lt;strong&gt;-&lt;/strong&gt; Take Highway 101 South (toward San Jose) to the Ralston Ave./Marine World Parkway exit. Take Marine World Parkway east which will loop you back over the freeway. Make a left at the first light onto Oracle Parkway. 350 Oracle Parkway will be on the right.&lt;/li&gt;
&lt;li&gt;[Northbound] &lt;strong&gt;-&lt;/strong&gt; Take Highway 101 North (toward San Francisco) to the Ralston Ave./Marine World Parkway exit. Take the first exit ramp onto Marine World Parkway. Make a left at the first light onto Oracle Parkway. 350 Oracle Parkway will be on the right.&lt;/li&gt;
&lt;/ul&gt;
&lt;h5 id=&#34;parking&#34;&gt;&lt;strong&gt;Parking&lt;/strong&gt;&lt;/h5&gt;
&lt;p&gt;The Conference Center has a designated parking lot located directly across from the building. If the lot is filled there is also additional parking in any of the parking garages located near by. No parking permits are needed.&lt;/p&gt;
&lt;h5 id=&#34;public-transport&#34;&gt;&lt;strong&gt;Public transport&lt;/strong&gt;&lt;/h5&gt;
&lt;p&gt;Take the Caltrain to either San Carlos or Hillsdale and take the free Oracle shuttle from there. Get off the Oracle shuttle at 100 Oracle Parkway (second stop) and walk 5 minutes to get to the Conference Center.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Caltrain timetables: &lt;a href=&#34;http://www.caltrain.com/schedules/weekdaytimetable.html&#34;&gt;http://www.caltrain.com/schedules/weekdaytimetable.html&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Oracle Shuttle timetables: &lt;a href=&#34;http://www.caltrain.com/schedules/weekdaytimetable.html&#34;&gt;http://www.caltrain.com/schedules/Shuttles/Oracle_Shuttle.html&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;You can also take the Caltrain to Belmont and walk 23 min, instead of taking the Oracle shuttle.&lt;/p&gt;
&lt;p&gt;Alternatively, SamTrans (San Mateo County&amp;rsquo;s Transit Agency) provides public bus service between the Millbrae BART station and Palo Alto with three stops on Oracle Parkway - one of which is directly in front of the Oracle Conference Center.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>LDBC and Apache Flink</title>
      <link>https://ldbcouncil.org/post/ldbc-and-apache-flink/</link>
      <pubDate>Mon, 16 Nov 2015 14:47:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/ldbc-and-apache-flink/</guid>
      <description>&lt;p&gt;Apache Flink &lt;a href=&#34;#references&#34;&gt;[1]&lt;/a&gt; is an open source platform for distributed stream and batch data processing. Flink&amp;rsquo;s core is a streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams. Flink also builds batch processing on top of the streaming engine, overlaying native iteration support, managed memory, and program optimization.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://flink.apache.org/img/flink-stack-small.png&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;Flink offers multiple APIs to process data from various data sources (e.g. HDFS, HBase, Kafka and JDBC). The DataStream and DataSet APIs allow the user to apply general-purpose data operations, like map, reduce, groupBy and join, on streams and static data respectively. In addition, Flink provides libraries for machine learning (Flink ML), graph processing (Gelly) and SQL-like operations (Table). All APIs can be used together in a single Flink program which enables the definition of powerful analytical workflows and the implementation of distributed algorithms.&lt;/p&gt;
&lt;p&gt;The following snippet shows how a wordcount program can be expressed in Flink using the DataSet API:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-java&#34; data-lang=&#34;java&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;DataSet&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;String&lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;&lt;/span&gt; text &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; env&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;fromElements&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;He who controls the past controls the future.&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;He who controls the present controls the past.&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;DataSet&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;Tuple2&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;String&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; Integer&lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;&amp;gt;&lt;/span&gt; wordCounts &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; text
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;flatMap&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;new&lt;/span&gt; LineSplitter&lt;span style=&#34;color:#f92672&#34;&gt;())&lt;/span&gt; &lt;span style=&#34;color:#75715e&#34;&gt;// splits the line and outputs (word,1)
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;tuples&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;groupBy&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;0&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;)&lt;/span&gt; &lt;span style=&#34;color:#75715e&#34;&gt;// group by word
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;sum&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;);&lt;/span&gt; &lt;span style=&#34;color:#75715e&#34;&gt;// sum the 1&amp;#39;s
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;wordCounts&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;print&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;();&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;At the Leipzig University, we use Apache Flink as execution layer for our graph analytics platform Gradoop &lt;a href=&#34;#references&#34;&gt;[2]&lt;/a&gt;. The LDBC datagen helps us to evaluate the scalability of our algorithms and operators in a distributed execution environment. To use the generated graph data in Flink, we wrote a tool that transforms the LDBC output files into Flink data sets for further processing &lt;a href=&#34;#references&#34;&gt;[3]&lt;/a&gt;. Using the class &lt;code&gt;LDBCToFlink&lt;/code&gt;, LDBC output files can be read directly from HDFS or from the local file system:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-java&#34; data-lang=&#34;java&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;final&lt;/span&gt; ExecutionEnvironment env &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  ExecutionEnvironment&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;getExecutionEnvironment&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;();&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;final&lt;/span&gt; LDBCToFlink ldbcToFlink &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;new&lt;/span&gt; LDBCToFlink&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;hdfs:///ldbc_snb_datagen/social_network&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; &lt;span style=&#34;color:#75715e&#34;&gt;// or &amp;#34;/path/to/social_network&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;  env&lt;span style=&#34;color:#f92672&#34;&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;DataSet&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;LDBCVertex&lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;&lt;/span&gt; vertices &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; ldbcToFlink&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;getVertices&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;();&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;DataSet&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;LDBCEdge&lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;&lt;/span&gt; edges &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; ldbcToFlink&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;getEdges&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;();&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The tuple classes &lt;code&gt;LDBCVertex&lt;/code&gt; and &lt;code&gt;LDBCEdge&lt;/code&gt; hold the information generated by the LDBC datagen and are created directly from its output files. During the transformation process, globally unique vertex identifiers are created based on the LDBC identifier and the vertex class. When reading edge files, source and target vertex identifiers are computed in the same way to ensure consistent linking between vertices.&lt;/p&gt;
&lt;p&gt;Each &lt;code&gt;LDBCVertex&lt;/code&gt; instance contains:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;an identifier, which is unique among all vertices * a vertex label (e.g. &lt;code&gt;Person&lt;/code&gt;, &lt;code&gt;Comment&lt;/code&gt;) * a key-value map of properties including also multivalued properties&lt;br&gt;
(e.g. &lt;code&gt;Person.email&lt;/code&gt;)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Each &lt;code&gt;LDBCEdge&lt;/code&gt; instance contains:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;an identifier, which is unique among all edges&lt;/li&gt;
&lt;li&gt;an edge label (e.g. &lt;code&gt;knows&lt;/code&gt;, &lt;code&gt;likes&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;a source vertex identifier&lt;/li&gt;
&lt;li&gt;a target vertex identifier&lt;/li&gt;
&lt;li&gt;a key-value map of properties&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The resulting datasets can be used by the DataSet API and all libraries that are built on top of it (i.e. Flink ML, Gelly and Table). In the following example, we load the LDBC graph from HDFS, filter vertices with the label &lt;code&gt;Person&lt;/code&gt; and edges with the label &lt;code&gt;knows&lt;/code&gt; and use Gelly to compute the connected components of that subgraph. The full source code is available on GitHub &lt;a href=&#34;#references&#34;&gt;[4]&lt;/a&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-java&#34; data-lang=&#34;java&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;final&lt;/span&gt; ExecutionEnvironment env &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  ExecutionEnvironment&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;getExecutionEnvironment&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;();&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;final&lt;/span&gt; LDBCToFlink ldbcToFlink &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;new&lt;/span&gt; LDBCToFlink&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;/home/s1ck/Devel/Java/ldbc_snb_datagen/social_network&amp;#34;&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  env&lt;span style=&#34;color:#f92672&#34;&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// filter vertices with label “Person”
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;DataSet&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;LDBCVertex&lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;&lt;/span&gt; ldbcVertices &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; ldbcToFlink&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;getVertices&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;filter&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;new&lt;/span&gt; VertexLabelFilter&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;LDBCConstants&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;VERTEX_CLASS_PERSON&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;));&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// filter edges with label “knows”
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;DataSet&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;LDBCEdge&lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;&lt;/span&gt; ldbcEdges &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; ldbcToFlink&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;getEdges&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;()&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;filter&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;new&lt;/span&gt; EdgeLabelFilter&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;LDBCConstants&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;EDGE_CLASS_KNOWS&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;));&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// create Gelly vertices suitable for connected components
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;DataSet&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;Vertex&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;Long&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; Long&lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;&amp;gt;&lt;/span&gt; vertices &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; ldbcVertices&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;map&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;new&lt;/span&gt; VertexInitializer&lt;span style=&#34;color:#f92672&#34;&gt;());&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// create Gelly edges suitable for connected components
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;DataSet&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;Edge&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;Long&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; NullValue&lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;&amp;gt;&lt;/span&gt; edges &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; ldbcEdges&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;map&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;new&lt;/span&gt; EdgeInitializer&lt;span style=&#34;color:#f92672&#34;&gt;());&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// create Gelly graph
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;Graph&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;Long&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; Long&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; NullValue&lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;&lt;/span&gt; g &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; Graph&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;fromDataSet&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;vertices&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; edges&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; env&lt;span style=&#34;color:#f92672&#34;&gt;);&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// run connected components on the subgraph for 10 iterations
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;DataSet&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;Vertex&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;Long&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; Long&lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;&amp;gt;&lt;/span&gt; components &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  g&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;run&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;new&lt;/span&gt; ConnectedComponents&lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;&lt;/span&gt;Long&lt;span style=&#34;color:#f92672&#34;&gt;,&lt;/span&gt; NullValue&lt;span style=&#34;color:#f92672&#34;&gt;&amp;gt;(&lt;/span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;10&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;));&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;// print the component id of the first 10 vertices
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;&lt;/span&gt;components&lt;span style=&#34;color:#f92672&#34;&gt;.&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;first&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;(&lt;/span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;10&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;).&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;print&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;();&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The ldbc-flink-import tool is available on Github &lt;a href=&#34;#references&#34;&gt;[3]&lt;/a&gt; and licensed under the GNU GPLv3. If you have any questions regarding the tool please feel free to contact me on GitHub. If you find bugs or have any ideas for improvements, please create an issue or a pull request.&lt;/p&gt;
&lt;p&gt;If you want to learn more about Apache Flink, a good starting point is the main documentation &lt;a href=&#34;#references&#34;&gt;[5]&lt;/a&gt; and if you have any question feel free to ask the official mailing lists.&lt;br&gt;
There is also a nice set of videos &lt;a href=&#34;#references&#34;&gt;[6]&lt;/a&gt; available from the latest Flink Forward conference.&lt;/p&gt;
&lt;h4 id=&#34;references&#34;&gt;References&lt;/h4&gt;
&lt;p&gt;[1] &lt;a href=&#34;http://flink.apache.org/&#34;&gt;http://flink.apache.org/&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;[2] &lt;a href=&#34;https://github.com/dbs-leipzig/gradoop&#34;&gt;https://github.com/dbs-leipzig/gradoop&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;[3] &lt;a href=&#34;https://github.com/s1ck/ldbc-flink-import&#34;&gt;https://github.com/s1ck/ldbc-flink-import&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;[4] &lt;a href=&#34;https://gist.github.com/s1ck/b33e6a4874c15c35cd16&#34;&gt;https://gist.github.com/s1ck/b33e6a4874c15c35cd16&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;[5] (link expired)&lt;/p&gt;
&lt;p&gt;[6] &lt;a href=&#34;https://www.youtube.com/channel/UCY8_lgiZLZErZPF47a2hXMA&#34;&gt;https://www.youtube.com/channel/UCY8_lgiZLZErZPF47a2hXMA&lt;/a&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Seventh TUC Meeting</title>
      <link>https://ldbcouncil.org/event/seventh-tuc-meeting/</link>
      <pubDate>Mon, 09 Nov 2015 14:17:30 -0400</pubDate>
      
      <guid>https://ldbcouncil.org/event/seventh-tuc-meeting/</guid>
      <description>&lt;p&gt;The LDBC consortium is pleased to announce its Seventh Technical User Community (TUC) meeting.&lt;/p&gt;
&lt;p&gt;This will be a two-day event at &lt;a href=&#34;http://www.research.ibm.com/labs/watson&#34;&gt;IBM&amp;rsquo;s TJ Watson&lt;/a&gt; facility on &lt;strong&gt;Monday and Tuesday November 9/10, 2015.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;This will be the first TUC meeting after the finalisation of the LDBC FP7 EC funded project. The event will basically set the following aspects:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Two day event with one day devoted to User&amp;rsquo;s experiences and one day devoted to benchmarking experiences.&lt;/li&gt;
&lt;li&gt;Presentation of the benchmarking results for the different benchmarks.&lt;/li&gt;
&lt;li&gt;Interaction with the new LDBC Board of Directors and the LDBC organisation officials.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We welcome all users of RDF and Graph technologies to attend. If you are interested, please, contact Damaris Coll (UPC) at &lt;a href=&#34;mailto:damaris@ac.upc.edu&#34;&gt;damaris@ac.upc.edu&lt;/a&gt;; in order to notify IBM security in advance, registration requests need to be in by Nov 1.&lt;/p&gt;
&lt;p&gt;In the agenda, there will be talks given by LDBC members and LDBC activities, but there will also be room for a number of short 20-minute talks by other participants. We are specifically interested in learning about new challenges in graph data management (where benchmarking would become useful) and on hearing about actual user stories and scenarios that could inspire benchmarks. Further, talks that provide feedback on existing benchmark (proposals) are very relevant. But nothing is excluded a priori if it is graph data management related. Talk proposals can be forwarded to Damaris as well and will be handled by Peter Boncz and Larri.&lt;/p&gt;
&lt;p&gt;Further, we call on you if you or your colleagues would happen to have contacts with companies that deal with graph data management scenarios to also attend and possibly present. LDBC is always looking to expand its circle of participants in TUCs meeting, its graph technology users contacts but also eventually its membership base.&lt;/p&gt;
&lt;p&gt;In this page, you&amp;rsquo;ll find information about the following items:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#agenda&#34;&gt;Agenda&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#logistics&#34;&gt;Logistics&lt;/a&gt;&lt;br&gt;
- &lt;a href=&#34;#date&#34;&gt;&lt;strong&gt;Date&lt;/strong&gt;&lt;/a&gt;&lt;br&gt;
- &lt;a href=&#34;#venue&#34;&gt;&lt;strong&gt;Venue&lt;/strong&gt;&lt;/a&gt;&lt;br&gt;
- &lt;a href=&#34;#maps-and-situation&#34;&gt;&lt;strong&gt;Maps and situation&lt;/strong&gt;&lt;/a&gt;&lt;br&gt;
- &lt;a href=&#34;#getting-there&#34;&gt;&lt;strong&gt;Getting there&lt;/strong&gt;&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;Monday, 9th of November 2015&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;8:45 - 9:15 Registration and welcome (Yinglong Xia and Josep L. Larriba Pey)&lt;/p&gt;
&lt;p&gt;9:15 - 9:30 LDBC introduction and status update (Josep L. Larriba-Pey)&lt;/p&gt;
&lt;p&gt;9:30 - 10:30 Details on the progress of LDBC Task Forces 1 (chair Josep L. Larriba-Pey)&lt;/p&gt;
&lt;p&gt;9:30 Arnau Prat (DAMA-UPC). Social Network Benchmark, Interactive workload&lt;/p&gt;
&lt;p&gt;10:00 Orri Erling (OpenLink Software). Social Network Benchmark, Business Intelligence workload&lt;/p&gt;
&lt;p&gt;10:30-11:00 Coffee break&lt;/p&gt;
&lt;p&gt;11:00 - 12:30 Details on the progress of LDBC Task Forces 2 (chair Yinglong Xia)&lt;/p&gt;
&lt;p&gt;11:00 Alexandru Iosup (TU Delft). Social Network Benchmark, Analytics workload.&lt;/p&gt;
&lt;p&gt;11:30 Claudio Gutierrez (U Chile). Query Language Task Force status.&lt;/p&gt;
&lt;p&gt;12:00 Atanas Kiryakov (Ontotext). Semantic Publishing Benchmark status&lt;/p&gt;
&lt;p&gt;12:30 - 14:00 Lunch break&lt;/p&gt;
&lt;p&gt;14:00 - 16:00 Technologies and benchmarking (chair Hassan Chafi)&lt;/p&gt;
&lt;p&gt;14:00 Molham Aref (LogicBlox). Graph Data Management with LogicBlox&lt;/p&gt;
&lt;p&gt;14:30 Peter Kogge (Notre Dame). BFS as in Graph500 on today&amp;rsquo;s architectures&lt;/p&gt;
&lt;p&gt;15:00 Ching-Yung Lin (IBM). Status and Demo of IBM System G&lt;/p&gt;
&lt;p&gt;15:30-16:00 Coffee break&lt;/p&gt;
&lt;p&gt;16:00 - 17:00 Technologies (chair Irini Fundulaki)&lt;/p&gt;
&lt;p&gt;16:00 Kavitha Srinivas (IBM). SQLGraph: An efficient relational based property graph store&lt;/p&gt;
&lt;p&gt;16:30 David Ediger (GeorgiaTech). STINGER&lt;/p&gt;
&lt;p&gt;17:00 Gary King (Franz Inc.). AllegroGraph&amp;rsquo;s SPARQL implementation with Social Network Analytics abilities using Magic Properties&lt;/p&gt;
&lt;p&gt;17:30 Manoj Kumar (IBM). Linear Algebra Formulation for Large Graph Analytics&lt;/p&gt;
&lt;p&gt;18:00 Reihaneh Amini (Wright State University) Linked Data in the GeoLink Usecase&lt;/p&gt;
&lt;p&gt;19:00 Social dinner&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Tuesday 10th November 2015&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;9:00 - 10:30 Technology, Applications and Benchmarking (chair Alexandru Iosup)&lt;/p&gt;
&lt;p&gt;9:00 Philip Rathle (Neo). On openCypher&lt;/p&gt;
&lt;p&gt;9:20 Morteza Shahriari (University of Florida). Multi-modal Probabilistic Knowledge Base for Remote Sensing Species Identification&lt;/p&gt;
&lt;p&gt;9:50 Peter Kogge (Notre Dame). Challenging problems with Lexis Nexis Risk Solutions&lt;/p&gt;
&lt;p&gt;10:10 Arnau Prat (DAMA-UPC). DATAGEN, status and perspectives for synthetic data generation&lt;/p&gt;
&lt;p&gt;10:30 - 11:00 Coffee break&lt;/p&gt;
&lt;p&gt;11:00 - 12:45 Applications and use of Graph Technologies (chair Atanas Kiryakov)&lt;/p&gt;
&lt;p&gt;11:00 Hassan Chafi (Oracle). Status and characteristics of PGQL&lt;/p&gt;
&lt;p&gt;11:20 David Guedalia (TAGIIO). Multi-tier distributed mobile applications and how they split their workload,&lt;/p&gt;
&lt;p&gt;11:40 Guojing Cong (IBM). Algorithmic technique and architectural support for fast graph analysis&lt;/p&gt;
&lt;p&gt;12:00 Josep Lluis Larriba-Pey. Conclusions for the TUC meeting and future perspectives&lt;/p&gt;
&lt;p&gt;12:30 - 14:00 Lunch break&lt;/p&gt;
&lt;p&gt;14:00 LDBC Board of Directors&lt;/p&gt;
&lt;h3 id=&#34;logistics&#34;&gt;Logistics&lt;/h3&gt;
&lt;h6 id=&#34;date&#34;&gt;&lt;strong&gt;Date&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;9th and 10th November 2015&lt;/p&gt;
&lt;h6 id=&#34;venue&#34;&gt;&lt;strong&gt;Venue&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;The TUC meeting will be held in the IBM Thomas J Watson Research Center.&lt;br&gt;
The address is:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;IBM Thomas J Watson Research Center&lt;/strong&gt;&lt;br&gt;
&lt;strong&gt;1101 Kitchawan Rd,&lt;/strong&gt;&lt;br&gt;
&lt;strong&gt;Yorktown Heights, NY 10598, USA&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;If you are using a &lt;em&gt;GPS system&lt;/em&gt;, please enter &lt;strong&gt;&amp;ldquo;200 Aqueduct Road, Ossining NY, 10562&amp;rdquo;&lt;/strong&gt; for accurate directions to the lab entrance. You may also want to check the routing online.&lt;/p&gt;
&lt;p&gt;The meeting will take place in the &lt;em&gt;Auditorium&lt;/em&gt; on November 9th, and in Meeting Room &lt;em&gt;20-043&lt;/em&gt; on November 10th.&lt;/p&gt;
&lt;h6 id=&#34;maps-and-situation&#34;&gt;&lt;strong&gt;Maps and situation&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;You are highly suggested to &lt;strong&gt;rent a car&lt;/strong&gt; for your convenience, since the public transportation system does not cover this area very well. Besides, there is no hotel within walkable distance to the IBM T.J. Watson Research Center. Feel free to find carpool with other attendees.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/seventh-tuc-meeting/attachments/6882333/15926330.png&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;h6 id=&#34;getting-there&#34;&gt;&lt;strong&gt;Getting there&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;&lt;strong&gt;Upper and Eastern New England&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Route I-84 west to Route I-684, south to Exit 6, west on Route 35 to Route 100, south to Route 134, west 2.5 miles. IBM is on the left.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;New Haven and Connecticut Shores&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Merritt Parkway or New England Thruway (Route I-95) west to Route I-287, west to Exit 3, north on Sprain Brook Parkway, which merges into Taconic State Parkway, north to Ossining/Route 134 exit. Turn right and proceed east on Route 134 several hundred yards. IBM is on the right.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;New Jersey&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Take New York State Thruway (Route I-87) east across the Tappan Zee Bridge and follow signs to the Saw Mill Parkway north. Proceed north on Saw Mill River Parkway to Taconic State Parkway exit, north to Ossining/Route 134 exit. Turn right and proceed east on Route 134 several hundred yards. IBM is on the right.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Upstate New York&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Route I-84 east across Newburgh-Beacon Bridge to Exit 16-S. Taconic State Parkway south to Route 134 East exit. Turn right and proceed east on Route 134 several hundred yards. IBM is on the right.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;New York City (Manhattan)&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Henry Hudson Parkway north, which becomes Saw Mill River Parkway, north to Taconic State Parkway exit. North on Taconic State Parkway to Ossining/Route 134 exit. Turn right and proceed east on Route 134 several hundred yards. IBM is on the right.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;John F. Kennedy International Airport&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;North on Van Wyck Expressway to the Whitestone Expressway and continue north across the Bronx-Whitestone Bridge to the Hutchinson River Parkway north to the Cross County Parkway exit and proceed west to the Bronx River Parkway. North on the Bronx River Parkway to the Sprain Brook Parkway, which merges into the Taconic State Parkway. Continue north to Ossining/Route 134 exit. Turn right and proceed east on Route 134 several hundred yards. IBM is on the right.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;LaGuardia Airport&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;East on the Grand Central Parkway, north on the Whitestone Expressway, and continue north across the Bronx-Whitestone Bridge. Continue with instructions from John F. Kennedy International Airport, above.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Newark International Airport&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;North on the New Jersey Turnpike (Route I-95). Stay in local lanes and take Exit 72 for Palisades Interstate Parkway. North on the Palisades Interstate Parkway to the New York State Thruway, Route I-87, and east across the Tappan Zee Bridge. Continue with instructions from New Jersey, above.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Stewart International Airport&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Route 207 east to Route I-84, east across Newburgh-Beacon Bridge to Taconic State Parkway, south. Continue with instructions from Upstate New York, above.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Westchester County Airport&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Right on Route 120, north. Turn left where Route 120 merges with Route 133. Continue on Route 120. Cross Route 100 and continue straight on Shingle House Road to Pines Bridge Road. Turn right and proceed several hundred yards. IBM is on the left.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Public Transportation&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Metropolitan Transportation Authority (MTA) train stations nearest to the Yorktown Heights location are the Croton-Harmon and White Plains stations. Taxi service is available at both locations.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Elements of Instance Matching Benchmarks: a Short Overview</title>
      <link>https://ldbcouncil.org/post/elements-of-instance-matching-benchmarks-a-short-overview/</link>
      <pubDate>Tue, 16 Jun 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/elements-of-instance-matching-benchmarks-a-short-overview/</guid>
      <description>&lt;p&gt;The number of datasets published in the Web of Data as part of the Linked Data Cloud is constantly increasing. The Linked Data paradigm is based on the unconstrained publication of information by different publishers, and the interlinking of web resources through “same-as” links which specify that two URIs correspond to the same real world object. In the vast number of data sources participating in the Linked Data Cloud, this information is not explicitly stated but is discovered using &lt;strong&gt;instance matching&lt;/strong&gt; techniques and tools. Instance matching is also known as &lt;strong&gt;record linkage&lt;/strong&gt; &lt;a href=&#34;#references&#34;&gt;[1]&lt;/a&gt;, &lt;strong&gt;duplicate detection&lt;/strong&gt; &lt;a href=&#34;#references&#34;&gt;[2]&lt;/a&gt;, &lt;strong&gt;entity resolution&lt;/strong&gt; &lt;a href=&#34;#references&#34;&gt;[3]&lt;/a&gt; and &lt;strong&gt;object identification&lt;/strong&gt; &lt;a href=&#34;#references&#34;&gt;[4]&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;For instance, a search in Geonames (&lt;a href=&#34;http://www.geonames.org/&#34;&gt;http://www.geonames.org/&lt;/a&gt;) for &amp;ldquo;Athens&amp;rdquo; would return a resource (i.e., URI) accompanied with a map of the area and information about the place; additional information for the city of Athens can be found in other datasets such as for instance DBpedia (&lt;a href=&#34;http://dbpedia.org/&#34;&gt;http://dbpedia.org/&lt;/a&gt;) or Open Government Datasets. To exploit all obtain all necessary information about the city of Athens we need to establish that the retrieved resources refer to the same real world object.&lt;/p&gt;
&lt;p&gt;Web resources are published by &amp;ldquo;autonomous agents&amp;rdquo; who choose their preferred information representation or the one that best fits the application of interest. Furthermore, different representations of the same real world entity are due to data acquisition errors or different acquisition techniques used to process scientific data. Moreover, real world entities evolve and change over time, and sources need to keep track of these developments, a task that is very hard and often not possible. Finally, when integrating data from multiple sources, the process itself may add new erroneous data. Clearly, these reasons are not limited to problems that did arise in the era of Web Data, it is thus not surprising that instance matching systems have been around for several years &lt;a href=&#34;#references&#34;&gt;[2]&lt;/a&gt;&lt;a href=&#34;#references&#34;&gt;[5]&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;It is though essential at this point to develop, along with instance and entity matching systems, &lt;em&gt;instance matching benchmarks to determine the weak and strong points of those systems, as well as their overall quality in order to support users in deciding the system to use for their needs&lt;/em&gt;. Hence, well defined, and good quality benchmarks are important for comparing the performance of the available or under development instance matching systems. Benchmarks are used not only to inform users of the strengths and weaknesses of systems, but also to motivate developers, researchers and technology vendors to deal with the weak points of their systems and to ameliorate their performance and functionality. They are also useful for identifying the settings in which each of the systems has optimal performance. Benchmarking aims at providing an objective basis for such assessments.&lt;/p&gt;
&lt;p&gt;An instance matching benchmark for Linked Data consists of a &lt;em&gt;source&lt;/em&gt; and &lt;em&gt;target dataset&lt;/em&gt; implementing a set of &lt;em&gt;test-cases&lt;/em&gt;, where each test case addresses a different kind of requirement regarding instance matching, a &lt;em&gt;ground truth&lt;/em&gt; or &lt;em&gt;gold standard&lt;/em&gt; and finally the &lt;em&gt;evaluation metrics&lt;/em&gt; used to &lt;em&gt;assess the benchmark.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Datasets are the raw material of a benchmark. A benchmark comprises of a &lt;em&gt;source&lt;/em&gt; and &lt;em&gt;target&lt;/em&gt; dataset and the objective of an instance matching system is to discover the matches of the two. Datasets are characterized by (a) their &lt;em&gt;nature&lt;/em&gt; (&lt;em&gt;real&lt;/em&gt; or &lt;em&gt;synthetic&lt;/em&gt;), (b) the &lt;em&gt;schemas/ontologies&lt;/em&gt; they use,  (c) their &lt;em&gt;domains&lt;/em&gt;,  (d) the &lt;em&gt;languages&lt;/em&gt; they are written in, and (e) the &lt;em&gt;variations/heterogeneities&lt;/em&gt; of the datasets. Real datasets are widely used in benchmarks since they offer realistic conditions for heterogeneity problems and they have realistic distributions. &lt;em&gt;Synthetic datasets&lt;/em&gt; are generated using automated data generators and  are useful because they offer fully controlled test conditions, have accurate gold standards and allow setting the focus on specific types of heterogeneity problems in a systematic manner&lt;/p&gt;
&lt;p&gt;Datasets (and benchmarks) may contain different &lt;em&gt;kinds of variations&lt;/em&gt; that correspond to &lt;em&gt;different test cases&lt;/em&gt;. According to Ferrara et.al. &lt;a href=&#34;#references&#34;&gt;[6]&lt;/a&gt;&lt;a href=&#34;#references&#34;&gt;[7]&lt;/a&gt;, three kinds of variations exist for Linked Data, namely &lt;em&gt;data variations&lt;/em&gt;, &lt;em&gt;structural variations&lt;/em&gt; and &lt;em&gt;logical variations&lt;/em&gt;. The first refers mainly to differences due to typographical errors, differences in the employed data formats, language etc. The second refers to the differences in the structure of the employed Linked Data schemas. Finally, the third  type derives from the use of semantically rich RDF and OWL constructs that enable one to define hierarchies and equivalence of classes and properties, (in)equality of instances, complex class definitions through union and intersection among others.&lt;/p&gt;
&lt;p&gt;The common case in real benchmarks is that the datasets to be matched contain different kinds (combinations) of variations. On the other hand, synthetic datasets may be purposefully designed to contain specific types (or combinations) of variations (e.g., only structural), or may be more general in an effort to illustrate all the common cases of discrepancies that appear in reality between individual descriptions.&lt;/p&gt;
&lt;p&gt;The &lt;em&gt;gold standard&lt;/em&gt; is considered as the “correct answer sheet” of the benchmark, and is used to judge the completeness and soundness of the result sets of the benchmarked systems. For instance matching benchmarks employing synthetic datasets, the gold standard is always automatically generated, as the errors (variations) that are added into the datasets are known and systematically created. When it comes to real datasets, the gold standard can be either manually curated or (semi-) automatically generated. In the first case, domain experts manually mark the matches between the datasets, whereas in the second, supervised and crowdsourcing techniques aid the process of finding the matches, a process that is often time consuming and error prone.&lt;/p&gt;
&lt;p&gt;Last, an instance matching benchmark uses &lt;em&gt;evaluation metrics&lt;/em&gt; to determine and assess the systems’ output quality and performance. For instance matching tools, performance is not a critical aspect.  On the other hand, an instance matching tool should return all and only the correct answers. So, what matters most is returning the relevant matches, rather than returning them quickly. For this reason, the evaluation metrics that are dominantly employed for instance matching benchmarks are the standard &lt;em&gt;precision&lt;/em&gt;, &lt;em&gt;recall&lt;/em&gt; and &lt;em&gt;f-measure&lt;/em&gt; metrics.&lt;/p&gt;
&lt;h4 id=&#34;references&#34;&gt;References&lt;/h4&gt;
&lt;p&gt;[1] Li, C., Jin, L., and Mehrotra, S. (2006) Supporting efficient record linkage for large data sets using mapping techniques. WWW 2006.&lt;/p&gt;
&lt;p&gt;[2] Dragisic, Z., Eckert, K., Euzenat, J., Faria, D., Ferrara, A., Granada, R., Ivanova, V., Jimenez-Ruiz, E., Oskar Kempf, A., Lambrix, P., Montanelli, S., Paulheim, H., Ritze, D., Shvaiko, P., Solimando, A., Trojahn, C., Zamaza, O., and Cuenca Grau,  B. (2014) Results of the Ontology Alignment Evaluation Initiative 2014. Proc. 9th ISWC workshop on ontology matching (OM 2014).&lt;/p&gt;
&lt;p&gt;[3] Bhattacharya, I. and Getoor, L. (2006) Entity resolution in graphs. Mining Graph Data. Wiley and Sons 2006.&lt;/p&gt;
&lt;p&gt;[4] Noessner, J., Niepert, M., Meilicke, C., and Stuckenschmidt, H. (2010) Leveraging Terminological Structure for Object Reconciliation. In ESWC 2010.&lt;/p&gt;
&lt;p&gt;[5] Flouris, G., Manakanatas, D., Kondylakis, H., Plexousakis, D., Antoniou, G. Ontology Change: Classification and Survey (2008) Knowledge Engineering Review (KER 2008), pages 117-152.&lt;/p&gt;
&lt;p&gt;[6] Ferrara, A., Lorusso, D., Montanelli, S., and Varese, G. (2008) Towards a Benchmark for Instance Matching. Proc. 3th ISWC workshop on ontology matching (OM 2008).&lt;/p&gt;
&lt;p&gt;[7] Ferrara, A., Montanelli, S., Noessner, J., and Stuckenschmidt, H. (2011) Benchmarking Matching Applications on the Semantic Web. In ESWC, 2011.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>SNB Interactive Part 3: Choke Points and Initial Run on Virtuoso</title>
      <link>https://ldbcouncil.org/post/snb-interactive-part-3-choke-points-and-initial-run-on-virtuoso/</link>
      <pubDate>Wed, 10 Jun 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/snb-interactive-part-3-choke-points-and-initial-run-on-virtuoso/</guid>
      <description>&lt;p&gt;In this post we will look at running the &lt;a href=&#34;https://ldbcouncil.org/developer/snb&#34;&gt;LDBC SNB&lt;/a&gt; on &lt;a href=&#34;https://virtuoso.openlinksw.com/&#34;&gt;Virtuoso&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;First, let&amp;rsquo;s recap what the benchmark is about:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;fairly frequent short updates, with no update contention worth mentioning&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;short random lookups&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;medium complex queries centered around a person&amp;rsquo;s social environment&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The updates exist so as to invalidate strategies that rely too heavily on precomputation. The short lookups exist for the sake of realism; after all, an online social application does lookups for the most part. The medium complex queries are to challenge the DBMS.&lt;/p&gt;
&lt;p&gt;The DBMS challenges have to do firstly with query optimization, and secondly with execution with a lot of non-local random access patterns. Query optimization is not a requirement, &lt;em&gt;per se,&lt;/em&gt; since imperative implementations are allowed, but we will see that these are no more free of the laws of nature than the declarative ones.&lt;/p&gt;
&lt;p&gt;The workload is arbitrarily parallel, so intra-query parallelization is not particularly useful, if also not harmful. There are latency constraints on operations which strongly encourage implementations to stay within a predictable time envelope regardless of specific query parameters. The parameters are a combination of person and date range, and sometimes tags or countries. The hardest queries have the potential to access all content created by people within 2 steps of a central person, so possibly thousands of people, times 2000 posts per person, times up to 4 tags per post. We are talking in the millions of key lookups, aiming for sub-second single-threaded execution.&lt;/p&gt;
&lt;p&gt;The test system is the same as used in the &lt;a href=&#34;http://www.openlinksw.com/weblog/oerling/?id=1739&#34;&gt;TPC-H series&lt;/a&gt;: dual Xeon E5-2630, 2x6 cores x 2 threads, 2.3GHz, 192 GB RAM. The software is the &lt;a href=&#34;https://github.com/v7fasttrack/virtuoso-opensource/tree/feature/analytics&#34;&gt;feature/analytics branch&lt;/a&gt; of &lt;a href=&#34;https://github.com/v7fasttrack/virtuoso-opensource/&#34;&gt;v7fasttrack, available from www.github.com&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The dataset is the SNB 300G set, with:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;1,136,127&lt;/th&gt;
&lt;th&gt;persons&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;125,249,604&lt;/td&gt;
&lt;td&gt;knows edges&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;847,886,644&lt;/td&gt;
&lt;td&gt;posts, including replies&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1,145,893,841&lt;/td&gt;
&lt;td&gt;tags of posts or replies&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1,140,226,235&lt;/td&gt;
&lt;td&gt;likes of posts or replies&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;As an initial step, we run the benchmark as fast as it will go. We use 32 threads on the driver side for 24 hardware threads.&lt;/p&gt;
&lt;p&gt;Below are the numerical quantities for a 400K operation run after 150K operations worth of warmup.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Duration:&lt;/strong&gt; 10:41.251&lt;br&gt;
&lt;strong&gt;Throughput:&lt;/strong&gt; 623.71 (op/s)&lt;/p&gt;
&lt;p&gt;The statistics that matter are detailed below, with operations ranked in order of descending client-side wait-time. All times are in milliseconds.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;% of total&lt;/th&gt;
&lt;th&gt;total_wait&lt;/th&gt;
&lt;th&gt;name&lt;/th&gt;
&lt;th&gt;count&lt;/th&gt;
&lt;th&gt;mean&lt;/th&gt;
&lt;th&gt;min&lt;/th&gt;
&lt;th&gt;max&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;20%&lt;/td&gt;
&lt;td&gt;4,231,130&lt;/td&gt;
&lt;td&gt;LdbcQuery5&lt;/td&gt;
&lt;td&gt;656&lt;/td&gt;
&lt;td&gt;6,449.89&lt;/td&gt;
&lt;td&gt;245&lt;/td&gt;
&lt;td&gt;10,311&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11%&lt;/td&gt;
&lt;td&gt;2,272,954&lt;/td&gt;
&lt;td&gt;LdbcQuery8&lt;/td&gt;
&lt;td&gt;18,354&lt;/td&gt;
&lt;td&gt;123.84&lt;/td&gt;
&lt;td&gt;14&lt;/td&gt;
&lt;td&gt;2,240&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10%&lt;/td&gt;
&lt;td&gt;2,200,718&lt;/td&gt;
&lt;td&gt;LdbcQuery3&lt;/td&gt;
&lt;td&gt;388&lt;/td&gt;
&lt;td&gt;5,671.95&lt;/td&gt;
&lt;td&gt;468&lt;/td&gt;
&lt;td&gt;17,368&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;7.3%&lt;/td&gt;
&lt;td&gt;1,561,382&lt;/td&gt;
&lt;td&gt;LdbcQuery14&lt;/td&gt;
&lt;td&gt;1,124&lt;/td&gt;
&lt;td&gt;1,389.13&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;5,724&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;6.7%&lt;/td&gt;
&lt;td&gt;1,441,575&lt;/td&gt;
&lt;td&gt;LdbcQuery12&lt;/td&gt;
&lt;td&gt;1,252&lt;/td&gt;
&lt;td&gt;1,151.42&lt;/td&gt;
&lt;td&gt;15&lt;/td&gt;
&lt;td&gt;3,273&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;6.5%&lt;/td&gt;
&lt;td&gt;1,396,932&lt;/td&gt;
&lt;td&gt;LdbcQuery10&lt;/td&gt;
&lt;td&gt;1,252&lt;/td&gt;
&lt;td&gt;1,115.76&lt;/td&gt;
&lt;td&gt;13&lt;/td&gt;
&lt;td&gt;4,743&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5%&lt;/td&gt;
&lt;td&gt;1,064,457&lt;/td&gt;
&lt;td&gt;LdbcShortQuery3PersonFriends&lt;/td&gt;
&lt;td&gt;46,285&lt;/td&gt;
&lt;td&gt;22.9979&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;2,287&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4.9%&lt;/td&gt;
&lt;td&gt;1,047,536&lt;/td&gt;
&lt;td&gt;LdbcShortQuery2PersonPosts&lt;/td&gt;
&lt;td&gt;46,285&lt;/td&gt;
&lt;td&gt;22.6323&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;2,156&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4.1%&lt;/td&gt;
&lt;td&gt;885,102&lt;/td&gt;
&lt;td&gt;LdbcQuery6&lt;/td&gt;
&lt;td&gt;1,721&lt;/td&gt;
&lt;td&gt;514.295&lt;/td&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;5,227&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3.3%&lt;/td&gt;
&lt;td&gt;707,901&lt;/td&gt;
&lt;td&gt;LdbcQuery1&lt;/td&gt;
&lt;td&gt;2,117&lt;/td&gt;
&lt;td&gt;334.389&lt;/td&gt;
&lt;td&gt;28&lt;/td&gt;
&lt;td&gt;3,467&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.4%&lt;/td&gt;
&lt;td&gt;521,738&lt;/td&gt;
&lt;td&gt;LdbcQuery4&lt;/td&gt;
&lt;td&gt;1,530&lt;/td&gt;
&lt;td&gt;341.005&lt;/td&gt;
&lt;td&gt;49&lt;/td&gt;
&lt;td&gt;2,774&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.1%&lt;/td&gt;
&lt;td&gt;440,197&lt;/td&gt;
&lt;td&gt;LdbcShortQuery4MessageContent&lt;/td&gt;
&lt;td&gt;46,302&lt;/td&gt;
&lt;td&gt;9.50708&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;2,015&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1.9%&lt;/td&gt;
&lt;td&gt;407,450&lt;/td&gt;
&lt;td&gt;LdbcUpdate5AddForumMembership&lt;/td&gt;
&lt;td&gt;14,338&lt;/td&gt;
&lt;td&gt;28.4175&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;2,008&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1.9%&lt;/td&gt;
&lt;td&gt;405,243&lt;/td&gt;
&lt;td&gt;LdbcShortQuery7MessageReplies&lt;/td&gt;
&lt;td&gt;46,302&lt;/td&gt;
&lt;td&gt;8.75217&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;2,112&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1.9%&lt;/td&gt;
&lt;td&gt;404,002&lt;/td&gt;
&lt;td&gt;LdbcShortQuery6MessageForum&lt;/td&gt;
&lt;td&gt;46,302&lt;/td&gt;
&lt;td&gt;8.72537&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;1,968&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1.8%&lt;/td&gt;
&lt;td&gt;387,044&lt;/td&gt;
&lt;td&gt;LdbcUpdate3AddCommentLike&lt;/td&gt;
&lt;td&gt;12,659&lt;/td&gt;
&lt;td&gt;30.5746&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;2,060&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1.7%&lt;/td&gt;
&lt;td&gt;361,290&lt;/td&gt;
&lt;td&gt;LdbcShortQuery1PersonProfile&lt;/td&gt;
&lt;td&gt;46,285&lt;/td&gt;
&lt;td&gt;7.80577&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;2,015&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1.6%&lt;/td&gt;
&lt;td&gt;334,409&lt;/td&gt;
&lt;td&gt;LdbcShortQuery5MessageCreator&lt;/td&gt;
&lt;td&gt;46,302&lt;/td&gt;
&lt;td&gt;7.22234&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;2,055&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1%&lt;/td&gt;
&lt;td&gt;220,740&lt;/td&gt;
&lt;td&gt;LdbcQuery2&lt;/td&gt;
&lt;td&gt;1,488&lt;/td&gt;
&lt;td&gt;148.347&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;2,504&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.96%&lt;/td&gt;
&lt;td&gt;205,910&lt;/td&gt;
&lt;td&gt;LdbcQuery7&lt;/td&gt;
&lt;td&gt;1,721&lt;/td&gt;
&lt;td&gt;119.646&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;2,295&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.93%&lt;/td&gt;
&lt;td&gt;198,971&lt;/td&gt;
&lt;td&gt;LdbcUpdate2AddPostLike&lt;/td&gt;
&lt;td&gt;5,974&lt;/td&gt;
&lt;td&gt;33.3062&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;1,987&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.88%&lt;/td&gt;
&lt;td&gt;189,871&lt;/td&gt;
&lt;td&gt;LdbcQuery11&lt;/td&gt;
&lt;td&gt;2,294&lt;/td&gt;
&lt;td&gt;82.7685&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;2,219&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.85%&lt;/td&gt;
&lt;td&gt;182,964&lt;/td&gt;
&lt;td&gt;LdbcQuery13&lt;/td&gt;
&lt;td&gt;2,898&lt;/td&gt;
&lt;td&gt;63.1346&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;2,201&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.74%&lt;/td&gt;
&lt;td&gt;158,188&lt;/td&gt;
&lt;td&gt;LdbcQuery9&lt;/td&gt;
&lt;td&gt;78&lt;/td&gt;
&lt;td&gt;2,028.05&lt;/td&gt;
&lt;td&gt;1,108&lt;/td&gt;
&lt;td&gt;4,183&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.67%&lt;/td&gt;
&lt;td&gt;143,457&lt;/td&gt;
&lt;td&gt;LdbcUpdate7AddComment&lt;/td&gt;
&lt;td&gt;3,986&lt;/td&gt;
&lt;td&gt;35.9902&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;1,912&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.26%&lt;/td&gt;
&lt;td&gt;54,947&lt;/td&gt;
&lt;td&gt;LdbcUpdate8AddFriendship&lt;/td&gt;
&lt;td&gt;571&lt;/td&gt;
&lt;td&gt;96.2294&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;988&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.2%&lt;/td&gt;
&lt;td&gt;43,451&lt;/td&gt;
&lt;td&gt;LdbcUpdate6AddPost&lt;/td&gt;
&lt;td&gt;1,386&lt;/td&gt;
&lt;td&gt;31.3499&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;2,060&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.01%&lt;/td&gt;
&lt;td&gt;1,848&lt;/td&gt;
&lt;td&gt;LdbcUpdate4AddForum&lt;/td&gt;
&lt;td&gt;103&lt;/td&gt;
&lt;td&gt;17.9417&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;65&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0.00%&lt;/td&gt;
&lt;td&gt;44&lt;/td&gt;
&lt;td&gt;LdbcUpdate1AddPerson&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;22&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;34&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;At this point we have in-depth knowledge of the choke points the benchmark stresses, and we can give a first assessment of whether the design meets its objectives for setting an agenda for the coming years of graph database development.&lt;/p&gt;
&lt;p&gt;The implementation is well optimized in general but still has maybe 30% room for improvement. We note that this is based on a compressed column store. One could think that alternative data representations, like in-memory graphs of structs and pointers between them, are better for the task. This is not necessarily so; at the least, a compressed column store is much more space efficient. Space efficiency is the root of cost efficiency, since as soon as the working set is not in memory, a random access workload is badly hit.&lt;/p&gt;
&lt;p&gt;The set of choke points (technical challenges) actually revealed by the benchmark is so far as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Cardinality estimation under heavy data skew —&lt;/em&gt; Many queries take a tag or a country as a parameter. The cardinalities associated with tags vary from 29M posts for the most common to 1 for the least common. Q6 has a common tag (in top few hundred) half the time and a random, most often very infrequent, one the rest of the time. A declarative implementation must recognize the cardinality implications from the literal and plan accordingly. An imperative one would have to count. Missing this makes Q6 take about 40% of the time instead of 4.1% when adapting.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Covering indices —&lt;/em&gt; Being able to make multi-column indices that duplicate some columns from the table often saves an entire table lookup. For example, an index onpost by author can also contain the post&amp;rsquo;s creation date.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Multi-hop graph traversal —&lt;/em&gt; Most queries access a two-hop environment starting at a person. Two queries look for shortest paths of unbounded length. For the two-hop case, it makes almost no difference whether this is done as a union or a special graph traversal operator. For shortest paths, this simply must be built into the engine; doing this client-side incurs prohibitive overheads. A bidirectional shortest path operation is a requirement for the benchmark.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Top &lt;em&gt;K&lt;/em&gt; —&lt;/em&gt; Most queries returning posts order results by descending date. Once there are at least &lt;em&gt;k&lt;/em&gt; results, anything older than the __k__th can be dropped, adding a dateselection as early as possible in the query. This interacts with vectored execution, so that starting with a short vector size more rapidly produces an initial top &lt;em&gt;k&lt;/em&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Late projection —&lt;/em&gt; Many queries access several columns and touch millions of rows but only return a few. The columns that are not used in sorting or selection can be retrieved only for the rows that are actually returned. This is especially useful with a column store, as this removes many large columns (e.g., text of a post) from the working set.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Materialization —&lt;/em&gt; Q14 accesses an expensive-to-compute edge weight, the number of post-reply pairs between two people. Keeping this precomputed drops Q14 from the top place. Other materialization would be possible, for example Q2 (top 20 posts by friends), but since Q2 is just 1% of the load, there is no need. One could of course argue that this should be 20x more frequent, in which case there could be a point to this.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Concurrency control —&lt;/em&gt; Read-write contention is rare, as updates are randomly spread over the database. However, some pages get read very frequently, e.g., some middle level index pages in the post table. Keeping a count of reading threads requires a mutex, and there is significant contention on this. Since the hot set can be one page, adding more mutexes does not always help. However, hash partitioning the index into many independent trees (as in the case of a cluster) helps for this. There is also contention on a mutex for assigning threads to client requests, as there are large numbers of short operations.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In subsequent posts, we will look at specific queries, what they in fact do, and what their theoretical performance limits would be. In this way we will have a precise understanding of which way SNB can steer the graph DB community.&lt;/p&gt;
&lt;h3 id=&#34;snb-interactive-series&#34;&gt;SNB Interactive Series&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://ldbcouncil.org/post/snb-interactive-part-1-what-is-snb-interactive-really-about&#34;&gt;SNB Interactive, Part 1: What is SNB Interactive Really About?&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://ldbcouncil.org/post/snb-interactive-part-2-modeling-choices&#34;&gt;SNB Interactive, Part 2: Modeling Choices&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://ldbcouncil.org/post/snb-interactive-part-3-choke-points-and-initial-run-on-virtuoso/&#34;&gt;SNB Interactive, Part 3: Choke Points and Initial Run on Virtuoso&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>SNB and Graphs Related Presentations at GRADES &#39;15</title>
      <link>https://ldbcouncil.org/post/snb-and-graphs-related-presentations-at-grades-15/</link>
      <pubDate>Fri, 29 May 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/snb-and-graphs-related-presentations-at-grades-15/</guid>
      <description>&lt;p&gt;Next 31st of  May the GRADES workshop will take place in Melbourne within the ACM/SIGMOD presentation. GRADES started as an initiative of the Linked Data Benchmark Council in the SIGMOD/PODS 2013 held in New York.&lt;/p&gt;
&lt;p&gt;Among the papers published in this edition we have &amp;ldquo;Graphalytics: A Big Data Benchmark for Graph-Processing Platforms&amp;rdquo;, which presents a new benchmark that uses the Social Network Benchmark data generator of LDBC (that can be found in &lt;a href=&#34;https://github.com/ldbc&#34;&gt;https://github.com/ldbc&lt;/a&gt;) as the base to execute the algorithms used for the benchmark, among which we have BFS, community detection and connected components. We also have &amp;ldquo;Microblogging Queries on Graph Databases: an Introspection&amp;rdquo; which benchmarks two of the most significant Graph Databases in the market, i.e. Neo4j and Sparksee using microblogging queries on top of twitter data. We can finally mention &amp;ldquo;Frappé: Querying the Linux Kernel Dependency Graph&amp;rdquo; which presents a framework for querying and visualising the dependencies of large C/C++ software systems.&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;http://event.cwi.nl/grades2015/program.shtml&#34;&gt;Check the complete agenda.&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Meet you in Melbourne!&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>SNB Interactive Part 2: Modeling Choices</title>
      <link>https://ldbcouncil.org/post/snb-interactive-part-2-modeling-choices/</link>
      <pubDate>Tue, 26 May 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/snb-interactive-part-2-modeling-choices/</guid>
      <description>&lt;p&gt;&lt;a href=&#34;https://ldbcouncil.org/benchmarks/snb&#34;&gt;​SNB Interactive&lt;/a&gt; is the wild frontier, with very few rules. This is necessary, among other reasons, because there is no standard property graph data model, and because the contestants support a broad mix of programming models, ranging from in-process APIs to declarative query.&lt;/p&gt;
&lt;p&gt;In the case of &lt;a href=&#34;http://dbpedia.org/resource/Virtuoso_Universal_Server&#34;&gt;Virtuoso&lt;/a&gt;, we have played with &lt;a href=&#34;http://dbpedia.org/resource/SQL&#34;&gt;SQL&lt;/a&gt; and &lt;a href=&#34;http://dbpedia.org/resource/SPARQL&#34;&gt;SPARQL&lt;/a&gt; implementations. For a fixed schema and well known workload, SQL will always win. The reason for this is that this allows to materialize multi-part indices and data orderings that make sense for the application. In other words, there is transparency into physical design. An RDF application may also have physical design by means ofstructure-aware storage but this is more complex and here we are just concerned with speed and having things work precisely as we intend.&lt;/p&gt;
&lt;h3 id=&#34;schema-design&#34;&gt;Schema Design&lt;/h3&gt;
&lt;p&gt;SNB has a regular schema described by a &lt;a href=&#34;https://en.wikipedia.org/wiki/Unified_Modeling_Language&#34;&gt;UML&lt;/a&gt; diagram. This has a number of relationships of which some have attributes. There are no heterogenous sets, e.g. no need for run-time typed attributes or graph edges with the same label but heterogeneous end points. Translation into SQL or RDF is straightforward. Edges with attributes, e.g. the knows relation between people would end up represented as a subject with the end points and the date since as properties. The relational implementation has a two-part primary key and the date since as a dependent column. A native property graph database would use an edge with an extra property for this, as such are typically supported.&lt;/p&gt;
&lt;p&gt;The only table-level choice has to do with whether &lt;code&gt;posts&lt;/code&gt; and &lt;code&gt;comments&lt;/code&gt; are kept in the same or different data structures. The Virtuoso schema has a single table for both, with nullable columns for the properties that occur only in one. This makes the queries more concise. There are cases where only non-reply posts of a given author are accessed. This is supported by having two author foreign key columns each with its own index. There is a single nullable foreign key from the reply to the post/comment being replied to.&lt;/p&gt;
&lt;p&gt;The workload has some frequent access paths that need to be supported by index. Some queries reward placing extra columns in indices. For example, a common pattern is accessing the most recent posts of an author or group of authors. There, having a composite key &lt;code&gt;of ps_creatorid&lt;/code&gt;, &lt;code&gt;ps_creationdate&lt;/code&gt;, &lt;code&gt;ps_postid&lt;/code&gt; pays off since the top-k on &lt;code&gt;creationdate&lt;/code&gt; can be pushed down into the index without needing a reference to the table.&lt;/p&gt;
&lt;p&gt;The implementation is free to choose data types for attributes, specifically datetimes. The Virtuoso implementation adopts the practice of the &lt;a href=&#34;http://dbpedia.org/resource/DEX_(Graph_database)&#34;&gt;Sparksee&lt;/a&gt; and &lt;a href=&#34;http://dbpedia.org/resource/Neo4j&#34;&gt;Neo4J&lt;/a&gt; implementations and represents this is a count of milliseconds since epoch. This is less confusing, faster to compare and more compact than a native datetime datatype that may or may not have timezones etc. Using a built-in datetime seems to be nearly always a bad idea. A dimension table or a number for a time dimension avoids the ambiguities of a calendar or at least makes these explicit.&lt;/p&gt;
&lt;p&gt;The benchmark allows procedurally maintaining materializations of intermediate results for use by queries as long as these are maintained transaction by transaction. For example, each person could have the 20 newest posts by immediate contacts precomputed. This would reduce Q2 &amp;ldquo;top of the wall&amp;rdquo; to a single lookup. This dows not however appear to be worthwhile. The Virtuoso implementation does do one such materialization for Q14: A connection weight is calculated for every pair of persons that know each other. This is related to the count of replies by one or the other to content generated by the other. If there does not exist a single reply in either direction, the weight is taken to be 0. This weight is precomputed after bulk load and subsequently maintained each time a reply is added. The table for this is the only row-wise structure in the schema and represents a half matrix of connected people, i.e. &lt;code&gt;person1&lt;/code&gt;, &lt;code&gt;person2&lt;/code&gt; -&amp;gt; &lt;code&gt;weight&lt;/code&gt;. &lt;code&gt;Person1&lt;/code&gt; is by convention the one with the smaller &lt;code&gt;p_personid&lt;/code&gt;. Note that comparing id&amp;rsquo;s in this way is useful but not normally supported by RDF systems. RDF would end up comparing strings of URI&amp;rsquo;s with disastrous performance implications unless an implementation specific trick were used.&lt;/p&gt;
&lt;p&gt;In the next installment we will analyze an actual run.&lt;/p&gt;
&lt;h3 id=&#34;snb-interactive-series&#34;&gt;SNB Interactive Series&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://ldbcouncil.org/post/snb-interactive-part-1-what-is-snb-interactive-really-about&#34;&gt;SNB Interactive, Part 1: What is SNB Interactive Really About?&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://ldbcouncil.org/post/snb-interactive-part-2-modeling-choices&#34;&gt;SNB Interactive, Part 2: Modeling Choices&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://ldbcouncil.org/post/snb-interactive-part-3-choke-points-and-initial-run-on-virtuoso/&#34;&gt;SNB Interactive, Part 3: Choke Points and Initial Run on Virtuoso&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>LDBC Participates in the 36th Edition of the ACM SIGMOD/PODS Conference</title>
      <link>https://ldbcouncil.org/post/ldbc-participates-in-the-36th-edition-of-the-acm-sigmod-pods-conference/</link>
      <pubDate>Mon, 25 May 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/ldbc-participates-in-the-36th-edition-of-the-acm-sigmod-pods-conference/</guid>
      <description>&lt;p&gt;LDBC is presenting two papers at the next edition of the ACM SIGMOD/PODS conference held in Melbourne from May 31st to June 4th, 2015. The annual ACM SIGMOD/PODS conference is a leading international forum for database researchers, practitioners, developers, and users to explore cutting-edge ideas and results, and to exchange techniques, tools and experiences.&lt;/p&gt;
&lt;p&gt;On the industry track, LDBC will be presenting the &lt;em&gt;Social Network Benchmark Interactive Workload&lt;/em&gt; by Orri Erling (OpenLink Software), Alex Averbuch (Neo Technology), Josep Larriba-Pey (Sparsity Technologies), Hassan Chafi (Oracle Labs), Andrey Gubichev (TU Munich), Arnau Prat (Universitat Politècnica de Catalunya), Minh-Duc Pham (VU University Amsterdam) and Peter Boncz (CWI).&lt;/p&gt;
&lt;p&gt;You can read more about the &lt;a href=&#34;https://ldbcouncil.org/benchmarks/snb&#34;&gt;Social Network Benchmark here&lt;/a&gt; and collaborate if you&amp;rsquo;re interested!&lt;/p&gt;
&lt;p&gt;The other presentation will be at the GRADES workshop within the SIGMOD program regarding &lt;em&gt;Graphalytics: A Big Data Benchmark for Graph-Processing platforms&lt;/em&gt; by Mihai Capotă, Tim Hegeman, Alexandru Iosup (Delft University of Technology), Arnau Prat (Universitat Politècnica de Catalunya), Orri Erling (OpenLink Sotware) and Peter Boncz (CWI). We will provide more information about GRADES and this specific presentation in a following post as GRADES is part of the events organized by LDBC.&lt;/p&gt;
&lt;p&gt;Don&amp;rsquo;t forget to check our presentations if you&amp;rsquo;re attending the SIGMOD!&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>SNB Interactive Part 1: What Is SNB Interactive Really About?</title>
      <link>https://ldbcouncil.org/post/snb-interactive-part-1-what-is-snb-interactive-really-about/</link>
      <pubDate>Thu, 14 May 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/snb-interactive-part-1-what-is-snb-interactive-really-about/</guid>
      <description>&lt;p&gt;This post is the first in a series of blogs analyzing the LDBC Social Network Benchmark Interactive workload. This is written from the dual perspective of participating in the benchmark design and of building the OpenLink Virtuoso implementation of same.&lt;/p&gt;
&lt;p&gt;With two implementations of SNB interactive at four different scales, we can take a first look at what the benchmark is really about. The hallmark of a benchmark implementation is that its performance characteristics are understood and even if these do not represent the maximum of the attainable, there are no glaring mistakes and the implementation represents a reasonable best effort by those who ought to know, namely the system vendors.&lt;/p&gt;
&lt;p&gt;The essence of a benchmark is a set of trick questions or choke points, as LDBC calls them. A number of these were planned from the start. It is then the role of experience to tell whether addressing these is really the key to winning the race. Unforeseen ones will also surface.&lt;/p&gt;
&lt;p&gt;So far, we see that SNB confronts the implementor with choices in the following areas:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Data model: Relational, RF, property graph?&lt;/li&gt;
&lt;li&gt;Physical model, e.g. row-wise vs. column wise storage&lt;/li&gt;
&lt;li&gt;Materialized data ordering: Sorted projections, composite keys, replicating columns in auxxiliary data structures&lt;/li&gt;
&lt;li&gt;Maintaining precomputed, materialized intermediate results, e.g. use of materialized views, triggers&lt;/li&gt;
&lt;li&gt;Query optimization: join order/type, interesting physical data orderings, late projection, top k, etc.&lt;/li&gt;
&lt;li&gt;Parameters vs. literals: Sometimes different parameter values result in different optimal query plans&lt;/li&gt;
&lt;li&gt;Predictable, uniform latency: The measurement rules stipulate the SUT must not fall behind the simulated workload&lt;/li&gt;
&lt;li&gt;Durability - how to make data durable while maintaining steady throughput? Logging vs. checkpointing.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In the process of making a benchmark implementation, one naturally encounters questions about the validity, reasonability and rationale of the benchmark definition itself. Additionally, even though the benchmark might not directly measure certain aspects of a system, making an implementation will take a system past its usual envelope and highlight some operational aspects.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Data generation - Generating a mid-size dataset takes time, e.g. 8 hours for 300G. In a cloud situation, keeping the dataset in S3 or similar is necessary, re-generating every time is not an option.&lt;/li&gt;
&lt;li&gt;Query mix - Are the relative frequencies of the operations reasonable? What bias does this introduce?&lt;/li&gt;
&lt;li&gt;Uniformity of parameters: Due to non-uniform data distributions in the dataset, there is easily a 100x difference between a &amp;lsquo;fast&amp;rsquo; and &amp;lsquo;slow&amp;rsquo; case of a single query template. How long does one need to run to balance these fluctuations?&lt;/li&gt;
&lt;li&gt;Working set: Experience shows that there is a large difference between almost warm and steady state of working set. This can be a factor of 1.5 in throughput.&lt;/li&gt;
&lt;li&gt;Are the latency constraints reasonable? In the present case, a qualifying run must have under 5% of all query executions starting over 1 second late. Each execution is scheduled beforehand and done at the intended time.  If the SUT does not keep up, it will have all available threads busy and must finish some work before accepting new work, so some queries will start late. Is this a good criterion for measuring consistency of response time? There are some obvious possibilities of abuse.&lt;/li&gt;
&lt;li&gt;Is the benchmark easy to implement/run? Perfection is open-ended and optimization possibilities infinite, albeit with diminishing returns. Still, getting startyed should not be too hard. Since systems will be highly diverse, testing that these in fact do the same thing is important. The SNB validation suite is good for this and given publicly available reference implementations, the effort of getting started is not unreasonable.&lt;/li&gt;
&lt;li&gt;Since a Qualifying run must meet latency constraints while going as fast as possible, setting the performance target involves trial and error. Does the tooling make this easy?&lt;/li&gt;
&lt;li&gt;Is the durability rule reasonable? Right now, one is not required to do checkpoints but must report the time to roll forward from the last checkpoint or initial state. Incenting vendors to build faster recovery is certainly good, but we are not through with all the implications. What about redundant clusters?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The following posts will look at the above in light of actual experience.&lt;/p&gt;
&lt;h3 id=&#34;snb-interactive-series&#34;&gt;SNB Interactive Series&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://ldbcouncil.org/post/snb-interactive-part-1-what-is-snb-interactive-really-about&#34;&gt;SNB Interactive, Part 1: What is SNB Interactive Really About?&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://ldbcouncil.org/post/snb-interactive-part-2-modeling-choices&#34;&gt;SNB Interactive, Part 2: Modeling Choices&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://ldbcouncil.org/post/snb-interactive-part-3-choke-points-and-initial-run-on-virtuoso/&#34;&gt;SNB Interactive, Part 3: Choke Points and Initial Run on Virtuoso&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>Why Do We Need an LDBC SNB-Specific Workload Driver?</title>
      <link>https://ldbcouncil.org/post/why-do-we-need-an-ldbc-snb-specific-workload-driver/</link>
      <pubDate>Tue, 21 Apr 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/why-do-we-need-an-ldbc-snb-specific-workload-driver/</guid>
      <description>&lt;p&gt;In a previous &lt;a href=&#34;https://ldbcouncil.org/tags/driver&#34;&gt;3-part blog series&lt;/a&gt; we touched upon the difficulties of executing the LDBC SNB Interactive (SNB) workload, while achieving good performance and scalability. What we didn&amp;rsquo;t discuss is why these difficulties were unique to SNB, and what aspects of the way we perform workload execution are scientific contributions - novel solutions to previously unsolved problems. This post will highlight the differences between SNB and more traditional database benchmark workloads. Additionally, it will motivate why we chose to develop a new workload driver as part of this work, rather than using existing tooling that was developed in other database benchmarking efforts. To briefly recap, the task of the driver is to run a transactional database benchmark against large synthetic graph datasets - &amp;ldquo;graph&amp;rdquo; is the word that best captures the novelty and difficulty of this work.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Workload Execution - Traditional vs Graph&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Transactional graph workloads differ from traditional relational workloads in several fundamental ways, one of them being the complex dependencies that exist between queries of a graph workload.&lt;/p&gt;
&lt;p&gt;To understand what is meant by &amp;ldquo;traditional relational workloads&amp;rdquo;, take the classical TPC-C benchmark as an example. In TPC-C Remote Terminal Emulators (emulators) are used to issue update transactions in parallel, where the transactions issued by these emulators do not depend on one another. Note, &amp;ldquo;dependency&amp;rdquo; is used here in the context of scheduling, i.e., one query is dependent on another if it can not start until the other completes. For example, a New-Order transaction does not depend on other orders from this or other users. Naturally, the results of Stock-Level transactions depend on the items that were previously sold, but in TPC-C it is not an emulator&amp;rsquo;s responsibility to enforce any such ordering. The scheduling strategy employed by TPC-C is tailored to the scenario where transactional updates do not depend on one another. In reality, one would expect to also have scheduling dependencies between transactions, e.g., checking the status of the order should only be done after the order is registered in the system.  TPC-C, however, does not do this and instead only asks for the status of the last order &lt;em&gt;for a given user&lt;/em&gt;. Furthermore, adding such dependencies to TPC-C would make scheduling only slightly more elaborate. Indeed, the Load Tester (LT) would need to make sure a New-Order transaction always precedes the read requests that check its status, but because users (and their orders) are partitioned across LTs, and orders belong to a particular user, this scheduling does not require inter-LT communication.&lt;/p&gt;
&lt;p&gt;A significantly more difficult scheduling problem arises when we consider the SNB benchmark that models a real-world social network. Its domain includes users that form a social friendship graph and which leave posts/comments/likes on each others walls (forums). The update transactions are generated (exported as a log) by the data generator, with assigned timestamps, e.g. user 123 added post 456 to forum 789 at time T. Suppose we partition this workload by user, such that each driver gets all the updates (friendship requests, posts, comments and likes on other user&amp;rsquo;s posts etc) initiated by a given user. Now, if the benchmark is to resemble a real-world social network, the update operations represent a highly connected (and dependent) network: a user should not create comments before she joins the network, a friendship request can not be sent to a non-existent user, a comment can only be added to a post that already exists, etc. Given a user partitioning scheme, most such dependencies would cross the boundaries between driver threads/processes, because the correct execution of update operations requires that the social network is in a particular state, and that state depends on the progress of other threads/processes.&lt;/p&gt;
&lt;p&gt;Such scheduling dependencies in the SNB workload essentially replicate the underlying graph-like shape of its dataset. That is, every time a user comments on a friend&amp;rsquo;s wall, for example, there is a dependency between two operations that is captured by an edge of the social graph. &lt;em&gt;Partitioning the workload among the LTs therefore becomes equivalent to graph partitioning, a known hard problem.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Because it&amp;rsquo;s a graph&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;In short, unlike previous database benchmarking efforts, the SNB workload has necessitated a redefining of the state-of-the-art in workload execution. It is no longer sufficient to rely solely on workload partitioning to safely capture inter-query dependencies in complex database benchmark workloads. The graph-centric nature of SNB introduces new challenges, and novel mechanisms had to be developed to overcome these challenges. To the best of our knowledge, the LDBC SNB Interactive benchmark is the first benchmark that requires a non-trivial partitioning of the workload, among the benchmark drivers. In the context of workload execution, our contribution is therefore the principled design of a driver that executes dependent update operations in a performant and scalable way, across parallel/distributed LTs, while providing repeatable, vendor-independent execution of the benchmark.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Event Driven Post Generation in Datagen</title>
      <link>https://ldbcouncil.org/post/event-driven-post-generation-in-datagen/</link>
      <pubDate>Fri, 10 Apr 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/event-driven-post-generation-in-datagen/</guid>
      <description>&lt;p&gt;As discussed in previous posts, one of the features that makes Datagen more realistic is the fact that the activity volume of the simulated Persons is not uniform, but forms spikes. In this blog entry I want to explain more in depth how this is actually implemented inside of the generator.&lt;/p&gt;
&lt;p&gt;First of all, I start with a few basics of how Datagen works internally. In Datagen, once the person graph has been created (persons and their relationships), the activity generation starts. Persons are divided into blocks of 10k, in the same way they are during friendship edges generation process. Then, for each person of the block, three types of forums are created:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;The wall of the person&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The albums of the person&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The groups where the person is a moderator&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We will put our attention to group generation, but the same concepts apply to the other types of forums. Once a group is created, the members of the group are selected. These are selected from either the friends of the moderator, or random persons within the same block.&lt;/p&gt;
&lt;p&gt;After assigning the members to the group, the post generation starts. We have two types of post generators, the uniform post generator and the event based post generator. Each post generator is responsible of, given a forum, generate a set of posts for the forum, whose authors are taken from the set of members of the forum. The uniform post generator distributes the dates of the generated posts uniformly in the time line (from the date of the membership until the end of the simulation time). On the other hand, the event based post generator assigns dates to posts, based on what we call “flashmob events”.&lt;/p&gt;
&lt;p&gt;Flashmob events are generated at the beginning of the execution. Their number is predefined by a configuration parameter which is set to 30 events per month of simulation, and the time of the event is distributed uniformly along all the time line. Also, each event has a volume level assigned (between 1 and 20) following a power law distribution, which determines how relevant or important the event is, and a tag representing the concept or topic of the event. Two different events can have the same tag. For example, one of the flashmob events created for SF1 is one related to &amp;ldquo;Enrique Iglesias&amp;rdquo; tag, whose level is 11 and occurs on 29th of May of 2012 at 09:33:47.&lt;/p&gt;
&lt;p&gt;Once the event based post generation starts for a given group, a subset of the generated flashmob events is extracted. These events must be correlated with the tag/topic of the group, and the set of selected events is restricted by the creation date of the group (in a group one cannot talk about an event previous to the creation of the group). Given this subset of events and their volume level, a cumulative probability distribution (using the events sorted by event date and their level) is computed, which is later used to determine to which event a given post is associated. Therefore, those events with a larger lavel will have a larger probability to receive posts, making their volume larger. Then, post generation starts, which can be summarized as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Determine the number of posts to generate&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Select a random member of the group that will generate the post&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Determine the event the post will be related to given the aforementioned cumulative distribution&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Assign the date of the post based on the event date&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In order to assign the date to the post, based on the date of the event the post is assigned to, we follow the following probability density, which has been extracted from &lt;a href=&#34;#references&#34;&gt;[1]&lt;/a&gt;. The shape of the probability density consists of a combination of an exponential function in the 8 hour interval around the peak, while the volume outside this interval follows a logarithmic function. The following figure shows the actual shape of the volume, centered at the date of the event.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;index.png&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;Following the example of &amp;ldquo;Enrique Iglesias&amp;rdquo;, the following figure shows the activity volume of posts around the event as generated by Datagen.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;index2.png&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;In this blog entry we have seen how datagen creates event driven user activity. This allows us to reproduce the heterogenous post creation density found in a real social network, where post creation is driven by real world events.&lt;/p&gt;
&lt;h4 id=&#34;references&#34;&gt;References&lt;/h4&gt;
&lt;p&gt;[1] Jure Leskovec, Lars Backstrom, Jon M. Kleinberg: Meme-tracking and the dynamics of the news cycle. KDD 2009: 497-506&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Sixth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/sixth-tuc-meeting/</link>
      <pubDate>Thu, 19 Mar 2015 13:53:33 -0400</pubDate>
      
      <guid>https://ldbcouncil.org/event/sixth-tuc-meeting/</guid>
      <description>&lt;p&gt;The LDBC consortium are pleased to announce its Sixth Technical User Community (TUC) meeting.&lt;/p&gt;
&lt;p&gt;This will be a two-day event at Universitat Politècnica de Catalunya, Barcelona on &lt;strong&gt;Thursday and Friday March 19/20, 2015.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The LDBC FP7 EC funded project is reaching its finalisation, and this will be the last event sponsored directly by the project. However, tasks within LDBC will continue based on the LDBC independent organisation. The event will basically set the following aspects:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Two day event with one day devoted to User&amp;rsquo;s experiences and one day devoted to benchmarking experiences.&lt;/li&gt;
&lt;li&gt;Presentation of the first benchmarking results for the different benchmarks.&lt;/li&gt;
&lt;li&gt;Interaction with the new LDBC Board of Directors and the whole new LDBC organisation officials.&lt;/li&gt;
&lt;li&gt;Pre-event with the 3rd Graph-TA workshop organised on March 18th at the same premises, with a lot of interaction and interesting research presentations.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We welcome all users of RDF and Graph technologies to attend. If you are interested, please, contact &lt;a href=&#34;mailto:damaris@ac.upc.edu&#34;&gt;damaris@ac.upc.edu&lt;/a&gt;.&lt;/p&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;Thursday 19th March&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;11:00 - 11:30 Registration, coffee break and welcome (Josep Larriba Pey)&lt;/p&gt;
&lt;p&gt;11:30 - 12:00 LDBC introduction and status update (Peter Boncz) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixth-tuc-meeting/attachments/6881717/6981131.pdf&#34;&gt;slides&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;12:00 - 13:30 Technology and benchmarking (chair: Peter Boncz)&lt;/p&gt;
&lt;p&gt;12:00 Venelin Kotsev (Ontotext). Semantic Publishing Benchmark v2.0. – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixth-tuc-meeting/attachments/6881717/6981137.pdf&#34;&gt;slides&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;12:30 Nina Saveta (FORTH). SPIMBENCH: A Scalable, Schema-Aware, Instance Matching Benchmark for the Semantic Publishing Domain&lt;/p&gt;
&lt;p&gt;12:50 Tomer Sagi (HP). Titan DB on LDBC SNB Interactive&lt;/p&gt;
&lt;p&gt;13:10 Claudio Martella (VUA): Giraph and Lighthouse&lt;/p&gt;
&lt;p&gt;13:30 - 14:30 Lunch break&lt;/p&gt;
&lt;p&gt;14:30 - 16:00 Applications and use of Graph Technologies (chair: Hassan Chafi)&lt;/p&gt;
&lt;p&gt;14:30 Jerven Bolleman (Swiss Institute of Bioinformatics): 20 billion triples in production &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixth-tuc-meeting/attachments/6881717/6981132.pdf&#34;&gt;slides&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;14:50 Mark Wilkinson (Universidad Politécnica de Madrid): Design principles for Linked-Data-native Semantic Web Services &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixth-tuc-meeting/attachments/6881717/6981133.pdf&#34;&gt;slides&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;15:10 Peter Haase (Metaphacts, Systap LLC): Querying the Wikidata Knowledge Graph &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixth-tuc-meeting/attachments/6881717/6981139.pdf&#34;&gt;slides&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;15:30 Esteban Sota (GNOSS): Human Interaction with Faceted Searching Systems for big or complex graphs&lt;/p&gt;
&lt;p&gt;18:30 - 20:00 Cultural visit Barcelona city center. Meet at Plaça Catalunya.&lt;/p&gt;
&lt;p&gt;20:00 Social dinner at &lt;a href=&#34;http://www.bastaix.com&#34;&gt;Bastaix Restaurant&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Friday 20th March&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;9:30 - 11:00 Technology and Benchmarking (chair: Josep L. Larriba-Pey)&lt;/p&gt;
&lt;p&gt;9:30 Yinglong Xia (IBM): Towards Temporal Graph Management and Analytics&lt;/p&gt;
&lt;p&gt;9:50 Alexandru Iosup (TU Delft). Graphalytics: A big data benchmark for graph-processing platforms&lt;/p&gt;
&lt;p&gt;10:10 John Snelson (MarkLogic): Introduction to MarkLogic&lt;/p&gt;
&lt;p&gt;10:30 Arnau Prat (UPC-Sparsity Technologies) and Alex Averbuch (Neo): Social Network Benchmark, Interactive Workload&lt;/p&gt;
&lt;p&gt;10:50 Moritz Kaufmann. &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixth-tuc-meeting/attachments/moritz-kaufmann-ldbc-snb-benchmark-auditing-6th-ldbc-tuc.pdf&#34;&gt;The auditing experience&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;11:15 - 11:45 Coffee break&lt;/p&gt;
&lt;p&gt;11:45 - 12:45 Applications and use of Graph Technologies (chair: Atanas Kiryakov)&lt;/p&gt;
&lt;p&gt;11:45 Boris Motik (Oxford University): Parallel and Incremental Materialisation of RDF/Datalog in RDFox&lt;/p&gt;
&lt;p&gt;12:05 Andreas Both (Unister): E-Commerce and Graph-driven Applications: Experiences and Optimizations while moving to Linked Data&lt;/p&gt;
&lt;p&gt;12:25 Smrati Gupta (CA Technologies). Modaclouds Decision Support System in multicloud environments&lt;/p&gt;
&lt;p&gt;12:45 Peter Boncz. Conclusions for the LDBC project and future perspectives. &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixth-tuc-meeting/attachments/6881717/6981138.pdf&#34;&gt;slides&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;13:30 - 14:30 Lunch break&lt;/p&gt;
&lt;p&gt;15:00  LDBC Board of Directors&lt;/p&gt;
&lt;h3 id=&#34;logistics&#34;&gt;Logistics&lt;/h3&gt;
&lt;h6 id=&#34;date&#34;&gt;&lt;strong&gt;Date&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;19th and 20th March 2015&lt;/p&gt;
&lt;h6 id=&#34;venue&#34;&gt;&lt;strong&gt;Venue&lt;/strong&gt;&lt;/h6&gt;
&lt;p&gt;The TUC meeting will be held at &amp;ldquo;Aula Master&amp;rdquo; at A3 building located inside the &amp;ldquo;Campus Nord UPC&amp;rdquo; in Barcelona. The address is:&lt;/p&gt;
&lt;p&gt;Aula Master&lt;br&gt;
Edifici A3, Campus Nord UPC&lt;br&gt;
C. Jordi Girona, 1-3&lt;br&gt;
08034 Barcelona, Spain&lt;/p&gt;
&lt;h5 id=&#34;maps-and-situation&#34;&gt;&lt;strong&gt;Maps and situation&lt;/strong&gt;&lt;/h5&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixth-tuc-meeting/attachments/1671180/1933315.jpg&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;h5 id=&#34;finding-upc&#34;&gt;&lt;strong&gt;Finding UPC&lt;/strong&gt;&lt;/h5&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixth-tuc-meeting/attachments/1671180/1933318.jpg&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;h5 id=&#34;finding-the-meeting-room&#34;&gt;&lt;strong&gt;Finding the meeting room&lt;/strong&gt;&lt;/h5&gt;
&lt;h5 id=&#34;getting-there&#34;&gt;Getting there&lt;/h5&gt;
&lt;p&gt;&lt;strong&gt;Flying:&lt;/strong&gt; Barcelona airport is situated 12 km from the city. There are several ways of getting from the airport to the centre of Barcelona, the cheapest of which is to take the train located outside just a few minutes walking distance past the parking lots at terminal 2 (there is a free bus between terminal 1 and terminal 2, see this &lt;a href=&#34;http://goo.gl/maps/iJqlj&#34;&gt;map of the airport&lt;/a&gt;). It is possible to buy 10 packs of train tickets which makes it cheaper. Taking the bus to the centre of town is more convenient as they leave directly from terminal 1 and 2, however it is more expensive than the train.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Rail:&lt;/strong&gt; The Renfe commuter train leaves the airport every 30 minutes from 6.13 a.m. to 11.40 p.m. Tickets cost around 3€ and the journey to&lt;br&gt;
the centre of Barcelona (Sants or Plaça Catalunya stations) takes 20 minutes.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Bus:&lt;/strong&gt; The Aerobus leaves the airport every 12 minutes, from 6.00 a.m. to 24.00, Monday to Friday, and from 6.30 a.m. to 24.00 on Saturdays, Sundays and public holidays. Tickets cost 6€ and the journey ends in Plaça Catalunya in the centre of Barcelona.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Taxi:&lt;/strong&gt; From the airport, you can take one of Barcelona&amp;rsquo;s typical black and yellow taxis. Taxis may not take more than four passengers. Unoccupied taxis display a green light and have a clearly visible sign showing LIBRE or LLIURE. The trip to Sants train station costs approximately €20 and trips to other destinations in the city cost approximately €25-30.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Train and bus:&lt;/strong&gt; Barcelona has two international train stations: Sants and França. Bus companies have different points of arrival in different parts of the city. You can find detailed information in the following link: &lt;a href=&#34;http://www.barcelona-airport.com/eng/transport_eng.htm&#34;&gt;http://www.barcelona-airport.com/eng/transport_eng.htm&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/sixth-tuc-meeting/attachments/1671180/1933316.jpg&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;h5 id=&#34;the-locations-of-the-airport-and-the-city-centre&#34;&gt;&lt;strong&gt;The locations of the airport and the city centre&lt;/strong&gt;&lt;/h5&gt;
</description>
    </item>
    
    <item>
      <title>The LDBC Datagen Community Structure</title>
      <link>https://ldbcouncil.org/post/the-ldbc-datagen-community-structure/</link>
      <pubDate>Sun, 15 Mar 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/the-ldbc-datagen-community-structure/</guid>
      <description>&lt;p&gt;This blog entry is about one of the features of DATAGEN that makes it different from other synthetic graph generators that can be found in the literature: the community structure of the graph.&lt;/p&gt;
&lt;p&gt;When generating synthetic graphs, one must not only pay attention to quantitative measures such as the number of nodes and edges, but also to other more qualitative characteristics such as the degree distribution, clustering coefficient. Real graphs, and specially social networks, have typically highly skewed degree distributions with a long tail, a moderatelly large clustering coefficient and an appreciable community structure.&lt;/p&gt;
&lt;p&gt;The first two characteristics are deliberately modeled in DATAGEN. DATAGEN generates persons with a degree distribution that matches that observed in Facebook, and thanks to the attribute correlated edge generation process, we obtain graphs with a moderately large clustering coefficient. But what about the community structure of graphs generated with DATAGEN? The answer can be found in the paper titled “How community-like is the structure of synthetically generated graphs”, which was published in GRADES 2014 &lt;a href=&#34;#references&#34;&gt;[1]&lt;/a&gt;. Here we summarize the paper and its contributions and findings.&lt;/p&gt;
&lt;p&gt;Existing synthetic graph generators such as Rmat &lt;a href=&#34;#references&#34;&gt;[1]&lt;/a&gt; and Mag &lt;a href=&#34;#references&#34;&gt;[2]&lt;/a&gt;, are graphs generators designed to produce graphs with long tailed distributions and large clustering coefficient, but completely ignore the fact that real graphs are structured into communities. For this reason, Lancichinetti et al. proposed LFR &lt;a href=&#34;#references&#34;&gt;[3]&lt;/a&gt;, a graph generator that did not only produced graphs with realistic high level characteristics, but enforced an appreciable community structure. This generator, has become the de facto standard for benchmarking community detection algorithms, as it does not only outputs a graph but also the communities present in that graph, hence it can be used to test the quality of a community detection algorithm.&lt;/p&gt;
&lt;p&gt;However, no one studied if the community structure produced by LFR, was in fact realistic compared to real graphs. Even though the community structure in LFR exhibit interesting properties, such as the expected larger internal density than external, or a longtailed distribution of community sizes, they lack the noise and inhomogeneities present in a real graph. And more importantly, how does the community structure of DATAGEN compares to that exhibited in LFR and reap graphs? Is it more or less realistic? The authors of &lt;a href=&#34;#references&#34;&gt;[1]&lt;/a&gt; set up an experiment where they analized the characteristics of the communities output by  LFR, and the groups (groups of people interested in a given topic) output by DATAGEN, and compared them to a set of real graphs with metadata. These real graphs, which can be downloaded from the Snap project website, are graphs that have recently become very popular in the field of community detection, as they contain ground truth communities extracted from their metadata. The ground truth graphs used in this experiment are shown in the following table. For more details about how this ground truth is generated, please refer to &lt;a href=&#34;#references&#34;&gt;[4]&lt;/a&gt;.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;&lt;em&gt;Nodes&lt;/em&gt;&lt;/th&gt;
&lt;th&gt;&lt;em&gt;Edges&lt;/em&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;em&gt;Amazon&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;334863&lt;/td&gt;
&lt;td&gt;925872&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;em&gt;Dblp&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;317080&lt;/td&gt;
&lt;td&gt;1049866&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;em&gt;Youtube&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;1134890&lt;/td&gt;
&lt;td&gt;2987624&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;em&gt;Livejournal&lt;/em&gt;&lt;/td&gt;
&lt;td&gt;3997962&lt;/td&gt;
&lt;td&gt;34681189&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The authors of &lt;a href=&#34;#references&#34;&gt;[1]&lt;/a&gt; selected  a set of statistical indicators to&lt;br&gt;
characterize the communities:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The clustering coefficient&lt;/li&gt;
&lt;li&gt;The triangle participation ration (TPR), which is the ratio of nodes that close at least one triangle in the community.&lt;/li&gt;
&lt;li&gt;The bridge ratio, which is the ratio of edges whose removal disconnects the community.&lt;/li&gt;
&lt;li&gt;The diameter&lt;/li&gt;
&lt;li&gt;The conductance&lt;/li&gt;
&lt;li&gt;The size&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The authors start by analyzing each community of the  ground truth graphs using the above statistical indicators and ploting the distributions of each of them. The following are the plots of the Livejournal graph. We summarize the findings of the authors regarding real graphs: + Several indicators (Clustering Coefficient, TPR and Bridge ratio) exihibit a multimodal distribution, with two peaks aht their extremes.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Many of the communities (44%) have a small clustering coefficient between 0 and 0.01. Out of them, 56% have just three vertices. On the other hand, 11% of the communities have a clustering coefficient between 0.99 and 1.0. In between, communities exhibit different values of clustering coefficients. This trend is also observed for TPR and Bridgeratio. This suggests that communities cannot be modeled using a single model. * 84% of the communities have a diameter smaller than five, suggesting that ground truth communities are small and compact * Ground truth communities are not very isolated, they have a lot of connections pointing outside of the community.&lt;/li&gt;
&lt;li&gt;Most of the communities are small (10 or less nodes).&lt;/li&gt;
&lt;li&gt;In general, ground truth communities are, small with a low diameter, not isolated and with different ranges of internal connectivity.&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index2.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Clustering Coefficient&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;TPR&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index3.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index4.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Bridge Ratio&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Diameter&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index5.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index6.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Conductance&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Size&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The authors performed the same experiment but for DATAGEN and LFR graphs. They generated a graph of 150k nodes, using their default parameters. In the case of LFR, they tested five different values of the mixing factor, which specifies the ratio of edges of the community pointing outside of the community, They ranged this value from 0 to 0.5. The following are the distributions for DATAGEN.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index8.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index9.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Clustering Coefficient&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;TPR&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index10.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index11.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Bridge Ratio&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;TPRDiameter&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index11.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index12.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Conductance&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Size&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The main conclusions that can be extracted from DATAGEN can be summarized asfollows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;DATAGEN is able to reproduce the multimodal distribution observed for clustering coefficient, TPR and bridge ratio.&lt;/li&gt;
&lt;li&gt;The central part of the clustering coefficient is biased towards the left, in a similar way as observed for the youtube and livejournal graphs.&lt;/li&gt;
&lt;li&gt;Communities of DATAGEN graphs are not, as in real graphs, isolated, but in this case their level of isolation if significantly larger.&lt;/li&gt;
&lt;li&gt;The diameter is small like in the real graphs.&lt;/li&gt;
&lt;li&gt;It is significant that communities in DATAGEN graphs are closer to those observed in Youtube and Livejournal, as these are social networks like the graphs produced by DATAGEN. We see that DATAGEN is able to reproduce many of their characteristics.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Finally, the authors repeat the same experiment for LFR graphs. The following are the plots for the LFR graph with mixing ratio 0.3. From them, the authors extract the following conclusions:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;LFR graphs donot show the multimodal distribution observed in real graphs&lt;/li&gt;
&lt;li&gt;Only the diameter shows a similar shape as in the ground truth.&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index13.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index14.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Clustering Coefficient&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;TPR&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index15.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index16.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Bridge Ratio&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;TPRDiameter&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index17.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index18.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Conductance&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Size&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;To better quanify how similar are the distribuions between the different graphs,  the authors also show the correlograms for each of the statisticsl indicators. These correlograms, contain the Spearman&amp;rsquo;s correlation coefficient between each pair of graphs for a given statistical indicator. The more blue the color, the better the correlation is. We see that DATAGEN distributions correlate very well with those observed in real graphs, specially as we commented above, with Youtube and Livejournal. On the other hand, LFR only succeds significantly in the case of the Diameter.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index19.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index20.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Clustering Coefficient&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;TPR&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index21.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index22.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Bridge Ratio&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;TPRDiameter&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index23.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img src=&#34;index24.png&#34; alt=&#34;&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Conductance&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Size&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;We see that DATAGEN is able to reproduce a realistics community structure, compared to existing graph generators. This feature, could be potentially exploited to define new benchmakrs to measure the quality of novel community detection algorithms. Stay tuned for future blog posts about his topic!&lt;/p&gt;
&lt;h4 id=&#34;references&#34;&gt;References&lt;/h4&gt;
&lt;p&gt;[1] Arnau Prat-Pérez, &lt;a href=&#34;http://dblp.uni-trier.de/pers/hd/d/Dom=iacute=nguez=Sal:David&#34;&gt;David Domínguez-Sal&lt;/a&gt;: How community-like is the structure of synthetically generated graphs? &lt;a href=&#34;http://dblp.uni-trier.de/db/conf/sigmod/grades2014.html#PratD14&#34;&gt;GRADES 2014&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;[2] Deepayan Chakrabarti, Yiping Zhan, and ChristosFaloutsos. R-mat: A recursive model for graph mining. SIAM 2014&lt;/p&gt;
&lt;p&gt;[3] Myunghwan Kim and Jure Leskovec. Multiplicative attribute graph model of real-world networks. Internet Mathematics&lt;/p&gt;
&lt;p&gt;[4] Andrea Lancichinetti, Santo Fortunato, and Filippo Radicchi. Benchmark graphs for testing community detection algorithms. Physical Review E 2008.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Industry Relevance of the Semantic Publishing Benchmark</title>
      <link>https://ldbcouncil.org/post/industry-relevance-of-the-semantic-publishing-benchmark/</link>
      <pubDate>Tue, 03 Mar 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/industry-relevance-of-the-semantic-publishing-benchmark/</guid>
      <description>&lt;h3 id=&#34;publishing-and-media-businesses-are-going-through-transformation&#34;&gt;Publishing and media businesses are going through transformation&lt;/h3&gt;
&lt;p&gt;I took this picture in June 2010 next to Union Square in San Francisco. I was smoking and wrestling my jetlag in front of Hilton. In the lobby inside the SemTech 2010 conference attendants were watching a game from the FIFA World Cup in South Africa. In the picture, the self-service newspaper stand is empty, except for one free paper. It was not long ago, in the year 2000, this stand was full. Back than the people in the Bay area were willing to pay for printed newspapers. But this is no longer true.&lt;/p&gt;
&lt;p&gt;What’s driving this change in publishing and media?&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Widespread and instantaneous distribution of information over the Internet has turned news into somewhat of a &amp;ldquo;commodity&amp;rdquo; and few people are willing to pay for it&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The wealth of free content on YouTube and similar services spoiled the comfort of many mainstream broadcasters;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Open access publishing has limited academic publishers to sell journals and books at prices that were considered fair ten years ago.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;em&gt;Alongside other changes in the industry, publishers figured out that it is critical to add value through better authoring, promotion, discoverability, delivery and presentation of precious content.&lt;/em&gt;&lt;/p&gt;
&lt;h3 id=&#34;imagine-instant-news-in-context-imagine-personal-channels-imagine--triplestores&#34;&gt;Imagine instant news in context, Imagine personal channels, Imagine &amp;hellip; triplestores&lt;/h3&gt;
&lt;p&gt;While plain news can be created repeatedly, premium content and services are not as easy to create. Think of an article that not only tells the new facts, but refers back to previous events and is complemented by an info-box of relevant facts. It allows one to interpret and comprehend news more effectively. This is the well-known journalistic aim to put news in context. It is also well-known that producing such news in &amp;ldquo;near real time&amp;rdquo; is difficult and expensive using legacy processes and content management technology.&lt;/p&gt;
&lt;p&gt;Another example would be a news feed that delivers good coverage of information relevant to a narrow subject – for example a company, a story line or a region. Judging by the demand for intelligent press clipping services like &lt;a href=&#34;http://new.dowjones.com/products/factiva/&#34;&gt;Factiva&lt;/a&gt;, such channels are in demand but are not straightforward to produce with today’s technology. Despite the common perception that automated recommendations for related content and personalized news are technology no-brainers, suggesting truly relevant content is far from trivial.&lt;/p&gt;
&lt;p&gt;Finally, if we use an example in life sciences, the ability to quickly find scientific articles discussing asthma and x-rays, while searching for respiration disorders and radiation, requires a search service that is not easy to deliver.&lt;/p&gt;
&lt;p&gt;Many publishers have been pressed to advance their business. This, in turn, had led to quest to innovate. And semantic technology can help publishers in two fundamental ways:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Generation of rich and &amp;ldquo;meaningful&amp;rdquo; (trying not to use &amp;ldquo;semantic&amp;rdquo; :-) metadata descriptions; 1. Dynamic retrieval of content, based on this rich metadata, enabling better delivery.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;In this post I write about &amp;ldquo;semantic annotation&amp;rdquo; and how it enables application scenarios like BBC’s Dynamic Semantic Publishing (DSP). I will also present the business case behind DSP. The final part of the post is about triplestores – semantic graph database engines, used in DSP. To be more concrete I write about the Semantic Publishing Benchmark (SPB), which evaluates the performance of triplestores in DSP scenarios.&lt;/p&gt;
&lt;h3 id=&#34;semantic-annotation-produces-rich-metadata-descriptions--the-fuel-for-semantic-publishing&#34;&gt;Semantic Annotation produces Rich Metadata Descriptions – the fuel for semantic publishing&lt;/h3&gt;
&lt;p&gt;The most popular meaning of &amp;ldquo;semantic annotation&amp;rdquo; is the process of enrichment of text with links to (descriptions of) concepts and entities mentioned in the text. This usually means tagging either the entire document or specific parts of it with identifiers of entities. These identifiers allow one to retrieve descriptions of the entities and relations to other entities – additional structured information that fuels better search and presentation.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;02_semantic_repository.png&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;The concept of using text-mining for automatic semantic annotation of text with respect to very large datasets, such as &lt;a href=&#34;http://dbpedia.org/&#34;&gt;DBPedia&lt;/a&gt;, emerged in early 2000. In practical terms it means using such large datasets as a sort of gigantic gazetteer (name lookup tool) and the ability to disambiguate. Figuring out whether &amp;ldquo;Paris&amp;rdquo; in the text refers to the capital of France or to Paris, Texas, or to Paris Hilton is crucial in such context. Sometimes this is massively difficult – try to instruct a computer how to guess whether &amp;ldquo;Hilton&amp;rdquo; in the second sentence of this post refers to a hotel from the chain founded by her grandfather or that I had the chance to meet Paris Hilton in person on the street in San Francisco.&lt;/p&gt;
&lt;p&gt;Today there are plenty of tools (such as the &lt;a href=&#34;https://www.ontotext.com/semantic-solutions/media-publishing/&#34;&gt;Ontotext Media and Publishing&lt;/a&gt; platform and &lt;a href=&#34;https://github.com/dbpedia-spotlight/dbpedia-spotlight/wiki&#34;&gt;DBPedia Spotlight&lt;/a&gt;) and services (such as Thomson Reuter’s &lt;a href=&#34;http://www.opencalais.com/&#34;&gt;OpenCalais&lt;/a&gt; and Ontotext’s S4 that offer automatic semantic annotation. Although text-mining cannot deliver 100% correct annotations, there are plenty of scenarios, where technology like this would revoluntionize a business. This is the case with the Dynamic Semantic Publishing scenario described below.&lt;/p&gt;
&lt;h3 id=&#34;the-bbcs-dynamic-semantic-publishing-dsp&#34;&gt;The BBC’s Dynamic Semantic Publishing (DSP)&lt;/h3&gt;
&lt;p&gt;Dynamic Semantic Publishing is a model for using semantic technology in media developed by a group led by John O’Donovan and Jem Rayfield at the BBC. The implementation of DSP behind BBC’s FIFA World Cup 2010 website was the first high-profile success story for usage of semantic technology in media. It is also the basis for the SPB benchmark – sufficient reasons to introduce this use case at length below.&lt;/p&gt;
&lt;p&gt;BBC Future Media &amp;amp; Technology department have transformed the BBC relational content management model and static publishing framework to a fully dynamic semantic publishing architecture. With minimal journalistic management, media assets are being enriched with links to concepts, semantically described in a triplestore. This novel semantic approach provides improved navigation, content re-use and re-purposing through automatic aggregation and rendering of links to relevant stories. At the end of the day DSP improves the user experience on BBC’s web site.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;&amp;ldquo;A high-performance dynamic semantic publishing framework facilitates the publication of automated metadata-driven web pages that are light-touch, requiring minimal journalistic management, as they automatically aggregate and render links to relevant stories&amp;rdquo;.&lt;/em&gt; &amp;ndash; &lt;a href=&#34;https://web.archive.org/web/20140420081627/https://www.bbc.co.uk/blogs/legacy/bbcinternet/2010/07/bbc_world_cup_2010_dynamic_sem.html&#34;&gt;Jem Rayfield, Senior Technical Architect&lt;/a&gt;, BBC News and Knowledge&lt;/p&gt;
&lt;p&gt;The Dynamic Semantic Publishing (DSP) architecture of the BBC curates and publishes content (e.g. articles or images) based on embedded Linked Data identifiers, ontologies and associated inference. It allows for journalists to determine levels of automation (&amp;ldquo;edited by exception&amp;rdquo;) and support semantic advertisement placement for audiences outside of the UK. The following quote explains the workflow when a new article gets into BBC’s content management system.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;&amp;ldquo;In addition to the manual selective tagging process, journalist-authored content is automatically analysed against the World Cup ontology. A &lt;a href=&#34;https://web.archive.org/web/20140420081627/https://www.bbc.co.uk/blogs/legacy/bbcinternet/2010/07/bbc_world_cup_2010_dynamic_sem.html#language&#34;&gt;natural language and ontological determiner process&lt;/a&gt; automatically extracts World Cup concepts embedded within a textual representation of a story. The concepts are moderated and, again, selectively applied before publication. Moderated, automated concept analysis improves the depth, breadth and quality of metadata publishing.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;03_bbc_sport.png&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Journalist-published metadata is captured and made persistent for querying using the resource description framework (&lt;a href=&#34;https://web.archive.org/web/20140420081627/https://www.bbc.co.uk/blogs/legacy/bbcinternet/2010/07/bbc_world_cup_2010_dynamic_sem.html#RDF&#34;&gt;&lt;em&gt;RDF&lt;/em&gt;&lt;/a&gt;) metadata representation and triple store technology. &lt;a href=&#34;https://web.archive.org/web/20140420081627/https://www.bbc.co.uk/blogs/legacy/bbcinternet/2010/07/bbc_world_cup_2010_dynamic_sem.html#BigOWLIM&#34;&gt;A RDF triplestore&lt;/a&gt; and &lt;a href=&#34;https://web.archive.org/web/20140420081627/https://www.bbc.co.uk/blogs/legacy/bbcinternet/2010/07/bbc_world_cup_2010_dynamic_sem.html#SPARQL&#34;&gt;SPARQL&lt;/a&gt; approach was chosen over and above traditional relational database technologies due to the requirements for interpretation of metadata with respect to an ontological domain model. The high level goal is that the domain ontology allows for intelligent mapping of journalist assets to concepts and queries. The chosen triplestore provides reasoning following the forward-chaining model and thus implied inferred statements are automatically derived from the explicitly applied journalist metadata concepts. For example, if a journalist selects and applies the single concept &amp;ldquo;Frank Lampard&amp;rdquo;, then the framework infers and applies concepts such as &amp;ldquo;England Squad&amp;rdquo;, &amp;ldquo;Group C&amp;rdquo; and &amp;ldquo;FIFA World Cup 2010&amp;rdquo; &amp;hellip;&amp;rdquo;&lt;/em&gt; &amp;ndash; Jem Rayfield&lt;/p&gt;
&lt;p&gt;One can consider each of the &amp;ldquo;aggregation pages&amp;rdquo; of BBC as a sort of feed or channel serving content related to a specific topic. If you take this perspective, with its World Cup 2010 website BBC was able to provide more than 700 thematic channels.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;&amp;ldquo;The World Cup site is a large site with over 700 aggregation pages (called index pages) designed to lead you on to the thousands of story pages and content&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;…&lt;/strong&gt;&lt;strong&gt;&lt;em&gt;we are not publishing pages, but publishing content&lt;/em&gt;&lt;/strong&gt; &lt;em&gt;as assets which are then organized by the metadata dynamically into pages, but could be re-organized into any format we want much more easily than we could before.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;04_content_tagging.png&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;… The index pages are published automatically. This process is what assures us of the highest quality output, but still &lt;strong&gt;save large amounts of time&lt;/strong&gt; in managing the site and &lt;strong&gt;makes it possible for us to efficiently run so many pages&lt;/strong&gt; for the World Cup.&amp;rdquo;&lt;/em&gt; &amp;ndash; &lt;a href=&#34;http://www.bbc.co.uk/blogs/bbcinternet/2010/07/the_world_cup_and_a_call_to_ac.html&#34;&gt;John O&amp;rsquo;Donovan, Chief Technical Architect, BBC Future Media &amp;amp; Technology&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;To get a real feeling about the load of the triplestore behind BBC&amp;rsquo;s World Cup web site, here are some statistics:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;800+ aggregation pages (Player, Team, Group, etc.), generated through SPARQL queries;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Average unique page requests/day: 2 million;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Average &lt;strong&gt;SPARQL queries/day: 1 million;&lt;/strong&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;100s repository updates/inserts per minute&lt;/strong&gt; with OWL 2 RL reasoning;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Multi data center that is fully resilient, clustered 6 node triplestore.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;the-semantic-publishing-benchmark&#34;&gt;The Semantic Publishing Benchmark&lt;/h3&gt;
&lt;p&gt;LDBC&amp;rsquo;s &lt;a href=&#34;https://ldbcouncil.org/developer/spb&#34;&gt;Semantic Publishing Benchmark&lt;/a&gt; (SPB) measures the performance of an RDF database in a load typical for metadata-based content publishing, such as the BBC Dynamic Semantic Publishing scenario. Such load combines tens of updates per second (e.g. adding metadata about new articles) with even higher volumes of read requests (SPARQL queries collecting recent content and data to generate web pages on a specific subject, e.g. Frank Lampard).&lt;/p&gt;
&lt;p&gt;SPB simulates a setup for media that deals with large volumes of streaming content, e.g. articles, pictures, videos. This content is being enriched with metadata that describes it through links to reference knowledge:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Reference knowledge:&lt;/em&gt; taxonomies and databases that include relevant concepts, entities and factual information (e.g. sport statistics);&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Metadata&lt;/em&gt; for each individual piece of content allows publishers to efficiently produce live streams of content relevant to specific subjects.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In this scenario the triplestore holds both reference knowledge and metadata. The main interactions with the repository are of two types:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Aggregation queries&lt;/em&gt; retrieve content according to various criteria. There are two sets (mixes) of aggregation queries. The basic one includes interactive queries that involve retrieval of concrete pieces of content, as well as aggregation functions, geo-spatial and full-text search constraints. The analytical query mix includes analytical queries, faceted search and drill-down queries;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Updates&lt;/em&gt;, adding new metadata or updating the reference knowledge. It is important that such updates should immediately impact the results of the aggregation queries. Imagine a fan checking the page for Frank Lampard right after he scored a goal – she will be very disappointed to see out of date statistics there.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;SPB v.1.0 directly reproduces the DSP setup at the BBC. The reference dataset consists of BBC Ontologies (Core, Sport, News), BBC datasets (list of F1 teams, MPs, etc.) and an excerpt from &lt;a href=&#34;http://www.geonames.org/&#34;&gt;Geonames&lt;/a&gt; for the UK. The benchmark is packed with metadata generator that allows one to set up experiments at different scales. The metadata generator produces 19 statements per Creative Work (BBC’s slang for all sorts of media assets). The standard scale factor is 50 million statements.&lt;/p&gt;
&lt;p&gt;A more technical introduction to SPB can be found in this &lt;a href=&#34;https://ldbcouncil.org/post/getting-started-with-the-semantic-publishing-benchmark&#34;&gt;post&lt;/a&gt;. Results from experiments with SPB on different hardware configurations, including AWS instances, are available in this &lt;a href=&#34;https://ldbcouncil.org/post/sizing-aws-instances-for-the-semantic-publishing-benchmark&#34;&gt;post&lt;/a&gt;. An interesting discovery is that given the current state of the technology (particularly the GraphDB v.6.1 engine) and today’s cloud infrastructure, the load of BBC’s World Cup 2010 website can be handled at AWS by a cluster that costs only $81/day.&lt;/p&gt;
&lt;p&gt;Despite the fact that SPB v.1.0 follows closely the usage scenario for triplestores in BBC’s DSP incarnations, it is relevant to a wide range of media and publishing scenarios, where large volumes of &amp;ldquo;fast flowing&amp;rdquo; content need to be &amp;ldquo;dispatched&amp;rdquo; to serve various information needs of a huge number of consumers. The main challenges can be summarized as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;The Triplestore is used as operational database serving a massive number of read queries (hundreds of queries per second) in parallel with tens of update transactions per second. Transactions need to be handled instantly and in a reliable and consistent manner;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Reasoning is needed to map content descriptions to queries in a flexible manner;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;There are specific requirements, such as efficient handling of full-text search, geo-spatial and temporal constraints.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;spb-v20--steeper-for-the-engines-closer-to-the-publishers&#34;&gt;SPB v.2.0 – steeper for the engines, closer to the publishers&lt;/h3&gt;
&lt;p&gt;We are in the final testing of the new version 2.0 of SPB. The benchmark has evolved to allow for retrieval of semantically relevant content in a more advanced manner and at the same time to demonstrate how triplestores can offer simplified and more efficient querying.&lt;/p&gt;
&lt;p&gt;The major changes in SPB v.2.0 can be summarized as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Much bigger reference dataset: from 170 thousand to 22 million statements. Now it includes GeoNames data about all of Europe (around 7 million statements) and DBPedia data about companies, people and events (14 million statements). This way we can simulate media archives described against datasets with good global coverage for specific types of objects. Such large reference sets also provide a better testing ground for experiments with very large content archives – think of 50 million documents (1 billion statements) or more;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Better interconnected reference data: more than 5 million links between entities, including 500,000 owl:sameAs links between DBPedia and Geonames descriptions. The latter evaluates the capabilities of the engine to deal with data coming from multiple sources, which use different identifiers for one and the same entity;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Retrieval of relevant content through links in the reference data, including inferred ones. To this end it is important than SPB v.2.0 involves much more comprehensive inference, particularly with respect to transitive closure of parent-company and geographic nesting chains.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>OWL-Empowered SPARQL Query Optimization</title>
      <link>https://ldbcouncil.org/post/owl-empowered-sparql-query-optimization/</link>
      <pubDate>Wed, 18 Feb 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/owl-empowered-sparql-query-optimization/</guid>
      <description>&lt;p&gt;The Linked Data paradigm has become the prominent enabler for sharing huge volumes of data using Semantic Web technologies, and has created novel challenges for non-relational data management systems, such as RDF and graph engines. Efficient data access through queries is perhaps the most important data management task, and is enabled through query optimization techniques, which amount to the discovery of optimal or close to optimal execution plans for a given query.&lt;/p&gt;
&lt;p&gt;In this post, we propose a different approach to query optimization, which is meant to complement (rather than replace) the standard optimization methodologies for SPARQL queries. Our approach is based on the use of schema information, encoded using OWL constructs, which often accompany Linked Data.&lt;/p&gt;
&lt;p&gt;OWL adopts the Open World Assumption and hence OWL axioms are perceived primarily to infer new knowledge. Nevertheless, ontology designers consider OWL as an expressive schema language used to express constraints for validating the datasets, hence following the Closed World Assumption when interpreting OWL ontologies. Such constraints include disjointness/equivalence of classes/properties, cardinality constraints, domain and range restrictions for properties and others.&lt;/p&gt;
&lt;p&gt;This richness of information carried over by OWL axioms can be the basis for the development of schema-aware techniques that will allow significant improvements in the performance of existing RDF query engines when used in tandem with data statistics or even other heuristics based on patterns found in SPARQL queries. As a simple example, a cardinality constraint at the schema level can provide a hint on the proper join ordering, even if data statistics are missing or incomplete.&lt;/p&gt;
&lt;p&gt;The aim of this post is to show that the richness of information carried over by OWL axioms under the Close World Assumption can be the basis for the development of schema-aware optimization techniques that will allow considerable improvement for query processing. To attain this objective, we discuss a small set of interesting cases of OWL axioms; a full list can be found &lt;a href=&#34;LDBC_D4.4.2_final.pdf&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;h3 id=&#34;schema-based-optimization-techniques&#34;&gt;Schema-Based Optimization Techniques&lt;/h3&gt;
&lt;p&gt;Here we provide some examples of queries, which, when combined with specific schema constraints expressed in OWL, can help the optimizer in formulating the (near to) optimal query plans.&lt;/p&gt;
&lt;p&gt;A simple first case is the case of constraint violation. Consider the query below, which returns all instances of class &lt;code&gt;&amp;lt;A&amp;gt;&lt;/code&gt; which are fillers of a specific property &lt;code&gt;&amp;lt;P&amp;gt;&lt;/code&gt;. If the underlying schema contains the information that the range of &lt;code&gt;&amp;lt;P&amp;gt;&lt;/code&gt; is class &lt;code&gt;&amp;lt;B&amp;gt;&lt;/code&gt;, and that class &lt;code&gt;&amp;lt;B&amp;gt;&lt;/code&gt; is disjoint from class &lt;code&gt;&amp;lt;A&amp;gt;&lt;/code&gt;, then this query should return the empty result, with no further evaluation (assuming that the constraints associated with the schema are satisfied by the data). An optimizer that takes into account schema information should return an empty result in constant time instead of trying to optimize or evaluate the large star join.&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;SELECT ?v 
WHERE { ?v rdf : type &amp;lt;A&amp;gt; .   
        ?u &amp;lt;P&amp;gt; ?v . ?u &amp;lt;P&amp;gt; ?v1 .   
        ?u &amp;lt;P1 &amp;gt; ?v2 . ?u &amp;lt;P2 &amp;gt; ?v3 .   
        ?u &amp;lt;P3 &amp;gt; ?v4 . ?u &amp;lt;P4 &amp;gt; ?v5}
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Schema-aware optimizers could also prune the search space by eliminating results that are known a priori not to be in the answer set of a query. The query above is an extreme such example (where all potential results are pruned), but other cases are possible, such as the case of the query below, where all subclasses of class &lt;code&gt;&amp;lt;A1&amp;gt;&lt;/code&gt; can immediately be identified as not being in the answer set.&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;SELECT ?c
WHERE { ?x rdf: type ?c . ?x &amp;lt;P&amp;gt; ?y . 
        FILTER NOT EXISTS \{ ?x rdf: type &amp;lt;A1 &amp;gt; }}
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Another category of schema-empowered optimizations has to do with improved selectivity estimation. In this respect, knowledge about the cardinality (minimum cardinality, maximum cardinality, exact cardinality, functionality) of a property can be exploited to formulate better query plans, even if data statistics are incomplete, missing or erroneous.&lt;/p&gt;
&lt;p&gt;Similarly, taking into account class hierarchies, or the definition of classes/properties via set theoretic constructs (union, intersection) at the schema level, can provide valuable information on the selectivity of certain triple patterns, thus facilitating the process of query optimization. Similar effects can be achieved using information about properties (functionality, transitivity, symmetry etc).&lt;/p&gt;
&lt;p&gt;As an example of these patterns, consider the query below, where class &lt;code&gt;&amp;lt;C&amp;gt;&lt;/code&gt; is defined as the intersection of classes &lt;code&gt;&amp;lt;C1&amp;gt;&lt;/code&gt;,&lt;code&gt; &amp;lt;C2&amp;gt;&lt;/code&gt;. Thus, the triple pattern &lt;code&gt;(?x rdf:type &amp;lt;C&amp;gt;)&lt;/code&gt; is more selective than &lt;code&gt;(?y rdf:type &amp;lt;C1&amp;gt;)&lt;/code&gt; and &lt;code&gt;(?z rdf:type &amp;lt;C2&amp;gt;)&lt;/code&gt; and this should be immediately recognizable by the optimizer, without having to resort to cost estimations. This example shows also how unnecessary triple patterns can be pruned from a query to reduce the number of necessary joins. Figure 1 illustrates the query plan obtained when the OWL intersectionOf construct is used.&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;SELECT ?x 
WHERE { ?x rdf: type &amp;lt;C&amp;gt; . ?x &amp;lt;P1 &amp;gt; ?y . 
        ?y rdf : type &amp;lt;C1 &amp;gt; . ?y &amp;lt;P2 &amp;gt; ?z . ?z rdf : type &amp;lt;C2 &amp;gt; }
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;&lt;img src=&#34;owl_constraints.png&#34; alt=&#34;image&#34;&gt;&lt;/p&gt;
&lt;p&gt;Schema information can also be used by the query optimizer to rewrite SPARQL queries to equivalent ones that are found in a form for which already known optimization techniques are easily applicable. For example, the query below could easily be transformed into a classical star-join query if we know (from the schema) that property &lt;code&gt;P4&lt;/code&gt; is a symmetric property.&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;SELECT ?y ?y1 ?y2 ?y3 
WHERE { ?x &amp;lt;P1 &amp;gt; ?y . ?x &amp;lt;P2 &amp;gt; ?y1 . 
        ?x &amp;lt;P3 &amp;gt; ?y2 . ?y3 &amp;lt;P4 &amp;gt; ?x }
&lt;/code&gt;&lt;/pre&gt;&lt;h3 id=&#34;conclusion&#34;&gt;Conclusion&lt;/h3&gt;
&lt;p&gt;In this post we argued that OWL-empowered optimization techniques can be beneficial for SPARQL query optimization when used in tandem with standard heuristics based on statistics. We provided some examples which showed the power of such optimizations in various cases, namely:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Cases where the search space can be pruned due to the schema and the associated constraints; an extreme special sub-case is the identification of queries that violate schema constraints and thus produce no results.&lt;/li&gt;
&lt;li&gt;Cases where the schema can help in the estimation of triple pattern selectivity, even if statistics are incomplete or missing.&lt;/li&gt;
&lt;li&gt;Cases where the schema can identify redundant triple patterns that do not affect the result and can be safely eliminated from the query.&lt;/li&gt;
&lt;li&gt;Cases where the schema can be used for rewriting a query in an equivalent form that would facilitate optimization using well-known optimization techniques.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This list is by no means complete, as further cases can be identified by optimizers. Our aim in this post was not to provide a complete listing, but to demonstrate the potential of the idea in various directions.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Person Activity Subgraph Features in LDBC DATAGEN</title>
      <link>https://ldbcouncil.org/post/person-activity-subgraph-features-in-ldbc-datagen/</link>
      <pubDate>Wed, 04 Feb 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/person-activity-subgraph-features-in-ldbc-datagen/</guid>
      <description>&lt;p&gt;When talking about DATAGEN and other graph generators with social network characteristics, our attention is typically borrowed by the friendship subgraph and/or its structure. However, a social graph is more than a bunch of people being connected by friendship relations, but has a lot more of other things is worth to look at. With a quick view to commercial social networks like Facebook, Twitter or Google+, one can easily identify a lot of other elements such as text images or even video assets. More importantly, all these elements form other subgraphs within the social network! For example, the person activity subgraph is composed by posts and their replies in the different forums/groups in a social network, and has a tree-like structure connecting people through their message interactions.&lt;/p&gt;
&lt;p&gt;When looking at the LDBC Social Network Benchmark (SNB) and its interactive workload, one realizes that these other subgraphs, and especially the person activity subgraph, play a role even more important than that played by the friendship subgraph. Just two numbers that illustrate this importance: 11 out of the 14 interactive workload queries needs traversing parts of the person activity subgraph, and about 80% of all the generated data by DATAGEN belongs to this subgraph. As a consequence, a lot of effort has been devoted to make sure that the person activity subgraph is realistic enough to fulfill the needs of the benchmark. In the rest of this post, I will discuss some of the features implemented in DATAGEN that make the person activity subgraph interesting.&lt;/p&gt;
&lt;h3 id=&#34;reaslistic-message-content&#34;&gt;Reaslistic Message Content&lt;/h3&gt;
&lt;p&gt;Messages&amp;rsquo; content in DATAGEN is not random, but contains snippets of text extracted from Dbpedia talking about the tags the message has. Furthermore, not all messages are the same size, depending on whether they are posts or replies to them. For example, the size of a post is selected uniformly between a minimum and a maximum, but also, there is a small probability that the content is very large (about 2000 characters). In the case of commets (replies to posts), there is a probability of 0.66 to be very short (“ok”, “good”, “cool”, “thanks”, etc.). Moreover, in real forum conversations, it is tipical to see conversations evolving from one topic to another. For this reason, there is a probability that the tags of comments replying posts to change during the flow of the conversation, moving from post&amp;rsquo;s tags to other related or randomly selected tags.&lt;/p&gt;
&lt;h3 id=&#34;non-uniform-activity-levels&#34;&gt;Non uniform activity levels&lt;/h3&gt;
&lt;p&gt;In a real social network, not all the members show the same level of activity. Some people post messages more sporadically than others, whose activity is significantly higher. DATAGEN reproduces this phenomena by correlating the activity level with the amount of friends the person has. That is, the larger the amount of friends a person has, the larger the number of posts it creates, and also, the larger the number of groups it belongs to.&lt;/p&gt;
&lt;h3 id=&#34;time-correlated-post-and-comment-generation&#34;&gt;Time correlated post and comment generation&lt;/h3&gt;
&lt;p&gt;In a real social network, user activity is driven by real world events such as sport events, elections or natural disasters, just to cite a few of them. For this reason, we observe spikes of activity around these events, where the amount of messages created increases significantly during a short period of time, reaching a maximum and then decreasing. DATAGEN emulates this behavior by generating a set of real world events about specific tags. Then, when dates of posts and comments are generated, these events are taken into account in such a way that posts and comments are clustered around them. Also not all the events are equally relevant, thus having spikes larger than others. The shape of the activity is modeled following the model described in &lt;a href=&#34;#references&#34;&gt;[1]&lt;/a&gt;. Furthermore, in order to represent the more normal and uniform person activity levels, we also generate uniformly distributed messages along the time line. The following figure shows the user activity volume along the time line.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;1.png&#34; alt=&#34;image&#34;&gt;&lt;/p&gt;
&lt;p&gt;As we see, the timeline contains spikes of activity, instead of being uniform. Note that the generally increasing volume activity is due to the fact that more people is added to the social network as time advances.&lt;/p&gt;
&lt;p&gt;In this post we have reviewed several interesting characteristics of the person activity generation process in DATAGEN. Stay tuned for future blog posts about this topic.&lt;/p&gt;
&lt;h4 id=&#34;references&#34;&gt;References&lt;/h4&gt;
&lt;p&gt;[1] Leskovec, J., Backstrom, L., &amp;amp; Kleinberg, J. (2009, June). Meme-tracking and the dynamics of the news cycle. In &lt;em&gt;Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining&lt;/em&gt; (pp. 497-506). ACM.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>SNB Driver - Part 2: Tracking Dependencies Between Queries</title>
      <link>https://ldbcouncil.org/post/snb-driver-part-2-tracking-dependencies-between-queries/</link>
      <pubDate>Fri, 23 Jan 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/snb-driver-part-2-tracking-dependencies-between-queries/</guid>
      <description>&lt;p&gt;The &lt;a href=&#34;https://ldbcouncil.org/post/snb-driver-part-1&#34;&gt;SNB Driver part 1&lt;/a&gt; post introduced, broadly, the challenges faced when developing a workload driver for the LDBC SNB benchmark. In this blog we&amp;rsquo;ll drill down deeper into the details of what it means to execute &amp;ldquo;dependent queries&amp;rdquo; during benchmark execution, and how this is handled in the driver. First of all, as many driver-specific terms will be used, below is a listing of their definitions. There is no need to read them in detail, it is just there to serve as a point of reference.&lt;/p&gt;
&lt;h3 id=&#34;definitions&#34;&gt;Definitions&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Simulation Time (ST)&lt;/em&gt;: notion of time created by data generator. All time stamps in the generated data set are in simulation time&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Real Time (RT)&lt;/em&gt;: wall clock time&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Time Compression Ratio&lt;/em&gt;: function that maps simulation time to real time, e.g., an offset in combination with a compression ratio. It is a static value, set in driver configuration. Real Time Ratio is reported along with benchmark results, allowing others to recreate the same benchmark&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Operation&lt;/em&gt;: read and/or write&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Dependencies&lt;/em&gt;: operations in this set introduce dependencies in the workload. That is, for every operation in this set there exists at least one other operation (in Dependents) that can not be executed until this operation has been processed&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Dependents&lt;/em&gt;: operations in this set are dependent on at least one other operation (in Dependencies) in the workload&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Due Time (DueT)&lt;/em&gt;: point in simulation time at which the execution of an operation should be initiated.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Dependent Time (DepT)&lt;/em&gt;: in addition to Due Time, every operation in Dependents also has a Dependent Time, which corresponds to the Due Time of the operation that it depends on. Dependent Time is always before Due Time. For operations with multiple dependencies Dependent Time is the maximum Due Time of all the operations it depends on.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Safe Time (SafeT)&lt;/em&gt;: time duration.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;when two operations have a necessary order in time (i.e., dependency) there is at least a SafeT interval between them&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;SafeT is the minimum duration between the Dependency Time and Due Time of any operations in Dependents&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;​&lt;em&gt;Operation Stream&lt;/em&gt;: sequence of operations ordered by Due Time (dependent operations must separated by at least SafeT)&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Initiated Operations&lt;/em&gt;: operations that have started executing but not yet finished&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Local Completion Time (per driver)&lt;/em&gt;: point in simulation time behind which there are no uncompleted operationsLocal Completion Time = min(min(Initiated Operations), max(Completed Operations))&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Global Completion Time (GCT)&lt;/em&gt;: minimum completion time of all drivers. Once GCT has advanced to the Dependent Time of some operation that operation is safe to execute, i.e., the operations it depends on have all completed executing. Global Completion Time = min(Local Completion Time)​&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Execution Window (Window)&lt;/em&gt;: a timespan within which all operations can be safely executed&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;All operations satisfying window.startTime &amp;lt;= operation.DueT &amp;lt; window.endTime may be executed&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Within a window no restrictions on operation ordering or operation execution time are enforced, driver has a freedom of choosing an arbitrary scheduling strategy inside the window&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;To ensure that execution order respects dependencies between operations, window size is bounded by SafeT, such that: 0 &amp;lt; window.duration &amp;lt;= SafeT&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Window duration is fixed, per operation stream; this is to simplify scheduling and make benchmark runs repeatable&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Before any operations within a window can start executing it is required that: GCT &amp;gt;= window.startTime - (SafeT - window.duration)&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;All operations within a window must initiate and complete between window start and end times: window.startTime &amp;lt;= operation.initiate &amp;lt; window.endTime and window.startTime &amp;lt;= operation.complete &amp;lt; window.endTime&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Dependency Mode&lt;/em&gt;: defines dependencies, constraints on operation execution order&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Execution Mode&lt;/em&gt;: defines how the runtime should execute operations of a given type&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;tracking-dependencies&#34;&gt;Tracking Dependencies&lt;/h3&gt;
&lt;p&gt;Now, the fun part, making sure dependent operations are executed in the correct order.&lt;/p&gt;
&lt;p&gt;Consider that every operation in a workload belongs to none, one, or both of the following sets: Dependencies and Dependents. As mentioned, the driver uses operation time stamps (Due Times) to ensure that dependencies are maintained. It keeps track of the latest point in time behind which every operation has completed. That is, every operation (i.e., dependency) with a Due Time lower or equal to this time is guaranteed to have completed execution. It does this by maintaining a monotonically increasing variable called Global Completion Time (GCT).&lt;/p&gt;
&lt;p&gt;Logically, every time the driver (via a database connector) begins execution of an operation from Dependencies that operation is added to Initiated Operations:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;the set of operations that have started executing but not yet finished.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Then, upon completion, the operation is removed from Initiated Operations and added to Completed Operations:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;the set of operations that have started and finished executing.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Using these sets, each driver process maintains its own view of GCT in the following way. Local progress is monitored and managed using a variable called Local Completion Time (LCT):&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;the point in time behind which there are no uncompleted operations. No operation in Initiated Operations has a lower or equal Due Time and no operation in Completed Operations has an equal or higher Due Time.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;LCT is periodically sent to all other driver processes, which all then (locally) set their view of GCT to the minimum LCT of all driver processes. At this point the driver has two, of the necessary three (third covered shortly), pieces of information required for knowing when to execute an operation:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;Due Time&lt;/em&gt;: point in time at which an operation should be executed, assuming all preconditions (e.g., dependencies) have been fulfilled&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;em&gt;GCT&lt;/em&gt;: every operation (from Dependencies) with a Due Time before this point in time has completed execution&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;However, with only GCT to track dependencies the driver has no way of knowing when it is safe to execute any particular dependent operation. What GCT communicates is that all dependencies up to some point in time have completed, but whether or not the dependencies for any particular operation are within these completed operations is unknown. The driver would have to wait until GCT has passed the Due Time (because Dependency Time is always lower) of an operation before that operation could be safely executed, which would result in the undesirable outcome of every operation missing its Due Time. The required information is which particular operation in Dependencies does any operation in Dependents depend on. More specifically, the Due Time of this operation. This is referred to as Dependent Time:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;in addition to Due Time, every operation in Dependents also has (read: must have) a Dependent Time, which corresponds to the latest Due Time of all the operations it depends on. Once GCT has advanced beyond the Dependent Time of an operation that operation is safe to execute.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Using these three mechanisms (Due Time, GCT, and Dependent Time) the driver is able to execute operations, while ensuring their dependencies are satisfied beforehand.&lt;/p&gt;
&lt;h3 id=&#34;scalable-execution-in-the-presence-of-dependencies&#34;&gt;Scalable execution in the Presence of Dependencies&lt;/h3&gt;
&lt;p&gt;The mechanisms introduced in part 1 guarantee that dependency constraints are not violated, but in doing so they unavoidably introduce overhead of communication/synchronization between driver threads/processes. To minimize the negative effects that synchronization has on scalability an additional Execution Mode was introduced (more about Execution Modes will be discussed shortly): Windowed Execution. Windowed Execution has two design goals:&lt;/p&gt;
&lt;p&gt;a) make the generated load less &amp;lsquo;bursty&amp;rsquo;&lt;/p&gt;
&lt;p&gt;b) allow the driver to &amp;lsquo;scale&amp;rsquo;, so when the driver is given more resources (CPUs, servers, etc.) it is able to generate more load.&lt;/p&gt;
&lt;p&gt;In the context of Windowed Execution, operations are executed in groups (Windows), where operations are grouped according to their Due Time. Every Window has a Start Time, a Duration, and an End Time, and Windows contain only those operations that have a Due Time between Window.startTime and Window.endTime. Logically, all operations within a Window are executed at the same time, some time within the Window. No guaranty is made regarding exactly when, or in what order, an operation will execute within its Window.&lt;/p&gt;
&lt;p&gt;The reasons this approach is correct are as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Operations belonging to the Dependencies set are never executed in this manner - the Due Times of Dependencies operations are never modified as this would affect how dependencies are tracked&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The minimum duration between the Dependency Time and Due Time of any operation in Dependents is known (can be calculated by scanning through workload once), this duration is referred to as Safe Time (SafeT)&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;A window does not start executing until the dependencies of all its operations have been fulfilled. This is ensured by enforcing that window execution does not start until&lt;/p&gt;
&lt;p&gt;GCT &amp;gt;= window.startTime - (SafeT - window.duration) = window.endTime - SafeT; that is, the duration between GCT and the end of the window is no longer than SafeT&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The advantages of such an execution mode are as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;As no guarantees are made regarding time or order of operation execution within a Window, GCT no longer needs to be read before the execution of every operation, only before the execution of every window&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Then, as GCT is read less frequently, it follows that it does not need to be communicated between driver processes as frequently. There is no need or benefit to communicating GCT protocol message more frequently than approximately Window.duration, the side effect of which is reduced network traffic&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Further, by making no guarantees regarding the order of execution the driver is free to reschedule operations (within Window bounds). The advantage being that operations can be rearranged in such a way as to reduce unwanted bursts of load during execution, which could otherwise occur while synchronizing GCT during demanding workloads. For example, a uniform scheduler may modify operation Due Times to be uniformly distributed across the Window timespan, to &amp;lsquo;smoothen&amp;rsquo; the load within a Window.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;As with any system, there are trade-offs to this design, particularly regarding Window.duration. The main trade-off is that between &amp;lsquo;workload resolution&amp;rsquo; and scalability. Increasing Window.duration reduces synchronization but also reduces the resolution at which the workload definition is followed. That is, the generated workload becomes less like the workload definition. However, as this is both bounded and configurable, it is not a major concern. This issue is illustrated in Figure 1, where the same stream of events is split into two different workloads based on different size of the Window. The workload with Window size 5 (on the right) has better resolution, especially for the &amp;lsquo;bursty&amp;rsquo; part of the event stream.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;window-scheduling.png&#34; alt=&#34;image&#34;&gt;&lt;br&gt;
Figure 1. Window scheduling&lt;/p&gt;
&lt;p&gt;This design also trades a small amount of repeatability for scalability: as there are no timing or ordering guarantees within a window, two executions of the same window are not guaranteed to be equivalent - &amp;lsquo;what happens in the window stays in the window&amp;rsquo;. Despite sacrificing this repeatability, the results of operations do not change. No dependency-altering operations occur during the execution of a Window, therefore results for all queries should be equivalent between two executions of the same workload, there is no effect on the expected result for any given operation.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>SNB Driver - Part 3: Workload Execution Putting It All Together</title>
      <link>https://ldbcouncil.org/post/snb-driver-part-3-workload-execution-putting-it-all-together/</link>
      <pubDate>Tue, 20 Jan 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/snb-driver-part-3-workload-execution-putting-it-all-together/</guid>
      <description>&lt;p&gt;Up until now we have introduced the &lt;a href=&#34;https://ldbcouncil.org/post/snb-driver-part-1&#34;&gt;challenges faced when executing the LDBC SNB benchmark&lt;/a&gt;, as well as explained &lt;a href=&#34;https://ldbcouncil.org/post/snb-driver-part-2-tracking-dependencies-between-queries&#34;&gt;how some of these are overcome&lt;/a&gt;. With the foundations laid, we can now explain precisely how operations are executed.&lt;/p&gt;
&lt;p&gt;Based on the dependencies certain operations have, and on the granularity of parallelism we wish to achieve while executing them, we assign a Dependency Mode and an Execution Mode to every operation type. Using these classifications the driver runtime then knows how each operation should be executed. These modes, as well as what they mean to the driver runtime, are described below.&lt;/p&gt;
&lt;h3 id=&#34;dependency-modes&#34;&gt;Dependency Modes&lt;/h3&gt;
&lt;p&gt;While executing a workload the driver treats operations differently, depending on their Dependency Mode. In the previous section operations were categorized by whether or not they are in the sets Dependencies and/or Dependents.&lt;/p&gt;
&lt;p&gt;Another way of communicating the same categorization is by assigning a Dependency Mode to operations - every operation type generated by a workload definition must be assigned to exactly one Dependency Mode. Dependency modes define dependencies, constraints on operation execution order. The driver supports a number of different Dependency Modes: None, Read Only, Write Only, Read Write. During workload execution, operations of each type are treated as follows:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;• None&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Depended On (NO): operations do not introduce dependencies with other operations (i.e., the correct execution of no other operation depends on these operations to have completed executing)&lt;/p&gt;
&lt;p&gt;– Prior Execution: do nothing – After Execution: do nothing&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;• Read Only&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Depended On (NO): operations do not introduce dependencies with other operations (i.e., the correct execution of no other operation depends on these operations to have completed executing)&lt;/p&gt;
&lt;p&gt;Dependent On (YES): operation execution does depend on GCT to have advanced sufficiently (i.e., correct execution of these operations requires that certain operations have completed execution)&lt;/p&gt;
&lt;p&gt;– Prior Execution: wait for GCT &amp;gt;= operation.DepTime – After Execution: do nothing&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;• Write Only&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Depended On (YES): operations do introduce dependencies with other operations (i.e., the correct execution of certain other operations requires that these operations to have completed executing, i.e., to advance GCT)&lt;/p&gt;
&lt;p&gt;Dependent On (NO): operation execution does not depend on GCT to have advanced sufficiently (i.e., correct execution of these operations does not depend on any other operations to have completed execution)&lt;/p&gt;
&lt;p&gt;– Prior Execution: add operation to Initiated Operations&lt;/p&gt;
&lt;p&gt;– After Execution: remove operation from Initiated Operations, add operation to Completed Operations&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;• Read Write&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Depended On (YES): operations do introduce dependencies with other operations (i.e., the correct execution of certain other operations requires that these operations to have completed executing, i.e., to advance GCT)&lt;/p&gt;
&lt;p&gt;Dependent On (YES): operation execution does depend on GCT to have advanced sufficiently (i.e., correct execution of these operations requires that certain operations have completed execution)&lt;/p&gt;
&lt;p&gt;– Prior Execution: add operation to Initiated Operations, wait for GCT &amp;lt; operation.DepT&lt;/p&gt;
&lt;p&gt;– After Execution: remove operation from Initiated Operations, add operation to Completed Operations&lt;/p&gt;
&lt;h3 id=&#34;execution-modes&#34;&gt;Execution Modes&lt;/h3&gt;
&lt;p&gt;Execution Modes relate to how operations are scheduled, when they are executed, and what their failure conditions are. Each operation type in a workload definition must be assigned to exactly one Execution Mode. The driver supports a number of different Execution Modes: Asynchronous, Synchronous, Partially Synchronous. It splits a single workload operation stream into multiple streams, zero or more steams per Execution Mode. During workload execution, operations from each of these streams are treated as follows.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;• Asynchronous&lt;/strong&gt;: operations are executed individually, when their Due Time arrives.&lt;/p&gt;
&lt;p&gt;Motivation: This is the default execution mode, it executes operations as true to the workload definition as possible.&lt;/p&gt;
&lt;p&gt;– Re-scheduling Before Execution: None: operation.DueT not modified by scheduler – Execute When time &amp;gt;= operation.DueT (and GCT &amp;gt;= operation.DepT)&lt;/p&gt;
&lt;p&gt;– Max Concurrent Executions: unbounded&lt;/p&gt;
&lt;p&gt;– Max Execution Time: unbounded&lt;/p&gt;
&lt;p&gt;– Failure: operation execution starts later than: operation.DueT  Tolerated Delay&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;• Synchronous&lt;/strong&gt;: operations are executed individually, sequentially, in blocking manner.&lt;/p&gt;
&lt;p&gt;Motivation: Some dependencies are difficult to capture efficiently with SafeT and GCT alone. For example, social applications often support conversations via posts and likes, where likes depend on the existence of posts. Furthermore, posts and likes also depend on the existence of the users that make them. However, users are created at a lower frequency than posts and likes, and it can be assumed they do not immediately start creating content. As such, a reasonably long SafeT can be used between the creation of a user and the first time that user creates posts or likes. Conversely, posts are often replied to and/or liked soon after their creation, meaning a short SafeT would be necessary to maintain the ordering dependency. Consequently, maintaining the dependencies related to conversations would require a short SafeT, and hence a small window. This results in windows containing fewer operations, leading to less potential for parallelism within windows, less freedom in scheduling, more synchronization, and greater likelihood of bursty behavior - all negative things.&lt;/p&gt;
&lt;p&gt;The alternative offered by Synchronous Execution is that, when practical, operations of certain types can be partitioned (e.g. posts and likes could be partitioned by the forum in which they appear), and partitions assigned to driver processes. Using the social application example from above, if all posts and likes were partitioned by forum the driver process that executes the operations from any partition could simply execute them sequentially. Then the only dependency to maintain would be on user operations, reducing synchronization dramatically, and parallelism could still be achieved as each partition would be executed independently, in parallel, by a different driver process.&lt;/p&gt;
&lt;p&gt;– Re-scheduling Before Execution: None: operation.DueT not modified by scheduler&lt;/p&gt;
&lt;p&gt;– Execute When time &amp;gt;= operation.DueT and previousOperation.completed == true (and GCT &amp;gt;= operation.DepT)&lt;/p&gt;
&lt;p&gt;– Max Concurrent Executions: 1&lt;/p&gt;
&lt;p&gt;– Max Execution Time: nextOperation.DueT - operation.DueT&lt;/p&gt;
&lt;p&gt;– Failure: operation execution starts later than: operation.DueT  Tolerated Delay E.g., if previousOperation did not complete in time, forcing current operation to wait for longer than the tolerated-delay&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;• Partially Synchronous&lt;/strong&gt; (Windowed Execution, described in Section 3.4 in more details), groups of operations from the same time window are executed together&lt;/p&gt;
&lt;p&gt;– Re-scheduling Before Execution: Yes, as long as the following still holds:&lt;/p&gt;
&lt;p&gt;window.startTime &amp;lt;= operation.DueT &amp;lt; window.startTime + window.duration&lt;/p&gt;
&lt;p&gt;Operations within a window may be scheduled in any way, as long as they remain in the window from which they originated: their Due Times, and therefore ordering, may be modified&lt;/p&gt;
&lt;p&gt;– Execute When time &amp;gt;= operation.DueT (and GCT &amp;gt;= operation.DepT)&lt;/p&gt;
&lt;p&gt;– Max Concurrent Executions: number of operations within window&lt;/p&gt;
&lt;p&gt;– Max Execution Time: (window.startTime + window.duration) - operation.DueT&lt;/p&gt;
&lt;p&gt;– Failure: operation execution starts later than: window.startTime  window.duration operation execution does not finish by: window.startTime + window.duration&lt;/p&gt;
&lt;h3 id=&#34;tying-it-back-to-ldbc-snb&#34;&gt;Tying it back to LDBC SNB&lt;/h3&gt;
&lt;p&gt;The driver was designed to execute the workload of LDBC SNB. As discussed, the main challenge of running queries in parallel on graph-shaped data stem from dependencies introduced by the graph structure. In other words, workload partitioning becomes as hard as graph partitioning.&lt;/p&gt;
&lt;p&gt;The LDBC SNB data can in fact be seen as a union of two parts:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Core Data: relatively small and dense friendship graph (not more than 10% of the data). Updates on this part are very hard to partition among driver threads, since the graph is essentially a single dense strongly connected component.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;User Activity Data: posts, replies, likes; this is by far the biggest part of the data. Updates on this part are easily partitioned as long as the dependencies with the &amp;ldquo;core&amp;rdquo; part are satisfied (i.e., users don&amp;rsquo;t post things before the profiles are created, etc.).&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;In order to avoid friendship graph partitioning, the driver introduces the concept SafeT, the minimal simulation time that should pass between two dependent events.&lt;/p&gt;
&lt;p&gt;This property is enforced by the data generator, i.e. the driver does not need to change or delay some operations in order to guarantee dependency safety. Respecting dependencies now means globally communicating the advances of the Global Completion Time, and making sure the operations do not start earlier than SafeT from their dependents.&lt;/p&gt;
&lt;p&gt;On the other hand, the driver exploits the fact that some of the dependencies in fact do not hinder partitioning: although replies to the post can only be sent after the post is created, these kinds of dependencies are satisfied if we partition workload by forums. This way, all (update) operations on posts and comments from one forum are assigned to one driver thread. Since there is typically a lot of forums, each driver thread gets multiple ones. Updates from one forum are then run in Synchronous Execution Mode, and parallelism is achieved by running many distinct forums in parallel. By doing so, we can add posts and replies to forums at very high frequency without the need to communicate the GCT across driver instances (i.e. we efficiently create the so-called flash-mob effects in the posting/replying workload).&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Running the Semantic Publishing Benchmark on Sesame, a Step by Step Guide</title>
      <link>https://ldbcouncil.org/post/running-the-semantic-publishing-benchmark-on-sesame-a-step-by-step-guide/</link>
      <pubDate>Tue, 13 Jan 2015 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/running-the-semantic-publishing-benchmark-on-sesame-a-step-by-step-guide/</guid>
      <description>&lt;p&gt;Until now we have discussed several aspects of the &lt;a href=&#34;https://ldbcouncil.org/benchmarks/spb&#34;&gt;Semantic Publishing Benchmark (SPB)&lt;/a&gt; such as the &lt;a href=&#34;https://ldbcouncil.org/post/sizing-aws-instances-for-the-semantic-publishing-benchmark&#34;&gt;difference in performance between virtual and real servers configuration&lt;/a&gt;, how to choose an &lt;a href=&#34;https://ldbcouncil.org/post/making-semantic-publishing-execution-rules&#34;&gt;appropriate query mix&lt;/a&gt; for a benchmark run and our experience with using SPB in the development process of GraphDB for &lt;a href=&#34;https://ldbcouncil.org/post/using-ldbc-spb-to-find-owlim-performance-issues&#34;&gt;finding performance issues&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;In this post we provide a step-by-step guide on how to run SPB using the &lt;a href=&#34;http://rdf4j.org/&#34;&gt;Sesame&lt;/a&gt; RDF data store on a fresh install of &lt;a href=&#34;http://releases.ubuntu.com/14.04.1/&#34;&gt;Ubuntu Server 14.04.1&lt;/a&gt;. The scenario is easy to adapt to other RDF triple stores which support the Sesame Framework used for querying and analyzing RDF data.&lt;/p&gt;
&lt;h3 id=&#34;prerequisites&#34;&gt;Prerequisites&lt;/h3&gt;
&lt;p&gt;We start with a fresh server installation, but before proceeding with setup of the Sesame Data Store and SPB benchmark we need the following pieces of software up and running:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Git&lt;/li&gt;
&lt;li&gt;Apache Ant 1.8 or higher&lt;/li&gt;
&lt;li&gt;OpenJDK 6 or Oracle JDK 6 or higher&lt;/li&gt;
&lt;li&gt;Apache Tomcat 7 or higher&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If you already have these components installed on your machine you can directly proceed to the next section: &lt;em&gt;Installing Sesame&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Following are sample commands which can be used to install the required software components:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;sudo apt-get install git
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;sudo apt-get install ant
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;sudo apt-get install default-jdk
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;sudo apt-get install tomcat7
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Optionally Apache Tomcat Server can be downloaded as a zipped file and extracted in a location of choice.&lt;/p&gt;
&lt;p&gt;After a successful installation of Apache Tomcat you should be able to get the default splash page &lt;em&gt;“It works”&lt;/em&gt; when you open your web browser and enter the following address:  http://&amp;lt;your_ip_address&amp;gt;:8080&lt;/p&gt;
&lt;h3 id=&#34;installing-sesame&#34;&gt;Installing Sesame&lt;/h3&gt;
&lt;p&gt;We will use current Sesame version 2.7.14. You can download it &lt;a href=&#34;http://sourceforge.net/projects/sesame/files/Sesame%202/&#34;&gt;here&lt;/a&gt; or run following command:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;wget &lt;span style=&#34;color:#ae81ff&#34;&gt;\\&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;http://sourceforge.net/projects/sesame/files/Sesame%202/2.7.14/openrdf-sesame-2.7.14-sdk.tar.gz/download&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#ae81ff&#34;&gt;\\&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  -O openrdf-sesame-2.7.14-sdk.tar.gz
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Then extract the Sesame tarball:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;tar -xvzf openrdf-sesame-2.7.14-sdk.tar.gz
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;To deploy sesame you have to copy the two war files that are in &lt;em&gt;openrdf-sesame-2.7.14/war&lt;/em&gt; to &lt;em&gt;/var/lib/tomcat7/webapps&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;From &lt;em&gt;openrdf-sesame-2.7.14/war&lt;/em&gt; you can do it with command:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;cp openrdf-*.war &amp;lt;tomcat_install&amp;gt;/webapps
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Sesame applications write and store configuration files in a single directory and the tomcat server needs permissions for it.&lt;/p&gt;
&lt;p&gt;By default the configuration directory is: &lt;em&gt;/usr/share/tomcat7/.aduna&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Create the directory:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;sudo mkdir /usr/share/tomcat7/.aduna
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Then change the ownership:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;sudo chown tomcat7 /usr/share/tomcat7/.aduna
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;And finally you should give the necessary permissions:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;sudo chmod o+rwx /usr/share/tomcat7/.aduna
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now when you go to: http://&amp;lt;your_ip_address&amp;gt;:8080/openrdf-workbench/repositories&lt;/p&gt;
&lt;p&gt;You should get a screen like this:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;01-Sesame-repo-list.png&#34; alt=&#34;image&#34;&gt;&lt;/p&gt;
&lt;h3 id=&#34;setup-spb&#34;&gt;Setup SPB&lt;/h3&gt;
&lt;p&gt;You can download the SPB code and find brief documentation on GitHub:&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://github.com/ldbc/ldbc_spb_bm&#34;&gt;https://github.com/ldbc/ldbc_spb_bm&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;A detailed documentation is located here:&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://github.com/ldbc/ldbc_spb_bm/blob/master/doc/LDBC_SPB_v0.3.pdf&#34;&gt;https://github.com/ldbc/ldbc_spb_bm/blob/master/doc/LDBC_SPB_v0.3.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;SPB offers many configuration options which control various features of the benchmark e.g.:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;query mixes&lt;/li&gt;
&lt;li&gt;dataset size&lt;/li&gt;
&lt;li&gt;loading datasets&lt;/li&gt;
&lt;li&gt;number of agents&lt;/li&gt;
&lt;li&gt;validating results&lt;/li&gt;
&lt;li&gt;test conformance to OWL2-RL ruleset&lt;/li&gt;
&lt;li&gt;update rate of agents&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Here we demonstrate how to generate a dataset and execute a simple test&lt;br&gt;
run with it.&lt;/p&gt;
&lt;p&gt;First download the SPB source code from the repository:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;git clone https://github.com/ldbc/ldbc_spb_bm.git
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Then in the ldbc_spb_bm directory build the project:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;ant build-basic-querymix
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;If you simply execute the command:&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;ant
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;you’ll get a list of all available build configurations for the SPB test driver, but for the purpose of this step-by-step guide, configuration shown above is sufficient.&lt;/p&gt;
&lt;p&gt;Depending on generated dataset size a bigger java heap size may be required for the Sesame Store. You can change it by adding following arguments to Tomcat&amp;rsquo;s startup files e.g. in &lt;em&gt;catalina.sh&lt;/em&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;export JAVA_OPTS&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;-d64 -Xmx4G&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;To run the Benchmark you need to create a repository in the Sesame Data Store, similar to the following screenshot:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;02-Sesame-create-repo.png&#34; alt=&#34;image&#34;&gt;&lt;/p&gt;
&lt;p&gt;Then we need to point the benchmark test driver to the SPARQL endpoint of that repository. This is done in &lt;em&gt;ldbc_spb_bm/dist/test.properties&lt;/em&gt; file.&lt;/p&gt;
&lt;p&gt;The default value of &lt;em&gt;datasetSize&lt;/em&gt; in the properties is set to be 10M, but for the purpose of this guide we will decrease it to 1M.&lt;/p&gt;
&lt;p&gt;You need to change&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;datasetSize=1000000
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Also the URLs of the SPARQL endpoint for the repository&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;endpointURL=http://localhost:8080/openrdf-sesame/repositories/ldbc1
endpointUpdateURL=http://localhost:8080/openrdf-sesame/repositories/ldbc1/statements
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;First step, before measuring the performance of a triple store, is to load the reference-knowledge data, generate a 1M dataset, load it into the repository and finally generate query substitution parameters.&lt;/p&gt;
&lt;p&gt;These are the settings to do that, following parameters will &amp;lsquo;instruct&amp;rsquo; the SPB test driver to perform all the actions described above:&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;#Benchmark Operational Phases
loadOntologies=true
loadReferenceDatasets=true
generateCreativeWorks=true
loadCreativeWorks=true
generateQuerySubstitutionParameters=true
validateQueryResults=false
warmUp=false
runBenchmark=false
runBenchmarkOnlineReplicationAndBackup=false
checkConformance=false
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;To run the benchmark execute the following:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;java -jar semantic_publishing_benchmark-basic-standard.jar
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;test.properties
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;When the initial run has finished, we should have a 1M dataset loaded into the repository and a set of files with query substitution parameters.&lt;/p&gt;
&lt;p&gt;Next we will measure the performance of Sesame Data Store by changing some configuration properties:&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;#Benchmark Configuration Parameters
warmupPeriodSeconds=60
benchmarkRunPeriodSeconds=300
...
#Benchmark Operational Phases
loadOntologies=false
loadReferenceDatasets=false
generateCreativeWorks=false
loadCreativeWorks=false
generateQuerySubstitutionParameters=false
validateQueryResults=false
warmUp=true
runBenchmark=true
runBenchmarkOnlineReplicationAndBackup=false
checkConformance=false
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;After the benchmark test run has finished result files are saved in folder: &lt;em&gt;dist/logs&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;There you will find three types of results: the result summary of the benchmark run (&lt;em&gt;semantic_publishing_benchmark_results.log),&lt;/em&gt; brief results and detailed results.&lt;/p&gt;
&lt;p&gt;In &lt;em&gt;semantic_publishing_benchmark_results.log&lt;/em&gt; you will find the results distributed per seconds. They should be similar to the listing bellow:&lt;/p&gt;
&lt;p&gt;Benchmark Results for the 300-th second&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;Seconds : 300 (completed query mixes : 0)
    Editorial:
        2 agents

        9     inserts (avg : 22484   ms, min : 115     ms, max : 81389   ms)
        0     updates (avg : 0       ms, min : 0       ms, max : 0       ms)
        0     deletes (avg : 0       ms, min : 0       ms, max : 0       ms)

        9 operations (9 CW Inserts (0 errors), 0 CW Updates (1 errors), 0 CW Deletions (2 errors))
        0.0300 average operations per second

    Aggregation:
        8 agents

        2     Q1   queries (avg : 319     ms, min : 188     ms, max : 451     ms, 0 errors)
        3     Q2   queries (avg : 550     ms, min : 256     ms, max : 937     ms, 0 errors)
        1     Q3   queries (avg : 58380   ms, min : 58380   ms, max : 58380   ms, 0 errors)
        2     Q4   queries (avg : 65250   ms, min : 40024   ms, max : 90476   ms, 0 errors)
        1     Q5   queries (avg : 84220   ms, min : 84220   ms, max : 84220   ms, 0 errors)
        2     Q6   queries (avg : 34620   ms, min : 24499   ms, max : 44741   ms, 0 errors)
        3     Q7   queries (avg : 5892    ms, min : 4410    ms, max : 8528    ms, 0 errors)
        2     Q8   queries (avg : 3537    ms, min : 546     ms, max : 6528    ms, 0 errors)
        4     Q9   queries (avg : 148573  ms, min : 139078  ms, max : 169559  ms, 0 errors)
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;This step-by-step guide gave an introduction on how to setup and run the SPB on a Sesame Data Store. Further details can be found in the reference documentation listed above.&lt;/p&gt;
&lt;p&gt;If you have any troubles running the benchmark, don&amp;rsquo;t hesitate to comment or use our social media channels.&lt;/p&gt;
&lt;p&gt;In a future post we will go through some of the parameters of SPB and check their performance implications.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Semantic Publishing Instance Matching Benchmark</title>
      <link>https://ldbcouncil.org/post/semantic-publishing-instance-matching-benchmark/</link>
      <pubDate>Tue, 30 Dec 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/semantic-publishing-instance-matching-benchmark/</guid>
      <description>&lt;p&gt;The Semantic Publishing Instance Matching Benchmark (SPIMBench) is a novel benchmark for the assessment of instance matching techniques for RDF data with an associated schema. SPIMBench extends the state-of-the art instance matching benchmarks for RDF data in three main aspects: it allows for systematic scalability testing, supports a wider range of test cases including semantics-aware ones, and provides an enriched gold standard.&lt;/p&gt;
&lt;p&gt;The SPIMBench test cases provide a systematic way for testing the performance of instance matching systems in different settings. SPIMBench supports the types of test cases already adopted by existing instance matching benchmarks:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;value-based test cases based on applying value transformations (e.g., blank character addition and deletion, change of date format, abbreviations, synonyms) on triples relating to given input entity&lt;/li&gt;
&lt;li&gt;structure-based test cases characterized by a structural transformation (e.g., different nesting levels for properties, property splitting, aggregation)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The novelty of SPIMBench lies in the support for the following semantics-aware test cases defined on the basis of OWL constructs:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;instance (in)equality (owl:sameAs, owl:differentFrom)&lt;/li&gt;
&lt;li&gt;class and property equivalence (owl:equivalentClass, owl:equivalentProperty)&lt;/li&gt;
&lt;li&gt;class and property disjointness (owl:disjointWith, owl:AllDisjointClasses, owl:propertyDisjointWith, owl:AllDisjointProperties)&lt;/li&gt;
&lt;li&gt;class and property hierarchies (rdfs:subClassOf, rdfs:subPropertyOf)&lt;/li&gt;
&lt;li&gt;property constraints (owl:FunctionalProperty, owl:InverseFunctionalProperty)&lt;/li&gt;
&lt;li&gt;complex class definitions (owl:unionOf, owl:intersectionOf)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;SPIMBench uses and extends the ontologies of LDBC&amp;rsquo;s Semantic Publishing Benchmark (SPB) to tackle the more complex schema constructs expressed in terms of OWL. It also extends SPB&amp;rsquo;s data generator to first generate a synthetic source dataset that does not contain any matches, and then to generate matches and non-matches to entities of the source dataset to address the supported transformations and OWL constructs. The data generation process allows the creation of arbitrary large datasets, thus supporting the evaluation of both the scalability and the matching quality of an instance matching system.&lt;/p&gt;
&lt;p&gt;Value and structure-based test cases are implemented using the SWING framework &lt;a href=&#34;#references&#34;&gt;[1]&lt;/a&gt; on data and object type properties respectively. These are produced by applying the appropriate transformation(s) on a source instance to obtain a target instance. Semantics-based test cases are produced in the same way as with the value and structure-based test cases with the difference that appropriate triples are constructed and added in the target dataset to consider the respective OWL constructs.&lt;/p&gt;
&lt;p&gt;SPIMBench, in addition to the semantics-based test cases that differentiate it from existing instance matching benchmarks, also offers a weighted gold standard used to judge the quality of answers of instance matching systems. It contains generated matches (a pair consisting of an entity of the source dataset and an entity of the target dataset) the type of test case it represents, the property on which a transformation was applied (in the case of value-based and structure-based test cases), and a weight that quantifies how easy it is to detect this match automatically. SPIMBench adopts an information-theoretical approach by applying multi-relational learning to compute the weight of the pair of matched instances by measuring the information loss that results from applying transformations to the source data to generate the target data. This detailed information, which is not provided by state of the art benchmarks, allows users of SPIMBench (e.g., developers of IM systems) to more easily identify the reasons underlying the performance results obtained using SPIMBench and thereby supports the debugging of instance matching systems.&lt;/p&gt;
&lt;p&gt;SPIMBench can be downloaded from &lt;a href=&#34;https://github.com/jsaveta/SPIMBench&#34;&gt;our repository&lt;/a&gt;.&lt;/p&gt;
&lt;h4 id=&#34;references&#34;&gt;References&lt;/h4&gt;
&lt;p&gt;[1] A. Ferrara, S. Montanelli, J. Noessner, and H. Stuckenschmidt. Benchmarking Matching Applications on the Semantic Web. In ESWC, 2011.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Further Developments in SNB BI Workload</title>
      <link>https://ldbcouncil.org/post/further-developments-in-snb-bi-workload/</link>
      <pubDate>Thu, 18 Dec 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/further-developments-in-snb-bi-workload/</guid>
      <description>&lt;p&gt;We are presently working on the SNB BI workload. Andrey Gubichev of TU Munchen and myself are going through the queries and are playing with two SQL based implementations, one on Virtuoso and the other on Hyper.&lt;/p&gt;
&lt;p&gt;As discussed before, the BI workload has the same choke points as TPC-H as a base but pushes further in terms of graphiness and query complexity.&lt;/p&gt;
&lt;p&gt;There are obvious marketing applications for a SNB-like dataset. There are also security related applications, ranging from fraud detection to intelligence analysis. The latter category is significant but harder to approach, as much of the detail of best practice is itself not in the open. In this post, I will outline some ideas discussed over time that might cristallize into a security related section in the SNB BI workload. We invite comments from practitioners for making the business questions more relevant while protecting sensitive details.&lt;/p&gt;
&lt;p&gt;Let’s look at what scenarios would fit with the dataset. We have people, different kinds of connections between people, organizations, places and messages. Messages (posts/replies), people and organizations are geo-tagged. Making a finer level of geo-tagging, with actual GPS coordinates, travel itineraries etc, all referring to real places would make the data even more interesting. The geo dimension will be explored separately in a forthcoming post.&lt;/p&gt;
&lt;p&gt;One of the first things to appear when approaching the question isthat the analysis of behavior patterns over time is not easily captured in purely declarative queries. For example, temporal sequence of events and the quantity and quality of interactions between players leads to intractably long queries which are hard to understand and debug. Therefore, views and intermediate materializations become increasingly necessary.&lt;/p&gt;
&lt;p&gt;Another feature of the scene is that information is never complete. Even if logs are complete for any particular system, there are always possible interactions outside of the system. Therefore we tend to get match scores more then strictly Boolean conditions. Since everybody is related to everybody else via a relative short path, the nature and stremgth of the relationship is key to interpreting its significance.&lt;/p&gt;
&lt;p&gt;Since a query consisting of scores and outer joins only is difficult to interpret and optimize, and since the information is seldom complete, some blanks may have to be filled in by guesses. The database must therefore contain metadata about this.&lt;/p&gt;
&lt;p&gt;An orthogonal aspect to security applications is the access control of the database itself. One might assume that if a data warehouse of analyzable information is put together, the analyst would have access to the entirety of it. This is however not necessarily the case since the information itself and its provenance may fall under different compartments.&lt;/p&gt;
&lt;p&gt;So, let’s see how some of these aspects could be captured in the SNB context.&lt;/p&gt;
&lt;p&gt;Geography - We materialize a table of travel events, so that an unbroken sequence of posts from the same location (e.g. country) other than the residence of the poster forms a travel event. The posts may have a fine grained position (IP, GPS coordinates of photos) that marks an itinerary. This is already beyond basicSQL, needing a procedure or window functions.&lt;/p&gt;
&lt;p&gt;The communication between people is implicit in reply threads and forum memberships. A reply is the closest that one comes to a person to person message in the dataset. Otherwise all content is posted to forumns with more or less participants. Membership in a high traffic forum with few participants would indicate a strong connection. Calculating these time varying connection strengths is a lot of work and a lot of text in queries. Keeping things simple requires materializing a sparse “adjacency cube,” i.e. a relation of person1, person2, time bucket -&amp;gt; connection strength. In the SNB case the connection strength may be derived from reciprocal replies, likes, being in the same forums, knowing each other etc. Selectivity is important, i.e. being in many small forumns together counts for more than being in ones where everybody else also participates.&lt;/p&gt;
&lt;p&gt;The behaviors of people in SNB is not identical from person to person but for the same person follows a preset pattern. Suppose a question like “ which person with access to secrets has a marked change of online behavior?” The change would be starting or stopping communication with a given set of people, for example. Think that the spy meets the future spymaster in a public occasion, has a series of exchanges, travels to an atypical destination, then stops all open contact with the spymaster or related individuals. Patterns like this do not occur in the data but can be introduced easily enough.&lt;/p&gt;
&lt;p&gt;In John Le Carre’s A Perfect Spy the main character is caught because it comes to light that his travel routes near always corresponded to his controller’s. This would make a query. This could be cast in marketing terms as a “(un)common shopping basket.”&lt;/p&gt;
&lt;p&gt;Analytics becomes prediction when one part of a pattern exists without the expected next stage. Thus the same query template can serve for detecting full or partial instances of a pattern, depending on how the scores are interpreted.&lt;/p&gt;
&lt;p&gt;From a database angle, these questions group on an item with internal structure. For the shopping basket this is a set. For the travel routes this is an ordered sequence of space/time points, with a match tolerance on the spatial and temporal elements. Another characteristic is that there is a baseline of expectations and the actual behavior. Both have structure, e.g. the occupation/location/interest/age of one’s social circle. These need to be condensed into a sort of metric space and then changes and rates of change can be observed. Again, this calls for a multidimensional cube to be created as a summary, then algorithms to be applied to this. The declarative BI query a la TPC-H does not easily capture this all.&lt;/p&gt;
&lt;p&gt;This leads us to graph analytics in a broader sense. Some of the questions addressed here will still fit in the materialized summaries+declarative queries pattern but the more complex summarization and clustering moves towards iterative algorithms.&lt;/p&gt;
&lt;p&gt;There is at present a strong interest in developing graph analytics benchmarks in LDBC. This is an activity that extends beyond the FP7 project duration and beyond the initial partners. To this effect I have implemented some SQL extensions for BSP style processing, as hinted at on my blog. These will be covered in more detail in January, when there are actual experiments.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Sizing AWS Instances for the Semantic Publishing Benchmark</title>
      <link>https://ldbcouncil.org/post/sizing-aws-instances-for-the-semantic-publishing-benchmark/</link>
      <pubDate>Wed, 17 Dec 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/sizing-aws-instances-for-the-semantic-publishing-benchmark/</guid>
      <description>&lt;p&gt;LDBC&amp;rsquo;s &lt;a href=&#34;https://ldbcouncil.org/developer/spb&#34;&gt;Semantic Publishing Benchmark&lt;/a&gt; (SPB) measures the performance of an RDF database in a load typical for metadata-based content publishing, such as the well-known &lt;a href=&#34;https://web.archive.org/web/20140420081627/https://www.bbc.co.uk/blogs/legacy/bbcinternet/2010/07/bbc_world_cup_2010_dynamic_sem.html&#34;&gt;BBC Dynamic Semantic Publishing&lt;/a&gt; scenario. Such load combines tens of updates per second (e.g. adding metadata about new articles) with even higher volume of read requests (SPARQL queries collecting recent content and data to generate web page on a specific subject, e.g. Frank Lampard). As we &lt;a href=&#34;https://ldbcouncil.org/post/using-ldbc-spb-to-find-owlim-performance-issues&#34;&gt;wrote earlier&lt;/a&gt;, SPB was already successfully used to help developers to identify performance issues and to introduce optimizations in SPARQL engines such as GraphDB and Virtuoso. Now we are at the point to experiment with different sizes of the benchmark and different hardware configurations.&lt;/p&gt;
&lt;p&gt;Lately we tested different Amazon Web Services (&lt;a href=&#34;https://aws.amazon.com/&#34;&gt;AWS&lt;/a&gt;) instance types for running SPB basic interactive query mix in parallel with the standard editorial updates – precisely the type of workload that &lt;a href=&#34;https://www.ontotext.com/products/ontotext-graphdb/&#34;&gt;GraphDB&lt;/a&gt; experiences in the backend of BBC Sport website. We discovered and report below a number of practical guidelines about the optimal instance types and configurations. We have proven that SPB 50M workloads can be executed efficiently on a mid-sized AWS instance – c3.2xlarge machine executes 16 read queries and 15 update operations per second. For $1 paid to Amazon for such instance GraphDB executes 140 000 queries and 120 000 updates. The most interesting discovery in this experiment is that if BBC were hosting the triplestore behind their Dynamic Semantic Publishing architecture at AWS, the total cost of the server infrastructure behind their Worldcup 2010 website would have been about $80/day.&lt;/p&gt;
&lt;h3 id=&#34;the-experiment&#34;&gt;The Experiment&lt;/h3&gt;
&lt;p&gt;For our tests we use:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;GraphDB Standard v6.1&lt;/li&gt;
&lt;li&gt;LDBC-SPB test driver (version 0.1.dc9a626 from 10.Nov.2014) configured as follows:
&lt;ul&gt;
&lt;li&gt;8 aggregation agents (read threads) and 2 editorial agents (write threads); for some configurations we experimented with different numbers of agents also&lt;/li&gt;
&lt;li&gt;50M dataset (SF1)&lt;/li&gt;
&lt;li&gt;40 minutes of benchmark run time (60 seconds of warm up)&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;5 different Amazon EC2 instances and one local server&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Each test run is cold, i.e. data is newly loaded for each run. We set a 5 GByte cache configuration, which is sufficient for the size of the generated dataset. We use the same query substitution parameters (the same randomization seed) for every run, so that we are sure that all test runs are identical.&lt;/p&gt;
&lt;p&gt;We use two types of instances – M3 and C3 instances. They both provide SSD storage for fast I/O performance. The M3 instances are with E5-2670v2, 2.50GHz CPU and provide good all-round performance, while the C3 instances are compute optimized with stronger CPU – E5-2680v2, 2.80GHz, but have half as much memory as the M3.&lt;/p&gt;
&lt;p&gt;We also use a local physical server with dual-CPU – E5-2650v2, 2.60Ghz; 256GB of RAM and RAID-0 array of SSD in order to provide ground for interpretation of the performance for the virtualized AWS instances. The CPU capacity of the AWS instances is measured in vCPUs (virtual CPU). A vCPU is a logical core – one hyper-thread of one physical core of the corresponding Intel Xeon processor used by Amazon. This means that a vCPU represents roughly half a physical core, even though the performance of a hyper-threaded core is not directly comparable with two non-hyper-threaded cores. We should keep this in mind comparing AWS instances to physical machines, i.e. our local server with two CPUs with 8 physical cores each has 32 logical cores, which is more than c3.4xlarge instance with 16 vCPUs.&lt;/p&gt;
&lt;h3 id=&#34;the-results&#34;&gt;The Results&lt;/h3&gt;
&lt;p&gt;For the tests we measured:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;em&gt;queries/s&lt;/em&gt; for the read threads, where queries include SELECT and CONSTRUCT&lt;/li&gt;
&lt;li&gt;&lt;em&gt;updates/s&lt;/em&gt; for the write threads, where an update operation is INSERT or DELETE&lt;/li&gt;
&lt;li&gt;&lt;em&gt;queries/$&lt;/em&gt; and &lt;em&gt;updates/$&lt;/em&gt; – respectively queries or updates per dollar is calculated for each AWS instance type based on price and update throughput&lt;/li&gt;
&lt;li&gt;&lt;em&gt;update/vCPU&lt;/em&gt; – modification operations per vCPU per second&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Results (Table 1.) provide strong evidence that performance depends mostly on processor power. This applies to both queries and updates - which in the current AWS setup go on par with one another. Comparing M3 and C3 instances with equal vCPUs we can see that performance is only slightly higher for the M3 machines and even lower for selects with 8 vCPUs. Taking into account the lower price of C3 because of their lower memory, it is clear that C3 machines are better suited for this type of workload and the sweet spot between price and performance is c3.2xlarge machine.&lt;/p&gt;
&lt;p&gt;The improvement in performance between the c3.xlarge and c3.2xlarge is more than twofold where the improvement between c3.2xlarge and c3.4xlarge is considerably lower. We also observe slower growth between c3.4xlarge and the local server machine. This is an indication that for SPB at this scale the difference between 7.5GB and 15GB of RAM is substantial, but RAM above this amount cannot be utilized efficiently by GraphDB.&lt;/p&gt;
&lt;p&gt;Table 1. SPB Measurement Results on AWS and Local Servers&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Server Type&lt;/th&gt;
&lt;th&gt;vCPUs&lt;/th&gt;
&lt;th&gt;R/W Agents&lt;/th&gt;
&lt;th&gt;RAM (GB)&lt;/th&gt;
&lt;th&gt;&amp;ldquo;Storage (GB, SSD)&amp;rdquo;&lt;/th&gt;
&lt;th&gt;Price USD/h&lt;/th&gt;
&lt;th&gt;Queries/ sec.&lt;/th&gt;
&lt;th&gt;Updates/ sec.&lt;/th&gt;
&lt;th&gt;Queries/ USD&lt;/th&gt;
&lt;th&gt;Updates/ USD&lt;/th&gt;
&lt;th&gt;Updates/ vCPU&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;m3.xlarge&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;8/2&lt;/td&gt;
&lt;td&gt;15&lt;/td&gt;
&lt;td&gt;2x 40&lt;/td&gt;
&lt;td&gt;0.28&lt;/td&gt;
&lt;td&gt;8.39&lt;/td&gt;
&lt;td&gt;8.23&lt;/td&gt;
&lt;td&gt;107 882&lt;/td&gt;
&lt;td&gt;105 873&lt;/td&gt;
&lt;td&gt;2.06&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;m3.2xlarge&lt;/td&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;8/2&lt;/td&gt;
&lt;td&gt;30&lt;/td&gt;
&lt;td&gt;2x 80&lt;/td&gt;
&lt;td&gt;0.56&lt;/td&gt;
&lt;td&gt;15.44&lt;/td&gt;
&lt;td&gt;15.67&lt;/td&gt;
&lt;td&gt;99 282&lt;/td&gt;
&lt;td&gt;100 752&lt;/td&gt;
&lt;td&gt;1.96&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;c3.xlarge&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;8/2&lt;/td&gt;
&lt;td&gt;7.5&lt;/td&gt;
&lt;td&gt;2x 40&lt;/td&gt;
&lt;td&gt;0.21&lt;/td&gt;
&lt;td&gt;7.17&lt;/td&gt;
&lt;td&gt;6.78&lt;/td&gt;
&lt;td&gt;122 890&lt;/td&gt;
&lt;td&gt;116 292&lt;/td&gt;
&lt;td&gt;1.7&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;c3.2xlarge&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;8&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;8/2&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;15&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;2x 80&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.42&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;16.46&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;14.56&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;141 107&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;124 839&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;1.82&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;c3.4xlarge&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;16&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;8/2&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;30&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;2x 160&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.84&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;23.23&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;21.17&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;99 578&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;90 736&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;1.32&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;c3.4xlarge&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;8/3&lt;/td&gt;
&lt;td&gt;30&lt;/td&gt;
&lt;td&gt;2x 160&lt;/td&gt;
&lt;td&gt;0.84&lt;/td&gt;
&lt;td&gt;22.89&lt;/td&gt;
&lt;td&gt;20.39&lt;/td&gt;
&lt;td&gt;98 100&lt;/td&gt;
&lt;td&gt;87 386&lt;/td&gt;
&lt;td&gt;1.27&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;c3.4xlarge&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;10/2&lt;/td&gt;
&lt;td&gt;30&lt;/td&gt;
&lt;td&gt;2x 160&lt;/td&gt;
&lt;td&gt;0.84&lt;/td&gt;
&lt;td&gt;26.6&lt;/td&gt;
&lt;td&gt;19.11&lt;/td&gt;
&lt;td&gt;114 000&lt;/td&gt;
&lt;td&gt;81 900&lt;/td&gt;
&lt;td&gt;1.19&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;c3.4xlarge&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;10/3&lt;/td&gt;
&lt;td&gt;30&lt;/td&gt;
&lt;td&gt;2x 160&lt;/td&gt;
&lt;td&gt;0.84&lt;/td&gt;
&lt;td&gt;26.19&lt;/td&gt;
&lt;td&gt;19.18&lt;/td&gt;
&lt;td&gt;112 243&lt;/td&gt;
&lt;td&gt;82 200&lt;/td&gt;
&lt;td&gt;1.2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;c3.4xlarge&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;16&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;14/2&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;30&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;2x 160&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.84&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;30.84&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;16.88&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;132 171&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;72 343&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;1.06&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;c3.4xlarge&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;14/3&lt;/td&gt;
&lt;td&gt;30&lt;/td&gt;
&lt;td&gt;2x 160&lt;/td&gt;
&lt;td&gt;0.84&lt;/td&gt;
&lt;td&gt;29.67&lt;/td&gt;
&lt;td&gt;17.8&lt;/td&gt;
&lt;td&gt;127 157&lt;/td&gt;
&lt;td&gt;76 286&lt;/td&gt;
&lt;td&gt;1.11&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Local&lt;/td&gt;
&lt;td&gt;32&lt;/td&gt;
&lt;td&gt;8/2&lt;/td&gt;
&lt;td&gt;256&lt;/td&gt;
&lt;td&gt;8x 256&lt;/td&gt;
&lt;td&gt;0.85&lt;/td&gt;
&lt;td&gt;37.11&lt;/td&gt;
&lt;td&gt;32.04&lt;/td&gt;
&lt;td&gt;156 712&lt;/td&gt;
&lt;td&gt;135 302&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Local&lt;/td&gt;
&lt;td&gt;32&lt;/td&gt;
&lt;td&gt;8/3&lt;/td&gt;
&lt;td&gt;256&lt;/td&gt;
&lt;td&gt;8x 256&lt;/td&gt;
&lt;td&gt;0.85&lt;/td&gt;
&lt;td&gt;37.31&lt;/td&gt;
&lt;td&gt;32.07&lt;/td&gt;
&lt;td&gt;157 557&lt;/td&gt;
&lt;td&gt;135 429&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Local&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;32&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;10/2&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;256&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;8x 256&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.85&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;40&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;31.01&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;168 916&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;130 952&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.97&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Local&lt;/td&gt;
&lt;td&gt;32&lt;/td&gt;
&lt;td&gt;14/2&lt;/td&gt;
&lt;td&gt;256&lt;/td&gt;
&lt;td&gt;8x 256&lt;/td&gt;
&lt;td&gt;0.85&lt;/td&gt;
&lt;td&gt;36.39&lt;/td&gt;
&lt;td&gt;26.42&lt;/td&gt;
&lt;td&gt;153 672&lt;/td&gt;
&lt;td&gt;111 569&lt;/td&gt;
&lt;td&gt;0.83&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Local&lt;/td&gt;
&lt;td&gt;32&lt;/td&gt;
&lt;td&gt;14/3&lt;/td&gt;
&lt;td&gt;256&lt;/td&gt;
&lt;td&gt;8x 256&lt;/td&gt;
&lt;td&gt;0.85&lt;/td&gt;
&lt;td&gt;36.22&lt;/td&gt;
&lt;td&gt;26.39&lt;/td&gt;
&lt;td&gt;152 954&lt;/td&gt;
&lt;td&gt;111 443&lt;/td&gt;
&lt;td&gt;0.82&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Local&lt;/td&gt;
&lt;td&gt;32&lt;/td&gt;
&lt;td&gt;20/2&lt;/td&gt;
&lt;td&gt;256&lt;/td&gt;
&lt;td&gt;8x 256&lt;/td&gt;
&lt;td&gt;0.85&lt;/td&gt;
&lt;td&gt;34.59&lt;/td&gt;
&lt;td&gt;23.86&lt;/td&gt;
&lt;td&gt;146 070&lt;/td&gt;
&lt;td&gt;100 759&lt;/td&gt;
&lt;td&gt;0.75&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h3 id=&#34;the-optimal-number-of-test-agents&#34;&gt;The Optimal Number of Test Agents&lt;/h3&gt;
&lt;p&gt;Experimenting with different number of aggregation (read) and editorial (write) agents at c3.4xlarge and the local server, we made some interesting observations:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;There is almost no benefit to use more than 2 write agents. This can be explained by the fact that certain aspects of handling writes in GraphDB are serialized, i.e. they cannot be executed in parallel across multiple write threads;&lt;/li&gt;
&lt;li&gt;Using more read agents can have negative impact on update performance. This is proven by the c3.4xlarge results with 8/2 and with 14/2 agents - while in the later case GraphDB handles a bit higher amount of queries (31 vs. 23) we see a drop in the updates rates (from 21 to 17);&lt;/li&gt;
&lt;li&gt;Overall, the configuration with 8 read agents and 2 write agents delivers good balanced results across various hardware configurations;&lt;/li&gt;
&lt;li&gt;For machines with more than 16 cores, a configuration like 10/2 or 14/2, would maximize the number of selects, still with good update rates. This way one can get 30 queries/sec. on c3.4xlarge and 40 queries/sec. on a local server;&lt;/li&gt;
&lt;li&gt;Launching more than 14 read agents does not help even on local server with 32 logical cores. This indicates that at this point we are reaching some constraints such as memory bandwidth or IO throughput and degree of parallelization.&lt;/li&gt;
&lt;li&gt;There is some overhead when handling bigger number of agents as the results for the local server tests with 14/3 and 20/2 show the worst results for both queries and updates.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;efficiency-and-cost&#34;&gt;Efficiency and Cost&lt;/h3&gt;
&lt;p&gt;AWS instance type c3.2xlarge provides the best price/performance ratio for applications where 15 updates/sec. are sufficient even at peak times. More intensive applications should use type c3.4xlarge, which guarantees more than 20 updates/sec.&lt;/p&gt;
&lt;p&gt;Cloud infrastructure providers like Amazon, allow one to have a very clear account of the full cost for the server infrastructure, including hardware, hosting, electricity, network, etc.&lt;/p&gt;
&lt;p&gt;$1 spent on c3.2xlarge ($0.41/hour) allows for handling 140 000 queries, along with more than 120 000 update operations!&lt;/p&gt;
&lt;p&gt;The full cost of the server infrastructure is harder to compute in the case of purchasing a server and hosting it in a proprietary data center. Still, one can estimate the upper limits - for machine, like the local server used in this benchmark, this price is way lower than $1/hour. One should consider that this machine is with 256GB of RAM, which is an overkill for Semantic Publishing Benchmark ran at 50M scale. Under all these assumptions we see that using local server is cheaper than the most cost-efficient AWS instance. This is expected - owning a car is always cheaper than renting it for 3 years in a row. Actually, the fact that the difference of the prices/query in this case are low indicates that using AWS services comes at very low extra cost.&lt;/p&gt;
&lt;p&gt;To put these figures in the context of a known real world application, let us model the case of a GraphDB Enterprise replication cluster with 2 master nodes and 6 worker nodes - the size of cluster that BBC used for their FIFA Worldcup 2010 project. Given c3.2xlarge instance type, the math works as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;100 queries/sec.&lt;/strong&gt; handled by the cluster. This means about 360 000 queries per hour or more than 4 million queries per day. This is at least 2 times more than the actual loads of GraphDB at BBC during the peak times of big sports events.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;10 updates/sec.&lt;/strong&gt; - the speed of updates in GraphDB Enterprise cluster is lower than the speed of each worker node in separation. There are relatively few content management applications that need more than 36 000 updates per hour.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;$81/day&lt;/strong&gt; is the full cost for the server infrastructure. This indicates an annual operational cost for cluster of this type in the range of $30 000, even without any effort to release some of the worker nodes in non-peak times.&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>DATAGEN: a Realistic Social Network Data Generator</title>
      <link>https://ldbcouncil.org/post/datagen-a-realistic-social-network-data-generator/</link>
      <pubDate>Sat, 06 Dec 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/datagen-a-realistic-social-network-data-generator/</guid>
      <description>&lt;p&gt;In previous posts (&lt;a href=&#34;https://ldbcouncil.org/post/getting-started-with-snb&#34;&gt;Getting started with snb&lt;/a&gt;, &lt;a href=&#34;https://ldbcouncil.org/post/datagen-data-generation-for-the-social-network-benchmark&#34;&gt;DATAGEN: data generation for the Social Network Benchmark&lt;/a&gt;), Arnau Prat discussed the main features and characteristics of DATAGEN: &lt;em&gt;realism&lt;/em&gt;, &lt;em&gt;scalability&lt;/em&gt;, &lt;em&gt;determinism&lt;/em&gt;, &lt;em&gt;usability&lt;/em&gt;. DATAGEN is the social network data generator used by the three LDBC-SNB workloads, which produces data simulating the activity in a social network site during a period of time. In this post, we conduct a series of experiments that will shed some light on how realistic data produced by DATAGEN looks. For our testing, we generated a dataset of scale factor 10 (i.e., social network of 73K users during 3 years) and loaded it into Virtuoso by following the &lt;a href=&#34;https://github.com/ldbc/ldbc_snb_datagen&#34;&gt;instructions for generating a SNB dataset&lt;/a&gt; and for loading the dataset into Virtuoso. In the following sections, we analyze several aspects of the generated dataset.&lt;/p&gt;
&lt;h3 id=&#34;a-realistic-social-graph&#34;&gt;A Realistic social graph&lt;/h3&gt;
&lt;p&gt;One of the most complexly structured graphs that can be found in the data produced by DATAGEN is the friends graph, formed by people and their &lt;em&gt;&lt;knows&gt;&lt;/em&gt; relationships. We used the R script after Figure 1 to draw the social degree distribution in the SNB friends graph. As shown in Figure 1, the cumulative social degree distribution of the friends graph is similar to that from Facebook (See the note about &lt;a href=&#34;https://www.facebook.com/notes/facebook-data-team/anatomy-of-facebook/10150388519243859&#34;&gt;Facebook Anatomy&lt;/a&gt;). This is not by chance, as DATAGEN has been designed to deliberately reproduce the Facebook&amp;rsquo;s graph distribution.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;Cumulative-distribution.png&#34; alt=&#34;image&#34;&gt; &lt;br&gt;
Figure 1: Cumulative distribution #friends per user&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;#R script for generating the social degree distribution &lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#75715e&#34;&gt;#Input files: person_knows_person_*.csv&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;library&lt;/span&gt;(data.table)
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;library&lt;/span&gt;(igraph)
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;library&lt;/span&gt;(plotrix)
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;require&lt;/span&gt;(bit64)
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;dflist &lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;lapply&lt;/span&gt;(&lt;span style=&#34;color:#a6e22e&#34;&gt;commandArgs&lt;/span&gt;(trailingOnly &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;TRUE&lt;/span&gt;), fread, sep&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;|&amp;#34;&lt;/span&gt;,
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  header&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;T, select&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#f92672&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#ae81ff&#34;&gt;2&lt;/span&gt;, colClasses&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;integer64&amp;#34;&lt;/span&gt;)
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  df &lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#a6e22e&#34;&gt;rbindlist&lt;/span&gt;(dflist) &lt;span style=&#34;color:#a6e22e&#34;&gt;setNames&lt;/span&gt;(df, &lt;span style=&#34;color:#a6e22e&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;P1&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;P2&amp;#34;&lt;/span&gt;))
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;d2 &lt;span style=&#34;color:#f92672&#34;&gt;&amp;lt;-&lt;/span&gt; df[,&lt;span style=&#34;color:#a6e22e&#34;&gt;length&lt;/span&gt;(P2),by&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;P1]
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;pdf&lt;/span&gt;(&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;socialdegreedist.pdf&amp;#34;&lt;/span&gt;)
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;plot&lt;/span&gt;(&lt;span style=&#34;color:#a6e22e&#34;&gt;ecdf&lt;/span&gt;(d2&lt;span style=&#34;color:#f92672&#34;&gt;$&lt;/span&gt;V1),main&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Cummulative distribution #friends per user&amp;#34;&lt;/span&gt;,
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  xlab&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Number of friends&amp;#34;&lt;/span&gt;, ylab&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;Percentage number of users&amp;#34;&lt;/span&gt;, log&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;x&amp;#34;&lt;/span&gt;,
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  xlim&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ae81ff&#34;&gt;0.8&lt;/span&gt;, &lt;span style=&#34;color:#a6e22e&#34;&gt;max&lt;/span&gt;(d2&lt;span style=&#34;color:#f92672&#34;&gt;$&lt;/span&gt;V1) &lt;span style=&#34;color:#f92672&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#ae81ff&#34;&gt;20&lt;/span&gt;))
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;dev.off&lt;/span&gt;()
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id=&#34;data-correlations&#34;&gt;Data Correlations&lt;/h3&gt;
&lt;p&gt;Data in real life as well as in a real social network is correlated; e.g. names of people living in Germany have a different distribution than those living in Netherlands, people who went to the same university in the same period have a much higher probability to be friends and so on and so forth. In this experiment we will analyze if data produced by DATAGEN also reproduces these phenomena.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Which are the most popular names of a country?&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;We run the following query on the database built in Virtuoso, which computes the distribution of the names of the people for a given country. In this query, &lt;em&gt;&amp;lsquo;A_country_name&amp;rsquo;&lt;/em&gt; is the name of a particular country such as  &lt;em&gt;&amp;lsquo;Germany&amp;rsquo;, &amp;lsquo;Netherlands&amp;rsquo;, or &amp;lsquo;Vietnam&amp;rsquo;&lt;/em&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-sql&#34; data-lang=&#34;sql&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;SELECT&lt;/span&gt; p_lastname, &lt;span style=&#34;color:#66d9ef&#34;&gt;count&lt;/span&gt; (p_lastname) &lt;span style=&#34;color:#66d9ef&#34;&gt;as&lt;/span&gt; namecnt 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;FROM&lt;/span&gt; person, country 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;WHERE&lt;/span&gt; p_placeid &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; ctry_city   
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;and&lt;/span&gt; ctry_name &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;A_country_name&amp;#39;&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;GROUP&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;BY&lt;/span&gt; p_lastname &lt;span style=&#34;color:#66d9ef&#34;&gt;order&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;by&lt;/span&gt; namecnt &lt;span style=&#34;color:#66d9ef&#34;&gt;desc&lt;/span&gt;;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;As we can see from Figures 2, 3, and 4, the distributions of names in Germany, Netherlands and Vietnam are different. A name that is popular in Germany such as &lt;em&gt;Muller&lt;/em&gt; is not popular in the Netherlands, and it even does not appear in the names of people in Vietnam.  We note that the names&amp;rsquo; distribution may not be exactly the same as the contemporary names&amp;rsquo; distribution in these countries, since the names resource files used in DATAGEN are extracted from Dbpedia, which may contain names from different periods of time.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;distribution-germany.png&#34; alt=&#34;image&#34;&gt; &lt;br&gt;
Figure 2. Distribution of names in Germany&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;distribution-netherlands.png&#34; alt=&#34;&#34;&gt; &lt;br&gt;
Figure 3. Distribution of names in Netherlands&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;distribution-vietnam.png&#34; alt=&#34;&#34;&gt; &lt;br&gt;
Figure 4. Distribution of names in Vietnam&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Where my friends are living?&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;We run the following query, which computes the locations of the friends of people living in China.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-sql&#34; data-lang=&#34;sql&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;SELECT&lt;/span&gt; top &lt;span style=&#34;color:#ae81ff&#34;&gt;10&lt;/span&gt; fctry.ctry_name, &lt;span style=&#34;color:#66d9ef&#34;&gt;count&lt;/span&gt; (&lt;span style=&#34;color:#f92672&#34;&gt;*&lt;/span&gt;) &lt;span style=&#34;color:#66d9ef&#34;&gt;from&lt;/span&gt; person &lt;span style=&#34;color:#66d9ef&#34;&gt;self&lt;/span&gt;, person
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;friend, country pctry, knows, country fctry 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;WHERE&lt;/span&gt; pctry.ctry_name &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;China&amp;#39;&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;and&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;self&lt;/span&gt;.p_placeid &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; pctry.ctry_city 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;and&lt;/span&gt; k_person1id &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;self&lt;/span&gt;.p_personid &lt;span style=&#34;color:#66d9ef&#34;&gt;and&lt;/span&gt; friend.p_personid &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; k_person2id 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;and&lt;/span&gt; fctry.ctry_city &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; friend.p_placeid 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;GROUP&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;BY&lt;/span&gt; fctry.ctry_name &lt;span style=&#34;color:#66d9ef&#34;&gt;ORDER&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;BY&lt;/span&gt; &lt;span style=&#34;color:#ae81ff&#34;&gt;2&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;desc&lt;/span&gt;;    
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;As shown in the graph, most of the friends of people living in China are also living in China. The rest comes predominantly from near-by countries such as India, Vietnam.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;chinese-friends.png&#34; alt=&#34;&#34;&gt; &lt;br&gt;
Figure 5. Locations of friends of people in China&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Where my friends are studying?&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Finally, we run the following query to find where the friends of people studying at a specific university (e.g., “Hangzhou_International_School”) are studying at.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-sql&#34; data-lang=&#34;sql&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;SELECT&lt;/span&gt; top &lt;span style=&#34;color:#ae81ff&#34;&gt;10&lt;/span&gt; o2.o_name, &lt;span style=&#34;color:#66d9ef&#34;&gt;count&lt;/span&gt;(o2.o_name) &lt;span style=&#34;color:#66d9ef&#34;&gt;from&lt;/span&gt; knows, person_university
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;p1, person_university p2, organisation o1, organisation o2 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;WHERE&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  p1.pu_organisationid &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; o1.o_organisationid 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;and&lt;/span&gt; o1.o_name&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;Hangzhou_International_School&amp;#39;&lt;/span&gt; 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;and&lt;/span&gt; k_person1id &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; p1.pu_personid &lt;span style=&#34;color:#66d9ef&#34;&gt;and&lt;/span&gt; p2.pu_personid &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; k_person2id 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;  &lt;span style=&#34;color:#66d9ef&#34;&gt;and&lt;/span&gt; p2.pu_organisationid &lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt; o2.o_organisationid 
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#66d9ef&#34;&gt;GROUP&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;BY&lt;/span&gt; o2.o_name &lt;span style=&#34;color:#66d9ef&#34;&gt;ORDER&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;BY&lt;/span&gt; &lt;span style=&#34;color:#ae81ff&#34;&gt;2&lt;/span&gt; &lt;span style=&#34;color:#66d9ef&#34;&gt;desc&lt;/span&gt;;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;As we see from Figure 6, most of the friends of the Hangzhou International School students also study at that university. This is a realistic correlation, as people studying at the same university have a much higher probability to be friends. Furthermore, top-10 universities for the friends of the Hangzhou School students’ are from China, while people from foreign universities have small number of friends that study in Hangzhou School (See Table 1).&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;friends-international-school.png&#34; alt=&#34;&#34;&gt; &lt;br&gt;
Figure 6. Top-10 universities where the friends of Hangzhou International School students are studying at.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Name&lt;/th&gt;
&lt;th&gt;# of friends&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Hangzhou_International_School&lt;/td&gt;
&lt;td&gt;12696&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Anhui_University_of_Science_and_Technology&lt;/td&gt;
&lt;td&gt;4071&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;China_Jiliang_University&lt;/td&gt;
&lt;td&gt;3519&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&amp;hellip;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Darmstadt_University_of_Applied_Sciences&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Calcutta_School_of_Tropical_Medicine&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Chettinad_Vidyashram&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Women&amp;rsquo;s_College_Shillong&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Universitas_Nasional&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Table 1. Universities where friends of Hangzhou International School students are studying at.&lt;/p&gt;
&lt;p&gt;In a real social network, data is riddled with many more correlations; it is a true data mining task to extract these.  Even though DATAGEN may not be able to model all the real life data correlations, it can generate a dataset that reproduce many of those important characteristics found in a real social network, and additionally introduce a series of plausible correlations in it. More and more interesting data correlations may also be found from playing with the SNB generated data.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>SNB Driver - Part 1</title>
      <link>https://ldbcouncil.org/post/snb-driver-part-1/</link>
      <pubDate>Thu, 27 Nov 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/snb-driver-part-1/</guid>
      <description>&lt;p&gt;In this multi-part blog we consider the challenge of running the LDBC Social Network Interactive Benchmark (LDBC SNB) workload in parallel, i.e. the design of the workload driver that will issue the queries against the System Under Test (SUT). We go through design principles that were implemented for the LDBC SNB workload generator/load tester (simply referred to as driver). Software and documentation for this driver is available here: &lt;a href=&#34;https://github.com/ldbc/ldbc_driver/&#34;&gt;https://github.com/ldbc/ldbc_driver/&lt;/a&gt;. Multiple reference implementations by two vendors are available here: &lt;a href=&#34;https://github.com/ldbc/ldbc_snb_implementations&#34;&gt;https://github.com/ldbc/ldbc_snb_implementations&lt;/a&gt;, and discussion of the schema, data properties, and related content is available here: &lt;a href=&#34;https://github.com/ldbc/ldbc_snb_docs&#34;&gt;https://github.com/ldbc/ldbc_snb_docs&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The following will concentrate on key decisions and techniques that were developed to support scalable, repeatable, distributed workload execution.&lt;/p&gt;
&lt;h3 id=&#34;problem-description&#34;&gt;Problem Description&lt;/h3&gt;
&lt;p&gt;The driver generates a stream of operations (e.g. create user, create post, create comment, retrieve person&amp;rsquo;s posts etc.) and then executes them using the provided database connector. To be capable of generating heavier loads, it executes the operations from this stream in parallel. If there were no dependencies between operations (e.g., reads that depend on the completion of writes) this would be trivial. This is the case, for example, for the classical TPC-C benchmark, where splitting transaction stream into parallel clients (terminals) is trivial. However, for LDBC SNB Interactive Workload this is not the case: some operations within the stream do depend on others, others are depended on, some both depend on others and are depended on, and some neither depend on others nor are they depended on.&lt;/p&gt;
&lt;p&gt;Consider, for example, a Social Network Benchmark scenario, where the data generator outputs a sequence of events such as User A posted a picture, User B left a comment to the picture of User A, etc. The second event depends on the first one in a sense that there is a causal ordering between them: User B can only leave a comment on the picture once it has been posted. The generated events are already ordered by their time stamp, so in case of the single-threaded execution this ordering is observed by default: the driver issues a request to the SUT with the first event (i.e., User A posts a picture), after its completion it issues the second event (create a comment). However, if events are executed in parallel, these two events may end up in different parallel sequences of events. Therefore, a driver needs a mechanism to ensure the dependency is observed even when the dependent events are in different parallel update streams.&lt;/p&gt;
&lt;p&gt;The next blog entries in this series will discuss the approaches used in the driver to deal with these challenges.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Making Semantic Publishing Execution Rules</title>
      <link>https://ldbcouncil.org/post/making-semantic-publishing-execution-rules/</link>
      <pubDate>Tue, 18 Nov 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/making-semantic-publishing-execution-rules/</guid>
      <description>&lt;p&gt;&lt;a href=&#34;https://ldbcouncil.org/&#34;&gt;LDBC&lt;/a&gt; &lt;a href=&#34;https://ldbcouncil.org/benchmarks/spb&#34;&gt;SPB (Semantic Publishing Benchmark)&lt;/a&gt; is based on the BBC linked data platform use case. Thus the data modelling and transaction mix reflects the BBC&amp;rsquo;s actual utilization of RDF. But a benchmark is not only a condensation of current best practices. The BBC linked data platform is an &lt;a href=&#34;https://www.ontotext.com/&#34;&gt;Ontotext Graph DB&lt;/a&gt; deployment. Graph DB was formerly known as OWLIM.&lt;/p&gt;
&lt;p&gt;So, in SPB we wanted to address substantially more complex queries than the lookups that the BBC linked data platform primarily serves. Diverse dataset summaries, timelines and faceted search qualified by keywords and/or geography are examples of online user experience that SPB needs to cover.&lt;/p&gt;
&lt;p&gt;SPB is not per se an analytical workload but we still find that the queries fall broadly in two categories:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Some queries are centred on a particular search or entity. The data touched by the query size does not grow at the same rate as the dataset.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Some queries cover whole cross sections of the dataset, e.g. find the most popular tags across the whole database.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These different classes of questions need to be separated in a metric, otherwise the short lookup dominates at small scales and the large query at large scales.&lt;/p&gt;
&lt;p&gt;Another guiding factor of SPB was the BBC&amp;rsquo;s and others&amp;rsquo; express wish to cover operational aspects such as online backups, replication and fail-over in a benchmark. True, most online installations have to deal with these things, which are yet as good as absent from present benchmark practice. We will look at these aspects in a different article, for now, I will just discuss the matter of workload mix and metric.&lt;/p&gt;
&lt;p&gt;Normally the lookup and analytics workloads are divided into different benchmarks. Here we will try something different. There are three things the benchmark does:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Updates - These sometimes insert a graph, sometimes delete and re-insert the same graph, sometimes just delete a graph. These are logarithmic to data size.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Short queries - These are lookups that most often touch on recent data and can drive page impressions. These are roughly logarithmic to data scale.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Analytics - These cover a large fraction of the dataset and are roughly linear to data size.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;A test sponsor can decide on the query mix within certain bounds. A qualifying run must sustain a minimum, scale-dependent update throughput and must execute a scale-dependent number of analytical query mixes or run for a scale-dependent duration. The minimum update rate, the minimum number of analytics mixes and the minimum duration all grow logarithmically to data size. Within these limits, the test sponsor can decide how to mix the workloads. Publishing several results, emphasizing different aspects is also possible. A given system may be specially good at one aspect, leading the test sponsor to accentuate this.&lt;/p&gt;
&lt;p&gt;The benchmark has been developed and tested at small scales, between 50 and 150M triples. Next we need to see how it actually scales. There we expect to see how the two query sets behave differently. One effect that we see right away when loading data is that creating the full text index on the literals is in fact the longest running part. For a SF 32 ( 1.6 billion triples) SPB database we have the following space consumption figures:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;46886 MB of RDF literal text&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;23924 MB of full text index for RDF literals&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;23598 MB of URI strings&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;21981 MB of quads, stored column-wise with default index scheme&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Clearly, applying column-wise compression to the strings is the best move for increasing scalability. The literals are individually short, so literal per literal compression will do little or nothing but applying this by the column is known to get a 2x size reduction with Google Snappy. The full text index does not get much from column store techniques, as it already consists of words followed by space efficient lists of word positions. The above numbers are measured with Virtuoso column store, with quads column wise and the rest row-wise. Each number includes the table(s) and any extra indices associated to them.&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s now look at a full run at unit scale, i.e. 50M triples.&lt;/p&gt;
&lt;p&gt;The run rules stipulate a minimum of 7 updates per second. The updates are comparatively fast, so we set the update rate to 70 updates per second. This is seen not to take too much CPU. We run 2 threads of updates, 20 of short queries and 2 of long queries. The minimum run time for the unit scale is 10 minutes, so we do 10 analytical mixes, as this is expected to take 10 a little over 10 minutes. The run stops by itself when the last of the analytical mixes finishes.&lt;/p&gt;
&lt;p&gt;The interactive driver reports:&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;Seconds run : 2144
    Editorial:
        2 agents

        68164 inserts (avg : 46      ms, min : 5       ms, max : 3002    ms)
        8440  updates (avg : 72      ms, min : 15      ms, max : 2471    ms)
        8539  deletes (avg : 37      ms, min : 4       ms, max : 2531    ms)

        85143 operations (68164 CW Inserts (98 errors), 8440 CW Updates (0 errors), 8539 CW Deletions (0 errors))
        39.7122 average operations per second

    Aggregation:
        20 agents

        4120  Q1   queries (avg : 789     ms, min : 197     ms, max : 6767    ms, 0 errors)
        4121  Q2   queries (avg : 85      ms, min : 26      ms, max : 3058    ms, 0 errors)
        4124  Q3   queries (avg : 67      ms, min : 5       ms, max : 3031    ms, 0 errors)
        4118  Q5   queries (avg : 354     ms, min : 3       ms, max : 8172    ms, 0 errors)
        4117  Q8   queries (avg : 975     ms, min : 25      ms, max : 7368    ms, 0 errors)
        4119  Q11  queries (avg : 221     ms, min : 75      ms, max : 3129    ms, 0 errors)
        4122  Q12  queries (avg : 131     ms, min : 45      ms, max : 1130    ms, 0 errors)
        4115  Q17  queries (avg : 5321    ms, min : 35      ms, max : 13144   ms, 0 errors)
        4119  Q18  queries (avg : 987     ms, min : 138     ms, max : 6738    ms, 0 errors)
        4121  Q24  queries (avg : 917     ms, min : 33      ms, max : 3653    ms, 0 errors)
        4122  Q25  queries (avg : 451     ms, min : 70      ms, max : 3695    ms, 0 errors)

        22.5239 average queries per second. Pool 0, queries [ Q1 Q2 Q3 Q5 Q8 Q11 Q12 Q17 Q18 Q24 Q25 ]

        45318 total retrieval queries (0 timed-out)
        22.5239 average queries per second
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;The analytical driver reports:&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;Aggregation:
        2 agents

        14    Q4   queries (avg : 9984    ms, min : 4832    ms, max : 17957   ms, 0 errors)
        12    Q6   queries (avg : 4173    ms, min : 46      ms, max : 7843    ms, 0 errors)
        13    Q7   queries (avg : 1855    ms, min : 1295    ms, max : 2415    ms, 0 errors)
        13    Q9   queries (avg : 561     ms, min : 446     ms, max : 662     ms, 0 errors)
        14    Q10  queries (avg : 2641    ms, min : 1652    ms, max : 4238    ms, 0 errors)
        12    Q13  queries (avg : 595     ms, min : 373     ms, max : 1167    ms, 0 errors)
        12    Q14  queries (avg : 65362   ms, min : 6127    ms, max : 136346  ms, 2 errors)
        13    Q15  queries (avg : 45737   ms, min : 12698   ms, max : 59935   ms, 0 errors)
        13    Q16  queries (avg : 30939   ms, min : 10224   ms, max : 38161   ms, 0 errors)
        13    Q19  queries (avg : 310     ms, min : 26      ms, max : 1733    ms, 0 errors)
        12    Q20  queries (avg : 13821   ms, min : 11092   ms, max : 15435   ms, 0 errors)
        13    Q21  queries (avg : 36611   ms, min : 14164   ms, max : 70954   ms, 0 errors)
        13    Q22  queries (avg : 42048   ms, min : 7106    ms, max : 74296   ms, 0 errors)
        13    Q23  queries (avg : 48474   ms, min : 18574   ms, max : 93656   ms, 0 errors)
        0.0862 average queries per second. Pool 0, queries [ Q4 Q6 Q7 Q9 Q10 Q13 Q14 Q15 Q16 Q19 Q20 Q21 Q22 Q23 ]

        180 total retrieval queries (2 timed-out)
        0.0862 average queries per second
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;The metric would be 22.52 qi/s, 310 qa/h, 39.7 u/s @ 50Mt (SF 1)&lt;/p&gt;
&lt;p&gt;The SUT is dual Xeon E5-2630, all in memory. The platform utilization is steadily above 2000% CPU (over 20/24 hardware threads busy on the DBMS). The DBMS is Virtuoso open source, (&lt;a href=&#34;https://github.com/v7fasttrack/virtuoso-opensource/&#34;&gt;v7fasttrack at github.com&lt;/a&gt;, &lt;a href=&#34;https://github.com/v7fasttrack/virtuoso-opensource/tree/feature/analytics&#34;&gt;feature/analytics&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;The minimum update rate of 7/s was sustained but fell short of the target of 70./s. In this run, most demand was put on the interactive queries. Different thread allocations would give different ratios of the metric components. The analytics mix is for example about 3x faster without other concurrent activity.&lt;/p&gt;
&lt;p&gt;Is this good or bad? I would say that this is possible but better can certainly be accomplished.&lt;/p&gt;
&lt;p&gt;The initial observation is that Q17 is the worst of the interactive lot. 3x better is easily accomplished by avoiding a basic stupidity. The query does the evil deed of checking for a substring in a URI. This is done in the wrong place and accounts for most of the time. The query is meant to test geo retrieval but ends up doing something quite different. Optimizing this right would almost double the interactive score. There are some timeouts in the analytical run, which as such disqualifies the run. This is not a fully compliant result but is close enough to give an idea of the dynamics. So we see that the experiment is definitely feasible, is reasonably defined and that the dynamics seen make sense.&lt;/p&gt;
&lt;p&gt;As an initial comment of the workload mix, I&amp;rsquo;d say that interactive should have a few more very short point lookups to stress compilation times and give a higher absolute score of queries per second.&lt;/p&gt;
&lt;p&gt;Adjustments to the mix will depend on what we find out about scaling. As with SNB, it is likely that the workload will shift a little, so this result might not be comparable with future ones.&lt;/p&gt;
&lt;p&gt;In the next SPB article, we will look closer at performance dynamics and choke points and will have an initial impression on scaling the workload.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Fifth TUC Meeting</title>
      <link>https://ldbcouncil.org/event/fifth-tuc-meeting/</link>
      <pubDate>Fri, 14 Nov 2014 12:32:22 -0400</pubDate>
      
      <guid>https://ldbcouncil.org/event/fifth-tuc-meeting/</guid>
      <description>&lt;p&gt;The LDBC consortium are pleased to announce its fifth Technical User&lt;br&gt;
Community (TUC) meeting.&lt;/p&gt;
&lt;p&gt;This will be a one-day event at the National Hellenic Research Institute&lt;br&gt;
in Athens, Greece on &lt;strong&gt;Friday November 14, 2014&lt;/strong&gt;.&lt;/p&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;10:30 - 11:00 Coffee Break&lt;/p&gt;
&lt;p&gt;11:00 - 11:10 Peter Boncz (VUA)  Welcome &amp;amp; LDBC project status update (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979841.pptx&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;11:10 - 11:25 Venelin Kotsev (ONTO) Semantic Publishing Benchmark:Short Presentation of SPB and Status&lt;/p&gt;
&lt;p&gt;Feedback &amp;amp; Roadmap for SPB &amp;amp; OWLIM (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979839.pdf&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;11:25 - 11:30 Orri Erling (OGL) Status, Feedback &amp;amp; Roadmap for SPB &amp;amp; Virtuoso (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979828.pdf&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;11:30 - 11:45 Alex Averbuch (NEO) Social Network Benchmark: Short Presentation of SNB and  Status, Feedback &amp;amp; Roadmap for SNB &amp;amp; Neo4J (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979830.pdf&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;11:45 - 12:00 Orri Erling (OGL) Status, Feedback &amp;amp; Roadmap for SNB &amp;amp; Virtuoso (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979829.pdf&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;12:00 - 12:20 Arnau Prat (UPC) &amp;amp; Andrey Gubichev Status, Feedback &amp;amp; Roadmap for SNB Interactive &amp;amp; Sparksee (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979836.pdf&#34;&gt;Presentation&lt;/a&gt; ) and Business Intelligence (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979837.pdf&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;12:20 - 12:40 Tomer Sagi,  &amp;ldquo;Experience with SNB and TitanDB at HP&amp;rdquo; (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979838.pptx&#34;&gt;Presentation&lt;/a&gt; )&lt;/p&gt;
&lt;p&gt;12:40 - 13:00 Jakob Nelson,   &amp;ldquo;graphbench.org on the SNB datagen&amp;rdquo;&lt;/p&gt;
&lt;p&gt;13:00 - 14:30 Lunch Break@Byzantine &amp;amp; Christian Museum (&lt;a href=&#34;http://www.byzantinemuseum.gr/en/&#34;&gt;link&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;14:30 - 14:50 Olaf Hartig, &amp;ldquo;Integrating the Property Graph and RDF data models&amp;rdquo; (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979831.pdf&#34;&gt;Presentation&lt;/a&gt;)\&lt;/p&gt;
&lt;p&gt;Documents: &lt;a href=&#34;http://arxiv.org/abs/1409.3288&#34;&gt;arxiv/1409.3288&lt;/a&gt;, &lt;a href=&#34;http://arxiv.org/abs/1406.3399&#34;&gt;arxiv/1406.3399&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;14:50 - 15:10 Maria-Esther Vidal and Maribel Acosta, &amp;ldquo;Challenges to be addressed during Benchmarking SPARQL Federated Engines&amp;rdquo; (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979842.pdf&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;15:10 - 15:30 Evaggelia Pitoura, &amp;ldquo;Historical Queries on Graphs&amp;rdquo; (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979835.pdf&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;15:30 - 16:00 Coffee Break&lt;/p&gt;
&lt;p&gt;16:00 - 16:20 Manolis Terrovitis, Giannis Liagos, George Papastefanatos, &amp;ldquo;Efficient Identification of Implicit Facts in Incomplete OWL2-EL Knowledge Bases&amp;rdquo; (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979843.pdf&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;16:20 - 16:40 Gunes Aluc, &amp;ldquo;WatDiv: How to Tune-up your RDF Data Management System&amp;rdquo; (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979832.pdf&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;16:40 - 17:00 Giorgos Kollias, Yannis Smaragdakis, &amp;ldquo;Benchmarking @LogicBlox&amp;rdquo; (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979840.pdf&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;17:00 - 17:15 Hassan Chafi, &amp;ldquo;Oracle Labs Graph Strategy&amp;rdquo;&lt;/p&gt;
&lt;p&gt;17:15 - 17:25 Yinglong Xia, &amp;ldquo;Property Graphs for Industry Solution at IBM&amp;rdquo; (&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/6979834.pdf&#34;&gt;Presentation&lt;/a&gt;)&lt;/p&gt;
&lt;p&gt;17:25 - 17:30 Arthur Keen, &amp;ldquo;Short Introduction to SPARQLcity&amp;rdquo;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;&lt;strong&gt;20:30 Dinner @ Konservokouti &lt;a href=&#34;https://plus.google.com/114240752029716758955/about?gl=gr&amp;amp;hl=en&#34;&gt;(link)&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;&lt;strong&gt;Get a Taxi, and go to Ippokratous 148, Athens, Neapoli Exarheion&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h4 id=&#34;logistics&#34;&gt;Logistics&lt;/h4&gt;
&lt;p&gt;The meeting will be held at the &lt;a href=&#34;http://www.eie.gr/index-en.html&#34;&gt;National Hellenic Research Foundation&lt;/a&gt; located in &lt;a href=&#34;http://www.eie.gr/location-en.html&#34;&gt;downtown Athens&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fifth-tuc-meeting/attachments/5996808/5964344.gif&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;h4 id=&#34;travel&#34;&gt;Travel&lt;/h4&gt;
&lt;p&gt;Athens, Greece&amp;rsquo;s capital city, is easily accessible by air. Travelers on flights to Athens will land at Athens Eleftherios Venizelos International Airport.&lt;/p&gt;
&lt;p&gt;To arrive in the city center, you can take the metro from the airport (Line #3) and stop at either stop Evangelismos (ΕΥΑΓΓΕΛΙΣΜΟΣ) or at Syntagma (ΣΥΝΤΑΓΜΑ) stations. You can also take express Bus X95 and stop again at either Evangelismos  (ΕΥΑΓΓΕΛΙΣΜΟΣ) or at Syntagma (ΣΥΝΤΑΓΜΑ) stations (the latter is the terminus for the bus).&lt;/p&gt;
&lt;p&gt;You can also take a taxi from the airport that runs on a fixed price for the city center (45 euros). More information on how to move around in Athens from the airport can be found here: &lt;a href=&#34;http://www.aia.gr/traveler/&#34;&gt;http://www.aia.gr/traveler/&lt;/a&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Getting Started With the Semantic Publishing Benchmark</title>
      <link>https://ldbcouncil.org/post/getting-started-with-the-semantic-publishing-benchmark/</link>
      <pubDate>Sun, 09 Nov 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/getting-started-with-the-semantic-publishing-benchmark/</guid>
      <description>&lt;p&gt;The Semantic Publishing Benchmark (SPB), developed in the context of LDBC, aims at measuring the read and write operations that can be performed in the context of a media organisation. It simulates the management and consumption of RDF metadata describing media assets and creative works. The scenario is based around a media organisation that maintains RDF descriptions of its catalogue of creative works. These descriptions use a set of ontologies proposed by BBC that define numerous properties for content; they contain asll RDFS schema constructs and certain OWL ones.&lt;/p&gt;
&lt;p&gt;The benchmark proposes a data generator that uses the ontologies provided by BBC and reference datasets (again provided by BBC) to produce a set of valid instances; it works with a predefined set of distributions derived from the reference datasets. In addition to these distributions, the data generator also models:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;clustering of creative works around certain entities from the reference datasets (e.g. the association of an entity with creative works would decay exponentially in time)&lt;/li&gt;
&lt;li&gt;correlations between entities -  there will be creative works about two entities for a certain period in time, that way a history of interactions is also modelled (e.g. J. Biden and B. Obama are tagged in creative works for a continuous period in time)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The driver proposed by the benchmark measures the performance of CRUD operations of a SPARQL endpoint by starting a number of concurrently running editorial and aggregation agents. The former executes a series of insert, update and delete operations, whereas the latter a set of construct, describe, and select queries on a SPARQL endpoint. The benchmark can access all SPARQL endpoints that support the SPARQL 1.1 protocol. Tests have been run on OWLIM and Virtuoso. Attempts were also made for Stardog.&lt;/p&gt;
&lt;p&gt;Currently, the benchmark offers two workloads: a base version that consists of a mix of nine queries of different complexity that consider nearly all the features of SPARQL 1.1 query language including sorting, subqueries, limit,  regular expressions and grouping. The queries aim at checking different choke points relevant to query optimisation such as:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;join ordering based on cardinality constraints - expressed by the different kinds of properties defined in the schema&lt;/li&gt;
&lt;li&gt;subselects that aggregate the query results that the optimiser should recognise and evaluate first&lt;/li&gt;
&lt;li&gt;optional and nested optional clauses where the optimiser is called to produce a plan where the execution of the optional triple patterns is performed last&lt;/li&gt;
&lt;li&gt;reasoning along the RDFS constructs (subclass, subproperty hierarchies, functional, object and transitive properties etc.)&lt;/li&gt;
&lt;li&gt;unions to be executed in parallel&lt;/li&gt;
&lt;li&gt;optionals that contain filter expressions that should be executed as early as possible in order to eliminate intermediate results&lt;/li&gt;
&lt;li&gt;ordering where the optimiser could consider the possibility to choose query plan(s) that facilitate the ordering of results&lt;/li&gt;
&lt;li&gt;handling of geo-spatial predicates&lt;/li&gt;
&lt;li&gt;full-text search optimisation&lt;/li&gt;
&lt;li&gt;asynchronous execution of the aggregate sub-queries&lt;/li&gt;
&lt;li&gt;use of distinct to choose the optimal query plan&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We give below Query 1 of the Semantic Publishing Benchmark.&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;PREFIX bbcevent:&amp;lt;http://www.bbc.co.uk/ontologies/event/&amp;gt;
PREFIX geo-pos:&amp;lt;http://www.w3.org/2003/01/geo/wgs84_pos#&amp;gt;
PREFIX bbc:&amp;lt;http://www.bbc.co.uk/ontologies/bbc/&amp;gt;
PREFIX time:&amp;lt;http://www.w3.org/2006/time#&amp;gt;
PREFIX event:&amp;lt;http://purl.org/NET/c4dm/event.owl#&amp;gt;
PREFIX music-ont:&amp;lt;http://purl.org/ontology/mo/&amp;gt;
PREFIX rdf:&amp;lt;http://www.w3.org/1999/02/22-rdf-syntax-ns#&amp;gt;
PREFIX foaf:&amp;lt;http://xmlns.com/foaf/0.1/&amp;gt;
PREFIX provenance:&amp;lt;http://www.bbc.co.uk/ontologies/provenance/&amp;gt;
PREFIX owl:&amp;lt;http://www.w3.org/2002/07/owl#&amp;gt;
PREFIX cms:&amp;lt;http://www.bbc.co.uk/ontologies/cms/&amp;gt;
PREFIX news:&amp;lt;http://www.bbc.co.uk/ontologies/news/&amp;gt;
PREFIX cnews:&amp;lt;http://www.bbc.co.uk/ontologies/news/cnews/&amp;gt;
PREFIX cconcepts:&amp;lt;http://www.bbc.co.uk/ontologies/coreconcepts/&amp;gt;
PREFIX dbp-prop:&amp;lt;http://dbpedia.org/property/&amp;gt;
PREFIX geonames:&amp;lt;http://sws.geonames.org/&amp;gt;
PREFIX rdfs:&amp;lt;http://www.w3.org/2000/01/rdf-schema#&amp;gt;
PREFIX domain:&amp;lt;http://www.bbc.co.uk/ontologies/domain/&amp;gt;
PREFIX dbpedia:&amp;lt;http://dbpedia.org/resource/&amp;gt;
PREFIX geo-ont:&amp;lt;http://www.geonames.org/ontology#&amp;gt;
PREFIX bbc-pont:&amp;lt;http://purl.org/ontology/po/&amp;gt;
PREFIX tagging:&amp;lt;http://www.bbc.co.uk/ontologies/tagging/&amp;gt;
PREFIX sport:&amp;lt;http://www.bbc.co.uk/ontologies/sport/&amp;gt;
PREFIX skosCore:&amp;lt;http://www.w3.org/2004/02/skos/core#&amp;gt;
PREFIX dbp-ont:&amp;lt;http://dbpedia.org/ontology/&amp;gt;
PREFIX xsd:&amp;lt;http://www.w3.org/2001/XMLSchema#&amp;gt;
PREFIX core:&amp;lt;http://www.bbc.co.uk/ontologies/coreconcepts/&amp;gt;
PREFIX curric:&amp;lt;http://www.bbc.co.uk/ontologies/curriculum/&amp;gt;
PREFIX skos:&amp;lt;http://www.w3.org/2004/02/skos/core#&amp;gt;
PREFIX cwork:&amp;lt;http://www.bbc.co.uk/ontologies/creativework/&amp;gt;
PREFIX fb:&amp;lt;http://rdf.freebase.com/ns/&amp;gt;

# Query Name : query1
# Query Description :
# Retrieve creative works about thing t (or that mention t)
# reasoning: rdfs:subClassOf, rdf:type
# join ordering: cwork:dateModified rdf:type owl:FunctionalProperty
# join ordering: cwork:dateCreated rdf:type owl:FunctionalProperty
# Choke Points :
# - join ordering based on cardinality of functional proerties cwork:dateCreated, cwork:dateModified
# Optimizer should use an efficient cost evaluation method for choosing the optimal join tree
# - A sub-select which aggregates results. Optimizer should recognize it and execute it first
# - OPTIONAL and nested OPTIONAL clauses (treated by query optimizer as nested sub-queries)
# Optimizer should decide to put optional triples on top of the join tree
# (i.e. delay their execution to the last possible moment) because OPTIONALs are treated as a left join
# - qiery optimizer has the chance to recognize the triple pattern : ?cWork a ?type . ?type rdfs:subClassOf cwork:CreativeWork
# and eliminate first triple (?cwork a ?type .) since ?cwork is a cwork:CreativeWork​

CONSTRUCT {
  ?creativeWork a cwork:CreativeWork ;
   a ?type ;
   cwork:title ?title ;
   cwork:shortTitle ?shortTitle ;
   cwork:about ?about ;
   cwork:mentions ?mentions ;
   cwork:dateCreated ?created ;
   cwork:dateModified ?modified ;
   cwork:description ?description ;
   cwork:primaryFormat ?primaryFormat ;
   bbc:primaryContentOf ?webDocument .
  ?webDocument bbc:webDocumentType ?webDocType .
  ?about rdfs:label ?aboutLabel ;
   bbc:shortLabel ?aboutShortLabel ;
   bbc:preferredLabel ?aboutPreferredLabel .
  ?mentions rdfs:label ?mentionsLabel ;
   bbc:shortLabel ?mentionsShortLabel ;
   bbc:preferredLabel ?mentionsPreferredLabel .
  ?creativeWork cwork:thumbnail ?thumbnail .
  ?thumbnail a cwork:Thumbnail ;
   cwork:altText ?thumbnailAltText ;
   cwork:thumbnailType ?thumbnailType .
}
WHERE {
  {
   SELECT ?creativeWork
    WHERE {
        ?creativeWork {{{cwAboutOrMentions}}} {{{cwAboutOrMentionsUri}}} .
        ?creativeWork a cwork:CreativeWork ;
        cwork:dateModified ?modified .
     }
    ORDER BY DESC(?modified)
    LIMIT 10
  }
  ?creativeWork a cwork:CreativeWork ;
         a ?type ;
         cwork:title ?title ;
         cwork:dateModified ?modified .
  OPTIONAL { ?creativeWork cwork:shortTitle ?shortTitle . }
  OPTIONAL { ?creativeWork cwork:description ?description . }
  OPTIONAL { ?creativeWork cwork:about ?about .
        OPTIONAL { ?about rdfs:label ?aboutLabel . }
        OPTIONAL { ?about bbc:shortLabel ?aboutShortLabel . }
        OPTIONAL { ?about bbc:preferredLabel ?aboutPreferredLabel . }
     }
  OPTIONAL {
         ?creativeWork cwork:mentions ?mentions .
         OPTIONAL { ?mentions rdfs:label ?mentionsLabel . }
         OPTIONAL { ?mentions bbc:shortLabel ?mentionsShortLabel . }
         OPTIONAL { ?mentions bbc:preferredLabel ?mentionsPreferredLabel . }
     }
   OPTIONAL { ?creativeWork cwork:dateCreated ?created . }
   OPTIONAL { ?creativeWork cwork:primaryFormat ?primaryFormat . }
   OPTIONAL { ?webDocument bbc:primaryContent ?creativeWork .
        OPTIONAL { ?webDocument bbc:webDocumentType ?webDocType . }
  }
  OPTIONAL { ?creativeWork bbc:primaryContentOf ?webDocument .
        OPTIONAL { ?webDocument bbc:webDocumentType ?webDocType . }
  }
  OPTIONAL { ?creativeWork cwork:thumbnail ?thumbnail .
        OPTIONAL { ?thumbnail cwork:altText ?thumbnailAltText . }
        OPTIONAL { ?thumbnail cwork:thumbnailType ?thumbnailType . }
  }
}
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Listing 1. Semantic Publishing Benchmark: Query 1&lt;/p&gt;
&lt;p&gt;The benchmark test driver is distributed as a jar file, but can also be built using an ant script. It is distributed with the BBC ontologies and reference datasets, the queries and update workloads discussed earlier and the configuration parameters for running the benchmark and for generating the data. It is organised in the following different phases: ontology loading and reference dataset loading, dataset generation and loading, warm up (where a series of aggregation queries are run for a predefined amount of time), benchmark where all queries (aggregation and editorial) are run, conformance checking (that allows one to check whether the employed RDF engine implements OWL reasoning) and finally cleanup that removes all the data from the repository. The benchmark provides a certain degree of freedom where each phase can run independently of the others.&lt;/p&gt;
&lt;p&gt;The data generator uses an RDF repository to load ontologies and reference datasets; actually, any system that will be benchmarked should have those ontologies loaded.  Any repository that will be used for the data generation should be set up with context indexing, and finally geo-spatial indexing, if available, to serve the spatial queries. The current version of the benchmark has been tested with Virtuoso and OWLIM.&lt;/p&gt;
&lt;p&gt;The generator uses configuration files that must be configured appropriately to set the values regarding the dataset size to produce, the number of aggregation and editorial agents, the query time out etc. The distributions used by the data generator could also be edited. The benchmark is very simple to run (once the RDF repository used to store the ontologies and the reference datasets is set up, and the configuration files updated appropriately) using the command: java -jar semantic_publishing_benchmark-*.jar test.properties. The benchmark produces three kinds of files that contain (a) brief information about each executed query, the size of the returned result, and the execution time (semantic_publishing_benchmark_queries_brief.log), (b) the detailed log of each executed query and its result (semantic_publishing_benchmark_queries_detailed.log) (c)  the benchmark results (semantic_publishing_benchmark_results.log ).&lt;/p&gt;
&lt;p&gt;Below we give an example of a run of the benchmark for OWLIM-SE. The benchmark reports the number of edit operations (inserts, updates, and writes) and queries executed at the Nth second of a benchmark run. It also reports that total number of retrieval queries as well as the average number of queries executed per second.&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;Seconds run : 600
        Editorial:
                0 agents

                0 operations (0 CW Inserts, 0 CW Updates, 0 CW Deletions)
                0.0000 average operations per second

        Aggregation:
                8 agents

                298   Q1   queries
                267   Q2   queries
                243   Q3   queries
                291   Q4   queries
                320   Q5   queries
                286   Q6   queries
                255   Q7   queries
                274   Q8   queries
                271   Q9   queries

                2505 total retrieval queries
                4.1750 average queries per second
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Listing 2. A snippet of semantic_publishing_benchmark_results.log&lt;/p&gt;
&lt;p&gt;We run the benchmark under the following configuration: we used 8 aggregation agents for query execution and 4 data generator workers all running in parallel. The warm up period is 120 seconds during which a number of aggregation agents is executed to prepare the tested systems for query execution. Aggregation agents run for a period of 600 seconds, and queries timeout after 90 seconds. We used 10 sets of substitution parameters for each query. For data generation, ontologies and reference datasets are loaded in the OWLIM-SE repository. We used OWLIM-SE,  Version 5.4.6287 with Sesame Version 2.6 and Tomcat Version 6. The results we obtained for the 10M, 100M and 1B triple datasets are given in the table below:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;#triples&lt;/th&gt;
&lt;th&gt;Q1&lt;/th&gt;
&lt;th&gt;Q2&lt;/th&gt;
&lt;th&gt;Q3&lt;/th&gt;
&lt;th&gt;Q4&lt;/th&gt;
&lt;th&gt;Q5&lt;/th&gt;
&lt;th&gt;Q6&lt;/th&gt;
&lt;th&gt;Q7&lt;/th&gt;
&lt;th&gt;Q8&lt;/th&gt;
&lt;th&gt;Q9&lt;/th&gt;
&lt;th&gt;#queries&lt;/th&gt;
&lt;th&gt;avg. #q. per sec.&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;10M&lt;/td&gt;
&lt;td&gt;298&lt;/td&gt;
&lt;td&gt;267&lt;/td&gt;
&lt;td&gt;243&lt;/td&gt;
&lt;td&gt;291&lt;/td&gt;
&lt;td&gt;320&lt;/td&gt;
&lt;td&gt;286&lt;/td&gt;
&lt;td&gt;255&lt;/td&gt;
&lt;td&gt;274&lt;/td&gt;
&lt;td&gt;271&lt;/td&gt;
&lt;td&gt;2505&lt;/td&gt;
&lt;td&gt;41,750&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;100M&lt;/td&gt;
&lt;td&gt;53&lt;/td&gt;
&lt;td&gt;62&lt;/td&gt;
&lt;td&gt;51&lt;/td&gt;
&lt;td&gt;52&lt;/td&gt;
&lt;td&gt;44&lt;/td&gt;
&lt;td&gt;62&lt;/td&gt;
&lt;td&gt;25&lt;/td&gt;
&lt;td&gt;55&lt;/td&gt;
&lt;td&gt;45&lt;/td&gt;
&lt;td&gt;449&lt;/td&gt;
&lt;td&gt;7,483&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1B&lt;/td&gt;
&lt;td&gt;34&lt;/td&gt;
&lt;td&gt;29&lt;/td&gt;
&lt;td&gt;22&lt;/td&gt;
&lt;td&gt;24&lt;/td&gt;
&lt;td&gt;25&lt;/td&gt;
&lt;td&gt;29&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;29&lt;/td&gt;
&lt;td&gt;28&lt;/td&gt;
&lt;td&gt;220&lt;/td&gt;
&lt;td&gt;3,667&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
</description>
    </item>
    
    <item>
      <title>Choke Point Based Benchmark Design</title>
      <link>https://ldbcouncil.org/post/choke-point-based-benchmark-design/</link>
      <pubDate>Tue, 14 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/choke-point-based-benchmark-design/</guid>
      <description>&lt;p&gt;The &lt;em&gt;Linked Data Benchmark Council&lt;/em&gt; (LDBC) mission is to design and maintain benchmarks for graph data management systems, and establish and enforce standards in running these benchmarks, and publish and arbitrate around the official benchmark results. The council and its &lt;a href=&#34;https://ldbcouncil.org&#34;&gt;https://ldbcouncil.org&lt;/a&gt; website just launched, and in its first 1.5 year of existence, most effort at LDBC has gone into investigating the needs of the field through interaction with the LDBC Technical User Community (&lt;a href=&#34;https://ldbcouncil.org/event/fifth-tuc-meeting&#34;&gt;next TUC meeting&lt;/a&gt; will be on October 5 in Athens) and indeed in &lt;em&gt;designing benchmarks&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;So, what makes a good benchmark design? Many talented people have paved our way in addressing this question and for relational database systems specifically the benchmarks produced by &lt;a href=&#34;http://www.tpc.org/&#34;&gt;TPC&lt;/a&gt; have been very helpful in maturing relational database technology, and making it successful. Good benchmarks are &lt;em&gt;relevant&lt;/em&gt; and &lt;em&gt;representative&lt;/em&gt; (address important challenges encountered in practice), &lt;em&gt;understandable&lt;/em&gt; , &lt;em&gt;economical&lt;/em&gt; (implementable on simple hardware), &lt;em&gt;fair&lt;/em&gt; (such as not to favor a particular product or approach), &lt;em&gt;scalable&lt;/em&gt;, &lt;em&gt;accepted&lt;/em&gt; by the community  and &lt;em&gt;public&lt;/em&gt; (e.g. all of its software is available in open source). This list stems from Jim Gray&amp;rsquo;s &lt;a href=&#34;http://research.microsoft.com/en-us/um/people/gray/BenchmarkHandbook/TOC.htm&#34;&gt;Benchmark Handbook&lt;/a&gt;. In this blogpost, I will share some thoughts on each of these aspects of good benchmark design.&lt;/p&gt;
&lt;p&gt;A very important aspect of benchmark development is making sure that the community &lt;em&gt;accepts&lt;/em&gt; a certain benchmark, and starts using it. A benchmark without published results and therefore opportunity to compare results, remains irrelevant. A European FP7 project is a good place to start gathering a critical mass of support (and consensus, in the process) for a new benchmark from the core group of benchmark designers in the joint work performed by the consortium. Since in LDBC multiple commercial graph and RDF vendors are on the table (Neo Technologies, Openlink, Ontotext and Sparsity) a minimal consensus on &lt;strong&gt;fairness&lt;/strong&gt; had to be established immediately. The Linked Data Benchmark Council itself is a noncommercial, neutral, entity which releases all its benchmark specifications, software, as well as many materials created during the design.  LDBC has spent a lot of time engaging interested parties (mainly through its &lt;a href=&#34;https://ldbcouncil.org/tags/tuc-meeting/&#34;&gt;Technical User Community gatherings&lt;/a&gt;) as well as lining up additional organizations as members of the Linked Data Benchmark Council. There is, in other words, a strong non-technical, human factor in getting benchmarks accepted.&lt;/p&gt;
&lt;p&gt;The need for &lt;em&gt;understandability&lt;/em&gt; for me means that a database benchmark should consist of a limited number of queries and result metrics. Hence I find TPC-H with its 22 queries more understandable than TPC-DS with its 99, because after (quite some) study and experience it is possible to understand the underlying challnges of all queries in TPC-H. It may also be possible for TPC-DS but the amount of effort is just much larger. Understandable also means for me that a particular query should behave similarly, regardless of the query parameters. Often, a particular query needs to be executed many times, and in order not to play into the hands of simple query caching and also enlarge the access footprint of the workload, different query parameters should be used. However, parameters can strongly change the nature of a query but this is not desirable for the understandability of the workload. For instance, we know that TPC-H Q01 tests raw computation power, as its selection predicate eliminates almost nothing from the main fact table (LINEITEM), that it scans and aggregates into a small 4-tuple result. Using a selection parameter that would select only 0.1% of the data instead, would seriously change the nature of Q01, e.g. making it amendable to indexing. This stability of parameter bindings is an interesting challenge for the &lt;a href=&#34;https://ldbcouncil.org/benchmarks/snb&#34;&gt;Social Network Benchmark&lt;/a&gt; (SNB) of LDBC which is not as uniform and uncorrelated as TPC-H. Addressing the challenge of obtaining parameter bindings that have similar execution characteristics will be the topic of a future blog post.&lt;/p&gt;
&lt;p&gt;The &lt;em&gt;economical&lt;/em&gt; aspect of benchmarking means that while rewarding high-end benchmark runs with higher scores, it is valuable if a meaningful run can also be done with small hardware. For this reason, it is good practice to use a performance-per-EURO (or $) metric, so small installations despite a lower absolute score can still do well on that metric. The economical aspect is right now hurting the (still) leading relational OLTP benchmark TPC-C. Its implementation rules are such that for higher reported rates of throughput, a higher number of warehouses (i.e. larger data size) is needed. In the current day and age of JIT-compiled machinecode SQL procedures and CPU-cache optimized main memory databases, the OLTP throughput numbers now obtainable on modern transactional systems like Hyper on even a single server (it reaches more than 100.000 transactions per second) are so high that they lead to petabyte storage requirements. Not only does this make TPC-C very expensive to run, just by the sheer amount of hardware needed according to the rules, but it also undermines it representativity, since OLTP data sizes encountered in the field are much smaller than OLAP data sizes and do not run in the petabytes.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Representative&lt;/em&gt; benchmarks can be designed by studying or even directly using real workload information, e.g. query logs. A rigorous example of this is the &lt;a href=&#34;http://aksw.org/Projects/DBPSB.html&#34;&gt;DBpedia benchmark&lt;/a&gt; whose workload is based on the query logs of dbpedia.org. However, this SPARQL endpoint is a single public Virtuoso instance that has been configured to interrupt all long running queries, such as to ensure the service remains responsive to as many users as possible. As a result, it is only practical to run small lookup queries on this database service, so the query log only contained solely such light queries. As a consequence, the DBpedia benchmark only tests small SPARQL queries that stress simple B-tree lookups only (and not joins, aggregations, path expressions or inference) and poses almost no technical challenges for either query optimization or execution. The lesson, thus, is to balance representativity with relevance (see later).&lt;/p&gt;
&lt;p&gt;The fact that a benchmark can be &lt;em&gt;scaled&lt;/em&gt; in size favors the use of synthetic data (i.e. created by a data generator) because data generators can produce any desired quantity of data. I hereby note that in this day and age,  data generators should be parallel. Single-threaded single-machine data generation just becomes unbearable even at terabyte scales. A criticism of synthetic data is that it may not be representative of real data, which e.g. tends to contain highly correlated data with skewed distributions. This may be addressed to a certain extent by injecting specific skew and correlations into synthetic data as well (but: which skew and which correlations?). An alternative is to use real data and somehow blow up or contract the data. This is the approach in the mentioned DBpedia benchmark, though such scaling will distort the original distributions and correlations. Scaling a benchmark is very useful to investigate the effect of data size on the metric, on individual queries, or even in micro-benchmark tests that are not part of the official query set. Typically OLTP database benchmarks have queries whose complexity is O(log(N)) of the data size N, whereas OLAP benchmarks have queries which are linear, O(N) or at most O(N.log(N))  &amp;ndash; otherwise executing the benchmark on large instances is infeasible. OLTP queries thus typically touch little data, in the order of log(N) tuples. In order not to measure fully cold query performance, OLTP benchmarks for that reason need a warmup phase with O(N/log(N)) queries in order to get the system into a representative state.&lt;/p&gt;
&lt;p&gt;Now, what makes a benchmark &lt;em&gt;relevant&lt;/em&gt;? In LDBC we think that benchmarks should be designed such that crucial areas of functionality are highlighted, and in turn system architects are stimulated to innovate. Either to catch up with competitors and bring the performance and functionality in line with the state-of-the-art but even to innovate and address technical challenges for which until now no good solutions exist, but which can give a decisive performance advantage in the benchmark. Inversely stated, benchmark design can thus be a powerful tool to influence the industry, as a benchmark design may set the agendas for multiple commercial design teams and database architects around the globe. To structure this design process, LDBC introduces the notion of &lt;em&gt;&amp;ldquo;choke points&amp;rdquo;&lt;/em&gt;: by which we mean problems that challenge current technology. These choke points are collected and described early in the LDBC design process, and the workloads developed later are scored in terms of their coverage of relevant choke points. In case of graph data querying, one of the choke points that is unique to the area is recursive Top-N query handling (e.g. shortest path queries). Another choke point that arises is the impact of correlations between attribute value of graph nodes (e.g. both employed by TUM) and the connectivity degree between nodes (the probability to be friends). The notion observed in practice is that people who are direct colleagues, often are in each others friend network. A query that selects people in a social graph that work for the same company, and then does a friendship traversal, may get a bad intermediate result size estimates and therefore suboptimal query plan, if optimizers remain unaware of value/structure correlations. So this is an area of functionality that the Social Network Benchmark (SNB) by LDBC will test.&lt;/p&gt;
&lt;p&gt;To illustrate what choke points are in more depth, we wrote a &lt;a href=&#34;https://ldbcouncil.org/docs/papers/tpc-h-analyzed-choke-points-tpctc2013.pdf&#34;&gt;paper in the TPCTC 2013&lt;/a&gt; conference that performs a post-mortem analysis of TPC-H and identified 28 such choke points. &lt;em&gt;&lt;a href=&#34;chokepoints.png&#34;&gt;This table&lt;/a&gt;&lt;/em&gt; lists them all, grouped into six Choke Point (CP) areas (CP1 Agregation, CP2 Join, CP3 Locality, CP4 Calculations, CP5 Subqueries and CP6 Parallelism). The classification also shows CP coverage over each of the 22 TPC-H queries (black is high impact, white is no impact):&lt;/p&gt;
&lt;p&gt;I would recommend reading this paper to anyone who is interested in improving the TPC-H score of a relational database system, since this paper contains the collected experience of three database architects who have worked with TPC-H at length: Orri Erling (of Virtuoso), Thomas Neumann (Hyper,RDF-3X), and me (MonetDB,Vectorwise).  Recently Orri Erling showed that this paper is not complete as he discovered one more choke-point area for TPC-H:  Top-N pushdown. In a detailed blog entry, Orri showed how this technique can trivialize Q18; and this optimization can single handedly improve the overall TPC-score by 10-15%. This is also a lesson for LDBC: even though we design benchmarks with choke points in mind, the queries themselves may bring to light unforeseen opportunities and choke-points that may give rise to yet unknown innovations.&lt;/p&gt;
&lt;p&gt;LDBC has just published two benchmarks as Public Drafts, which essentially means that you are cordially invited to download and try out the RDF-focused Semantic Publishing Benchmark &lt;a href=&#34;https://ldbcouncil.org/developer/spb&#34;&gt;(SPB)&lt;/a&gt; and the more graph-focused Social Network Benchmark (&lt;a href=&#34;https://ldbcouncil.org/developer/snb&#34;&gt;SNB&lt;/a&gt;),  and &lt;a href=&#34;https://groups.google.com/forum/#!forum/ldbcouncil&#34;&gt;tell us what you think&lt;/a&gt;. Stay tuned for the coming detailed blog posts about these benchmarks, which will explain the graph and RDF processing choke-points that they test.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;(for more posts from Peter Boncz, see also &lt;a href=&#34;https://databasearchitects.blogspot.com&#34;&gt;Database Architects&lt;/a&gt;, a blog about data management challenges and techniques written by people who design and implement database systems)&lt;/em&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>New Website Online LDBC Benchmarks Reach Public Draft</title>
      <link>https://ldbcouncil.org/post/new-website-online-ldbc-benchmarks-reach-public-draft/</link>
      <pubDate>Tue, 14 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/new-website-online-ldbc-benchmarks-reach-public-draft/</guid>
      <description>&lt;p&gt;The Linked Data Benchmark Council  (LDBC) is reaching a milestone today, June 23 2014, in announcing that two of the benchmarks that it has been developing since 1.5 years have now reached the status of Public Draft. This concerns the Semantic Publishing Benchmark (SPB) and the interactive workload of the Social Network Benchmark (SNB). In case of LDBC, the release is staged: now the benchmark software just runs read-only queries. This will be expanded in a few weeks with a mix of read- and insert-queries. Also, query validation will be added later. Watch this blog for the announcements to come, as this will be a matter of weeks to add.&lt;/p&gt;
&lt;p&gt;The Public Draft stage means that the initial software (data generator, query driver) work and an initial technical specification and documentation has been written. In other words, there is a testable version of the benchmark available for anyone who is interested. Public Draft status does not mean that the benchmark has been adopted yet, it rather means that LDBC has come closer to adopting them, but is now soliciting feedback from the users. The benchmarks will remain in this stage at least until October 6. On that date, LDBC is organizing its fifth &lt;a href=&#34;https://ldbcouncil.org/event/fifth-tuc-meeting&#34;&gt;Technical User Community meeting&lt;/a&gt;. One of the themes for that meeting is collecting user feedback on the Public Drafts; which input will be used to either further evolve the benchmarks, or adopt them.&lt;/p&gt;
&lt;p&gt;You can also see that we created a this new website and a new logo. This website is different from &lt;code&gt;http://ldbc.eu&lt;/code&gt; that describes the EU project which kick-starts LDBC. The ldbcouncil.org is a website maintained by the Linked Data Benchmark Council legal entity, which will live on after the EU project stops (in less than a year). The Linked Data Benchmark Council is an independent, impartial, member-sustained organization dedicated to the creation of RDF and graph data management benchmarks and benchmark practices.&lt;/p&gt;
&lt;p&gt;In the next weeks, you will see many contributors in LDBC post items on this blog. Some of these blog entries will be very technical, others not, but all aim to explain what LDBC is doing for RDF and graph benchmarking, and why.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Social Network Benchmark Goals</title>
      <link>https://ldbcouncil.org/post/social-network-benchmark-goals/</link>
      <pubDate>Tue, 14 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/social-network-benchmark-goals/</guid>
      <description>&lt;p&gt;Social Network interaction is amongst the most natural and widely spread activities in the internet society, and it has turned out to be a very useful way for people to socialise at different levels (friendship, professional, hobby, etc.). As such, Social Networks are well understood from the point of view of the data involved and the interaction required by their actors. Thus, the concepts of friends of friends, or retweet are well established for the data attributes they represent, and queries such as “find the friend of a specified person who has long worked in a company in a specified country” are natural for the users and easy to understand from a functional point of view.&lt;/p&gt;
&lt;p&gt;From a totally different perspective, Social Networks are challenging technologically, being part of the Big Data arena, and require the execution of queries that involve complex relationship search and data traversal computations that turn out to be choke points for the data management solutions in the market.&lt;/p&gt;
&lt;p&gt;With the objective of shaping a benchmark which is up to date as a use case, well understood by everybody and poses significant technological challenges, the LDBC consortium decided to create the Social Network Benchmark, &lt;a href=&#34;https://ldbcouncil.org/benchmarks/snb&#34;&gt;SNB&lt;/a&gt;, which is eventually going to include three workloads: the Interactive, the Business Intelligence and the Analytical. Those workloads are going to share a unique synthetic data generation tool that will mimic the data managed by real Social Networks.&lt;/p&gt;
&lt;p&gt;The SNB data generator created by LDBC is an evolution of the S3G2 data generator. The data generator is unique because it generates data that contains realistic distributions and correlations among variables that were not taken into consideration before. It also allows generating large datasets because it uses a Hadoop based implementation to compute the complex data generated. The SNB data generator has already been used in different situations like the &lt;a href=&#34;https://arxiv.org/pdf/2010.12243.pdf&#34;&gt;ACM SIGMOD programming contest 2014&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The SNB presents the Interactive workload as first of a breed with the objective to resemble the queries that users may place to a Social Network portal. Those are a combination of read and write small queries that express the needs of a user who is interacting with her friends and connections through the Social Network. Queries like that explained above (Q12 in the workload) are examples that set up choke points like pattern recognition or full traversals.&lt;/p&gt;
&lt;p&gt;More details will be given in blogs to follow both for the data generator as well as for the specific characteristics of the workloads allowing the users to obtain a first contact with the benchmarks.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Welcome to the New Industry Oriented LDBC Organisation for Benchmarking RDF and Graph Technologies</title>
      <link>https://ldbcouncil.org/post/welcome-to-the-new-industry-oriented-ldbc-organisation-for-benchmarking-rdf-and-graph-technologies/</link>
      <pubDate>Tue, 14 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/welcome-to-the-new-industry-oriented-ldbc-organisation-for-benchmarking-rdf-and-graph-technologies/</guid>
      <description>&lt;p&gt;It is with great pleasure that we announce the new LDBC organisation site at &lt;a href=&#34;https://www.ldbcouncil.org&#34;&gt;www.ldbcouncil.org&lt;/a&gt;. The LDBC started as a European Community FP7 funded project with the objective to create, foster and become an industry reference for benchmarking RDF and Graph technologies. A period of more than one and a half years has led us to the creation of the first two workloads, the Semantic Publishing Benchmark and the Social Network Benchmark in its interactive workload, which you will find in the &lt;em&gt;benchmarks&lt;/em&gt; menu on this site.&lt;/p&gt;
&lt;p&gt;Those benchmarks will allow all the actors in the RDF and Graph industry to know who is who and how the different technology players are reacting to the results of their competing industry companies. Thus, the users will have results to compare the technologies and vendors will have a clear idea of how their products evolve compared to other vendors, all with the objective to foster the technological growth of the RDF and Graph arena.&lt;/p&gt;
&lt;p&gt;While the main objective of LDBC is to create benchmarks, we know that we need a strong community to grow and evolve those benchmarks taking into consideration all the market and technology needs. With this objective, we have created a special section to engage all the interested community through a blog, forums to discuss interesting issues and a lot of information on benchmarking, including links to other benchmarks, pointers to interesting conferences and venues and all the publications on benchmarking RDF and Graph technologies.&lt;/p&gt;
&lt;p&gt;We want to make sure that we all know what benchmarking and the LDBC effort means, both historically, and from the global needs perspective. To make sure that this is accomplished, we set up a section open to the public with in depth explanations of the history of industry benchmarking, LDBC and why our society needs such efforts globally.&lt;/p&gt;
&lt;p&gt;Finally, we want to invite you to our Fifth Technical Users Community (TUC) meeting to be held in Athens next Monday Oct. 6th 2014. This event will have as its main objective to allow for presentations on experiences with the two already released benchmarks, SNB and SPB. You’ll find updated information here.&lt;/p&gt;
&lt;p&gt;In all, we expect that the LDBC organisation site engages all of you and that the growth of RDF and Graph technologies in the future is secured by the benchmarks fostered by us.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>2nd International Workshop on Benchmarking RDF Systems</title>
      <link>https://ldbcouncil.org/post/2nd-international-workshop-on-benchmarking-rdf-systems/</link>
      <pubDate>Thu, 09 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/2nd-international-workshop-on-benchmarking-rdf-systems/</guid>
      <description>&lt;p&gt;Following the 1st International workshop on Benchmarking RDF Systems (BeRSys 2013) the aim of the BeRSys 2014 workshop is to provide a discussion forum where researchers and industrials can meet to discuss topics related to the performance of RDF systems. BeRSys 2014 is the only workshop dedicated to benchmarking different aspects of RDF engines - in the line of TPCTC series of workshops.The focus of the workshop is to expose and initiate discussions on best practices, different application needs and scenarios related to different aspects of RDF data management.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>DATAGEN: Data Generation for the Social Network Benchmark</title>
      <link>https://ldbcouncil.org/post/datagen-data-generation-for-the-social-network-benchmark/</link>
      <pubDate>Thu, 09 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/datagen-data-generation-for-the-social-network-benchmark/</guid>
      <description>&lt;p&gt;As explained in a previous post, the LDBC Social Network Benchmark (LDBC-SNB) has the objective to provide a realistic yet challenging workload, consisting of a social network and a set of queries. Both have to be realistic, easy to understand and easy to generate. This post has the objective to discuss the main features of DATAGEN, the social network data generator provided by LDBC-SNB, which is an evolution of S3G2 &lt;a href=&#34;#references&#34;&gt;[1]&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;One of the most important components of a benchmark is the dataset. However, directly using real data in a benchmark is not always possible. On the one hand, it is difficult to find data with all the scaling characteristics the benchmark requires. On the other hand, collecting real data can be expensive or simply not possible due to privacy concerns.&lt;/p&gt;
&lt;p&gt;For these reasons, LDBC-SNB provides DATAGEN which is the synthetic data generator responsible for generating the datasets for the three LDBC-SNB workloads: the Interactive, the Business Intelligence and the Analytical. DATAGEN has been carefully designed with the following goals in mind:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Realism.&lt;/strong&gt; The data generated by DATAGEN has to mimic the features of those found in a real social network. In DATAGEN, output attributes, cardinalities, correlations and distributions have been finely tuned to reproduce a real social network in each of its aspects. DATAGEN is aware of the data and link distributions found in a real social network such as Facebook &lt;a href=&#34;#references&#34;&gt;[2]&lt;/a&gt;. Also, it uses real data from DBPedia, such as property dictionaries, which ensure that the content is realistic and correlated.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Scalability.&lt;/strong&gt; Since LDBC-SNB is targeting systems of different scales and budgets, DBGEN must be capable of generating datasets of different sizes, from a few Gigabytes to Terabytes. DATAGEN is implemented following the MapReduce paradigm, allowing for the generation of large datasets on commodity clusters.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Determinism.&lt;/strong&gt; DATAGEN is deterministic regardless of the number of cores/machines used to produce the data. This important feature guarantees that all Test Sponsors will face the same dataset, thus, making the comparisons between different systems fair and the benchmarks’ results reproducible.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Usability.&lt;/strong&gt; LDBC-SNB has been designed to have an affordable entry point. As such, DATAGEN has been severely influenced by this philosophy, and therefore it has been designed to be as easy to use as possible.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Finally, the area of action of DATAGEN is not only limited to the scope of LDBC-SNB. Several researchers and practitioners are already using DATAGEN in a wide variety of situations. If you are interested on the internals and possibilities of DATAGEN, please visit its &lt;a href=&#34;https://github.com/ldbc/ldbc_snb_datagen&#34;&gt;official repository&lt;/a&gt;.&lt;/p&gt;
&lt;h4 id=&#34;references&#34;&gt;References&lt;/h4&gt;
&lt;p&gt;[1] Pham, Minh-Duc, Peter Boncz, and Orri Erling. &amp;ldquo;S3g2: A scalable structure-correlated social graph generator.&amp;rdquo; Selected Topics in Performance Evaluation and Benchmarking. Springer Berlin Heidelberg, 2013. 156-172.&lt;/p&gt;
&lt;p&gt;[2] Prat-Pérez, Arnau, and David Dominguez-Sal. &amp;ldquo;How community-like is the structure of synthetically generated graphs?.&amp;rdquo; Proceedings of Workshop on GRAph Data management Experiences and Systems. ACM, 2014.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Getting Started With SNB</title>
      <link>https://ldbcouncil.org/post/getting-started-with-snb/</link>
      <pubDate>Thu, 09 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/getting-started-with-snb/</guid>
      <description>&lt;p&gt;In a previous blog post titled &amp;ldquo;&lt;a href=&#34;https://ldbcouncil.org/post/is-snb-like-facebooks-linkbench/&#34;&gt;Is SNB like Facebook&amp;rsquo;s LinkBench?&lt;/a&gt;&amp;rdquo;, Peter Boncz discusses the design philosophy that shapes SNB and how it compares to other existing benchmarks such as LinkBench. In this post, I will briefly introduce the essential parts forming SNB, which are DATAGEN, the LDBC execution driver and the workloads.&lt;/p&gt;
&lt;h3 id=&#34;datagen&#34;&gt;DATAGEN&lt;/h3&gt;
&lt;p&gt;DATAGEN is the data generator used by all the workloads of SNB. &lt;a href=&#34;https://ldbcouncil.org/post/datagen-data-generation-for-the-social-network-benchmark/&#34;&gt;Here&lt;/a&gt; we introduced the design goals that drive the development of DATAGEN, which can be summarized as: &lt;em&gt;Realism, Scalability, Determinism and Usability.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;DATAGEN produces datasets with the following schema, in terms of entities and their relations. Data generated represents a snapshot of the activity of a social network similar to real social networks such as Facebook, during a period of time. Data includes entities such as Persons, Organizations, and Places. The schema also models the way persons interact, by means of the friendship relations established with other persons, and the sharing of content such as messages (both textual and images), replies to messages and likes to messages. People form groups to talk about specific topics, which are represented as tags.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;schema.png&#34; alt=&#34;image&#34;&gt;&lt;/p&gt;
&lt;p&gt;For the sake of credibility, data produced by DATAGEN has to be realistic. In this sense, data produced by DATAGEN not only has a realistic schema, but also pays attention to the following items:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Realistic distributions. The degree distribution of friendship relationships has been modeled to reproduce that found in the Facebook graph. Also, other distributions such as the number of replies to a post, the number of persons per country or the popularity of a tag has been realistically modeled either using known distributions or data extracted from real sources such as Dbpedia.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Correlated attributes and relations. Attribute values are not chosen at random, but follow correlations. For instance, people from a specific country have a larger probability to have names typical from that country, to work on companies from that country or to study at universities of that country. Also, we DATAGEN implements a relationship creation process that tries to reproduce the homophily principle, that is, people with similar characteristics tend to be connected.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;DATAGEN is built on top of Hadoop, to generate datasets of different sizes. It works either on single node SMP machines or a cluster environment. DATAGEN supports different output formats targeting different systems. On the one hand, we have the CSV format, where each entity and relation is output into a different comma separated value file. On the other hand, it also supports the Turtle format for RDF systems.&lt;/p&gt;
&lt;p&gt;Finally, DATAGEN outputs two other things:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Update Streams, which will be used in the future to implement updates in the workloads.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Substitution parameters, which are the parameters of the query instances the LDBC driver will issue. These are select so the query plans of the resulting query executions do not differ significantly.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Configuring and using DATAGEN is easy. Please visit &lt;a href=&#34;https://github.com/ldbc/ldbc_snb_datagen&#34;&gt;this page&lt;/a&gt; for more information.&lt;/p&gt;
&lt;h3 id=&#34;ldbc-driver&#34;&gt;LDBC driver&lt;/h3&gt;
&lt;p&gt;SNB is designed to be as easier to adopt as possible. Therefore, SNB provides the LDBC execution driver, which is designed to automatically generated the benchmark workload and gather the benchmark results. It then generates a stream of operations in conformance with a workload definition, and executes those operations against some system using the provided database connector, and with the substitution parameters produced by DATAGEN. During execution, the driver continuously measures performance metrics, then upon completion it generates a report of those metrics.&lt;/p&gt;
&lt;p&gt;It is capable of generating parallel workloads (e.g. concurrent reads and writes), while respecting the configured operation mix and ensuring that ordering between dependent operations is maintained. For further details on how the driver achieves that, please visit the Documentation &lt;a href=&#34;https://github.com/ldbc/ldbc_driver/wiki&#34;&gt;page&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The test sponsor (aka the implementer of the benchmark), has to provide a set of implemented interfaces, that form a benchmark implementation to plug into the driver, and then the benchmark is automatically executed.&lt;/p&gt;
&lt;p&gt;Given a workload consisting of a series of &lt;em&gt;Operations&lt;/em&gt;, the test sponsor implements &lt;em&gt;OperationHandlers&lt;/em&gt; __ for them. &lt;em&gt;OperationHandlers&lt;/em&gt; are responsible of executing instances of an specific operation (query) type. This is done by overriding the method &lt;em&gt;executeOperation&lt;/em&gt;(), which receives as input parameter an &lt;em&gt;Operation&lt;/em&gt; instance and returns the result. From &lt;em&gt;Operation&lt;/em&gt; __ instance, the operation&amp;rsquo;s input parameters can be retrieved, as well as the database connection state.&lt;/p&gt;
&lt;p&gt;The database connector is used to initialize, cleanup and get the database connection state. The database connector must implement the &lt;em&gt;Db&lt;/em&gt; interface, which consists of three methods: &lt;em&gt;onInit&lt;/em&gt;(), &lt;em&gt;onCleanup&lt;/em&gt;() and &lt;em&gt;getConnectionState&lt;/em&gt;(). &lt;em&gt;onInit&lt;/em&gt;() is called before the benchmark is executed, and is responsible of initializing the database and registering the different &lt;em&gt;OperationHandlers&lt;/em&gt;. &lt;em&gt;onCleanup&lt;/em&gt;() is called after the benchmark has completed. Any resources that need to be released should be released here.&lt;/p&gt;
&lt;p&gt;Finally, &lt;em&gt;getConnectionState&lt;/em&gt;() returns an instance of &lt;em&gt;DbConnectionState&lt;/em&gt;, which encapsulates any state that needs to be shared between &lt;em&gt;OperationHandler&lt;/em&gt; instances. For instance, this state could contain the necessary classes used to execute a given query for the implementing system.&lt;/p&gt;
&lt;p&gt;A good example on how to implement the benchmark can be found &lt;a href=&#34;https://github.com/ldbc/ldbc_driver/wiki/Implementing%20a%20Database%20Connector&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;h3 id=&#34;workloads&#34;&gt;Workloads&lt;/h3&gt;
&lt;p&gt;Currently, LDBC has only released the first draft of the Interactive workload, but the business intelligence and analytical workloads are on the works. Workloads are designed to mimic the different usage scenarios found in operating a real social network site, and each of them targets one or more types of systems. Each workload defines a set of queries and query mixes, designed to stress the systems under test in different choke-point areas, while being credible and realistic.&lt;/p&gt;
&lt;p&gt;Interactive workload reproduces the interaction between the users of the social network by including lookups and transactions that update small portions of the data base. These queries are designed to be interactive and target systems capable of responding such queries with low latency for multiple concurrent users. Examples of Interactive queries are, given a user, retrieve those friends with a specific name, or finding the most recent post and comments created by your friends.&lt;/p&gt;
&lt;p&gt;Business Intelligence workload, will represent those business intelligence analytics a social network company would like to perform in the social network, in order to take advantage of the data to discover new business opportunities. This workload will explore moderate portions of data from different entities, and will perform more complex and data intensive operations compared to the Interactive ones.&lt;/p&gt;
&lt;p&gt;Examples of possible Business Intelligence queries could be finding trending topics in country in a given moment, or looking for fraudulent “likers”.&lt;/p&gt;
&lt;p&gt;Finally, the Analytical workload will aim at exploring the characteristics of the underlying structure of the network. Shortest paths, community detection or centrality, are representative queries of this workload, and will imply touching a vast amount of the dataset.&lt;/p&gt;
&lt;h3 id=&#34;final-remarks&#34;&gt;Final remarks&lt;/h3&gt;
&lt;p&gt;This is just a quick overview of the SNB benchmark. For a more detailed description, do not hesitate to read the official SNB specification &lt;a href=&#34;https://github.com/ldbc/ldbc_snb_docs&#34;&gt;draft&lt;/a&gt;, and stay tunned to the LDBC blog for future blog posts detailing all of the SNB parts in depth.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Introducing SNB Interactive, the LDBC Social Network Benchmark Online Workload</title>
      <link>https://ldbcouncil.org/post/introducing-snb-interactive-the-ldbc-social-network-benchmark-online-workload/</link>
      <pubDate>Thu, 09 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/introducing-snb-interactive-the-ldbc-social-network-benchmark-online-workload/</guid>
      <description>&lt;p&gt;The LDBC Social Network Benchmark (SNB) is composed of three distinct workloads, interactive, business intelligence and graph analytics. This post introduces the interactive workload.&lt;/p&gt;
&lt;p&gt;The benchmark measures the speed of queries of medium complexity against a social network being constantly updated. The queries are scoped to a user&amp;rsquo;s social environment and potentially access data associated with the friends or a user and their friends.&lt;/p&gt;
&lt;p&gt;This is representative of an operational application. This goes beyond OLTP (On Line Transaction Processing) by having substantially more complex queries touching much more data than the point lookups and short reports in TPC-C or E. The emphasis is presenting a rich and timely view of a constantly changing environment.&lt;/p&gt;
&lt;p&gt;SNB Interactive gives end users and application developers a reference workload for comparing the relative merits of different technologies for graph data management. These range from dedicated graph databases to RDF stores and relational databases. There are graph serving benchmarks such as the Facebook Linkbench but SMB Interactive goes well beyond this in richness of schema and queries.&lt;/p&gt;
&lt;p&gt;The challenge to implementors is handling the user facing logic of a social network in a single system as the scale increases. The present practice in large social networks is massive sharding and use of different SQL and key value stores for different aspects of the service. The SNB workload is not intended to replicate this situation but to look for ways forward, so that one system can keep up with transactions and offer user rich and varied insight into their environment. The present practice relies on massive precomputation but SNB interactive seeks more agility and adhoc capability also on the operational side.&lt;/p&gt;
&lt;p&gt;The dataset is scaled in buckets, with distinct scales for 10, 30, 100, 300GB and so forth. A 100GB dataset has approximately 500,000 simulated users with their connections and online history. This is a convenient low-end single server size while 500 million users is 100TB, which is a data center scale requiring significant scale-out.&lt;/p&gt;
&lt;p&gt;The metric is operations per minute at scale. Online benchmarks typically have a fixed ratio between throughput and dataset size. Here we depart from this, thus one can report arbitrarily high throughputs at any scale. This makes main memory approaches feasible, which corresponds to present online practices.  The benchmark makes transactions and queries on a simulated timeline of social interactions. The challenge for the systm is to run this as fast as possible at the  selected  scale while providing fast and predictable response times. Throughput can be increased at the cost of latency but here the system must satisfy response time criteria while running at the reported throughput.&lt;/p&gt;
&lt;p&gt;Different technologies can be used for implementing SNB interactive. The workload is defined in natural language with sample implementations in SPARQL and Cypher. Other possibilities include SQL and graph database API&amp;rsquo;s.&lt;/p&gt;
&lt;p&gt;SNB Interactive is an example of LDBC&amp;rsquo;s choke point driven design methodology, where we draw on the combined knowledge and experience of several database system architects for defining realistic, yet ambitious challenges whose solution will advance the state of the art&lt;/p&gt;
&lt;p&gt;The benchmark specification and associated tools are now offered for public feedback. The LDBC partners working on SNB nteractive will provide sample implementations of the workload on their systems, including Virtuoso, Neo4J and Sparsity. Specifics of availability and coverage may vary.&lt;/p&gt;
&lt;p&gt;Subsequent posts will address the workload in more detail.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Is SNB Like Facebooks LinkBench</title>
      <link>https://ldbcouncil.org/post/is-snb-like-facebooks-linkbench/</link>
      <pubDate>Thu, 09 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/is-snb-like-facebooks-linkbench/</guid>
      <description>&lt;p&gt;In this post, I will discuss in some detail the rationale and goals of the design of the &lt;a href=&#34;https://ldbcouncil.org/benchmarks/snb&#34;&gt;Social Network Benchmark&lt;/a&gt; (SNB) and explain how it relates to real social network data as in Facebook, and in particular FaceBook&amp;rsquo;s own graph benchmark called &lt;a href=&#34;https://www.facebook.com/notes/facebook-engineering/linkbench-a-database-benchmark-for-the-social-graph/10151391496443920&#34;&gt;LinkBench&lt;/a&gt;. We think SNB is the most intricate graph database benchmark to date (it&amp;rsquo;s also available in RDF!), that already has made some waves. SNB recently received praise at the most important database systems conference &lt;a href=&#34;http://www.sigmod2014.org/&#34;&gt;SIGMOD in Snowbird&lt;/a&gt; after being used for this year&amp;rsquo;s &lt;a href=&#34;https://arxiv.org/pdf/2010.12243.pdf&#34;&gt;ACM SIGMOD Programming Contest&lt;/a&gt;, which was about graph analytics.&lt;/p&gt;
&lt;p&gt;SNB is intended to provide the following &lt;strong&gt;value&lt;/strong&gt; to different stakeholders:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;For end users facing graph processing tasks, SNB provides a recognizable scenario against which it is possible to &lt;em&gt;compare merits of different products&lt;/em&gt; and technologies.  By covering a wide variety of scales and price points, SNB can serve as an aid to technology selection.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;For vendors of graph database technology, SNB provides a &lt;em&gt;checklist of features&lt;/em&gt; and performance characteristics that helps in product positioning and can serve to guide new development.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;For researchers, both industrial and academic, the SNB dataset and workload provide &lt;em&gt;interesting challenges&lt;/em&gt; in multiple technical areas, such as query optimization, (distributed) graph analysis, transactional throughput, and provides a way to objectively compare the effectiveness and efficiency of new and existing technology in these areas.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;I should clarify that even though the data model of SNB resembles Facebook (and we&amp;rsquo;re extending it to also look more like Twitter), the goal of SNB is not to advise Facebook or Twitter what systems to use, they don&amp;rsquo;t need LDBC for that. Rather, we take social network data as a model for the much more broader graph data management problems that IT practitioners face. The particular characteristic of a graph data management problem is that the queries and analysis is not just about finding data by value, but about learning about the &lt;em&gt;connection patterns&lt;/em&gt; between data. The scenario of the SNB, a social network, was chosen with the following goals in mind:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;the benchmark scenario should be &lt;strong&gt;understandable&lt;/strong&gt; to a large audience, and this audience should also understand the relevance of managing such data.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;the scenario in the benchmark should cover the complete range of challenges &lt;strong&gt;relevant&lt;/strong&gt; for graph data management, according to the benchmark scope.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;the query challenges in it should be &lt;strong&gt;realistic&lt;/strong&gt; in the sense that, though synthetic, similar data and workloads are encountered in practice.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The SNB is in fact three distinct benchmarks with a common dataset, since there are &lt;em&gt;three different workloads&lt;/em&gt;. Each workload produces a single metric for performance at the given scale and a price/performance metric at the scale.  The full disclosure further breaks down the composition of the metric into its constituent parts, e.g. single query execution times.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Interactive Workload.&lt;/strong&gt;  The Interactive SNB workload is the first one we are releasing. It is defined in plain text, yet we have example implementations in Neo4j&amp;rsquo;s Cypher, SPARQL and SQL. The interactive workloads tests a system&amp;rsquo;s throughput with relatively simple queries with concurrent updates.  The system under test (SUT) is expected to run in a steady state, providing durable storage with smooth response times.  Inserts are typically small, affecting a few nodes at a time, e.g. uploading of a post and its tags.  Transactions may require serializability, e.g. verifying that something does not exist before committing the transaction.   Reads do not typically require more than read committed isolation. One could call the Interactive Workload an OLTP workload, but while queries typically touch a small fraction of the database, this can still be up to hundreds of thousands of values (the two-step neighborhood of a person in the social graph, often). Note that in order to support the read-queries, there is a lot of liberty to create indexing structures or materialized views, however such structures need to be maintained with regards to the continues inserts that also part of the workload. This workload is now in draft stage, which means that the data generator and &lt;a href=&#34;https://github.com/ldbc/ldbc_driver&#34;&gt;driver software stack&lt;/a&gt; are ready and the purpose is to obtain user feedback, as well as develop good system implementations.  The first implementations of this workload are now running on Openlink Virtuoso, Neo4j and Sparsity Sparksee, and we are eager to see people try these, and optimize and involve these.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Business Intelligence Workload.&lt;/strong&gt; There is a first stab at this workload formulated in SPARQL, tested against Openlink Virtuoso. The BI workload consists of complex structured queries for analyzing online behavior of users for marketing purposes.  The workload stresses query execution and optimization. Queries typically touch a large fraction of the data and do not require repeatable read.  The queries will be concurrent with trickle load (not out yet). Unlike the interactive workload, the queries touch more data as the database grows.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Graph Analytics Workload.&lt;/strong&gt; This workload is not yet available. It will test the functionality and scalability of the SUT for graph analytics that typically cannot be expressed in a query language. As such it is the natural domain for graph programming frameworks like Giraph. The workload is still under development, but will consist of algorithms like PageRank, Clustering and Breadth First Search. The analytics is done on most of the data in the graph as a single operation.  The analysis itself produces large intermediate results.  The analysis is not expected to be transactional or to have isolation from possible concurrent updates.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;All the SNB scenarios share a common scalable synthetic data set, generated by a state-of-the art data generator. We strongly believe in a single dataset that makes sense for all workloads, that is, the interactive and BI workloads will traverse data that has sensible PageRank outcomes, and graph clustering structure, etc. This is in contrast to &lt;a href=&#34;http://people.cs.uchicago.edu/~tga/pubs/sigmod-linkbench-2013.pdf&#34;&gt;LinkBench&lt;/a&gt;, released by the team of Facebook that manages the OLTP workload on the Facebook Graph, which closely tunes to the &lt;strong&gt;low-level&lt;/strong&gt; MySQL query patterns Facebook sees, but whose graph structure does not attempt to be realistic beyond average out degree of the nodes (so, it makes no attempts to create realistic community patterns or correlations) . The authors of LinkBench may be right that  the graph structure does not make a difference for simple insert/update/delete/lookup actions which LinkBench itself tests, but for the SNB queries in the Interactive and BI workloads this is not true. Note that &lt;a href=&#34;http://borthakur.com/ftp/sigmod2013.pdf&#34;&gt;Facebook&amp;rsquo;s IT infrastructure&lt;/a&gt; does not store all user data in MySQL and its modified memcached (&amp;quot;&lt;a href=&#34;http://www.cs.cmu.edu/~pavlo/courses/fall2013/static/papers/11730-atc13-bronson.pdf&#34;&gt;TAO&lt;/a&gt;&amp;quot;), some of it ends up in separate subsystems (using HDFS and HBase), which is outside of the scope of LinkBench. However, for queries like in the SNB Interactive and BI workloads it &lt;strong&gt;does&lt;/strong&gt; matter how people are connected, and how the attribute values  of connected people correlate. In fact, the SNB data generator is unique in that it generates a huge graph with &lt;em&gt;correlations&lt;/em&gt;, where people who live together, have the same interests or work for the same company have greater chance to be connected, and people from Germany have mostly German names, etc. Correlations frequently occur in practice and can strongly influence the quality of query optimization and execution, therefore LDBC wants to test their effects on graph data management systems (the impact of correlation among values and structure on query optimization and execution are a &amp;ldquo;choke point&amp;rdquo; for graph data management system where LDBC wants to stimulate innovation).&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Making It Interactive</title>
      <link>https://ldbcouncil.org/post/making-it-interactive/</link>
      <pubDate>Thu, 09 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/making-it-interactive/</guid>
      <description>&lt;p&gt;&lt;em&gt;Synopsis:&lt;/em&gt; Now is the time to finalize the interactive part of the Social Network Benchmark (SNB). The benchmark must be both credible in a real social network setting and pose new challenges. There are many hard queries but not enough representation for what online systems in fact do. So, the workload mix must strike a balance between the practice and presenting new challenges.&lt;/p&gt;
&lt;p&gt;It is about to be showtime for LDBC. The initial installment of the LDBC Social Network Benchmark (SNB) is the full data generator, test driver, workload and reference implementation for the interactive workload. SNB will further acquire business intelligence and graph analytics workloads but this post is about the interactive workload.&lt;/p&gt;
&lt;p&gt;As part of finalizing the interactive workload, we need to determine precise mixes of the component queries and updates. We note that the interactive mix so far consists of very heavy queries. These touch, depending on the scale upwards of a million entities in the database.&lt;/p&gt;
&lt;p&gt;Now, rendering a page view in a social network site does not touch millions of entities. The query that needs to be correct and up to date touches tens or hundreds of entities, e.g. posts or social connections for a single page impression. There are also statistical views like the count of people within so many steps or contact recommendations but these are not real time and not recalculated each time they are shown.&lt;/p&gt;
&lt;p&gt;So, LDBC SNB has a twofold task:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;In order to be a credible interactive workload, it must in fact have characteristics of one&lt;/li&gt;
&lt;li&gt;In order to stimulate progress it must have queries that are harder than those that go in routine page views but are still not database-wide analytics.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Designing a workload presents specific challenges:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;The workload must be realistic enough for users to identify with it.&lt;/li&gt;
&lt;li&gt;The workload must pose challenges and drive innovation in a useful direction.&lt;/li&gt;
&lt;li&gt;The component operations must all play a noticeable role in it.  If the operation&amp;rsquo;s relative performance doe does not affect the score, why is it in the workload?&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The interactive mix now has 14 queries that are interesting from a query optimization and execution viewpoint but touch millions of entities. This is not what drives page inpressions in online sites. Many users of GDB and RDF are about online sites, so this aspect must not be ignored.&lt;/p&gt;
&lt;p&gt;Very roughly, the choke points (technical challenges) of SNB interactive are as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Random access - Traversing between people, content makes large numbers of random lookups. These can be variously parallelized and/or vectored.&lt;/li&gt;
&lt;li&gt;Query optmization must produce right plans - The primary point isjoin order and join type.  Index vs. hash based joins have very different performance properties and the right choice depends on corectly guessing the number of rows and of distinct keys on either side of the join.&lt;/li&gt;
&lt;li&gt;When doing updates and lookups, the execution plan is obvious but there the choke point is the scheduling of large numbers of short operations.&lt;/li&gt;
&lt;li&gt;Many queries have aggregation, many have distinct, all have result ordering and a limit on result count. The diverse interactions of these operators produce optimization opportunities.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Dreaming up a scenario and workload is not enough for a benchmark. There must also be a strong indication that the job is do-able and plausible in the scenario.&lt;/p&gt;
&lt;p&gt;In online benchmarks different operations have different frequencies and the operations are repeated large numbers of times. There is a notion of steady state, so that the reported result represents a level of performance a system can sustain indefinitely.&lt;/p&gt;
&lt;p&gt;A key part of the workload definition is the workload mix, i.e. the relative frequencies of the operations. This decides in fact what the benchmark measures.&lt;/p&gt;
&lt;p&gt;The other aspect is the metric, typically some variation on operations per unit of time.&lt;/p&gt;
&lt;p&gt;All these are interrelated. Here we can take clicks per second as a metric, which is easy to understand. We wish to avoid the pitfall of TPC-C which ties the metric to a data size, so that for a high metric one must have a correspondingly larger database. This rule makes memory-only implementations in practice unworkable, while in reality many online systems in fact run from memory. So, here we scale in buckets, like in TPC-H but we still have an online workload. The scenario of the benchmark has its own timeline, here called simulation time. A benchmark run produces events in the simulation time but takes place in real time. This defines an accelration ratio. For example we could say that a system does 1000 operations per second at 300G scale, with an acceleration of 7x, i.e. 7 hours worth of simulation time are done in one hour of real time. A metric of this form is directly understandable for sizing a system, as long as the workload mix is realistic. We note that online sites usually are provisioned so that servers do not run anywhere near their peak throughput at a busy time.&lt;/p&gt;
&lt;p&gt;So how to define the actual mix? By measuring. But measuring requires a reference implementation that is generally up to date for the database science of the time and where the individual workload pieces are implemented in a reasonable manner, so no bad query plans or bad schema design. For the reference implementation, we use Virtuoso column store in SQL.&lt;/p&gt;
&lt;p&gt;But SQL is not graphy! Why not SPARQL? Because SPARQL has diverse fixed overheads and this is not a RDF-only workload. We do not want SPARQL overheads to bias the metric, we just want an implementation where we know exactly what goes on and how it works, with control of physical data placement so we know there are no obvious stupidities in any of this. SPARQL will come. Anyway, as said elsewhere, we believe that SPARQL will outgrow its overheads, at which point SQL or SPARQL is a matter of esthetic preference.  For now, it is SQL and all we want is transparency into the metal.&lt;/p&gt;
&lt;p&gt;Having this, we peg the operation mix to the update stream generated by the data generator. At the 30G scale, there are 3.5M new posts/replies per month of simulation time.  For each such, a query mix will be run, so as to establish a realistic read/write ratio. The query mix will have fractional queries, for example 0.2 friends recommendations per new post, but that is not a problem, since we run large numbers of these and at the end of the run can check that the ratios of counts are as expected.  Next, we run this as fast as it will go on the test system. Then we adjust the ratio of short and long queries to get two objectives:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Short queries should collectively be about 45% of the CPU load.&lt;/li&gt;
&lt;li&gt;Updates will be under 5%&lt;/li&gt;
&lt;li&gt;Long queries will take up the rest.  For long queries, we further tune the relative frequencies so that each represents a roughly equal slice of the time. Having a query that does not influence the metric is useless, so each gets enough showtime to have an impact but by their nature some are longer than others.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The reason why short queries should have a large slice is the fact that this is so in real interactive systems. The reason why long queries are important is driving innovation.  Like this we get both scheduling (short lookup/update) and optimization choke points covered. As a bonus be make the mix so that we get a high metric, so many clicks per second, since this is what the operator of an online site wants.&lt;/p&gt;
&lt;p&gt;There is a further catch: Different scales have different degrees of the friends graph and this will have a different influence on different queries.  To see whether this twists the metric out of shape we must experiment. For example, one must not have ogarithmic and linear complexity queries in the same mix, as BSBM for example has. So this is to be kept in mind as we proceed.&lt;/p&gt;
&lt;p&gt;In the next post we will look at the actual mix and execution times on the test system.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>SNB Data Generator - Getting Started</title>
      <link>https://ldbcouncil.org/post/snb-data-generator-getting-started/</link>
      <pubDate>Thu, 09 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/snb-data-generator-getting-started/</guid>
      <description>&lt;p&gt;In previous posts (&lt;a href=&#34;https://ldbcouncil.org/post/datagen-data-generation-for-the-social-network-benchmark&#34;&gt;this&lt;/a&gt; and &lt;a href=&#34;https://ldbcouncil.org/post/getting-started-with-snb&#34;&gt;this&lt;/a&gt;) we briefly introduced the design goals and philosophy behind DATAGEN, the data generator used in LDBC-SNB. In this post, I will explain how to use DATAGEN to generate the necessary datatsets to run LDBC-SNB. Of course, as DATAGEN is continuously under development,  the instructions given in this tutorial might change in the future.&lt;/p&gt;
&lt;h3 id=&#34;getting-and-configuring-hadoop&#34;&gt;Getting and Configuring Hadoop&lt;/h3&gt;
&lt;p&gt;DATAGEN runs on top of hadoop 1.2.1  to be scale. You can download it from here. Open a console and type the following commands to decompress hadoop into /home/user folder:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;$ cd /home/user
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;$ tar xvfz hadoop-1.2.1.tar.gz
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;For simplicity, in this tutorial we will run DATAGEN in standalone mode, that is, only one machine will be used, using only one thread at a time to run the mappers and reducers. This is the default configuration, and therefore anything else needs to be done for configuring it. For other configurations, such as Pseudo-Distributed (multiple threads on a single node) or Distributed (a cluster machine), visit the &lt;a href=&#34;https://github.com/ldbc/ldbc_snb_datagen_hadoop/wiki/Configuration&#34;&gt;LDBC DATAGEN wiki&lt;/a&gt;.&lt;/p&gt;
&lt;h3 id=&#34;getting-and-configuring-datagen&#34;&gt;Getting and configuring DATAGEN&lt;/h3&gt;
&lt;p&gt;Before downloading DATAGEN, be sure to fulfill the following requirements:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Linux based machine&lt;/li&gt;
&lt;li&gt;java 1.6 or greater&lt;/li&gt;
&lt;li&gt;python 2.7.X&lt;/li&gt;
&lt;li&gt;maven 3&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;After configuring hadoop, now is the time to get DATAGEN from the LDBC-SNB official repositories. Always download the latest release, which at this time is v0.1.2. Releases page is be found &lt;a href=&#34;https://github.com/ldbc/ldbc_snb_datagen_hadoop/releases&#34;&gt;here&lt;/a&gt;. Again, decompress the downloaded file with the following commands:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;$ cd /home/user
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;$ tar xvfz ldbc_snb_datagen-0.1.2.tar.gz
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This will create a folder called “ldbc_snb_datagen-0.1.2”.&lt;/p&gt;
&lt;p&gt;DATAGEN provides a &lt;em&gt;run.sh&lt;/em&gt; is a script to automate the compilation and execution of DATAGEN. It needs to be configured for your environment, so open it and set the two variables at the top of the script to the corresponding paths.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;HADOOP_HOME&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;/home/user/hadoop-1.2.1
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;LDBC_SNB_DATAGEN_HOME&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;/home/user/ldbc_snb_datagen
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;HADOOP_HOME points to the path where hadoop-1.2.1 is installed, while LDBC_SNB_DATAGEN_HOME points to where DATAGEN is installed. Change these variables to the appropriate values. Now, we can execute &lt;em&gt;run.sh&lt;/em&gt; script to compile and execute DATAGEN using default parameters. Type the following commands:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;$ cd /home/user/ldbc_snb_datagen-0.1.2
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;$ ./run.sh
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This will run DATAGEN, and two folders will be created at the same directory: &lt;em&gt;social_network&lt;/em&gt; containing the scale factor 1 dataset with csv uncompressed files, and &lt;em&gt;substitution_parameters&lt;/em&gt; containing the substituion parameters needed by the driver to execute the benchmark.&lt;/p&gt;
&lt;h3 id=&#34;changing-the-generated-dataset&#34;&gt;Changing the generated dataset&lt;/h3&gt;
&lt;p&gt;The characteristics of the dataset to be generated are specified in the &lt;em&gt;params.ini&lt;/em&gt; file. By default, this file has the following content:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-ini&#34; data-lang=&#34;ini&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;scaleFactor:1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;compressed:false&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;serializer:csv&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;numThreads:1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The following is the list of options and their default values supported by DATAGEN:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Option&lt;/th&gt;
&lt;th&gt;Default value&lt;/th&gt;
&lt;th&gt;Description&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;scaleFactor&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;&amp;ldquo;The scale factor of the data to generate. Possible values are: 1, 3, 10, 30, 100, 300 and 1000&amp;rdquo;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;serializer&lt;/td&gt;
&lt;td&gt;csv&lt;/td&gt;
&lt;td&gt;&amp;ldquo;The format of the output data. Options are: csv, csv_merge_foreign, ttl&amp;rdquo;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;compressed&lt;/td&gt;
&lt;td&gt;FALSE&lt;/td&gt;
&lt;td&gt;Specifies to compress the output data in gzip.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;outputDir&lt;/td&gt;
&lt;td&gt;./&lt;/td&gt;
&lt;td&gt;Specifies the folder to output the data.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;updateStreams&lt;/td&gt;
&lt;td&gt;FALSE&lt;/td&gt;
&lt;td&gt;&amp;ldquo;Specifies to generate the update streams of the network. If set to false, then the update portion of the network is output as static&amp;rdquo;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;numThreads&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;Sets the number of threads to use. Only works for pseudo-distributed mode&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;For instance, a possible &lt;em&gt;params.ini&lt;/em&gt; file could be the following:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-ini&#34; data-lang=&#34;ini&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;scaleFactor:30&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;serializer:ttl&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;compressed:true&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;updateStreams:false&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;outputDir:/home/user/output&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;numThreads:4&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;For those not interested on generating a dataset for a given predefined scale factor, but for other applications, the following parameters can be specified (they need to be specified all together):&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Option&lt;/th&gt;
&lt;th&gt;Default value&lt;/th&gt;
&lt;th&gt;Description&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;numPersons&lt;/td&gt;
&lt;td&gt;-&lt;/td&gt;
&lt;td&gt;The number of persons to generate&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;numYears&lt;/td&gt;
&lt;td&gt;-&lt;/td&gt;
&lt;td&gt;The amount of years of activity&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;startYear&lt;/td&gt;
&lt;td&gt;-&lt;/td&gt;
&lt;td&gt;The start year of simulation.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The following is an example of another possible &lt;em&gt;params.ini&lt;/em&gt; file&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-ini&#34; data-lang=&#34;ini&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;numPersons:100000&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;numYears:3&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;startYear:2010&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;serializer:csv_merge_foreign&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;compressed:false&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;updateStreams:true&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;outputDir:/home/user/output&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;&lt;span style=&#34;color:#a6e22e&#34;&gt;numThreads:4&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;For more information about the schema of the generated data, the different scale factors and serializers, please visit the wiki page of DATAGEN at &lt;a href=&#34;https://github.com/ldbc/ldbc_snb_datagen_hadoop/&#34;&gt;GitHub&lt;/a&gt;!&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>The Day of Graph Analytics</title>
      <link>https://ldbcouncil.org/post/the-day-of-graph-analytics/</link>
      <pubDate>Thu, 09 Oct 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/the-day-of-graph-analytics/</guid>
      <description>&lt;p&gt;&lt;em&gt;Note: consider this post as a continuation of the &amp;ldquo;&lt;a href=&#34;https://ldbcouncil.org/post/making-it-interactive&#34;&gt;Making it interactive&lt;/a&gt;&amp;rdquo; post by Orri Erling.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;I have now completed the &lt;a href=&#34;https://github.com/openlink/virtuoso-opensource&#34;&gt;Virtuoso&lt;/a&gt; TPC-H work, including scale out. Optimization possibilities extend to infinity but the present level is good enough. &lt;a href=&#34;http://www.tpc.org/tpch/&#34;&gt;TPC-H&lt;/a&gt; is the classic of all analytics benchmarks and is difficult enough, I have extensive commentary on this on my blog (In Hoc Signo Vinces series), including experimental results. This is, as it were, the cornerstone of the true science. This is however not the totality of it. From the LDBC angle, we might liken this to the last camp before attempting a mountain peak.&lt;/p&gt;
&lt;p&gt;So, we may now seriously turn to graph analytics. The project has enough left to run in order to get a good BI and graph analytics workload. In LDBC in general, as in the following, BI or business intelligence means complex analytical queries. Graph analytics means graph algorithms that are typically done in graph programming frameworks or libraries.&lt;/p&gt;
&lt;p&gt;The BI part is like TPC-H, except for adding the following challenges:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Joins of derived tables with group by, e.g. comparing popularity of items on consecutive time periods.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Transitive dimensions - A geographical or tag hierarchy can be seen as a dimension table. To get the star schema plan with the selective hash join, the count of the transitive traversal of the hierarchy (hash build side) must be correctly guessed.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Transitivity in fact table, i.e. average length of reply thread. There the cost model must figure that the reply link is much too high cardinality for hash build side, besides a transitive operation is not a good candidate for a build in multiple passes, hence the plan will have to be by index.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Graph traversal with condition on end point and navigation step. The hierarchical dimensions and reply threads are in fact trees, the social graph is not. Again the system must know some properties of connectedness (in/out degree, count of vertices) to guess a traversal fanout. This dictates the join type in the step (hash or index). An example is a transitive closure with steps satisfying a condition, e.g. all connected persons have a specific clearance.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Running one query with parameters from different buckets, implying different best plan.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Data correlations, e.g. high selectivity arising from two interests seldom occurring together, in places where the correct estimation makes the difference between a good and a bad plan.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Large intermediate results stored in tables, as in materializing complex summaries of data for use in follow up queries.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;More unions and outer joins.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The idea is to cover the base competences the world has come to expect and to build in challenges to last another 10-15 years.&lt;/p&gt;
&lt;p&gt;For rules and metric, we can use the TPC-H or &lt;a href=&#34;http://www.tpc.org/tpcds/default.asp&#34;&gt;TPC-DS&lt;/a&gt; ones as a template. The schema may differ from an implementation of the interactive workload, as these things would normally run on different systems anyway. As another activity that is not directly LDBC, I will do a merge of SNB and &lt;a href=&#34;http://www.openstreetmap.org/&#34;&gt;Open Street Map&lt;/a&gt;. The geolocated things (persons, posts) will get real coordinates from their vicinity and diverse geo analytics will become possible. This is of some significant interest to Geoknow, another FP7 where OpenLink is participating.&lt;/p&gt;
&lt;p&gt;Doing the BI mix and even optimizing the interactive part involves some redoing of the present support for transitivity in Virtuoso. The partitioned group by with some custom aggregates is the right tool for the job, with all parallelization, scale-out, etc ready. You see, TPC-H is very useful also in places one does not immediately associate with it.&lt;/p&gt;
&lt;p&gt;As a matter of fact, this becomes a BSP (bulk synchronous processing) control structure. Run any number of steps, each item produces results/effects scattered across partitions. The output of the previous is the input of the next. We might say BSP is an attractor or &amp;ldquo;Platonic&amp;rdquo; control structure to which certain paths inevitably lead. Last year I did a BSP implementation in SQL, reading and writing tables and using transactions for serializable update of the border. This is possible but will not compete with a memory based framework and not enough of the optimization potential, e.g. message combining, is visible to the engine in this formulation. So, now we will get this right, as suggested.&lt;/p&gt;
&lt;p&gt;So, the transitive derived table construct can have pluggable aggregations, e.g. remembering a path, a minimum length or such), reduction like a scalar-valued aggregate (min/max), different grouping sets like in a group by with cube or grouping sets, some group-by like reduction for message combining and so forth. If there is a gather phase that is not just the result of the scatter of the previous step, this can be expressed as an arbitrary database query, also cross partition in a scale-out setting.&lt;/p&gt;
&lt;p&gt;The distributed/partitioned group by hash table will be a first class citizen, like a procedure scoped temporary table to facilitate returning multiple results and passing large data between multiple steps with different vertex operations, e.g. forward and backward in betweenness centrality.&lt;/p&gt;
&lt;p&gt;This brings us to the graph analytics proper, which is often done in BSP style, e.g. &lt;a href=&#34;http://es.slideshare.net/shatteredNirvana/pregel-a-system-for-largescale-graph-processing&#34;&gt;Pregel&lt;/a&gt;, &lt;a href=&#34;http://giraph.apache.org&#34;&gt;Giraph&lt;/a&gt;, &lt;a href=&#34;http://uzh.github.io/signal-collect/&#34;&gt;Signal-Collect&lt;/a&gt;, some but not all Green-Marl applications. In fact, a Green-Marl back end for Virtuoso is conceivable, whether one will be made is a different matter.&lt;/p&gt;
&lt;p&gt;With BSP in the database engine, a reference implementation of many standard algorithms is readily feasible and performant enough to do reasonable sizing for the workload and to have a metric. This could be edges or vertices per unit of time, across a mix of algorithms, for example. Some experimentation will be needed. The algorithms themselves may be had from the Green-Marl sample programs or other implementations. Among others, Oracle would presumably agree that this sort of functionality will in time migrate into core database. We will here have a go at this and along the way formulate some benchmark tasks for a graph analytics workload. Whenever feasible, this will derive from existing work such as &lt;a href=&#34;http://graphbench.org/&#34;&gt;graphbench.org&lt;/a&gt; but will be adapted to the SNB dataset.&lt;/p&gt;
&lt;p&gt;The analytics part will be done with more community outreach than the interactive one. I will blog about the business questions, queries and choke points as we go through them. The interested may pitch in as the matter comes up.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Using LDBC SPB to Find OWLIM Performance Issues</title>
      <link>https://ldbcouncil.org/post/using-ldbc-spb-to-find-owlim-performance-issues/</link>
      <pubDate>Wed, 20 Aug 2014 00:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/post/using-ldbc-spb-to-find-owlim-performance-issues/</guid>
      <description>&lt;p&gt;During the past six months we (the OWLIM Team at Ontotext) have integrated the LDBC &lt;a href=&#34;https://ldbcouncil.org/developer/spb&#34;&gt;Semantic Publishing Benchmark&lt;/a&gt; (LDBC-SPB) as a part of our development and release process.&lt;/p&gt;
&lt;p&gt;First thing we’ve started using the LDBC-SPB for is to monitor the performance of our RDF Store when a new release is about to come out.&lt;/p&gt;
&lt;p&gt;Initially we’ve decided to fix some of the benchmark parameters :&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;the dataset size - 50 million triples (LDBC-SPB50) * benchmark warmup and benchmark run times - 60s and 600s respectively.  * maximum number of Editorail Agents (E) : 2 (threads that will execute INSERT/UPDATE operations) * maximum number of Aggregation Agents (A) : 16 (threads that will execute SELECT operations) * generated data by the benchmark driver to be “freshly” deployed before each benchmark run - benchmark driver can be configured to generate the data and stop. We’re using that option and have a fresh copy of it put aside ready for each run.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Having those parameters fixed, running LDBC-SPB is a straight-forward task. The hardware we’re using for benchmarking is a machine with 2 Intel Xeon CPUs, 8 cores each, 256 GB of memory and SSD storage, running Linux. Another piece of hardware we’ve tested with is a regular desktop machine with Intel i7, 32 GB of memory and HDD storage. During our experiments we have allowed a deviation in results of 5% to 10% because of the multi-threaded nature of the benchmark driver.&lt;/p&gt;
&lt;p&gt;We’ve also decided to produce some benchmark results on Amazon’s EC2 Instances and compare with the results we’ve had so far. Starting with m3.2xlarge instance (8 vCPUs, 30GB of memory and 2x80GB SSD storage) on a 50M dataset we’ve achieved more than 50% lower results than ones on our own hardware. On a largrer Amazon Instance c3.4xlarge (16 vCPUs, 30GB of memory and doubled SSD storage) we’ve achieved the same performance in terms of aggregation operations and even worse performance in terms for editorial operations, which we give to the fact that Amazon instances are not providing consistent performance all the time.&lt;/p&gt;
&lt;p&gt;Following two charts are showing how OWLIM performs on different hardware and with different configurations. They also give an indication of Amazon’s capabilities compared to the results achieved on a bare-metal hardware.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;16-2-Performance.png&#34; alt=&#34;image&#34;&gt;&lt;/p&gt;
&lt;p&gt;Figure 1 : OWLIM Performance : 2 amazon instances and 2 local machines. 16 aggregation and 2 editorial agents running simultaneously. Aggregation and editorial operations displayed here should  be considered independently, i.e. even though editorial opeartions graph shows higher results on Amazon m3.2xlarge instance, values are normalized and are referring to corresponding type of operation.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;8-0-Performance.png&#34; alt=&#34;image&#34;&gt;&lt;/p&gt;
&lt;p&gt;Figure 2 : OWLIM Performance : 2 amazon instances and 2 local machines. 8 aggregation running simultaneously. Read-only mode.&lt;/p&gt;
&lt;p&gt;Another thing that we’re using LDBC-SPB for is to monitor load performance speeds. Loading of generated data can be done either manually by creating some sort of a script (CURL), or by the benchmark driver itself which will execute a standard POST request against a provided SPARQL endpoint. Benchmark&amp;rsquo;s data generator can be configured to produce chunks of generated data in various sizes, which can be used for exeperiments on load performance. Of course load times of forward-chaining reasoners can not be compared to backward-chaining ones which is not the goal of the benchmark. Loading performances is not measured “officially“ by LDBC-SPB (although time for loading the data is reported), but its good thing to have when comparing RDF Stores.&lt;/p&gt;
&lt;p&gt;An additional and interesting feature of the SPB is the test for conformance to OWL2-RL rule-set. It is a part of the LDBC-SPB benchmark and that phase is called &lt;em&gt;checkConformance&lt;/em&gt;. The phase is run independently of the benchmark phase itself. It requires no data generation or loading except the initial set of ontologies. It tests RDF store’s capabilities for conformance to the rules in OWL2-RL rule-set by executing a number of INSERT/ASK queries specific for each rule. The result of that phase is a list of all rules that have been passed or failed which is very useful for regression testing.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Fourth TUC meeting</title>
      <link>https://ldbcouncil.org/event/fourth-tuc-meeting/</link>
      <pubDate>Thu, 03 Apr 2014 12:32:22 -0400</pubDate>
      
      <guid>https://ldbcouncil.org/event/fourth-tuc-meeting/</guid>
      <description>&lt;p&gt;The LDBC consortium are pleased to announce the fourth Technical User Community (TUC) meeting.&lt;/p&gt;
&lt;p&gt;This will be a one-day event at CWI in Amsterdam on &lt;em&gt;Thursday April 3, 2014&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;The event will include:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Introduction to the objectives and progress of the LDBC project.&lt;/li&gt;
&lt;li&gt;Description of the progress of the benchmarks being evolved through Task Forces.&lt;/li&gt;
&lt;li&gt;Users explaining their use-cases and describing the limitations they have found in current technology.&lt;/li&gt;
&lt;li&gt;Industry discussions on the contents of the benchmarks.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;All users of RDF and graph databases are welcome to attend. If you are interested, please contact: ldbc AT ac DOT upc DOT edu&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;For presenters please limit your talks to just 15 minutes&lt;/strong&gt;&lt;/p&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;April 3rd&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;10:00 Peter Boncz (VUA) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506371.pptx&#34;&gt;pptx&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=JYWVgrP1kVY&#34;&gt;video&lt;/a&gt;: &lt;em&gt;LDBC project status update&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;10:20 Norbert Martinez (UPC) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506375.pdf&#34;&gt;pdf&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=4yREJQ3yDr0&#34;&gt;video&lt;/a&gt;: &lt;em&gt;Status update on the LDBC Social  Network Benchmark (SNB) task force&lt;/em&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;10:50 Alexandru Iosup (TU Delft) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506363.ppt&#34;&gt;ppt&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=ulT-RFwKpOE&#34;&gt;video&lt;/a&gt;: &lt;em&gt;Towards Benchmarking Graph-Processing Platforms&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;11:10 Mike Bryant (Kings College) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506364.pptx&#34;&gt;pptx&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=KiHRTu9xx0A&#34;&gt;video&lt;/a&gt;: &lt;em&gt;EHRI Project: Archival Integration with Neo4j&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;11:30 coffee&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;11:50 Thilo Muth (University of Magdeburg) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506369.pptx&#34;&gt;pptx&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=5xH3UDLP6Oc&#34;&gt;video&lt;/a&gt;: &lt;em&gt;MetaProteomeAnalyzer: a graph database backed software for functional and taxonomic protein data analysis&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;12:10 Davy Suvee (Janssen Pharmaceutica / Johnson &amp;amp; Johnson) – &lt;a href=&#34;https://www.youtube.com/watch?v=XN3LRJUfJIU&#34;&gt;video&lt;/a&gt;: &lt;em&gt;Euretos Brain - Experiences on using a graph database to analyse data stored as a scientific knowledge graph&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;12:30 Yongming Luo (TU Eindhoven) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506366.pdf&#34;&gt;pdf&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=g_my3tBB2_s&#34;&gt;video&lt;/a&gt;: &lt;em&gt;Regularities and dynamics in bisimulation reductions of big graphs&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;12:50 Christopher Davis (TU Delft) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506370.pdf&#34;&gt;pdf&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/channel/UC6HbzfJ4016Vez-2HKNeDag&#34;&gt;video&lt;/a&gt;: &lt;em&gt;Enipedia - Enipedia is an active exploration into the applications of wikis and the semantic web for energy and industry issues&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;13:10 - 14:30 lunch @ restaurant Polder&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;14:30 &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506365.pptx&#34;&gt;SPB task force report&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;15:00 Bastiaan Bijl (Sysunite) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506373.pdf&#34;&gt;pdf&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=TsCeKDHShMY&#34;&gt;video&lt;/a&gt;: &lt;em&gt;Using a semantic approach for monitoring applications in large engineering projects&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;15:20 Frans Knibbe (Geodan) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506372.pptx&#34;&gt;pptx&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=uAX-m4OewPM&#34;&gt;video&lt;/a&gt;: &lt;em&gt;Benchmarks for geographical data&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;15:40 Armando Stellato (University of Rome, Tor Vergata &amp;amp; UN Food and Agriculture Organization) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506374.pptx&#34;&gt;pptx&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=mfA4csAs72Y&#34;&gt;video&lt;/a&gt;: &lt;em&gt;VocBench2.0, a Collaborative Environment for SKOS/SKOS-XL Management: scalability and (inter)operatibility challenges&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;16:00 coffee&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;16:20 Ralph Hodgson (TopQuadrant) – [pdf](https://pu b-3834 10a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachment s/5538064/5506367.pdf), &lt;a href=&#34;https://www.youtube.com/watch?v=ZUDnVw9P_Rc&#34;&gt;video&lt;/a&gt;:&lt;em&gt;Customer experiences in implementing SKOS-based vocabularymanagement systems&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;16:40 Simon Jupp (European Bioinformatics Institute) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506368.pdf&#34;&gt;pdf&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=CgTuOGK92W8&#34;&gt;video&lt;/a&gt;: &lt;em&gt;[Delivering RDF for the life science at the European Bioinformatics Institute: Six months in.]&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;17:00 Jerven Bolleman (Swiss Institute of Bioinformatics) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506381.pdf&#34;&gt;pdf&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=QTc3yOgoEsg&#34;&gt;video&lt;/a&gt;: &lt;em&gt;Breakmarking UniProt RDF. SPARQL queries that make your database cry&amp;hellip;&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;17:20 Rein van &amp;rsquo;t Veer (Digital Heritage Netherlands) – &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506380.pptx&#34;&gt;pptx&lt;/a&gt;, &lt;a href=&#34;https://www.youtube.com/watch?v=2vDrZoskGyQ&#34;&gt;video&lt;/a&gt; &lt;em&gt;Time and space for heritage&lt;/em&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;17:40 &lt;strong&gt;end of meeting&lt;/strong&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;19:00 - 21:30 Social Dinner in restaurant Boom&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;April 4th&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;LDBC plenary meeting for project partners.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5506362.ppt&#34;&gt;Benchmarking Graph-Processing Platforms: A Vision&lt;/a&gt; – Alexandru Iosup&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;logistics&#34;&gt;Logistics&lt;/h3&gt;
&lt;p&gt;The meeting will be held at the Dutch national research institute for computer science and mathematics (&lt;a href=&#34;http://www.cwi.nl&#34;&gt;CWI&lt;/a&gt; - Centrum voor Wiskunde en Informatica). It is located at &lt;a href=&#34;http://www.amsterdamsciencepark.nl/&#34;&gt;Amsterdam Science Park&lt;/a&gt;:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5505821.jpg&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;(&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/fourth-tuc-meeting/attachments/5538064/5505820.pdf&#34;&gt;A5 map&lt;/a&gt;)&lt;/p&gt;
&lt;h6 id=&#34;travel&#34;&gt;Travel&lt;/h6&gt;
&lt;p&gt;&lt;strong&gt;Arriving &amp;amp; departing:&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Amsterdam has a well-functioning and nearby airport called Schiphol (AMS, &lt;a href=&#34;http://www.schiphol.com/&#34;&gt;www.schiphol.nl&lt;/a&gt;) that serves all main European carriers and also very many low-fare carriers.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Trains&lt;/strong&gt; (~5 per hour) are the most convenient means of transport between Schiphol airport and Amsterdam city center, the Centraal Station (17 minutes, a train every 15 minutes) &amp;ndash; which station you are also likely arriving at in case of an international train trip.&lt;/p&gt;
&lt;p&gt;From the Centraal Station in Amsterdam, there is a direct train (every half an hour, runs 11 minutes) to the Science Park station, which is walking distance of CWI.  If you go from the Centraal Station to one of the hotels, you should take tram 9 &amp;ndash; it starts at Centraal Station (exception: for Hotel Casa 400, you should take the metro to Amstel station - any of the metros will do).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Taxi&lt;/strong&gt; is an alternative, though expensive. The price from Schiphol will be around 45 EUR to the CWI or another point in the city center (depending on traffic, the ride is 20-30 minutes).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Public transportation&lt;/strong&gt; (tram, bus, metro) tickets for a single ride and 1-day (24 hour) passes can be purchased from the driver/conductor on trams and buses (cash only) and from vending machines in the metro stations.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Only the &amp;ldquo;disposable&amp;rdquo; cards are interesting for you as visitor.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Multi-day (up to 7-days/168 hours) passes can only be purchased from the vending machines or from the ticket office opposite of Centraal Station.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Getting Around:&lt;/strong&gt; the fastest way to move in the city of Amsterdam generally is by bicycle. Consider renting such a device at your hotel. For getting from your hotel to the CWI, you can either take a taxi (expensive), have a long walk (35min), use public transportation (for NH Tropen/The Manor take bus 40 from Muiderpoort Station, for Hotel Casa 400 same bus 40 but from Amstel station, and for the Rembrandt Hotel it is tram 9 until Middenweg/Kruislaan and then bus 40), or indeed bike for 12 minutes.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Cars&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;In case you plan to arrive by car, please be aware that parking space in Amsterdam is scarce and hence very expensive. But, you can park your car on the &amp;ldquo;WCW&amp;rdquo; terrain where CWI is located. To enter the terrain by car, you have to get a ticket from the machine at the gate. To leave the terrain, again, you can get an exit ticket from the CWI reception.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Arriving at CWI:&lt;/strong&gt; Once you arrive at CWI, you need to meet the reception, and tell them that you are attending the LDBC TUC meeting. Then, you&amp;rsquo;ll receive a visitor&amp;rsquo;s pass that allows you to enter our building.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Social Dinner&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The social dinner will take place at 7pm on April 3 in Restaurant Boom, Linneausstraat 63, Amsterdam.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Third TUC Meeting</title>
      <link>https://ldbcouncil.org/event/third-tuc-meeting/</link>
      <pubDate>Tue, 19 Nov 2013 08:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/event/third-tuc-meeting/</guid>
      <description>&lt;p&gt;The LDBC consortium is pleased to announce the third Technical User Community (TUC) meeting!&lt;/p&gt;
&lt;p&gt;This will be a one day event in London on the &lt;strong&gt;19 November 2013&lt;/strong&gt; running in collaboration with the &lt;a href=&#34;http://www.graphconnect.com/&#34;&gt;GraphConnect&lt;/a&gt; event (18/19 November). Registered TUC participants that would like a free pass to all of GraphConnect should register for GraphConnect using this following coupon code: &lt;strong&gt;LDBCTUC&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The TUC event will include:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Introduction to the objectives and progress of the LDBC project&lt;/li&gt;
&lt;li&gt;Description of the progress of the benchmarks being evolved through Task Forces&lt;/li&gt;
&lt;li&gt;Users explaining their use-cases and describing the limitations they have found in current technology&lt;/li&gt;
&lt;li&gt;Industry discussions on the contents of the benchmarks&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We will also be launching the LDBC non-profit organization, so anyone outside the EU project will be able to join as a member.&lt;/p&gt;
&lt;p&gt;We will kick off  new benchmark development task forces in the coming year, and talks at this coming TUC will play an important role in deciding the use case scenarios that will drive those benchmarks.&lt;/p&gt;
&lt;p&gt;All users of RDF and graph databases are welcome to attend. If you are interested, please contact: ldbc AT ac DOT upc DOT edu&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#agenda&#34;&gt;Agenda&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#logistics&#34;&gt;Logistics&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#ldbctuc-background&#34;&gt;LDBC/TUC Background&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#social-network-benchmark&#34;&gt;Social Network Benchmark&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#semantic-publishing-benchmark&#34;&gt;Semantic Publishing Benchmark&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;November 19th - Public TUC Meeting&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;8:00 Breakfast and registration will open for Graph Connect/TUC at 8:00 am (Dexter House)&lt;/p&gt;
&lt;p&gt;short LDBC presentation (Peter Boncz) during GraphConnect keynote by Emil Eifrem (09:00-09:30 Dexter House)&lt;/p&gt;
&lt;p&gt;NOTE: the TUC meeting is at the Tower Hotel, nearby Dexter House.&lt;/p&gt;
&lt;p&gt;10:00 TUC Meeting Opening (Peter Boncz)&lt;/p&gt;
&lt;p&gt;10:10 TUC Presentations (RDF Application Descriptions)&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Johan Hjerling (BBC): &lt;em&gt;&lt;strong&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/5275669.pdf&#34;&gt;BBC Linked Data and the Semantic Publishing Benchmark&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;Andreas Both (Unister): &lt;em&gt;&lt;strong&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/5505027.pdf&#34;&gt;Ontology-driven applications in an e-commerce context&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;Nuno Carvalho (Fujitsu Laboratories Europe): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/5275666.pdf&#34;&gt;&lt;em&gt;&lt;strong&gt;Fujitsu RDF use cases and benchmarking requirements&lt;/strong&gt;&lt;/em&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Robina Clayphan (Europeana): &lt;em&gt;&lt;strong&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/4816977.ppt&#34;&gt;Europeana and Open Data&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;11:30 Semantic Publishing Benchmark (SPB)&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Venelin Kotsev (Ontotext - LDBC): &lt;em&gt;&lt;strong&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/4816974.pdf&#34;&gt;Semantic Publishing Benchmark Task Force Update&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt; and &lt;em&gt;&lt;strong&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/4816974.pdf&#34;&gt;report&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;12:00-13:00 Lunch at the Graph Connect venue&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Talks During Lunch:&lt;/em&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Pedro Furtado, Jorge Bernardino (Univ. Coimbra): &lt;strong&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/5275671.pdf&#34;&gt;KEYSTONE Cost Action&lt;/a&gt;&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;13:00 TUC Presentations (Graph Application Descriptions)&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Minqi Zhou / Weining Qian (East China Normal University): &lt;em&gt;&lt;strong&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/5275670.pdf&#34;&gt;Elastic and realistic social media data generation&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;Andrew Sherlock (Shapespace): &lt;em&gt;&lt;strong&gt;Shapespace Use Case&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;Sebastian Verheughe (Telenor): &lt;em&gt;&lt;strong&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/5275667.pdf&#34;&gt;Real-time Resource Authorization&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;14:00 Social Network Benchmark (SNB)&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Norbert Martinez (UPC - LDBC): &lt;em&gt;&lt;strong&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/5505025.pdf&#34;&gt;Social Network Benchmark Task Force Update&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt; and &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/4816975.pdf&#34;&gt;Report&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;em&gt;14:30 Break&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;14:45 TUC Presentations (Graph Analytics)&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Keith Houck (IBM): &lt;em&gt;&lt;strong&gt;Benchmarking experiences with [System G Native Store (tentative title)]&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;Abraham Bernstein (University of Zurich): &lt;em&gt;&lt;strong&gt;Streams and Advanced Processing: Benchmarking RDF querying beyond the Standard SPARQL Triple Store&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;Luis Ceze (University of Washington): &lt;em&gt;&lt;strong&gt;Grappa and GraphBench Status Update&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;em&gt;15:45 Break&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;16:00 TUC Presentations* (Possible Future RDF Benchmarking Topics)*&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Christian-Emil Ore (Unit for Digital Documentation, University of Oslo, Norway): &lt;em&gt;&lt;strong&gt;CIDOC-CRM&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;Atanas Kiryakov (Ontotext): &lt;em&gt;&lt;strong&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/5275672.pdf&#34;&gt;Large-scale Reasoning with a Complex Cultural Heritage Ontology (CIDOC CRM)&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;Kostis Kyzirakos (National and Kapodistrian University of Athens / CWI): &lt;em&gt;&lt;strong&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/5275668.pdf&#34;&gt;Geographica: A Benchmark for Geospatial RDF Stores&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;Xavier Lopez (Oracle): &lt;em&gt;&lt;strong&gt;W3C Property Graph progress&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;Thomas Scharrenbach (University Zurich) &lt;em&gt;&lt;strong&gt;PCKS:  Benchmarking Semantic Flow Processing Systems&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;17:20 Meeting Conclusion (Josep Larriba Pey)&lt;/p&gt;
&lt;p&gt;17:30 End of TUC meeting&lt;/p&gt;
&lt;p&gt;19:00 Social dinner&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;November 20th - Internal LDBC Meeting&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;10:00 Start&lt;/p&gt;
&lt;p&gt;12:30 &lt;em&gt;End of meeting&lt;/em&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;coffee and lunch provided&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;logistics&#34;&gt;Logistics&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;Date&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;19th November 2013&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Location&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The TUC meeting will be held in &lt;strong&gt;The Tower&lt;/strong&gt; hotel (&lt;a href=&#34;http://goo.gl/qZt8Fz&#34;&gt;Google Maps link&lt;/a&gt;) approximately 4 minutes walk from the &lt;a href=&#34;http://www.graphconnect.com/&#34;&gt;GraphConnect&lt;/a&gt; conference in London.&lt;/p&gt;
&lt;p&gt;Getting there&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;From City Airport is the easiest: short ride on the DLR to Tower Gateway. Easy.&lt;/li&gt;
&lt;li&gt;From London Heathrow: first need to take the Heathrow Express to Paddington. Then take the Circle line to Tower Hill. &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/4554995.pdf&#34;&gt;See attached&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;ldbctuc-background&#34;&gt;LDBC/TUC Background&lt;/h3&gt;
&lt;p&gt;Looking back, we have been working on two benchmarks for the past year: a Social Network Benchmark (SNB) and a Semantic Publishing Benchmark (SPB). While below we provide a short summary, all the details of the work on these benchmark development efforts can be found in the first yearly progress reports:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/4816974.pdf&#34;&gt;LDBC_SNB_Report_Nov2013.pdf&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/4816974.pdf&#34;&gt;LDBC_SPB_Report_Nov2013.pdf&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;A summary of these efforts can be read below or, for a more detailed account, please refer to: &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/4554967.pdf&#34;&gt;The Linked Data Benchmark Council: a Graph and RDF industry benchmarking effort&lt;/a&gt;. Annual reports about the progress, results, and future work of these two efforts will soon be available for download here, and will be discussed in depth at the TUC.&lt;/p&gt;
&lt;h4 id=&#34;social-network-benchmark&#34;&gt;Social Network Benchmark&lt;/h4&gt;
&lt;p&gt;The Social Network Benchmark (SNB) is designed for evaluating a broad range of technologies for tackling graph data management workloads. The systems targeted are quite broad: from graph, RDF, and relational database systems to Pregel-like graph compute frameworks. The social network scenario was chosen with the following goals in mind:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;it should be understandable, and the relevance of managing such data should be understandable&lt;/li&gt;
&lt;li&gt;it should cover the complete range of interesting challenges, according to the benchmark scope&lt;/li&gt;
&lt;li&gt;the queries should be realistic, i.e., similar data and workloads are encountered in practice&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;SNB includes a data generator for creation of synthetic social network data with the following characteristics:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;data schema is representative of real social networks&lt;/li&gt;
&lt;li&gt;data generated includes properties occurring in real data, e.g. irregular structure, structure/value correlations, power-law distributions&lt;/li&gt;
&lt;li&gt;the software generator is easy-to-use, configurable and scalable&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;SNB is intended to cover a broad range of aspects of social network data management, and therefore includes three distinct workloads:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Interactive&lt;/strong&gt;
&lt;ul&gt;
&lt;li&gt;Tests system throughput with relatively simple queries and concurrent updates, it is designed to test ACID features and scalability in an online operational setting.&lt;/li&gt;
&lt;li&gt;The targeted systems are expected to be those that offer transactional functionality.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Business Intelligence&lt;/strong&gt;
&lt;ul&gt;
&lt;li&gt;Consists of complex structured queries for analyzing online behavior of users for marketing purposes, it is designed to stress query execution and optimization.&lt;/li&gt;
&lt;li&gt;The targeted systems are expected to be those that offer an abstract query language.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Graph Analytics&lt;/strong&gt;
&lt;ul&gt;
&lt;li&gt;Tests the functionality and scalability of systems for graph analytics, which typically cannot be expressed in a query language.&lt;/li&gt;
&lt;li&gt;Analytics is performed on most/all of the data in the graph as a single operation and produces large intermediate results, and it is not not expected to be transactional or need isolation.&lt;/li&gt;
&lt;li&gt;The targeted systems are graph compute frameworks though database systems may compete, for example by using iterative implementations that repeatedly execute queries and keep intermediate results in temporary data structures.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id=&#34;semantic-publishing-benchmark&#34;&gt;Semantic Publishing Benchmark&lt;/h4&gt;
&lt;p&gt;The Semantic Publishing Benchmark (SPB) simulates the management and consumption of RDF metadata that describes media assets, or creative works.&lt;/p&gt;
&lt;p&gt;The scenario is a media organization that maintains RDF descriptions of its catalogue of creative works &amp;ndash; input was provided by actual media organizations which make heavy use of RDF, including the BBC. The benchmark is designed to reflect a scenario where a large number of aggregation agents provide the heavy query workload, while at the same time a steady stream of creative work description management operations are in progress. This benchmark only targets RDF databases, which support at least basic forms of semantic inference. A tagging ontology is used to connect individual creative work descriptions to instances from reference datasets, e.g. sports, geographical, or political information. The data used will fall under the following categories: reference data, which is a combination of several Linked Open Data datasets, e.g. GeoNames and DBpedia; domain ontologies, that are specialist ontologies used to describe certain areas of expertise of the publishing, e.g., sport and education; publication asset ontologies, that describe the structure and form of the assets that are published, e.g., news stories, photos, video, audio, etc.; and tagging ontologies and the metadata, that links assets with reference/domain ontologies.&lt;/p&gt;
&lt;p&gt;The data generator is initialized by using several ontologies and datasets. The instance data collected from these datasets are then used at several points during the execution of the benchmark. Data generation is performed by generating SPARQL fragments for create operations on creative works and executing them against the RDF database system.&lt;/p&gt;
&lt;p&gt;Two separate workloads are modeled in SPB:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Editorial:&lt;/strong&gt; Simulates creating, updating and deleting creative work metadata descriptions.  Media companies use both manual and semi-automated processes for efficiently and correctly managing asset descriptions, as well as annotating them with relevant instances from reference ontologies.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Aggregation:&lt;/strong&gt; Simulates the dynamic aggregation of content for consumption by the distribution pipelines (e.g. a web-site). The publishing activity is described as &amp;ldquo;dynamic&amp;rdquo;, because the content is not manually selected and arranged on, say, a web page. Instead, templates for pages are defined and the content is selected when a consumer accesses the page.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/third-tuc-meeting/attachments/4325436/5505026.pdf&#34;&gt;Status of the Semantic Publishing Benchmark&lt;/a&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Second TUC Meeting</title>
      <link>https://ldbcouncil.org/event/second-tuc-meeting/</link>
      <pubDate>Mon, 22 Apr 2013 10:00:00 +0000</pubDate>
      
      <guid>https://ldbcouncil.org/event/second-tuc-meeting/</guid>
      <description>&lt;p&gt;The LDBC consortium are pleased to announce the second Technical User Community (TUC) meeting.&lt;/p&gt;
&lt;p&gt;This will be a two day event in Munich on the &lt;strong&gt;22/23rd April 2013&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The event will include:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Introduction to the objectives and progress of the LDBC project.&lt;/li&gt;
&lt;li&gt;Description of the progress of the benchmarks being evolved through Task Forces.&lt;/li&gt;
&lt;li&gt;Users explaining their use-cases and describing the limitations they have found in current technology.&lt;/li&gt;
&lt;li&gt;Industry discussions on the contents of the benchmarks.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;All users of RDF and graph databases are welcome to attend. If you are interested, please contact: ldbc AT ac DOT upc DOT edu&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#agenda&#34;&gt;Agenda&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#logistics&#34;&gt;Logistics&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#date&#34;&gt;Date&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#location&#34;&gt;Location&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#venue&#34;&gt;Venue&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#getting-to-the-tum-campus-from-the-munich-city-center-subway-u-bahn&#34;&gt;Getting to the TUM Campus from the Munich city center: Subway (U-Bahn)&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#getting-to-the-tum-campus-from-the-munich-airport&#34;&gt;Getting to the TUM Campus from the Munich Airport&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#getting-to-the-tum-campus-from-garching-u-bahn&#34;&gt;Getting to the TUM Campus from Garching: U-Bahn&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#getting-there&#34;&gt;Getting there&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#social-dinner&#34;&gt;Social Dinner&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;April 22nd&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;10:00 &lt;em&gt;Registration.&lt;/em&gt;&lt;br&gt;
10:30 Josep Lluis Larriba Pey (UPC) - &lt;em&gt;Welcome and Introduction.&lt;/em&gt;&lt;br&gt;
10:30 Peter Boncz (VUA): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/second-tuc-meeting/attachments/2523698/2687373.pptx&#34;&gt;LDBC: goals and status&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Social Network Use Cases (with discussion moderated by Josep Lluis Larriba Pey)&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;11:00 Josep Lluis Larriba Pey (UPC): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/second-tuc-meeting/attachments/2523698/2687372.pdf&#34;&gt;Social Network Benchmark Task Force&lt;/a&gt;&lt;br&gt;
11:30 Gustavo González (Mediapro): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/second-tuc-meeting/attachments/2523698/2687367.pdf&#34;&gt;Graph-based User Modeling through Real-time Social Streams&lt;/a&gt;&lt;br&gt;
12:00 Klaus Großmann (Dshini): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/second-tuc-meeting/attachments/2523698/2687365.pdf&#34;&gt;Neo4j at Dshini&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;12:30 Lunch&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Semantic Publishing Use Cases (with discussion moderated by Barry Bishop)&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;13:30 Barry Bishop (Ontotext): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/second-tuc-meeting/attachments/2523698/2687366.pptx&#34;&gt;Semantic Publishing Benchmark Task Force&lt;/a&gt;&lt;br&gt;
14:00 Dave Rogers (BBC): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/second-tuc-meeting/attachments/2523698/2687364.pptx&#34;&gt;Linked Data Platform at the BBC&lt;/a&gt;&lt;br&gt;
14:30 Edward Thomas (Wolters Kluwer): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/second-tuc-meeting/attachments/2523698/2687374.pdf&#34;&gt;Semantic Publishing at Wolters Kluwer&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;15:00 Coffee break&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Projects Related to LDBC&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;15:30 Fabian Suchanek (MPI): &amp;ldquo;YAGO: A large knowledge base from Wikipedia and WordNet&amp;rdquo;&lt;br&gt;
16:00 Antonis Loziou (VUA): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/second-tuc-meeting/attachments/2523698/2687375.pptx&#34;&gt;The OpenPHACTS approach to data integration&lt;/a&gt;&lt;br&gt;
16:30 Mirko Kämpf (Brox): &amp;ldquo;GeoKnow - Spatial Data Web project and Supply Chain Use Case&amp;rdquo;&lt;/p&gt;
&lt;p&gt;17:00 &lt;em&gt;End of first day&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;19:00 Social dinner&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;April 23rd&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Industry &amp;amp; Hardware Aspects&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;10:00 Xavier Lopez (Oracle): &lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/second-tuc-meeting/attachments/2523698/2687384.pdf&#34;&gt;Graph Database Performance an Oracle Perspective.pdf&lt;/a&gt;&lt;br&gt;
10:30 Pedro Trancoso (University of Cyprus): &amp;ldquo;Benchmarking and computer architecture: the research side&amp;rdquo;&lt;/p&gt;
&lt;p&gt;11:00 Coffee break&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Future Steps and TUC feedback session&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;11:30 Peter Boncz (VUA) moderates: next steps in the Social Networking Task Force&lt;br&gt;
12:00 Barry Bishop (Ontotext) moderates: next steps in the Semantic Publishing Task Force&amp;quot;&lt;/p&gt;
&lt;p&gt;12:30 &lt;em&gt;End of meeting&lt;/em&gt;&lt;/p&gt;
&lt;h3 id=&#34;logistics&#34;&gt;Logistics&lt;/h3&gt;
&lt;h4 id=&#34;date&#34;&gt;Date&lt;/h4&gt;
&lt;p&gt;22nd and 23th April 2013&lt;/p&gt;
&lt;h4 id=&#34;location&#34;&gt;Location&lt;/h4&gt;
&lt;p&gt;The TUC meeting will be held at LE009 room at LRZ (Leibniz-Rechenzentrum) located inside the TU Munich campus in Garching, Germany. The address is:&lt;/p&gt;
&lt;p&gt;LRZ (Leibniz-Rechenzentrum)&lt;br&gt;
Boltzmannstraße 1&lt;br&gt;
85748 Garching, Germany&lt;/p&gt;
&lt;h4 id=&#34;venue&#34;&gt;Venue&lt;/h4&gt;
&lt;p&gt;To reach the campus, there are several options, including Taxi and Subway &lt;a href=&#34;http://www.in.tum.de/fileadmin/user_upload/Sonstiges/anfahrt_garching.pdf&#34;&gt;Ubahn&lt;/a&gt;&lt;/p&gt;
&lt;h5 id=&#34;getting-to-the-tum-campus-from-the-munich-city-center-subway-u-bahn&#34;&gt;Getting to the TUM Campus from the Munich city center: Subway (U-Bahn)&lt;/h5&gt;
&lt;p&gt;Take the U-bahn line U6 in the direction of Garching-Forschungszentrum, exit at the end station. Take the south exit to MI-Building and LRZ on the Garching Campus. The time of the journey from the city center is approx. 25-30 minutes. In order to get here from the City Center, you need the Munich XXL ticket that costs around 7.50 euros and covers all types of transportation for one day. The ticket has to be validated before ride.&lt;/p&gt;
&lt;h5 id=&#34;getting-to-the-tum-campus-from-the-munich-airport&#34;&gt;Getting to the TUM Campus from the Munich Airport&lt;/h5&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;(except weekends) S-Bahn S8 line in the direction of (Hauptbahnhof) Munich Central Station until the third stop, Ismaning (approx. 13 minutes). From here Bus Nr. 230 until stop MI-Building on the Garching Campus. Alternatively: S1 line until Neufahrn, then with the Bus 690, which stops at Boltzmannstraße.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;S-Bahn lines S8 or S1 towards City Center until Marienplatz stop. Then change to U-bahn U6 line towards Garching-Forschungszentrum, exit at the last station. Take the south exit to MI-Building and LRZ.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Taxi: fare is ca. 30-40 euros.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;For cases 1 and 2, before the trip get the One-day Munich Airport ticket and validate it. It will cover all public transportation for that day.&lt;/p&gt;
&lt;h5 id=&#34;getting-to-the-tum-campus-from-garching-u-bahn&#34;&gt;Getting to the TUM Campus from Garching: U-Bahn&lt;/h5&gt;
&lt;p&gt;The city of Garching is located on the U6 line, one stop before the Garching-Forschungszentrum. In order to get from Garching to Garching-Forschungszentrum with the U-bahn, a special one-way ticket called Kurzstrecke (1.30 euros) can be purchased.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Finding LRZ@TUM&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;http://www.openstreetmap.org/?mlat=48.2615702464&amp;amp;mlon=11.6686558264&amp;amp;zoom=32&#34;&gt;OpenStreetMap link&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://maps.google.com/maps?q=48.2615702464,11.6686558264&amp;amp;spn=0.005,0.005&amp;amp;t=k&#34;&gt;Google Maps link&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/second-tuc-meeting/attachments/2523698/2687268.gif&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/second-tuc-meeting/attachments/2523698/2687269.gif&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;h4 id=&#34;getting-there&#34;&gt;Getting there&lt;/h4&gt;
&lt;p&gt;&lt;strong&gt;Flying: Munich&lt;/strong&gt; airport is located 28.5 km northeast of Munich. There are two ways to get from the airport to the city center: suburban train (S-bahn) and Taxi.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;S-Bahn:&lt;/strong&gt; S-bahn lines S1 and S8 will get you from the Munich airport to the city center, stopping at both Munich Central Station (Hauptbahnhof) and Marienplatz. One-day Airport-City ticket costs 11.20 euros and is valid for the entire Munich area public transportation during the day of purchase (the tickets needs to be validated before the journey). S-bahn leaves every 5-20 minutes and reaches the city center in approx. 40 minutes.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Taxi:&lt;/strong&gt; taxi from the airport to the city center costs approximately 50 euros&lt;/p&gt;
&lt;h4 id=&#34;social-dinner&#34;&gt;Social Dinner&lt;/h4&gt;
&lt;p&gt;The social dinner will take place at 7 pm on April 22 in Hofbräuhaus (second floor)&lt;/p&gt;
&lt;p&gt;Address: Hofbräuhaus, Platzl 9, Munich&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>First TUC Meeting</title>
      <link>https://ldbcouncil.org/event/first-tuc-meeting/</link>
      <pubDate>Mon, 19 Nov 2012 09:00:00 +0100</pubDate>
      
      <guid>https://ldbcouncil.org/event/first-tuc-meeting/</guid>
      <description>&lt;p&gt;The LDBC consortium are pleased to announce the first Technical User Community (TUC) meeting. This will be a two day event in Barcelona on the &lt;strong&gt;19/20th November 2012&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;So far more than six commercial consumers of graph/RDF database technology have expressed an interest in attending the event and more are welcome. The proposed format of the event wil include:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Introduction by the coordinator and technical director explaining the objectives of the LDBC project&lt;/li&gt;
&lt;li&gt;Invitation to users to explain their use-cases and describe the limitations they have found in current technology&lt;/li&gt;
&lt;li&gt;Brain-storming session for identifying trends and mapping out strategies to tackle existing choke-points&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The exact agenda will be published here as things get finalised before the event.&lt;/p&gt;
&lt;p&gt;All users of RDF and graph databases are welcome to attend. If you are interested, please contact: ldbc AT ac DOT upc DOT edu&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#agenda&#34;&gt;Agenda&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#slide&#34;&gt;Slide&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#logistics&#34;&gt;Logistics&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;#date&#34;&gt;Date&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#location&#34;&gt;Location&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#venue&#34;&gt;Venue&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;#getting-there&#34;&gt;Getting there&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;agenda&#34;&gt;Agenda&lt;/h3&gt;
&lt;p&gt;We will start at 9:00 on Monday for a full day, followed by a half a day on Tuesday to allow attendees to travel home on the evening of the 20th.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Day 1&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;09:00 Welcome (Location: Aula Master)&lt;br&gt;
09:30 Project overview (Emphasis on task forces?) + Questionnaire results?&lt;br&gt;
10:30 Coffee break&lt;br&gt;
11:00 User talks (To gather information for use cases?)&lt;/p&gt;
&lt;p&gt;13:00 Lunch&lt;/p&gt;
&lt;p&gt;14:00 User talks (cont.)&lt;br&gt;
15:00 Use case discussions (based on questionnaire results + consortium proposal + user talks).&lt;br&gt;
16:00 Task force proposals (consortium)&lt;br&gt;
17:00 Finish first day&lt;/p&gt;
&lt;p&gt;20:00 Social dinner&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Day 2&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;10:00 Task force discussion (consortium + TUC)&lt;br&gt;
11:00 Coffe break&lt;br&gt;
11:30 Task force discussion (consortium + TUC)&lt;br&gt;
12:30 Summaries (Task forces, use cases, &amp;hellip;) and actions&lt;/p&gt;
&lt;p&gt;13:00 Lunch and farewell&lt;/p&gt;
&lt;p&gt;15:00 LDBC Internal meeting&lt;/p&gt;
&lt;h3 id=&#34;slide&#34;&gt;Slide&lt;/h3&gt;
&lt;p&gt;Opening session:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/2686995.pptx&#34;&gt;CWI – Peter Boncz&lt;/a&gt; – Objectives&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/2687001.pdf&#34;&gt;UPC – Larri&lt;/a&gt; – Questionnaire&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;User stories:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/2686998.pdf&#34;&gt;BBC – Jem Rayfield&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;CA Technologies – Victor Muntés&lt;/li&gt;
&lt;li&gt;Connected Discovery (Open Phacts) – Bryn Williams-Jones&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/2687003.pptx&#34;&gt;Elsevier – Alan Yagoda&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/2687000.pptx&#34;&gt;ERA7 Bioinformatics – Eduardo Pareja&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/2687005.pptx&#34;&gt;Press Association – Jarred McGinnis&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/2687004.pptx&#34;&gt;RJLee – David Neuer&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/2686994.pdf&#34;&gt;Yale – Lec Maj&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Benchmark proposals:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/2686991.pdf&#34;&gt;Publishing benchmark proposal – Ontotext – Barry Bishop&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/2687002.pdf&#34;&gt;Social Network Benchmark Proposal – UPC – Larri&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id=&#34;logistics&#34;&gt;Logistics&lt;/h4&gt;
&lt;h5 id=&#34;date&#34;&gt;Date&lt;/h5&gt;
&lt;p&gt;19th and 20th November 2012&lt;/p&gt;
&lt;h5 id=&#34;location&#34;&gt;Location&lt;/h5&gt;
&lt;p&gt;The TUC meeting will be held at “Aula Master” at A3 building located inside the “Campus Nord de la UPC” in Barcelona. The address is:&lt;/p&gt;
&lt;p&gt;Aula Master&lt;br&gt;
Edifici A3, Campus Nord UPC&lt;br&gt;
C. Jordi Girona, 1-3&lt;br&gt;
08034 Barcelona, Spain&lt;/p&gt;
&lt;h4 id=&#34;venue&#34;&gt;Venue&lt;/h4&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/1933315.jpg&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Finding UPC&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/1933318.jpg&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Finding the meeting room&lt;/strong&gt;&lt;/p&gt;
&lt;h4 id=&#34;getting-there&#34;&gt;Getting there&lt;/h4&gt;
&lt;p&gt;&lt;strong&gt;Flying:&lt;/strong&gt; Barcelona airport is situated 12 km from the city. There are several ways of getting from the airport to the centre of Barcelona, the cheapest of which is to take the train located outside just a few minutes walking distance past the parking lots at terminal 2 (there is a free bus between terminal 1 and terminal 2, see this &lt;a href=&#34;http://goo.gl/maps/iJqlj&#34;&gt;map of the airport&lt;/a&gt;). It is possible to buy 10 packs of train tickets which makes it cheaper. Taking the bus to the centre of town is more convenient as they leave directly from terminal 1 and 2, however it is more expensive than the train.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Rail:&lt;/strong&gt; The Renfe commuter train leaves the airport every 30 minutes from 6.13 a.m. to 11.40 p.m. Tickets cost around 3€ and the journey to the centre of Barcelona (Sants or Plaça Catalunya stations) takes 20 minutes.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Bus:&lt;/strong&gt; The Aerobus leaves the airport every 12 minutes, from 6.00 a.m. to 24.00, Monday to Friday, and from 6.30 a.m. to 24.00 on Saturdays, Sundays and public holidays. Tickets cost 6€ and the journey ends in Plaça Catalunya in the centre of Barcelona.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Taxi:&lt;/strong&gt; From the airport, you can take one of Barcelona&amp;rsquo;s typical black and yellow taxis. Taxis may not take more than four passengers. Unoccupied taxis display a green light and have a clearly visible sign showing LIBRE or LLIURE. The trip to Sants train station costs approximately €16 and trips to other destinations in the city cost approximately €18.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Train and bus:&lt;/strong&gt; Barcelona has two international train stations: Sants and França. Bus companies have different points of arrival in different parts of the city. You can find detailed information in the following link: &lt;a href=&#34;http://www.barcelona-airport.com/eng/transport_eng.htm&#34;&gt;http://www.barcelona-airport.com/eng/transport_eng.htm&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/1933316.jpg&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The locations of the airport and the city centre&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://pub-383410a98aef4cb686f0c7601eddd25f.r2.dev/event/first-tuc-meeting/attachments/1671180/1933317.jpg&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Bus map&lt;/strong&gt;&lt;/p&gt;
</description>
    </item>
    
  </channel>
</rss>