rfc for now() in INSERT INTO

Options_for_a_now_function/README.md

@@ -0,0 +1,95 @@
+# Options For A Now() Function
+
+## Introduction
+
+This document outlines the options for a `now()` function in TS SQL
+
+## Purpose
+
+To outline the various technical options and their uses cases, pros and cons and enable the correct product decision to be made
+
+## Scope
+
+The scope of this document is a `now()` function in an `INSERT` statement.
+
+```sql
+INSERT INTO mytable VALUES("myfamily", "myseries", now(), "mypayload");
+```
+
+The use of `now()` in a `SELECT` statement is out of scope (and assumed to be fine and jim dandy).
+
+# Statement Of The Problem
+
+A **naïve** implementation of `now()` in an `INSERT` statement would result in inconsistent data with a very high frequency.
+
+For the purposes of exposition we shall assume a 3 node Riak cluster with a client writing to it and a remote `riak-shell`
+
+The three Riak nodes suffer from a 1 hour clock drift.
+
+The client writes to the cluster with a load balancer.
+
+Consider this execution path:
+
+![Problem Scenario](./problem_scenario.png)
+
+We see that the results return:
+```
+{b, 13:00:00}
+{a, 12:00:01}
+{c, 11:00:02}
+```
+
+These are out of order.
+
+This problem never occurs with `riak-shell` because it always connects to the same riak node:
+
+![Problem Scenario](./riak_shell_basic.png)
+
+## The Correct Way
+
+The correct way is to implement a consistent server-side timestamp path. That timestamp could be used with a SQL command like:
+
+```sql
+INSERT INTO mytable VALUES("a", now());
+```
+
+or in the `CREATE TABLE` command like:
+
+```sql
+CREATE TABLE GeoCheckin
+(
+   id           SINT64    NOT NULL,
+   region       VARCHAR   NOT NULL,
+   state        VARCHAR   NOT NULL,
+   time         TIMESTAMP NOT NULL AUTO,
+   weather      VARCHAR   NOT NULL,
+   temperature  DOUBLE,
+   PRIMARY KEY (
+     (id, QUANTUM(time, 15, 'm')),
+      id, time
+   )
+);
+```
+
+The execution flow is the same but different routing questions are asked - let us use the `INSERT INTO` statement for exposition purposes. We will only show the `client` path - the `riak-shell` one is trivially identical:
+
+![Correct Way](./correct_way.png)
+
+There are two variants on the ***correct way***:
+
+* **strictly correct** if the canonical node isn't available the write fails
+* **bit shit mode** if the canonical node isn't available go to fall back and let the guarantees go to hell in the proverbial
+
+**Note**: oh, yeah, if you go ***bit shit*** yon John Daily runs you down in a banana-yella jeep, so there is that...
+
+## Convenience Option
+
+In this world `now()` in insert is seen as a convenience for play about and is optimised for onboarding and not production.
+
+In this world we introduce a lexical token in the `riak-shell` that forces time serialisation in the `riak-client` only:
+
+![Convenience Option](./convenience_option.png)
+
+## Choices
+
+I go for convenience option - targetted at onboarding - and if we are going to go down the correct route we should design and implement the `CREATE TABLE` version first and then add `now()` to the `INSERT INTO` path.

Options_for_a_now_function/convenience_option.msc

@@ -0,0 +1,61 @@
+// diagram for upgrade_downgrade_specs.md
+//
+// Upgrade/Downgrade Scenario 1 diagram
+//
+// This diagram is generated by mscgen
+// which can be installed on ubuntu by
+// sudo apt-get install mscgen
+//
+// Generate this diagram on the command line with:
+// > mscgen -T png -i problem_scenario.msc -F ./fonts/DejaVuSansMono.ttf
+//
+msc {
+
+	// Image defaults
+	width="1200", wordwraparcs="1";
+
+	// declare objects
+	// This is a cluster with 3 nodes for exposition purposes
+	// a client and a riak-shell 
+
+	"Client", "Node 1", "Node 2", "Node 3", "Riak Shell";
+
+    "Node 1" box "Node 1" [label="\nTime: T + 1h\n"];
+    "Node 2" box "Node 2" [label="\nTime: T\n"];
+    "Node 3" box "Node 3" [label="\nTime: T - 1h\n"];
+
+    ...;
+
+    |||;
+
+    ---;
+
+    "Riak Shell" box "Riak Shell" [label="\nINSERT INTO mytable VALUES(a, $now);\n"],
+
+	|||;
+
+	"Riak Shell" => "Riak Shell" [label = "\nexpand $now =>'2016-04-23 11:12:13'\n"];
+
+	|||;
+
+	"Riak Shell" => "Node 2" [label = "send INSERT INTO\n\nmytable VALUES(a, '2016-04-23 11:12:13');"];
+
+    |||;
+
+	"Node 2" => "Node 2" [label = "write {a, '2016-04-23 11:12:13'}"];
+
+    |||;
+
+    ...;
+
+    |||;
+
+	"Client" => "Node 1" [label = "send INSERT INTO\n\nmytable VALUES(a, $now);"];
+
+    |||;
+
+ 	"Node 1" => "Client" [label = "Error: syntax error"];
+
+    |||;
+
+}

Options_for_a_now_function/correct_way.msc

@@ -0,0 +1,69 @@
+// diagram for upgrade_downgrade_specs.md
+//
+// Upgrade/Downgrade Scenario 1 diagram
+//
+// This diagram is generated by mscgen
+// which can be installed on ubuntu by
+// sudo apt-get install mscgen
+//
+// Generate this diagram on the command line with:
+// > mscgen -T png -i problem_scenario.msc -F ./fonts/DejaVuSansMono.ttf
+//
+msc {
+
+	// Image defaults
+	width="1200", wordwraparcs="1";
+
+	// declare objects
+	// This is a cluster with 3 nodes for exposition purposes
+	// a client and a riak-shell 
+
+	"Client", "Node 1", "Node 2", "Node 3", "Riak Shell";
+
+    "Node 1" box "Node 1" [label="\nTime: T + 1h\n"];
+    "Node 2" box "Node 2" [label="\nTime: T\n"];
+    "Node 3" box "Node 3" [label="\nTime: T - 1h\n"];
+
+    ...;
+
+    |||;
+
+    ---;
+
+	"Client" => "Node 1" [label = "send INSERT INTO\n\nmytable VALUES(a, now())"];
+
+    |||;
+
+    "Node 1" box "Node 1" [label="\nidentify canonical vnode\n"],
+
+	|||;
+
+	"Node 1" => "Node 2" [label = "send INSERT INTO\n\nmytable VALUES(a, now())"];
+
+    |||;
+
+	"Node 2" box "Node 2" [label="\n evaluate now() => 12:00:00\n"],
+
+    |||;
+
+	"Node 2" => "Node 2" [label = "write {a, 12:00:00}"];
+
+    |||;
+
+	"Client" => "Node 2" [label = "send INSERT INTO\n\nmytable VALUES(b, now())"];
+
+    |||;
+
+    "Node 2" box "Node 2" [label="\nidentify canonical vnode\n"],
+
+	|||;
+
+	"Node 2" box "Node 2" [label="\n evaluate now() => 12:00:01\n"],
+
+    |||;
+
+	"Node 2" => "Node 2" [label = "write {b, 12:00:01}"];
+
+    |||;
+
+}

Options_for_a_now_function/problem_scenario.msc

@@ -0,0 +1,69 @@
+// diagram for upgrade_downgrade_specs.md
+//
+// Upgrade/Downgrade Scenario 1 diagram
+//
+// This diagram is generated by mscgen
+// which can be installed on ubuntu by
+// sudo apt-get install mscgen
+//
+// Generate this diagram on the command line with:
+// > mscgen -T png -i problem_scenario.msc -F ./fonts/DejaVuSansMono.ttf
+//
+msc {
+
+	// Image defaults
+	width="1200", wordwraparcs="1";
+
+	// declare objects
+	// This is a cluster with 3 nodes for exposition purposes
+	// a client and a riak-shell 
+
+	"Client", "Node 1", "Node 2", "Node 3", "Riak Shell";
+
+    "Node 1" box "Node 1" [label="\nTime: T + 1h\n"];
+    "Node 2" box "Node 2" [label="\nTime: T\n"];
+    "Node 3" box "Node 3" [label="\nTime: T - 1h\n"];
+
+    ...;
+
+    |||;
+
+    ---;
+
+	"Client" => "Node 1" [label = "send INSERT INTO\n\nmytable VALUES(a, now())"];
+
+    |||;
+
+    "Node 1" box "Node 1" [label="\n evaluate now() => 13:00:00\n"],
+
+	|||;
+
+	"Node 1" => "Node 3" [label = "write {a, 13:00:00}"];
+
+    |||;
+
+	"Client" => "Node 2" [label = "send INSERT INTO mytable\n\nVALUES(b, now())"];
+
+    |||;
+
+    "Node 2" box "Node 2" [label="\n evaluate now() => 12:00:01\n"],
+
+	|||;
+
+	"Node 2" => "Node 3" [label = "write {b, 12:00:01}"];
+
+    |||;
+
+	"Client" => "Node 3" [label = "send INSERT INTO mytable\n\nVALUES(c, now())"];
+
+    |||;
+
+    "Node 3" box "Node 3" [label="\n evaluate now() => 11:00:02\n"],
+
+	|||;
+
+	"Node 3" => "Node 3" [label = "write {c, 11:00:02}"];
+
+    |||;
+
+}

Options_for_a_now_function/riak_shell_basic.msc

@@ -0,0 +1,62 @@
+// diagram for upgrade_downgrade_specs.md
+//
+// Upgrade/Downgrade Scenario 1 diagram
+//
+// This diagram is generated by mscgen
+// which can be installed on ubuntu by
+// sudo apt-get install mscgen
+//
+// Generate this diagram on the command line with:
+// > mscgen -T png -i problem_scenario.msc -F ./fonts/DejaVuSansMono.ttf
+//
+msc {
+
+	// Image defaults
+	width="1200", wordwraparcs="1";
+
+	// declare objects
+	// This is a cluster with 3 nodes for exposition purposes
+	// a client and a riak-shell 
+
+	"Client", "Node 1", "Node 2", "Node 3", "Riak Shell";
+
+    "Node 1" box "Node 1" [label="\nTime: T + 1s\n"];
+    "Node 2" box "Node 2" [label="\nTime: T\n"];
+    "Node 3" box "Node 3" [label="\nTime: T - 1s\n"];
+
+    ...;
+
+    |||;
+
+    ---;
+
+	"Riak Shell" => "Node 2" [label = "send INSERT INTO\n\nmytable VALUES(a, now())"];
+
+    |||;
+
+    "Node 2" box "Node 2" [label="\n evaluate now() => 12:00:00\n"],
+
+	|||;
+
+	"Node 2" => "Node 3" [label = "write {a, 12:00:00}"];
+
+    |||;
+
+    ...;
+
+    |||;
+
+	"Riak Shell" => "Node 2" [label = "send INSERT INTO\n\nmytable VALUES(b, now())"];
+
+    |||;
+
+    "Node 2" box "Node 2" [label="\n evaluate now() => 12:00:01\n"],
+
+	|||;
+
+	"Node 2" => "Node 3" [label = "write {b, 12:00:01}"];
+
+    |||;
+
+
+}