Merge pull request #105 from panchul/alekp_kubeflow_install

Adding ssh-port-forwarding instructions, and other updates.

Research/kubeflow-on-azure-stack/Readme.md

@@ -264,6 +264,11 @@ command (default editor is vi, to edit you need to press `i`, and to save and ex
 
     $ ./edit_external_access.sh
 
+Or you can patch it like so:
+
+    $ kubectl patch svc/istio-ingressgateway -p '{"spec":{"type": "LoadBalancer"}}' -n istio-system
+    service/istio-ingressgateway patched
+
 It will look something like this: 
 
 ![pics/ingress_loadbalancer.png](pics/ingress_loadbalancer.png)
@@ -650,6 +655,18 @@ Here is an example of how you can connect to the model you trained:
     MAE: 69.52472687854122
     R2: -0.02072575078015859
 
+In our case, if you built the image from MLFlow section and pushed it to your account,
+
+    $ curl --header "Content-Type: application/json" \
+                    --request POST \
+                    --data '{"AGE":123,"SEX":1,"BMI":32.1,"BP":101.0}' \
+             http://localhost:5050/invocations
+    Elasticnet model (alpha=0.050000, l1_ratio=0.050000):
+
+    RMSE: 82.16359959591213
+    MAE: 69.52472687854122
+    R2: -0.02072575078015859
+
 ## Next Steps
 
 Proceed to [TensorFlow on Kubeflow Tutorial](tensorflow-on-kubeflow/Readme.md#tensorflow-on-kubeflow-on-azure-stack)
@@ -658,7 +675,8 @@ to learn how to execute `TFJob`s on Kubeflow, in the environment that we just cr
 And then run [PyTorch on Kubeflow Tutorial](pytorch-on-kubeflow/Readme.md#pytorch-on-kubeflow-on-azure-stack) tutorial to learn running
 `PyTorchJob`s.
 
-The PyTorch example we run will log data for TensorBoard, you will see something like this:
+Some of our examples provide data to Tensorboard, see [Working with TensorBoard](working_with_tensorboard.md) for more information.
+The simplier PyTorch example we run will log data that looks something like this:
 
 ![pytorch-on-kubeflow/images/tensorboard_scalars.png](pytorch-on-kubeflow/images/tensorboard_scalars.png)
 

Research/kubeflow-on-azure-stack/pytorch-on-kubeflow/Readme.md

@@ -321,9 +321,31 @@ Look at the logs to see the progress:
     accuracy=0.9872
     saving model to 'mnist_cnn.pt'...
 
+# Tracking your model via Tensorboard
 
-And, since in our example we used tensorboardX to write summaries, if you connected the `Tensorboard` logs properly,
-you will see your data in the Tensorboard:
+One of optional features of our example is logging of the data in Tensorboard's format,
+we used `tensorboardX` to write summaries.
+
+You will need instantiate Tensorboard like so:
+
+    $ kubectl create -f tb_pytorch.yaml
+
+You might contact your cloud administrator to help you establish network access, or you can
+use ssh port forwarding to see it via your desktop's `localhost` address and port 6006.
+This is how it looks like(run it on the machine where your web browser is):
+
+     $ ssh -NfL 6006:localhost:6006 -i id_rsa_for_kubernetes azureuser@<public_ip_address_or_dns_name>
+
+An alternative would be to create an RDP and XWindows server at the master node and RDP to it.
+If you did the ssh port fowarding, you do not need it.
+
+Now you can access the port you forward from your Kubernetes environment:
+
+    $ export PODNAME=$(kubectl get pod -l app=tensorboard -o jsonpath='{.items[0].metadata.name}')
+    $ kubectl port-forward ${PODNAME} 6006:6006
+
+If you have done everything correctly, you will see your data in the subfolder of your
+shared drive, and in the Tensorboard, and if not, please re-visit [Working with Tensorboard](../working_with_tensorboard.md):
 
 ![images/tensorboard_scalars.png](images/tensorboard_scalars.png)
 

Research/kubeflow-on-azure-stack/pytorch-on-kubeflow/docker/mnist.py

@@ -124,7 +124,7 @@ def main():
     if use_cuda:
         print('Using CUDA')
 
-    writer = SummaryWriter(args.dir)
+    writer = SummaryWriter(f"{args.dir}/logs")
 
     torch.manual_seed(args.seed)
 

Research/kubeflow-on-azure-stack/pytorch-on-kubeflow/tb_pytorch.yaml

@@ -0,0 +1,38 @@
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  labels:
+    app: tensorboard
+  name: tensorboard
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: tensorboard
+  template:
+    metadata:
+      labels:
+        app: tensorboard
+    spec:
+      volumes:
+        - name: samba-share-volume2
+          persistentVolumeClaim:
+            claimName: samba-share-claim
+      containers:
+      - name: tensorboard
+        image: tensorflow/tensorflow:1.10.0
+        imagePullPolicy: Always
+        command:
+         - /usr/local/bin/tensorboard
+        args:
+        - --logdir
+        - /tmp/tensorflow/logs
+        volumeMounts:
+          - mountPath: /tmp/tensorflow
+            subPath: pytorch-tb
+            name: samba-share-volume2
+        ports:
+        - containerPort: 6006
+          protocol: TCP
+      dnsPolicy: ClusterFirst
+      restartPolicy: Always

Research/kubeflow-on-azure-stack/tensorflow-on-kubeflow/Readme.md

@@ -259,17 +259,25 @@ For more tutorials and How-Tos, see TensorFlow's [save_and_load.ipynb](https://g
 
 There is another useful tool to monitor some ML applications if they support it. We provided a sample file to start it in your Kubernetes cluster, `tensorboard.yaml`.
 
-A concrete example of a tensorboard-using script is in folder `mnist-w-tb`. You will need your
+A concrete example of a tensorboard-using script is in folder `tf-mnist-w-tb`. You will need your
 github account to build the image(substitute `rollingstone` for yours) and run:
 
     $ cd mnist-w-tb
     $ docker build -t rollingstone/tf-mnist-w-tb:latest .
     $ docker push rollingstone/tf-mnist-w-tb:latest
-    $ kubectl create -f tb.yaml
+    $ kubectl create -f tb_tf.yaml
     $ kubectl create -f tfjob.yaml
 
 You might contact your cloud administrator to help you establish network access, or you can
 use ssh port forwarding to see it via your desktop's `localhost` address and port 6006.
+This is how it looks like(run it on the machine where your web browser is):
+
+     $ ssh -NfL 6006:localhost:6006 -i id_rsa_for_kubernetes azureuser@<public_ip_address_or_dns_name>
+
+An alternative would be to create an RDP and XWindows server at the master node and RDP to it.
+If you did the ssh port fowarding, you do not need it.
+
+Now you can access the port you forward from your Kubernetes environment:
 
     $ export PODNAME=$(kubectl get pod -l app=tensorboard -o jsonpath='{.items[0].metadata.name}')
     $ kubectl port-forward ${PODNAME} 6006:6006

Research/kubeflow-on-azure-stack/tensorflow-on-kubeflow/mnist-w-tb/tb.yaml → Research/kubeflow-on-azure-stack/tensorflow-on-kubeflow/mnist-w-tb/tb_tf.yaml

@@ -1,4 +1,4 @@
-apiVersion: extensions/v1
+apiVersion: extensions/v1beta1
 kind: Deployment
 metadata:
   labels:

Research/kubeflow-on-azure-stack/working_with_tensorboard.md

@@ -0,0 +1,36 @@
+# Working with Tensorboard
+
+Tensorboard is an application that helps visualizing data. It was built to visualize
+TensorFlow, but could be used more broadly. For example, in our tutorial we demo
+how to use it for TensorFlow and Pytorch.
+
+We could use a generic Tensorboard deplolyment, see `tb_generic.yaml`: 
+
+    $ kubectl create -f tb_generic.yaml
+
+Then as long as your application logs to the same folder that Tensorboard loads from, you will
+see the UI. 
+
+You might contact your cloud administrator to help you establish network access, or you can
+use ssh port forwarding to see it via your desktop's `localhost` address and port 6006.
+This is how it looks like(run it on the machine where your web browser is):
+
+     $ ssh -NfL 6006:localhost:6006 -i id_rsa_for_kubernetes azureuser@<public_ip_address_or_dns_name>
+
+An alternative would be to create an RDP and XWindows server at the master node and RDP to it.
+If you did the ssh port fowarding, you do not need it.
+
+Now you can access the port you forward from your Kubernetes environment:
+
+    $ export PODNAME=$(kubectl get pod -l app=tensorboard -o jsonpath='{.items[0].metadata.name}')
+    $ kubectl port-forward ${PODNAME} 6006:6006
+
+It will look something like this:
+
+![pics/tensorboard_graph.png](pics/tensorboard_graph.png)
+
+
+# Links
+
+- https://www.tensorflow.org/tensorboard/
+