Add TensorFlow Serving Cluster tutorial

[RodgerZhu]

Add TensorFlow Serving Cluster tutorial Based on Graphene SGX.

---

This change is 

[dimakuv]

Reviewed 3 of 3 files at r1.
Reviewable status: all files reviewed, 12 unresolved discussions, not enough approvals from maintainers (3 more required), not enough approvals from different teams (2 more required, approved so far: ) (waiting on @RodgerZhu)

a discussion (no related file):
@RodgerZhu Could you add a sign-off in the commit message? Simply do git commit --amend --signoff and then git push --force-with-lease.

---

a discussion (no related file):
You forgot to update Documentation/index.rst file -- it must have the link to your new file index.rst. Just like you did in the previous PR: https://github.com/oscarlab/graphene/pull/2194/files#diff-65f82f80279ec12ce6d19e0a686d262618611be80d1a264e1cffed0d14c814a4

---

a discussion (no related file):
Your tutorial and the scripts use an old Graphene version. Recently we renamed all Graphene utilities (and the build system) to graphene-direct (instead of pal_loader), graphene-sgx (instead of SGX=1 pal_loader), and so on.

I suggest to wait with making these changes until around July, when the Graphene names will stabilize. And then you can re-update your scripts (in #1) and the text (in this PR) with new Graphene names.

---

---

Documentation/tutorials/tensorflow-serving-cluster/index.rst, line 14 at r1 (raw file):

   This tutorial is an external contribution and wasn't reviewed as thoroughly
   as the rest of the code. Please be careful, especially in terms of security!

Please replace as the rest of the code with this: as the core Graphene documentation.

---

Documentation/tutorials/tensorflow-serving-cluster/index.rst, line 130 at r1 (raw file):

  designed for production environments. Install::

     pip3 install -r ./tensorflow-serving/client/requirements.txt

Please replace ./tensorflow-serving/client/requirements.txt with this: <tensorflow-serving-cluster-dir>/tensorflow-serving/client/requirements.txt

---

Documentation/tutorials/tensorflow-serving-cluster/index.rst, line 316 at r1 (raw file):

The way these Trusted Files work is before Graphene runs TensorFlow Serving inside
the SGX enclave, Graphene generates the final SGX manifest file using ``pal-sgx-
sign`` Graphene utility. This utility calculates hashes of each trusted file and

We renamed pal-sgx-sign to graphene-sgx-sign. Please rename here.

---

Documentation/tutorials/tensorflow-serving-cluster/index.rst, line 450 at r1 (raw file):

In ``deploy.yaml``, it mainly configures the parameters passed to containers.
You need to replace the graphene repository path with your own in the host and

Graphene (with capital letter)

---

Documentation/tutorials/tensorflow-serving-cluster/index.rst, line 513 at r1 (raw file):

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Under the untrusted machine B, in the host, please download graphene source code.

Graphene (with capital letter)

---

Documentation/tutorials/tensorflow-serving-cluster/index.rst, line 551 at r1 (raw file):

Under trusted machine A, the user must prepare the secret provisioning server
and start it. We can build and run the secret provisioning server in the Docker,
here for simplicity, we run it on the host::

here for simplicity we run it on the host (remove the comma)

---

Documentation/tutorials/tensorflow-serving-cluster/index.rst, line 572 at r1 (raw file):

provisioning server (verifier), and will be sent to the client during TLS
handshake, but it was designed for local (single-machine) test.
We need to regenerate the ``server2-sha256.crt to support remote (two different

You forgot closing backticks, you should have this:

``server2-sha256.crt``

---

Documentation/tutorials/tensorflow-serving-cluster/index.rst, line 575 at r1 (raw file):

machines) test. For ``server2.key`` and ``test-ca-sha256.crt``, we keep them as-is.

Generate new `server2-sha256.crt`::

You should have double backticks

---

Documentation/tutorials/tensorflow-serving-cluster/index.rst, line 586 at r1 (raw file):

           $(MBEDTLS_CERT_REQ) output_file=$@ filename=$< subject_name="C=NL,O=PolarSSL,CN=attestation.service.com" md=SHA256

You can set your special ``CN`` value::

The following command line doesn't correspond to this text. Maybe this command line should be moved after Then we will get...?

[mkow]

Reviewable status: 2 of 4 files reviewed, 13 unresolved discussions, not enough approvals from maintainers (3 more required), not enough approvals from different teams (2 more required, approved so far: ) (waiting on @dimakuv and @RodgerZhu)

a discussion (no related file):
Shouldn't this and #1 be just a single PR? Why splitting them?

---

[dimakuv]

Reviewed 2 of 2 files at r2.
Reviewable status: all files reviewed, 3 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (2 more required, approved so far: ) (waiting on @RodgerZhu)

a discussion (no related file):

Previously, mkow (Michał Kowalczyk) wrote…

Shouldn't this and #1 be just a single PR? Why splitting them?

Historical reasons :) But Michal is right. @RodgerZhu Could you add the changes from #1 in this PR and close #1? This will make it more meaningful -- both the documentation and the accompanying scripts will be in one PR and in one commit.

Please also update the commit message accordingly.

---

---

Documentation/tutorials/tensorflow-serving-cluster/index.rst, line 591 at r2 (raw file):

   LD_LIBRARY_PATH=../../install/lib/ make server2-sha256.crt

Please use the new created ``server2-sha256.crt`` to replace the one under

the newly created

[dimakuv]

Reviewed 27 of 27 files at r3.
Reviewable status: all files reviewed, 7 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (2 more required, approved so far: ) (waiting on @RodgerZhu)

a discussion (no related file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Your tutorial and the scripts use an old Graphene version. Recently we renamed all Graphene utilities (and the build system) to graphene-direct (instead of pal_loader), graphene-sgx (instead of SGX=1 pal_loader), and so on.

I suggest to wait with making these changes until around July, when the Graphene names will stabilize. And then you can re-update your scripts (in #1) and the text (in this PR) with new Graphene names.

Actually, I just noticed that your scripts use ENV GRAPHENE_VERSION=303528131c67f58aeee677397ade9593f222ae88. So you download and build an old version of Graphene that still uses pal_loader, pal-sgx-sign, etc.

You'll need to update the Graphene version (maybe in July, when Graphene is stabilized) in this tutorial, otherwise it will be very outdated.

---

---

tensorflow-serving-cluster/README.md, line 3 at r3 (raw file):

# TensorFlow Serving Cluster PPML Tutorial Scripts

This directory contains the scripts package for [TensorFlow Serving cluster PPML tutorial](https://graphene.readthedocs.io/en/latest/tutorials/tensorflow-serving-cluster/index.html), split into two sub-directories:

The link is wrong now. It should be https://graphene-contrib.readthedocs.io/...

---

tensorflow-serving-cluster/README.md, line 18 at r3 (raw file):

Simply running a TensorFlow Serving system inside Graphene is not enough for a
safe & secure end-user experience. Thus, there is a need to build a complete
secure inferencing flow. [The linked tutorial](https://graphene.readthedocs.io/en/latest/tutorials/tensorflow-serving-cluster/index.html)

ditto

---

tensorflow-serving-cluster/README.md, line 61 at r3 (raw file):

- Graphene. Follow [Quick Start](https://graphene.readthedocs.io/en/latest/quickstart.html)
  to build Graphene. [In the linked tutorial](https://graphene.readthedocs.io/en/latest/tutorials/tensorflow-serving-cluster/index.html),

ditto

---

tensorflow-serving-cluster/README.md, line 67 at r3 (raw file):

# Run the PPML

Please follow the [TensorFlow Serving cluster PPML tutorial](https://graphene.readthedocs.io/en/latest/tutorials/tensorflow-serving-cluster/index.html)

ditto

---

tensorflow-serving-cluster/kubernetes/ingress-nginx/README.md, line 4 at r3 (raw file):

Kubernetes has a built in configuration object for HTTP load balancing, called
[Ingress Nginx](https://kubernetes.io/docs/concepts/services-networking/ingress/)
, that defines rules for external connectivity to the pods represented by one

Please move , to a previous line, otherwise you have a redundant space, see how it's rendered: https://github.com/RodgerZhu/graphene-contrib/tree/yunzhu/tensorflow-serving-cluster/tensorflow-serving-cluster/kubernetes/ingress-nginx