|
2 | 2 |
|
3 | 3 | ## Building a model and running inference on it.
|
4 | 4 |
|
5 |
| -You can run inferencing using pytorchserver. See [KFServing PyTorch demo](https://github.com/kubeflow/kfserving/tree/master/docs/samples/pytorch) for more information if needed. |
| 5 | +You can run inferencing using pytorchserver, a part of Kubeflow KFServing GitHub reposotory. |
| 6 | +See [KFServing PyTorch demo](https://github.com/kubeflow/kfserving/tree/master/docs/samples/pytorch) for more information if needed. |
6 | 7 |
|
7 |
| -You need to have the pytorchserver installed. Clone KFServing repository and install the pre-requisites. |
| 8 | +You need to have the `pytorchserver` installed. You may need to install the prerequisites manually, specifying |
| 9 | +versions and hardware nuances(CUDA version, etc.) |
| 10 | + |
| 11 | +In simple case: |
| 12 | + |
| 13 | + $ pip install torch torchvision |
| 14 | + |
| 15 | +Clone KFServing repository and install the pre-requisites. See KFServing's |
| 16 | +[python/pytorchserver](https://github.com/kubeflow/kfserving/tree/master/python/pytorchserver) |
| 17 | +if you have any issues. |
8 | 18 |
|
9 | 19 | $ git clone https://github.com/kubeflow/kfserving.git
|
10 | 20 | $ cd kfserving/python/pytorchserver
|
11 | 21 | $ pip install -e .
|
12 | 22 |
|
13 |
| -See KFServing's [python/pytorchserver](https://github.com/kubeflow/kfserving/tree/master/python/pytorchserver) if you have any issues. |
14 |
| - |
15 | 23 | Verify that it works:
|
16 | 24 |
|
17 |
| - /kfserving/python/pytorchserver$ python3python3 -m pytorchserver -h |
| 25 | + /kfserving/python/pytorchserver$ python3 -m pytorchserver -h |
18 | 26 | usage: __main__.py [-h] [--http_port HTTP_PORT] [--grpc_port GRPC_PORT]
|
19 | 27 | [--max_buffer_size MAX_BUFFER_SIZE] [--workers WORKERS]
|
20 | 28 | --model_dir MODEL_DIR [--model_name MODEL_NAME]
|
@@ -102,9 +110,17 @@ Wait until the pods are running and the service is 'ready' and has URL:
|
102 | 110 |
|
103 | 111 | Define the parameters you will be using in your requests:
|
104 | 112 |
|
105 |
| - $ MODEL_NAME=pytorch-cifar10 |
106 |
| - $ INPUT_PATH=@./pytorch_input.json |
107 |
| - $ SERVICE_HOSTNAME=$(kubectl get inferenceservice pytorch-cifar10 -n kfserving-test -o jsonpath='{.status.url}' | cut -d "/" -f 3) |
| 113 | + $ export MODEL_NAME=pytorch-cifar10 |
| 114 | + $ export INPUT_PATH=@./pytorch_input.json |
| 115 | + $ export SERVICE_HOSTNAME=$(kubectl get inferenceservice pytorch-cifar10 -n kfserving-test -o jsonpath='{.status.url}' | cut -d "/" -f 3) |
| 116 | + |
| 117 | +Depending on your environment, if you run on KFServing that is part of Kubeflow instalation(this is what we do thuought this lab): |
| 118 | + |
| 119 | + $ export INGRESS_HOST=$(kubectl -n istio-system get service kfserving-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}') |
| 120 | + $ export INGRESS_PORT=$(kubectl -n istio-system get service kfserving-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}') |
| 121 | + |
| 122 | +Or for more generic case: |
| 123 | + |
108 | 124 | $ export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
|
109 | 125 | $ export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
|
110 | 126 |
|
|
0 commit comments