Skip to content
This repository was archived by the owner on Nov 16, 2023. It is now read-only.

Commit 77a3c0a

Browse files
committed
adding remarks of ingressgateway options
1 parent ac7009b commit 77a3c0a

File tree

1 file changed

+24
-8
lines changed
  • Research/kubeflow-on-azure-stack-lab/04-KFServing

1 file changed

+24
-8
lines changed

Research/kubeflow-on-azure-stack-lab/04-KFServing/pytorch.md

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,27 @@
22

33
## Building a model and running inference on it.
44

5-
You can run inferencing using pytorchserver. See [KFServing PyTorch demo](https://github.com/kubeflow/kfserving/tree/master/docs/samples/pytorch) for more information if needed.
5+
You can run inferencing using pytorchserver, a part of Kubeflow KFServing GitHub reposotory.
6+
See [KFServing PyTorch demo](https://github.com/kubeflow/kfserving/tree/master/docs/samples/pytorch) for more information if needed.
67

7-
You need to have the pytorchserver installed. Clone KFServing repository and install the pre-requisites.
8+
You need to have the `pytorchserver` installed. You may need to install the prerequisites manually, specifying
9+
versions and hardware nuances(CUDA version, etc.)
10+
11+
In simple case:
12+
13+
$ pip install torch torchvision
14+
15+
Clone KFServing repository and install the pre-requisites. See KFServing's
16+
[python/pytorchserver](https://github.com/kubeflow/kfserving/tree/master/python/pytorchserver)
17+
if you have any issues.
818

919
$ git clone https://github.com/kubeflow/kfserving.git
1020
$ cd kfserving/python/pytorchserver
1121
$ pip install -e .
1222

13-
See KFServing's [python/pytorchserver](https://github.com/kubeflow/kfserving/tree/master/python/pytorchserver) if you have any issues.
14-
1523
Verify that it works:
1624

17-
/kfserving/python/pytorchserver$ python3python3 -m pytorchserver -h
25+
/kfserving/python/pytorchserver$ python3 -m pytorchserver -h
1826
usage: __main__.py [-h] [--http_port HTTP_PORT] [--grpc_port GRPC_PORT]
1927
[--max_buffer_size MAX_BUFFER_SIZE] [--workers WORKERS]
2028
--model_dir MODEL_DIR [--model_name MODEL_NAME]
@@ -102,9 +110,17 @@ Wait until the pods are running and the service is 'ready' and has URL:
102110

103111
Define the parameters you will be using in your requests:
104112

105-
$ MODEL_NAME=pytorch-cifar10
106-
$ INPUT_PATH=@./pytorch_input.json
107-
$ SERVICE_HOSTNAME=$(kubectl get inferenceservice pytorch-cifar10 -n kfserving-test -o jsonpath='{.status.url}' | cut -d "/" -f 3)
113+
$ export MODEL_NAME=pytorch-cifar10
114+
$ export INPUT_PATH=@./pytorch_input.json
115+
$ export SERVICE_HOSTNAME=$(kubectl get inferenceservice pytorch-cifar10 -n kfserving-test -o jsonpath='{.status.url}' | cut -d "/" -f 3)
116+
117+
Depending on your environment, if you run on KFServing that is part of Kubeflow instalation(this is what we do thuought this lab):
118+
119+
$ export INGRESS_HOST=$(kubectl -n istio-system get service kfserving-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
120+
$ export INGRESS_PORT=$(kubectl -n istio-system get service kfserving-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
121+
122+
Or for more generic case:
123+
108124
$ export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
109125
$ export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
110126

0 commit comments

Comments
 (0)