notes on: triton model adapter in prediction pipeline

[FynnBe]

In order to implement a create_prediction_pipeline (for a triton model adapter) that takes in a RAW model description I had a look at the current use case:

bioengine-model-runner/src/bioengine-model-runner/1/model.py

Lines 142 to 153 in e470edb

	model_resource = bioimageio.core.load_raw_resource_description(
	model_id
	)
	pred_pipeline = create_prediction_pipeline(
	bioimageio_model=model_resource,
	model_adapter=TritonModelAdapter(
	server_url="127.0.0.1:8000",
	model_id=model_id,
	model_version="1",
	model_resource=model_resource,
	),
	)

What strikes me as odd here is the parallel use of model_id and model_resource.

As discussed "offline" with @k-dominik it might make sense to separate a create_server_prediction_pipeline from the create_prediction_pipeline due to the different requirements and use-cases. It is important to keep the prediction pipeline interface the same across model adapters, but a separate creation function shouldn't wreak too much havoc...

I'll take a close look and make a PR soon

cc @oeway

[oeway]

What strikes me as odd here is the parallel use of model_id and model_resource.

Not sure what you mean, odd, but the model_id passed in TritonModelAdapter is the id used in the triton server, which can be different but in this particular case they are the same (think nickname vs zenodo id).

It seems not necessary to make a separate function, we just need to get the raw resource description work. Later we can make another model adapter which talks to a remote server to do inference too, it's all fits in the definition of prediction pipeline.

[FynnBe]

odd because the model_resource contains all information about the model.. why would we want to use a different id in triton?
Anyway, that is not the main problem at hand...

[oeway]

Well, not really, the model resource is not aware of the triton server, so we cannot know the triton model id from the model resource.

[FynnBe]

but the triton server is certainly aware of the model zoo? so why not reuse the model id?

[oeway]

We are using that, but there are more complications with different model versions. I'd rather pass it again so I can easily rewrite it to match different strategy we may come up with.

And I am not sure it worth a lot of discussion on the function signature of an internal class too. It's not working yet, we can always change later.