Restart model service process without reloading the container

[kyujin-cho]

Motivation  

In early bootstrap phases, model service needs to be restarted frequently to find out the working and performant parameter. In such cases following the original model service restart - terminating and creating a fresh new session - would be a bottleneck, especially when compared to just restarting the inference framework process itself. By extending current kernel runner spec a little bit mode, we can think achieve restarting the model service process itself while not touching the container lifecycle.

Required Features

• Create a new REST API to trigger restarting the model service process
• Create a new RPC Handler to handle model service process restart request
• Add a new kernel runner interface which shuts down ongoing model service process

Impact  

• Backend.AI Agent: new RPC interface
• Backend.AI Manager: new REST API

Testing Scenarios  

• Trigger restart of the model service process and check if the inference worker loads like before
• Reiterate step 1, this time modifying the model-definition.yaml file. Check if the change is reflected to the runner.