What are plugin resource limits?

[jennydaman]

A plugin's JSON representation may define a memory_limit value. When creating a plugin instance of a particular plugin, this memory_limit value may be explicitly specified. But how exactly is memory_limit to be interpreted (by pman)?

The kubernetesmgr.py of pman is the first implementation of AbstractManager to respect the memory_limit value. So the amount of memory requested is hard-coded to be 100Mi, while a limit on memory is set to the value of memory_limit.

pman/pman/kubernetesmgr.py

Lines 116 to 117 in 898b8a1

	requests = {'memory': '150Mi', 'cpu': '250m'}
	limits = {'memory': memory_limit, 'cpu': cpu_limit}

So a plugin instance which sets memory_limit as 100Mi cannot run.

[ERROR][manager 8 140461197596480] [CODE01,chris-jid-9]: Error submitting job to pfcon url -->http://tunnels.local:6004/api/v1/<--, detail: {"message": "Error response from pman service while submitting job chris-jid-9, detail: {\"message\": \"(422)\\nReason: Unprocessable Entity\\nHTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '68c1055e-085f-4a7f-9938-4254064d5ce6', 'X-Kubernetes-Pf-Prioritylevel-Uid': '296737dd-595c-4f96-93b6-bb9c25ac5bcb', 'Date': 'Tue, 29 Mar 2022 21:28:15 GMT', 'Content-Length': '518'})\\nHTTP response body: {\\\"kind\\\":\\\"Status\\\",\\\"apiVersion\\\":\\\"v1\\\",\\\"metadata\\\":{},\\\"status\\\":\\\"Failure\\\",\\\"message\\\":\\\"Job.batch \\\\\\\"chris-jid-9\\\\\\\" is invalid: spec.template.spec.containers[0].resources.requests: Invalid value: \\\\\\\"150Mi\\\\\\\": must be less than or equal to memory limit\\\",\\\"reason\\\":\\\"Invalid\\\",\\\"details\\\":{\\\"name\\\":\\\"chris-jid-9\\\",\\\"group\\\":\\\"batch\\\",\\\"kind\\\":\\\"Job\\\",\\\"causes\\\":[{\\\"reason\\\":\\\"FieldValueInvalid\\\",\\\"message\\\":\\\"Invalid value: \\\\\\\"150Mi\\\\\\\": must be less than or equal to memory limit\\\",\\\"field\\\":\\\"spec.template.spec.containers[0].resources.requests\\\"}]},\\\"code\\\":422}\\n\\n\"}\n"}

Solution: set memory_limit as requested memory.