ValueError: Invalid value for steps, must not be None when listing workflows

[brtasavpatel]

Hello I am trying to list workflows from within a Pod which is in my k8s cluster and have following service account applied on it.

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-api
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: my-api-argo-roles
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: 'true'
rules:
  - verbs:
      - create
      - delete
      - deletecollection
      - get
      - list
      - patch
      - update
      - watch
    apiGroups:
      - argoproj.io
    resources:
      - workflows
      - workflows/finalizers
      - workflowtemplates
      - workflowtemplates/finalizers
      - cronworkflows
      - cronworkflows/finalizers
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: mu-api-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: my-api-argo-roles
subjects:
- kind: ServiceAccount
  name: my-api
  namespace: default

when I try to list workflows from default namespace I get this error :

In [9]: from argo.workflows.client import V1alpha1Api
   ...: from argo.workflows.config import load_incluster_config
   ...:
   ...: load_incluster_config()  # loads local configuration from ~/.kube/confi
   ...: v1alpha1 = V1alpha1Api()
   ...:
   ...: wfs = v1alpha1.list_namespaced_workflows(namespace='default')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-9-c36ca0651faf> in <module>
      5 v1alpha1 = V1alpha1Api()
      6
----> 7 wfs = v1alpha1.list_namespaced_workflows(namespace='default')

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api/v1alpha1_api.py in list_namespaced_workflows(self, namespace, **kwargs)
   1613             return self.list_namespaced_workflows_with_http_info(namespace, **kwargs)  # noqa: E501
   1614         else:
-> 1615             (data) = self.list_namespaced_workflows_with_http_info(namespace, **kwargs)  # noqa: E501
   1616             return data
   1617

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api/v1alpha1_api.py in list_namespaced_workflows_with_http_info(self, namespace, **kwargs)
   1706             _preload_content=params.get('_preload_content', True),
   1707             _request_timeout=params.get('_request_timeout'),
-> 1708             collection_formats=collection_formats)
   1709
   1710     def list_namespaced_workflowtemplates(self, namespace, **kwargs):  # noqa: E501

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, async_req, _return_http_data_only, collection_formats, _preload_content, _request_timeout)
    328                                    response_type, auth_settings,
    329                                    _return_http_data_only, collection_formats,
--> 330                                    _preload_content, _request_timeout)
    331         else:
    332             thread = self.pool.apply_async(self.__call_api, (resource_path,

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, _return_http_data_only, collection_formats, _preload_content, _request_timeout)
    167             # deserialize response data
    168             if response_type:
--> 169                 return_data = self.deserialize(response_data, response_type)
    170             else:
    171                 return_data = None

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in deserialize(self, response, response_type)
    239             data = response.data
    240
--> 241         return self.__deserialize(data, response_type)
    242
    243     def __deserialize(self, data, klass):

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize(self, data, klass)
    278             return self.__deserialize_datatime(data)
    279         else:
--> 280             return self.__deserialize_model(data, klass)
    281
    282     def call_api(self, resource_path, method,

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize_model(self, data, klass)
    622                         isinstance(data, (list, dict))):
    623                     value = data[klass.attribute_map[attr]]
--> 624                     kwargs[attr] = self.__deserialize(value, attr_type)
    625
    626         instance = klass(**kwargs)

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize(self, data, klass)
    256                 sub_kls = re.match(r'list\[(.*)\]', klass).group(1)
    257                 return [self.__deserialize(sub_data, sub_kls)
--> 258                         for sub_data in data]
    259
    260             if klass.startswith('dict('):

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in <listcomp>(.0)
    256                 sub_kls = re.match(r'list\[(.*)\]', klass).group(1)
    257                 return [self.__deserialize(sub_data, sub_kls)
--> 258                         for sub_data in data]
    259
    260             if klass.startswith('dict('):

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize(self, data, klass)
    278             return self.__deserialize_datatime(data)
    279         else:
--> 280             return self.__deserialize_model(data, klass)
    281
    282     def call_api(self, resource_path, method,

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize_model(self, data, klass)
    622                         isinstance(data, (list, dict))):
    623                     value = data[klass.attribute_map[attr]]
--> 624                     kwargs[attr] = self.__deserialize(value, attr_type)
    625
    626         instance = klass(**kwargs)

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize(self, data, klass)
    278             return self.__deserialize_datatime(data)
    279         else:
--> 280             return self.__deserialize_model(data, klass)
    281
    282     def call_api(self, resource_path, method,

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize_model(self, data, klass)
    622                         isinstance(data, (list, dict))):
    623                     value = data[klass.attribute_map[attr]]
--> 624                     kwargs[attr] = self.__deserialize(value, attr_type)
    625
    626         instance = klass(**kwargs)

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize(self, data, klass)
    256                 sub_kls = re.match(r'list\[(.*)\]', klass).group(1)
    257                 return [self.__deserialize(sub_data, sub_kls)
--> 258                         for sub_data in data]
    259
    260             if klass.startswith('dict('):

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in <listcomp>(.0)
    256                 sub_kls = re.match(r'list\[(.*)\]', klass).group(1)
    257                 return [self.__deserialize(sub_data, sub_kls)
--> 258                         for sub_data in data]
    259
    260             if klass.startswith('dict('):

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize(self, data, klass)
    278             return self.__deserialize_datatime(data)
    279         else:
--> 280             return self.__deserialize_model(data, klass)
    281
    282     def call_api(self, resource_path, method,

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize_model(self, data, klass)
    622                         isinstance(data, (list, dict))):
    623                     value = data[klass.attribute_map[attr]]
--> 624                     kwargs[attr] = self.__deserialize(value, attr_type)
    625
    626         instance = klass(**kwargs)

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize(self, data, klass)
    256                 sub_kls = re.match(r'list\[(.*)\]', klass).group(1)
    257                 return [self.__deserialize(sub_data, sub_kls)
--> 258                         for sub_data in data]
    259
    260             if klass.startswith('dict('):

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in <listcomp>(.0)
    256                 sub_kls = re.match(r'list\[(.*)\]', klass).group(1)
    257                 return [self.__deserialize(sub_data, sub_kls)
--> 258                         for sub_data in data]
    259
    260             if klass.startswith('dict('):

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize(self, data, klass)
    278             return self.__deserialize_datatime(data)
    279         else:
--> 280             return self.__deserialize_model(data, klass)
    281
    282     def call_api(self, resource_path, method,

/usr/local/lib/python3.6/site-packages/argo/workflows/client/api_client.py in __deserialize_model(self, data, klass)
    624                     kwargs[attr] = self.__deserialize(value, attr_type)
    625
--> 626         instance = klass(**kwargs)
    627
    628         if (isinstance(instance, dict) and

/usr/local/lib/python3.6/site-packages/argo/workflows/client/models/v1alpha1_parallel_steps.py in __init__(self, steps)
     45         self.discriminator = None
     46
---> 47         self.steps = steps
     48
     49     @property

/usr/local/lib/python3.6/site-packages/argo/workflows/client/models/v1alpha1_parallel_steps.py in steps(self, steps)
     66         """
     67         if steps is None:
---> 68             raise ValueError("Invalid value for `steps`, must not be `None`")  # noqa: E501
     69
     70         self._steps = steps

ValueError: Invalid value for `steps`, must not be `None`

but if I do list_namespaced_workflows on namespace='argo' it works fine :

In [19]: from argo.workflows.client import V1alpha1Api
    ...: from argo.workflows.config import load_incluster_config
    ...:
    ...: load_incluster_config()  # loads local configuration from ~/.kube/confi
    ...: v1alpha1 = V1alpha1Api()
    ...:
    ...: wfs = v1alpha1.list_namespaced_workflows(namespace='argo')

In [20]: wfs.items[0]
Out[20]:
{'api_version': 'argoproj.io/v1alpha1',
 'kind': 'Workflow',
 'metadata': {'annotations': None,
              'cluster_name': None,
...
...

my environment :

Python v3.6.10

pip list | grep -i 'kubernetes\|argo'
argo-workflows       3.0.2
argo-workflows-dsl   0.1.0.dev0
kubernetes           10.0.1

[fvdnabee]

I created a fork that builds the python SDK from the latest openapi spec of argo (2.8.1), available here. In its v3.4 release, the following is working for me:

from argo.workflows.client import V1alpha1Api

from argo.workflows.client.__about__ import __version__
print(f"Using Argo workflows python client v{__version__}")

from argo.workflows.config import load_kube_config
load_kube_config()  # loads local configuration from ~/.kube/config
v1alpha1 = V1alpha1Api()

wfs = v1alpha1.list_namespaced_workflows(namespace="default")
print(wfs)

Using Argo workflows python client v3.4.0
{'api_version': 'argoproj.io/v1alpha1',
 'items': [{'api_version': 'argoproj.io/v1alpha1',
            'kind': 'Workflow',
            'metadata': {'annotations': None,
                         'cluster_name': None,
                         'creation_timestamp': datetime.datetime(2020, 5, 29, 19, 31, 4, tzinfo=tzutc()),
...

kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"archive", BuildDate:"2020-05-22T20:04:08Z", GoVersion:"go1.14.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.8-eks-e16311", GitCommit:"e163110a04dcb2f39c3325af96d019b4925419eb", GitTreeState:"clean", BuildDate:"2020-03-27T22:37:12Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}

If you want to try it, first a warning: the changes in my fork remain largely untested at the moment. The fork isn't available on pypi, but you can install the wheels/tar from the release page.

[Jane85]

@fvdnabee Hi, thanks a lot for the fix. Do you have a rough estimation about when this can be released officially?

[fvdnabee]

@Jane85 there has been some discussion on the argo-sdk slack channel towards searching a new maintainer for argo-client-python. Some people volunteered, but nothing concrete has come up.

Note I did publish the 3.4.0 release of my fork to a separate pypi repo (as I don't own the argo-workflows pypi repo) here: https://pypi.org/project/argo-workflows-fvdnabee/3.4.0/ -> pip install argo-workflows-fvdnabee==3.4.0