Ok_Test ConceptCase has bug that cannot run without choices

[WillLin22]

Issue

It seems that the ConceptCase in client/sources/ok_test/concept.py is originally designed to support both choice questions and questions without choices. But in reality only choice question is implemented and the other one will meet an Exception saying that the NoValue has no len()

$ python ok -u
...
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "(path_to_the_project)/__main__.py", line 46, in <module>
  File "(path_to_the_project)/client/cli/ok.py", line 283, in main
  File "(path_to_the_project)/client/protocols/unlock.py", line 77, in run
  File "(path_to_the_project)/client/sources/ok_test/models.py", line 144, in unlock
  File "(path_to_the_project)/client/sources/ok_test/concept.py", line 76, in unlock
  File "(path_to_the_project)/client/protocols/unlock.py", line 126, in interact
TypeError: object of type 'NoValue' has no len()

The corresponding configuration is as follows.

# tests/q.py
test = {
  'name': 'Q',
  'points': 4,
  'suites': [
    {
      'cases': [
        {
          'answer': '6fc1d2bf205571bd49961b546ba1d431',
          'hidden': False,
          'locked': True,
          'multiline': False,
          'question': 'Q'
        }
      ],
      'scored': False,
      'type': 'concept'
    }
  ]
}

The .ok file is as follows:

{
    "name": "Example",
    "src": [
        "*.py"
    ],
    "tests": {
        "tests/*.py": "ok_test"
    }
}

Explanation

In deep check, I found a bug that lies inside the NoValue class and the interact method of unlock.

The code of NoValue is nothing more than an object:

python
class NoValue(object):
    pass

NoValue = NoValue()

In the original design of the ConceptCase, when it is instantiated, the choices is set to be optional. When it is not in the values of suite['cases'] , the choices will be instantiated as an NoValue object rather than None.

# client/sources/concept.py
class ConceptCase(common_models.Case):
    question = core.String()
    answer = core.String()
    choices = core.List(type=str, optional=True) # it is set here to be optional
def ...
# client/sources/common/core.py
class _SerializeMeta(type):
    def __init__(cls, name, bases, attrs):
         ...
    def __call__(cls, *args, **kargs):
        obj = type.__call__(cls, *args, **kargs)
        # Validate existing arguments
        for attr, value in kargs.items():
            if attr not in cls._fields:
                raise ex.SerializeException('__init__() got an unexpected '
                                'keyword argument: {}'.format(attr))
            else:
                setattr(obj, attr, value)
        # Check for missing/default fields
        for attr, value in cls._fields.items():
            if attr in kargs:
                continue
            elif value.optional:
# Since the choices of the ConceptCase is set to be optional, it is set to be the default value which is NoValue set in the Field class shown below.
                setattr(obj, attr, value.default) 
            else:
                raise ex.SerializeException('__init__() missing expected '
                                'argument {}'.format(attr))
        obj.post_instantiation()
        return obj

# client/sources/common/core.py
class Field(object):
    _default = NoValue
@property
    def default(self):
        return self._default

But the real problem is not here. The problem is that the interact method of the Unlock protocol failed to consider this! It considered the existance of the choice option so it is declared in the interface. But when it is not exist, it is just considered to be None. So when the protocol like unlock check whether it exists the problem occurs:

# client/protocols/unlock.py
class UnlockProtocol(models.Protocol):
  ...
      def interact(self, unique_id, case_id, question_prompt, answer, choices=None, randomize=True,
                 *, multiline=False, normalizer=lambda x: x):
        if randomize and choices: # Note that the choices is set as a NoValue instance so it returns True!
            choices = random.sample(choices, len(choices))
        ... # multiple if choices like this

Solution

The solution is pretty simple: override the bool method of the NoValue so that it behave like False when using it inside if

python
class NoValue(object):
    # pass
    def __bool__(self):
        return False