Living in a Box

[a-v-popov]

Netlab uses the python-box package. If anyone is unsure what it is, here is a 2023 article from its author:
https://codecalamity.com/box-7-is-out-but-should-you-use-it/
Or you can skip to When should I use Box? at the end. My read on this is that it might have a place in quick IPython experimentation or in a Jupyter Notebook exploring unpredictable, unvalidated external data. However, for any decently sized project, its drawbacks (some listed under Amazing! what can’t it do?) far outweigh any savings in typing.

What netlab is actually using

python-box can have very different behaviour depending on the parameters passed to the constructor. The most common flavour used by netlab seems to be Box(init,default_box=True,default_box_none_transform=False,box_dots=True). Parameters are described on the Types of Boxes page. default_box=True enables autovivification, which is contrary to what any contributor familiar with Python would expect. box_dots=True treats strings with dots as a deep object access pattern. The two together have a shorthand alias, DDBox, and significantly reduce the effectiveness of code readability, code completion, refactoring tools, and static analysis. AI tools might have better luck, but they will suffer as well, as will human readers.

Just to give a few examples:

>>> b = DDBox({'x':1})
>>>
>>> b
DDBox({'x': 1})
>>> if b.get('y.z.data'):
...     print('there is data')
... else:
...     print('there is NO data')
...
there is NO data
>>>
>>> b
DDBox({'x': 1, 'y': {'z': {'data': {}}}})

Would it be reasonable to expect anyone to anticipate that?
Also, if you pass an explicit default value, get() does not create the key.

>>> b = DDBox({'x':1})
>>> if b.get('y.z.data', False):
...     print('there is data')
... else:
...     print('there is NO data')
...
there is NO data
>>> b
DDBox({'x': 1})

According to the documentation page, default_box_none_transform = False should restore more predictable behaviour: "If a key exists, but has a value of None return the default attribute instead". Sounds good, but it also stops get() from creating the key.

>>> b = DDBox({'x':1}, default_box_none_transform = False)
>>> b
DDBox({'x': 1})
>>> if b.get('y.z.data'):
...     print('there is data')
... else:
...     print('there is NO data')
...
there is NO data
>>> b
DDBox({'x': 1})

But nested attribute access still creates the missing path.

>>> b = DDBox({'x':1}, default_box_none_transform = False)
>>> if b.y.z.data:
...     print('there is data')
... else:
...     print('there is NO data')
...
there is NO data
>>> b
DDBox({'x': 1, 'y': {'z': {'data': {}}}})

To stop this behaviour one would need default_box_create_on_get = False.
In the cases I tested, it prevents mutation during missing-key reads, including chained attribute reads.

>>> b = DDBox({'x':1}, default_box_none_transform = False, default_box_create_on_get=False)
>>> b
DDBox({'x': 1})
>>> if b.y.z.data:
...     print('there is data')
... else:
...     print('there is NO data')
...
there is NO data
>>> b
DDBox({'x': 1})

I do not see a point in investigating whether any of these behaviours are bugs or intended behaviour, as the problem is more general.
But this is just one flavour of the Box used in netlab. There are a number of direct object instantiations bypassing get_box.
So you would never know which behaviour you would get.

What should be done

In my eyes python-box is not the library of choice for a long-term project, especially if netlab is planning to rely substantially on automated testing or wants to attract new contributors. I would suggest avoiding implicit Box behaviour in new and updated code:

• avoid implicit object creation and dotted-string path access
• prefer other types when implicit Box behaviour is not required
• consistently use get_box() helper whenever Box is required
• explicitly disable autovivification on get with default_box_create_on_get = False

I would also try disabling multilevel autovivification with default_box_attr = None, which changes failure mode.

>>> b = DDBox({'x':1}, default_box_none_transform = False, default_box_create_on_get=False, default_box_attr=None)
>>> if b.y.z.data:
...     print('there is data')
... else:
...     print('there is NO data')
...
Traceback (most recent call last):
  File "<python-input-83>", line 1, in <module>
    if b.y.z.data:
       ^^^^^
AttributeError: 'NoneType' object has no attribute 'z'

But it might turn out rather challenging.

If for whatever reason the project stakeholders decide that Box is the way the project wants to evolve, I would suggest documenting it explicitly with a proper dedicated PR.

prefer concise code like "a.b.c|default(None) == 1" over bloated stuff like "a is defined and a.b is defined and a.b.c is defined and a.b.c == 1"

My understanding is that it is a Jinja example and that, with netlab's custom j2_Undefined class, it would work for other types, including dataclasses.

It should be noted that Jinja undefined chaining is read-only template evaluation, which is relatively harmless. However, it would be a mistake to carry that intuition back into Python Box code. Box is mutable Python data, with several constructor flavours in the repository, and attribute access can autovivify. Those are materially different situations.

From readability and "fixing that code" perspective, I would personally prefer to rely on standard Python patterns rather than give up static analysis. If code looks convoluted, it is usually a sign that it needs some refactoring, not implicit Box behaviour.