Uncaught MemoryError and OverflowError in EXTRACT

[sam-xif]

Summary of the problem

Hello manticore community,

I am working as part of a research team developing a code analysis tool for Python. One of the issues the tool discovered in manticore's codebase is that core.smtlib.operators.EXTRACT has the potential to throw uncaught OverflowError and MemoryError. These errors are caused by large values for the size argument.

If you are interested in learning more about the tool and how it found this issue, let me know down in the comments, or you can contact me at [email protected]. We are primarily curious about whether you find that this issue is legitimate and worth reporting and fixing. If not, we would be interested in understanding why.

Thank you for your consideration!

-Sam

Manticore version

Latest master (commit hash: 8861005)

Python version

Python 3.8

OS / Environment

Linux (kernel version 5.10.218)

Dependencies

N/A

Step to reproduce the behavior

Call EXTRACT with a large value for the size argument.

Expected behavior

Magnitude of size is appropriately limited.

Actual behavior

Traceback:

Traceback (most recent call last):
  ...
  File ".../repos/manticore/manticore/core/smtlib/operators.py", line 134, in EXTRACT
    return (x >> offset) & ((1 << size) - 1)
MemoryError

Traceback (most recent call last):
  ...
  File ".../repos/manticore/manticore/core/smtlib/operators.py", line 134, in EXTRACT
    return (x >> offset) & ((1 << size) - 1)
OverflowError: too many digits in integer