Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Python deserialization documentation and add unit test #763

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 58 additions & 1 deletion Insecure Deserialization/Python.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,16 @@
* [Pickle](#pickle)
* [PyYAML](#pyyaml)
* [References](#references)
* [Common Pitfalls](#common-pitfalls)
* [Testing for Insecure Deserialization](#testing-for-insecure-deserialization)


## Tools

* [j0lt-github/python-deserialization-attack-payload-generator](https://github.com/j0lt-github/python-deserialization-attack-payload-generator) - Serialized payload for deserialization RCE attack on python driven applications where pickle,PyYAML, ruamel.yaml or jsonpickle module is used for deserialization of serialized data.
* [Bandit](https://github.com/PyCQA/bandit) - A tool designed to find common security issues in Python code, including insecure deserialization.
* [PyYAML](https://pyyaml.org/wiki/PyYAMLDocumentation) - A YAML parser and emitter for Python.
* [jsonpickle](https://jsonpickle.github.io/) - A library for serializing and deserializing complex Python objects to and from JSON.


## Methodology
Expand Down Expand Up @@ -71,6 +76,29 @@ evil_token = b64encode(cPickle.dumps(e))
print("Your Evil Token : {}").format(evil_token)
```

#### Secure Alternative

To avoid using `pickle` for untrusted data, consider using `json` for serialization and deserialization, as it is safer and more secure.

```python
import json
from base64 import b64encode, b64decode

class User:
def __init__(self):
self.username = "anonymous"
self.password = "anonymous"
self.rank = "guest"

h = User()
auth_token = b64encode(json.dumps(h.__dict__).encode())
print("Your Auth Token : {}").format(auth_token)

new_token = input("New Auth Token : ")
token = json.loads(b64decode(new_token).decode())
print("Welcome {}".format(token['username']))
```


### PyYAML

Expand Down Expand Up @@ -108,11 +136,40 @@ with open('exploit_unsafeloader.yml') as file:
data = yaml.load(file,Loader=yaml.UnsafeLoader)
```

#### Secure Alternative

To avoid using `unsafe_load`, always use `safe_load` when working with untrusted YAML data.

```py
import yaml

with open('safe_data.yml') as file:
data = yaml.safe_load(file)
```


## Common Pitfalls

1. **Using `pickle` with untrusted data**: The `pickle` module is not secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.
2. **Using `yaml.load` without specifying a safe loader**: Always use `yaml.safe_load` when working with untrusted YAML data to avoid remote code execution vulnerabilities.
3. **Ignoring security warnings**: Always pay attention to security warnings and best practices when working with serialization and deserialization in Python.


## Testing for Insecure Deserialization

1. **Manual Testing**:
- Review the codebase for the use of insecure deserialization functions such as `pickle.loads`, `yaml.load`, and `jsonpickle.decode`.
- Identify the sources of input data and ensure they are properly validated and sanitized before deserialization.

2. **Automated Testing**:
- Use static analysis tools like [Bandit](https://github.com/PyCQA/bandit) to scan the codebase for insecure deserialization functions and patterns.
- Implement unit tests to verify that deserialization functions are not used with untrusted data and that proper input validation is in place.


## References

- [CVE-2019-20477 - 0Day YAML Deserialization Attack on PyYAML version <= 5.1.2 - Manmeet Singh (@_j0lt) - June 21, 2020](https://thej0lt.com/2020/06/21/cve-2019-20477-0day-yaml-deserialization-attack-on-pyyaml-version/)
- [Exploiting misuse of Python's "pickle" - Nelson Elhage - March 20, 2011](https://blog.nelhage.com/2011/03/exploiting-pickle/)
- [Python Yaml Deserialization - HackTricks - July 19, 2024](https://book.hacktricks.xyz/pentesting-web/deserialization/python-yaml-deserialization)
- [PyYAML Documentation - PyYAML - April 29, 2006](https://pyyaml.org/wiki/PyYAMLDocumentation)
- [YAML Deserialization Attack in Python - Manmeet Singh & Ashish Kukret - November 13, 2021](https://www.exploit-db.com/docs/english/47655-yaml-deserialization-attack-in-python.pdf)
- [YAML Deserialization Attack in Python - Manmeet Singh & Ashish Kukret - November 13, 2021](https://www.exploit-db.com/docs/english/47655-yaml-deserialization-attack-in-python.pdf)
19 changes: 19 additions & 0 deletions test_python_md.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import unittest
import re

class TestPythonMd(unittest.TestCase):
def test_python_code_blocks(self):
with open('Insecure Deserialization/Python.md', 'r') as file:
content = file.read()

# Extract Python code blocks
code_blocks = re.findall(r'```python(.*?)```', content, re.DOTALL)

for code in code_blocks:
try:
exec(code)
except Exception as e:
self.fail(f"Code block failed to execute: {e}")

if __name__ == '__main__':
unittest.main()