Skip to content

8.6.0 #557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open

8.6.0 #557

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
b25f726
Colored view
mauvilsa May 10, 2025
eca5dfe
Colored compact view
mauvilsa May 19, 2025
77fa8d3
Fix CLI
mauvilsa May 20, 2025
18a0334
Fix bool evaluation
mauvilsa May 20, 2025
82f890c
Fixes and more tests
mauvilsa May 22, 2025
50617f9
adding uv
seperman May 27, 2025
bce6343
Fix array bugs
mauvilsa May 28, 2025
5a7ded3
adding support to ip addresses
seperman Jun 7, 2025
662f9ba
adding ip address modules safe for import
seperman Jun 7, 2025
c7e581f
adding support to ip addresses
seperman Jun 7, 2025
d5f23df
Adding support for applying deltas to NamedTuple
paulsc Jun 20, 2025
10cb342
Handle one more case where NamedTuple has a frozenset.
paulsc Jun 23, 2025
f40fa0e
Fix test_delta.py with Python 3.14.
Romain-Geissler-1A Jun 29, 2025
10baf2a
serializing properties
seperman Jul 2, 2025
07e7d0a
Merge pull request #556 from Romain-Geissler-1A/master
seperman Jul 2, 2025
1c30c5a
Merge pull request #549 from mauvilsa/colored-view
seperman Jul 2, 2025
4c02bf9
Merge pull request #554 from paulsc/namedtuple-add-delta
seperman Jul 2, 2025
8d1399b
adding Claude.md
seperman Jul 2, 2025
2b1a039
gh-535: UUID set comparison failure
akshat62 Jul 8, 2025
b9af75d
Adding test cases
akshat62 Jul 9, 2025
5c815c8
updating docs
seperman Jul 13, 2025
6b01e71
adding ignore_uuid_types flag
seperman Jul 13, 2025
66eb8f6
Merge branch 'gh-535/UUID-set-comparison-failure' of github.com:aksha…
seperman Jul 13, 2025
791bdfa
fixing the implementation for hashing uuid
seperman Jul 13, 2025
8121be3
supporing memoryview
seperman Jul 14, 2025
5e514c5
serializing memoryview
seperman Jul 14, 2025
687ea04
adding one more test case
seperman Jul 15, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ __pycache__/
# Distribution / packaging
.Python
env/
.venv
build/
develop-eggs/
dist/
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# DeepDiff Change log

- [Unreleased]
- Colored View: Output pretty-printed JSON with color-coded differences (added in green, removed in red, changed values show old in red and new in green).

- v8-5-0
- Updating deprecated pydantic calls
- Switching to pyproject.toml
Expand Down
82 changes: 82 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

DeepDiff is a Python library for deep comparison, searching, and hashing of Python objects. It provides:
- **DeepDiff**: Deep difference detection between objects
- **DeepSearch**: Search for objects within other objects
- **DeepHash**: Content-based hashing for any object
- **Delta**: Git-like diff objects that can be applied to other objects
- **CLI**: Command-line interface via `deep` command

## Development Commands

### Setup
```bash
# Install with all development dependencies
uv pip install -e ".[cli,coverage,dev,docs,static,test]"
# OR using uv (recommended)
uv sync --all-extras
```

**Virtual Environment**: Activate with `source ~/.venvs/atlas/bin/activate` before running tests or Python commands


### Testing
```bash
# Run tests with coverage
pytest --cov=deepdiff --cov-report term-missing

# Run tests including slow ones
pytest --cov=deepdiff --runslow

# Run single test file
pytest tests/test_diff_text.py

# Run tests across multiple Python versions
nox -s pytest
```

### Quality Checks
```bash
# Linting (max line length: 120)
nox -s flake8

# Type checking
nox -s mypy

# Run all quality checks
nox
```

## Architecture

### Core Structure
- **deepdiff/diff.py**: Main DeepDiff implementation (most complex component)
- **deepdiff/deephash.py**: DeepHash functionality
- **deepdiff/base.py**: Shared base classes and functionality
- **deepdiff/model.py**: Core data structures and result objects
- **deepdiff/helper.py**: Utility functions and type definitions
- **deepdiff/delta.py**: Delta objects for applying changes

### Key Patterns
- **Inheritance**: `Base` class provides common functionality with mixins
- **Result Objects**: Different result formats (`ResultDict`, `TreeResult`, `TextResult`)
- **Path Navigation**: Consistent path notation for nested object access
- **Performance**: LRU caching and numpy array optimization

### Testing
- Located in `/tests/` directory
- Organized by functionality (e.g., `test_diff_text.py`, `test_hash.py`)
- Aims for ~100% test coverage
- Uses pytest with comprehensive fixtures

## Development Notes

- **Python Support**: 3.9+ and PyPy3
- **Main Branch**: `master` (PRs typically go to `dev` branch)
- **Build System**: Modern `pyproject.toml` with `flit_core`
- **Dependencies**: Core dependency is `orderly-set>=5.4.1,<6`
- **CLI Tool**: Available as `deep` command after installation with `[cli]` extra
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ include *.txt
include *.sh
include pytest.ini
include *.py
exclude uv.lock
recursive-include docs/ *.rst
recursive-include docs/ *.png
recursive-include tests *.csv
Expand Down
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,17 @@ Please take a look at the [CHANGELOG](CHANGELOG.md) file.

:mega: **Please fill out our [fast 5-question survey](https://forms.gle/E6qXexcgjoKnSzjB8)** so that we can learn how & why you use DeepDiff, and what improvements we should make. Thank you! :dancers:

# Local dev

1. Clone the repo
2. Switch to the dev branch
3. Create your own branch
4. Install dependencies

- Method 1: Use [`uv`](https://github.com/astral-sh/uv) to install the dependencies: `uv sync --all-extras`.
- Method 2: Use pip: `pip install -e ".[cli,coverage,dev,docs,static,test]"`
5. Build `flit build`

# Contribute

1. Please make your PR against the dev branch
Expand Down
10 changes: 9 additions & 1 deletion deepdiff/base.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import uuid
from deepdiff.helper import strings, numbers, SetOrdered


Expand All @@ -21,7 +22,8 @@ def get_significant_digits(self, significant_digits, ignore_numeric_type_changes
def get_ignore_types_in_groups(self, ignore_type_in_groups,
ignore_string_type_changes,
ignore_numeric_type_changes,
ignore_type_subclasses):
ignore_type_subclasses,
ignore_uuid_types=False):
if ignore_type_in_groups:
if isinstance(ignore_type_in_groups[0], type):
ignore_type_in_groups = [ignore_type_in_groups]
Expand All @@ -43,6 +45,12 @@ def get_ignore_types_in_groups(self, ignore_type_in_groups,
if ignore_numeric_type_changes and self.numbers not in ignore_type_in_groups:
ignore_type_in_groups.append(SetOrdered(self.numbers))

if ignore_uuid_types:
# Create a group containing both UUID and str types
uuid_str_group = SetOrdered([uuid.UUID, str])
if uuid_str_group not in ignore_type_in_groups:
ignore_type_in_groups.append(uuid_str_group)

if not ignore_type_subclasses:
# is_instance method needs tuples. When we look for subclasses, we need them to be tuples
ignore_type_in_groups = list(map(tuple, ignore_type_in_groups))
Expand Down
139 changes: 139 additions & 0 deletions deepdiff/colored_view.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
import json
import os
from ast import literal_eval
from importlib.util import find_spec
from typing import Any, Dict

from deepdiff.model import TextResult, TreeResult


if os.name == "nt" and find_spec("colorama"):
import colorama

colorama.init()


# ANSI color codes
RED = '\033[31m'
GREEN = '\033[32m'
RESET = '\033[0m'


class ColoredView:
"""A view that shows JSON with color-coded differences."""

def __init__(self, t2: Any, tree_result: TreeResult, compact: bool = False):
self.t2 = t2
self.tree = tree_result
self.compact = compact
self.diff_paths = self._collect_diff_paths()

def _collect_diff_paths(self) -> Dict[str, str]:
"""Collect all paths that have differences and their types."""
text_result = TextResult(tree_results=self.tree, verbose_level=2)
diff_paths = {}
for diff_type, items in text_result.items():
if not items:
continue
try:
iter(items)
except TypeError:
continue
for path, item in items.items():
if diff_type in ("values_changed", "type_changes"):
changed_path = item.get("new_path") or path
diff_paths[changed_path] = ("changed", item["old_value"], item["new_value"])
elif diff_type in ("dictionary_item_added", "iterable_item_added", "set_item_added"):
diff_paths[path] = ("added", None, item)
elif diff_type in ("dictionary_item_removed", "iterable_item_removed", "set_item_removed"):
diff_paths[path] = ("removed", item, None)
return diff_paths

def _format_value(self, value: Any) -> str:
"""Format a value for display."""
if isinstance(value, bool):
return 'true' if value else 'false'
elif isinstance(value, str):
return f'"{value}"'
elif isinstance(value, (dict, list, tuple)):
return json.dumps(value)
else:
return str(value)

def _get_path_removed(self, path: str) -> dict:
"""Get all removed items for a given path."""
removed = {}
for key, value in self.diff_paths.items():
if value[0] == 'removed' and key.startswith(path + "["):
key_suffix = key[len(path):]
if key_suffix.count("[") == 1 and key_suffix.endswith("]"):
removed[literal_eval(key_suffix[1:-1])] = value[1]
return removed

def _has_differences(self, path_prefix: str) -> bool:
"""Check if a path prefix has any differences under it."""
return any(diff_path.startswith(path_prefix + "[") for diff_path in self.diff_paths)

def _colorize_json(self, obj: Any, path: str = 'root', indent: int = 0) -> str:
"""Recursively colorize JSON based on differences, with pretty-printing."""
INDENT = ' '
current_indent = INDENT * indent
next_indent = INDENT * (indent + 1)

if path in self.diff_paths and path not in self._colorize_skip_paths:
diff_type, old, new = self.diff_paths[path]
if diff_type == 'changed':
return f"{RED}{self._format_value(old)}{RESET} -> {GREEN}{self._format_value(new)}{RESET}"
elif diff_type == 'added':
return f"{GREEN}{self._format_value(new)}{RESET}"
elif diff_type == 'removed':
return f"{RED}{self._format_value(old)}{RESET}"

if isinstance(obj, (dict, list)) and self.compact and not self._has_differences(path):
return '{...}' if isinstance(obj, dict) else '[...]'

if isinstance(obj, dict):
if not obj:
return '{}'
items = []
for key, value in obj.items():
new_path = f"{path}['{key}']" if isinstance(key, str) else f"{path}[{key}]"
if new_path in self.diff_paths and self.diff_paths[new_path][0] == 'added':
# Colorize both key and value for added fields
items.append(f'{next_indent}{GREEN}"{key}": {self._colorize_json(value, new_path, indent + 1)}{RESET}')
else:
items.append(f'{next_indent}"{key}": {self._colorize_json(value, new_path, indent + 1)}')
for key, value in self._get_path_removed(path).items():
new_path = f"{path}['{key}']" if isinstance(key, str) else f"{path}[{key}]"
items.append(f'{next_indent}{RED}"{key}": {self._colorize_json(value, new_path, indent + 1)}{RESET}')
return '{\n' + ',\n'.join(items) + f'\n{current_indent}' + '}'

elif isinstance(obj, (list, tuple)):
if not obj:
return '[]'
removed_map = self._get_path_removed(path)
for index in removed_map:
self._colorize_skip_paths.add(f"{path}[{index}]")

items = []
remove_index = 0
for index, value in enumerate(obj):
while remove_index == next(iter(removed_map), None):
items.append(f'{next_indent}{RED}{self._format_value(removed_map.pop(remove_index))}{RESET}')
remove_index += 1
items.append(f'{next_indent}{self._colorize_json(value, f"{path}[{index}]", indent + 1)}')
remove_index += 1
for value in removed_map.values():
items.append(f'{next_indent}{RED}{self._format_value(value)}{RESET}')
return '[\n' + ',\n'.join(items) + f'\n{current_indent}' + ']'
else:
return self._format_value(obj)

def __str__(self) -> str:
"""Return the colorized, pretty-printed JSON string."""
self._colorize_skip_paths = set()
return self._colorize_json(self.t2)

def __iter__(self):
"""Make the view iterable by yielding the tree results."""
yield from self.tree.items()
8 changes: 7 additions & 1 deletion deepdiff/commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ def cli():
@click.option('--significant-digits', required=False, default=None, type=int, show_default=True)
@click.option('--truncate-datetime', required=False, type=click.Choice(['second', 'minute', 'hour', 'day'], case_sensitive=True), show_default=True, default=None)
@click.option('--verbose-level', required=False, default=1, type=click.IntRange(0, 2), show_default=True)
@click.option('--view', required=False, type=click.Choice(['tree', 'colored', 'colored_compact'], case_sensitive=True), show_default=True, default='tree')
@click.option('--debug', is_flag=True, show_default=False)
def diff(
*args, **kwargs
Expand All @@ -74,6 +75,8 @@ def diff(
t2_path = kwargs.pop("t2")
t1_extension = t1_path.split('.')[-1]
t2_extension = t2_path.split('.')[-1]
if "view" in kwargs and kwargs["view"] is None:
kwargs.pop("view")

for name, t_path, t_extension in [('t1', t1_path, t1_extension), ('t2', t2_path, t2_extension)]:
try:
Expand Down Expand Up @@ -112,7 +115,10 @@ def diff(
sys.stdout.buffer.write(delta.dumps())
else:
try:
print(diff.to_json(indent=2))
if kwargs["view"] in {'colored', 'colored_compact'}:
print(diff)
else:
print(diff.to_json(indent=2))
except Exception:
pprint(diff, indent=2)

Expand Down
16 changes: 13 additions & 3 deletions deepdiff/deephash.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/usr/bin/env python
import logging
import datetime
import uuid
from typing import Union, Optional, Any, List, TYPE_CHECKING
from collections.abc import Iterable, MutableMapping
from collections import defaultdict
Expand Down Expand Up @@ -99,6 +100,8 @@ def prepare_string_for_hashing(
original_type = obj.__class__.__name__
# https://docs.python.org/3/library/codecs.html#codecs.decode
errors_mode = 'ignore' if ignore_encoding_errors else 'strict'
if isinstance(obj, memoryview):
obj = obj.tobytes()
if isinstance(obj, bytes):
err = None
encodings = ['utf-8'] if encodings is None else encodings
Expand Down Expand Up @@ -163,6 +166,7 @@ def __init__(self,
ignore_string_type_changes=False,
ignore_type_in_groups=None,
ignore_type_subclasses=False,
ignore_uuid_types=False,
include_paths=None,
number_format_notation="f",
number_to_string_func=None,
Expand All @@ -177,7 +181,7 @@ def __init__(self,
"The valid parameters are obj, hashes, exclude_types, significant_digits, truncate_datetime,"
"exclude_paths, include_paths, exclude_regex_paths, hasher, ignore_repetition, "
"number_format_notation, apply_hash, ignore_type_in_groups, ignore_string_type_changes, "
"ignore_numeric_type_changes, ignore_type_subclasses, ignore_string_case "
"ignore_numeric_type_changes, ignore_type_subclasses, ignore_string_case, ignore_uuid_types, "
"number_to_string_func, ignore_private_variables, parent, use_enum_value, default_timezone "
"encodings, ignore_encoding_errors") % ', '.join(kwargs.keys()))
if isinstance(hashes, MutableMapping):
Expand All @@ -203,7 +207,9 @@ def __init__(self,
ignore_type_in_groups=ignore_type_in_groups,
ignore_string_type_changes=ignore_string_type_changes,
ignore_numeric_type_changes=ignore_numeric_type_changes,
ignore_type_subclasses=ignore_type_subclasses)
ignore_type_subclasses=ignore_type_subclasses,
ignore_uuid_types=ignore_uuid_types,
)
self.ignore_string_type_changes = ignore_string_type_changes
self.ignore_numeric_type_changes = ignore_numeric_type_changes
self.ignore_string_case = ignore_string_case
Expand Down Expand Up @@ -484,7 +490,7 @@ def _prep_number(self, obj):
number_format_notation=self.number_format_notation)
return KEY_TO_VAL_STR.format(type_, obj)

def _prep_ipranges(self, obj):
def _prep_ipranges(self, obj) -> str:
type_ = 'iprange'
obj = str(obj)
return KEY_TO_VAL_STR.format(type_, obj)
Expand Down Expand Up @@ -566,6 +572,10 @@ def _hash(self, obj, parent, parents_ids=EMPTY_FROZENSET):
elif isinstance(obj, ipranges):
result = self._prep_ipranges(obj)

elif isinstance(obj, uuid.UUID):
# Handle UUID objects (including uuid6.UUID) by using their integer value
result = str(obj)

elif isinstance(obj, MutableMapping):
result, counts = self._prep_dict(obj=obj, parent=parent, parents_ids=parents_ids)

Expand Down
Loading
Loading