Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance error messages in parameter operations #348

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
26 changes: 26 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# PolicyEngine Core - Development Guide 📚

## Common Commands
- `make all`: Full development cycle (install, format, test, build)
- `make install`: Install package in dev mode with dependencies
- `make format`: Format code with Black (79 char line limit)
- `make test`: Run all tests with coverage
- `make mypy`: Run type checking
- `pytest tests/path/to/test_file.py::test_function_name -v`: Run single test

## Code Style Guidelines
- **Formatting**: Black with 79 character line length
- **Imports**: Standard library, third-party, local modules (standard Python grouping)
- **Types**: Use type annotations, checked with mypy
- **Naming**: snake_case for functions/variables, CamelCase for classes
- **Error Handling**: Use custom error classes from policyengine_core/errors/
- **Testing**: All new features need tests, both unit and YAML-based tests supported
- **PRs**: Include "Fixes #{issue}" in description, add changelog entry, pass all checks
- **Changelog**: Update changelog_entry.yaml with proper bump type and change description
- **Dependencies**: All versions must be capped to avoid breaking changes
- **Python**: 3.10+ required

## Git Workflow
- First push to a new branch: `git push -u origin feature/branch-name` to set up tracking
- Subsequent pushes: just use `git push` to update the same PR
- Always run `make format` before committing to ensure code passes style checks
6 changes: 6 additions & 0 deletions changelog_entry.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
- bump: minor
changes:
added:
- A new ParameterPathError class for improved error handling in parameter operations
changed:
- Enhanced get_parameter error handling with specific error types and helpful suggestions
60 changes: 56 additions & 4 deletions policyengine_core/parameters/operations/get_parameter.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,64 @@ def get_parameter(root: ParameterNode, parameter: str) -> Parameter:
name, index = name.split("[")
index = int(index[:-1])
node = node.children[name].brackets[index]
except:
except Exception:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hard to review/blocking? Claude took a $ to create this, but it would take me a lot of time to review. If I had to change it at some point, it would take a long time to figure out what to change. I don't know claude actually could follow it's own logic here to maintain either.

Why is this hard to review/mantain: The level of nesting of if/else/try/catch/throw/re-catch is very hard to follow and contains at least redudnancies (AFAICT) so probably some bugs.

Why don't I think Claude will maintain this: Maybe it's worth spending the time to prove this, but based on my previous interactions with Claude: I bet if I take the time to fully understand this nested logic and talk to Claude about it, Claude will be demonstrably wrong in describing what it does.

What would be better Clean Code (book) has a lot of suggestions around how to write this kind of logic so it's clear/easy to read/follow/maintain.

suggestion Can you get Claude to clean this up to follow best principals from Clean Code for readability without additional feedback? (i.e. can we fix this for cheap or do I actually have to dig into it)

Copy link
Collaborator

@mikesmit mikesmit Feb 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, did Claude code write this? I was assuming that's the case (I picked this up assuming it was the change you referenced on LinkedIn yesterday), but maybe I'm wrong.

I also see copilot failed to flag any of this as a problem. How's that configured/setup? It seems like it could identify lots of confusing nesting even if not neccessarily correct the issue.

Copy link
Contributor Author

@MaxGhenis MaxGhenis Feb 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes claude code wrote it, i'll ask it to address your comments. i gave it this task to test its fully autonomy, just asked this after opening the repo:

find some opportunity to improve error handling, file an issue for it, then file a pr to do it linked to the issue

# Enhanced error message for bracket syntax errors
raise ValueError(
"Invalid bracket syntax (should be e.g. tax.brackets[3].rate"
f"Invalid bracket syntax at '{name}' in parameter path '{parameter}'. "
"Use format: parameter_name[index], e.g., brackets[0].rate"
)
except:
except KeyError:
# Enhanced error message for parameter not found
similar_params = []
if hasattr(node, "children"):
similar_params = [
key
for key in node.children.keys()
if (
name.lower() in key.lower()
or key.lower().startswith(name.lower())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

example I believe this is redundant (I.e. if name is in key, that already includes if key starts with name).

If I was trying to bug fix and assumed a person wrote this, I would now pause because people have intent and I would assume they actually did this for a specific reason which I should now understand before moving on.

)
and "[" not in name
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

example I'm pretty sure, based on the nested if logic above, that this cannot happen. (specifically we would have already caught any error related to a name with a "[" above and then rethrown that as a ValueError).

Again, if I'm maintaining this and assume a human wrote it, I now wonder what I didn't understand that leads to this edge case the coder obviously explicitly added for some reason.

]

suggestions = (
f" Did you mean: {', '.join(similar_params)}?"
if similar_params
else ""
)
raise ValueError(
f"Parameter '{name}' not found in path '{parameter}'.{suggestions}"
)
except (IndexError, AttributeError) as e:
# Enhanced error message for bracket index errors
if isinstance(e, IndexError) and "[" in name:
try:
name_part = name.split("[")[0]
max_index = len(node.children[name_part].brackets) - 1
raise ValueError(
f"Bracket index out of range in '{name}' of parameter path '{parameter}'. "
f"Valid indices are 0 to {max_index}."
)
except Exception:
pass
elif isinstance(e, AttributeError) and "[" in name:
name_part = name.split("[")[0]
raise ValueError(
f"'{name_part}' in parameter path '{parameter}' does not support bracket indexing. "
"Only scale parameters (like brackets) support indexing."
)

# Generic error message
raise ValueError(
f"Could not find the parameter (failed at {name})."
f"Could not find the parameter '{parameter}' (failed at '{name}'). {str(e)}"
)
except ValueError:
# Re-raise ValueError directly
raise
except Exception as e:
# Catch-all for other errors
raise ValueError(
f"Error accessing parameter '{parameter}' at '{name}': {str(e)}"
)

return node
74 changes: 74 additions & 0 deletions tests/core/parameters/operations/test_get_parameter.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
import pytest
from policyengine_core.parameters import ParameterNode
from policyengine_core.parameters.operations import get_parameter


def test_parameter_not_found_message():
"""Test that not found errors have better messages."""
parameters = ParameterNode(
data={
"tax": {
"income_tax": {
"rate": {
"values": {
"2022-01-01": 0.2,
}
},
},
"sales_tax": {
"rate": {
"values": {
"2022-01-01": 0.1,
}
},
},
}
}
)

# Test parameter not found
with pytest.raises(ValueError) as excinfo:
get_parameter(parameters, "tax.property_tax.rate")

# Ensure the error message contains useful information
error_message = str(excinfo.value)
assert "property_tax" in error_message
assert "not found" in error_message

# Test parameter not found but with similar names
with pytest.raises(ValueError) as excinfo:
get_parameter(parameters, "tax.income")

# Check that the suggestion for "income" includes "income_tax"
error_message = str(excinfo.value)
assert "income" in error_message
assert "Did you mean" in error_message
assert "income_tax" in error_message


def test_invalid_bracket_syntax_message():
"""Test error handling with invalid bracket syntax."""
parameters = ParameterNode(
data={
"tax": {
"brackets": [
{
"threshold": {"values": {"2022-01-01": 0}},
"rate": {"values": {"2022-01-01": 0.1}},
},
],
}
}
)

# Test invalid bracket syntax (missing closing bracket)
with pytest.raises(ValueError) as excinfo:
get_parameter(parameters, "tax.brackets[0.rate")

assert "Invalid bracket syntax" in str(excinfo.value)

# Test invalid bracket index (not an integer)
with pytest.raises(ValueError) as excinfo:
get_parameter(parameters, "tax.brackets[a].rate")

assert "Invalid bracket syntax" in str(excinfo.value)