Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for discounted properties in MDPs and DTMCs #621

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

AlexBork
Copy link
Contributor

This PR adds support for checking cumulative and total reward properties with a given discount factor on MDPs and DTMCs.

  • Extends the parser to allow discounted properties, proposed syntax is R=? [Cdiscount=_factor_]
  • Introduces helper classes for checking discounted properties
  • Extends the respective PCTL model checking classes with support for discounted cumulative and total reward properties, implemented analogous to the existing undiscounted implementations

@AlexBork AlexBork added this to the 1.10 milestone Sep 10, 2024
@sjunges
Copy link
Contributor

sjunges commented Sep 10, 2024

This is absolutely great.

I have two questions:

  • Does this code support convergence detection based on the Bellman residual error?
  • Do we throw relevant error messages when doing, e.g., weak bismulation?

Best,
Sebastian

@AlexBork
Copy link
Contributor Author

Yes, convergence is detected using the Bellman residual, see src/storm/solver/helper/DiscountedValueIterationHelper.cpp. I have also tested convergence using only the number of iterations as the criterion, i.e. letting the iteration run until it is guaranteed that we are precise enough. That was however highly inefficient as the number of iterations for guaranteed convergence is usually far larger than the actual number needed to reach the precision.

The error messages are not implemented, I will take care of it! I'm happy about any pointers where such problems might occur.

@AlexBork
Copy link
Contributor Author

Thanks @sjunges for the review! I incorporated your comments now.

@sjunges
Copy link
Contributor

sjunges commented Oct 29, 2024

Thanks! @tquatmann do you want to have a look before merging?

@sjunges sjunges requested a review from tquatmann November 28, 2024 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants