Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Normalize Acceptance Rate Plot by ESS #345

Merged
merged 4 commits into from
Sep 11, 2020
Merged

Normalize Acceptance Rate Plot by ESS #345

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a9b845f
normalize acceptance rate by ess
Sep 8, 2020
2dbdbef
switch to pop size
Sep 9, 2020
f68f83c
adjust test
Sep 9, 2020
9b70038
additional test
Sep 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions pyabc/visualization/sample.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

from ..storage import History
from .util import to_lists_or_default
from ..weighted_statistics import effective_sample_size


def plot_sample_numbers(
Expand Down Expand Up @@ -263,7 +264,8 @@ def plot_acceptance_rates_trajectory(
yscale: str = 'lin',
size: tuple = None,
ax: mpl.axes.Axes = None,
colors: List[str] = None):
colors: List[str] = None,
normalize_by_ess: bool = False):
"""
Plot of acceptance rates over all iterations, i.e. one trajectory
per history.
Expand All @@ -288,6 +290,9 @@ def plot_acceptance_rates_trajectory(
The size of the plot in inches.
ax: matplotlib.axes.Axes, optional
The axis object to use.
normalize_by_ess: bool, optional (default = False)
Indicator to use effective sample size for the acceptance rate in
place of the population size.

Returns
-------
Expand All @@ -312,9 +317,16 @@ def plot_acceptance_rates_trajectory(
# note: the first entry of time -1 is trivial and is thus ignored here
h_info = history.get_all_populations()
times.append(np.array(h_info['t'])[1:])
samples.append(np.array(h_info['samples'])[1:])
pop_sizes.append(np.array(
if normalize_by_ess:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is one error here: Not samples should be replaced by ESS, but pop_sizes should be replaced by ESS, t.t. the resulting rations are usually <= 1. So, just move this code block up a bit ;) Sorry, if I miscommunicated that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also update the docstring accordingly.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!

ess = np.zeros(len(h_info['t']) - 1)
for t in np.array(h_info['t'])[1:]:
w = history.get_weighted_distances(t=t)['w']
ess[t-1] = effective_sample_size(w)
pop_sizes.append(ess)
else:
pop_sizes.append(np.array(
history.get_nr_particles_per_population().values[1:]))
samples.append(np.array(h_info['samples'])[1:])

# compute acceptance rates
rates = []
Expand Down
6 changes: 5 additions & 1 deletion test/visualization/test_visualization.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,11 @@ def test_acceptance_rates_trajectory():
histories, labels, yscale='log', rotation=76)
_, ax = plt.subplots()
pyabc.visualization.plot_acceptance_rates_trajectory(
histories, labels, yscale='log10', rotation=76, size=(10, 5), ax=ax)
histories, labels, yscale='log10', rotation=76, size=(10, 5), ax=ax
normalize_by_ess=True)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last thing: Could you once run this with normalize_by_ess=False, and once with True? So as to test both modes.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

pyabc.visualization.plot_acceptance_rates_trajectory(
histories, labels, yscale='log10', rotation=76, size=(10, 5), ax=ax
normalize_by_ess=False)
plt.close()


Expand Down