Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Nethack Benchmarking Script #22

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

aholyoke
Copy link

@aholyoke aholyoke commented Oct 4, 2017

Setup

Download https://alt.org/nethack/xlogfile.full.txt
Run this script like:
python mk-nethack.py --generate --benchmark --name <name> /path/to/xlogfile.full.txt

--generate Creates a new tdb from the nethack log named <name>.tdb (Not included in benchmark timing)
The input can be passed as a file path or piped in from stdin.

eg. head -1000 /path/to/xlogfile.full.txt | python mk-nethack.py --generate --name sample

--benchmark Runs the benchmark on <name>.tdb

The Benchmark

The nethack data has a bunch of causes of death in the death field.
eg,

killed by invisible Ms. Possogroenoe, the shopkeeper
slipped while mounting a saddled pony called Argenti
petrified by a cockatrice, while helpless

We take the first word (Usually a verb) and track the average number of points for that death.

The Results

On the full nethack data, this takes about 2 minutes 20 seconds on master and about 48s using the #21 perf-improvements branch.

@Noeda
Copy link

Noeda commented Oct 5, 2017

Hey, I'm a kind of thinking maybe it would make sense to have a separate repository for example TrailDBs. The TrailDBs themselves aren't tied to any particular traildb bindings, are they?

@aholyoke
Copy link
Author

aholyoke commented Oct 5, 2017

This is a good idea. We should make a repository of example traildbs along with the script that generated them and benchmarks.

@tuulos
Copy link
Member

tuulos commented Oct 5, 2017

a separate repo of example traildbs is a great idea!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants