Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simulator: implement --load and --watch flags #720

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

alpaylan
Copy link
Contributor

The current status of the PR is halfway. The new framing of simulation runner where setup_simulation is separated from run_simulation allows for injecting custom plans easily. The PR is currently missing the functionality to update the SimulatorEnv ad hoc from the plan, as the environment tables were typically created during the planning phase.

The next steps will be to implement a function fn mk_env(InteractionPlan, SimulatorEnv) -> SimulatorEnv, add --load flag to the CLI for loading a serialized plan file, making a corresponding environment and running the simulation.

We can optionally combine this with a --save option, in which we keep a seed-vault as part of limbo simulator, corresponding each seed with its generated plan and save the time to regenerate existing seeds by just loading them into memory. I am curious to hear thoughts on this? Would the maintainers be open to adding such a seed-vault? Do you think the saved time would be worth the complexity of the approach?

- makes interaction plans serializable
- fixes the shadowing bug where non-created tables were assumed to be created in the shadow tables map
- makes small changes to make clippy happy
- reorganizes simulation running flow to remove unnecessary plan regenerations while shrinking and double checking
…xecutes it instead of generating one from scratch

- save a json serialization of the generated plans to `<tempdir>/simulator.plan.json`
@alpaylan alpaylan marked this pull request as ready for review January 17, 2025 19:10
@alpaylan
Copy link
Contributor Author

Continuing with --watch.

Screen.Recording.2025-01-18.at.6.04.26.PM.mov

- add `--watch` flag
- start saving seeds in persistent storage
- make a separate version of execution functions that use `vector of interaction` instead of `InteractionPlan`
@alpaylan alpaylan changed the title simulator: implement --load flag simulator: implement --load and --watch flags Jan 18, 2025
@alpaylan
Copy link
Contributor Author

Watch mode is here. We can run --watch that checks the saved plan file and reruns it as it changes. Right now, it's pretty hacky in the following ways;

  • There is no custom parsing for plan files. We compare the JSON representation of the Interaction Plan against the text version, detect the removed interactions and delete them from the plan. This means we cannot change the file or add anything, only add things. In the future, we can use the Limbo SQL parser for also changing values.
  • It's not very well tested, as I'm not sure what kind of testing could give useful results, I just hand tested it.
  • There's a lot of code duplication for the execution, as that code is not very reusable with the existing abstractions. We should look into deduplicating them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant