Rerun a subset of keys

[pepeiborra]

I want a version of shakeRunDatabase that takes in a set of rule keys to use as a dirtiness seed. Shake should then rerun only these rules and anything that depends on them, and assume that all other rule keys haven't changed.

The expected Shake overhead should be proportional to the number of rules dirtied and not to the number of rule dependencies.

It is ok if a Shake option is needed to enable this functionality and results in a higher space/time overhead in other operations

[pepeiborra]

I assume that this will require extending Shake to track reverse dependencies. @hsenag given your dag experience, do you know what are the pitfalls here?

[pepeiborra]

I have a rough, but working implementation of this. PR soon!

[hsenag]

I assume that this will require extending Shake to track reverse dependencies. @hsenag given your dag experience, do you know what are the pitfalls here?

I guess the main thing is that it adds more overhead. Firstly the obvious point about space/time to store and update the reverse dependencies. But also having a bidirectionally connected graph significantly increases the chances of memory leaks and means you need more complicated locking and update protocols to avoid races. I don't know much about the Shake graph and haven't looked at the detail of your PR, though.