Performance for loading facts (engine << [fact])

[waveofphase]

I was experimenting with the load times for large fact bases, and was wondering if there is a way to optimize appending facts to the engine. Is a way to load multiple facts at once and run a final assert and process_cascade call?

[ulfurinn]

The biggest issue I'm aware of is that alpha memory cells are not selective enough, and matchers have to repeatedly iterate through long lists of things they're not even interested in. I started rewriting the alpha memory to maintain smaller, more selective indexes some time ago but ran out of steam before I could finish :(

[waveofphase]

@ulfurinn did you happen to push the code for that? Otherwise, I'm curious if you've thought of any other ways to optimize this structure?

[ulfurinn]

No, I never did, but it was just a raw experiment even at its latest.

[ulfurinn]

The general idea was that permanently assigning an alpha node to a beta node causes issues when the beta node's template has only variables or blanks, so it gets assigned a very generic alpha node because you can't constrain it at compile time without having constants in the matcher rule, so most of that alpha's content will eventually be discarded; so instead alpha memory could index the data on lots of different combinations (like "all facts matching {a _ _}" and "all facts matching {a b _}" etc) and thus amortise a large chunk of the filtering work, and then a beta node could just say "give me your most specific set of data that matches this template".

[waveofphase]

That may be an option for us - do you have a specific spot in mind where is would be best to apply a fact-matching set?