You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The fanout is currently built during the merge operation, by registering all addition in its internal array, then it is both written in the header of the file, and kept in memory for future find to use.
Whenever a merge occurs, the old fannout is dropped, and a new one is created from scratch. Although quite light, this introduces a lot of unnecessary computations (also keep in mind that almost all changes by calls to update are overwritten by the following call anyway, which could also be improved).
Since merges only introduce shifts in the data file, the Fan module should be able to update an existing fanout instead of rewriting it from scratch every time. Special care should be taken when the fanout requires resizing (see the size computed in Fan.v).
The text was updated successfully, but these errors were encountered:
Reporting an offline discussion with @craigfe:
Given the current datastructure (a simple array) used for the fanout, it is unlikely that we can create a fanout from an existing one, and just update the log entries, especially when a resizing occurs (and accuracy is lost and can cumulate in time, resulting in very unhelpful states).
It appears that we will still have to register data entries, but we don't have to register them all. The bucket size used in the fanout is known, which means we could only register the last entry of each bucket in the merge.
The fanout is currently built during the
merge
operation, by registering all addition in its internal array, then it is both written in the header of the file, and kept in memory for futurefind
to use.Whenever a
merge
occurs, the old fannout is dropped, and a new one is created from scratch. Although quite light, this introduces a lot of unnecessary computations (also keep in mind that almost all changes by calls toupdate
are overwritten by the following call anyway, which could also be improved).Since merges only introduce shifts in the
data
file, theFan
module should be able to update an existing fanout instead of rewriting it from scratch every time. Special care should be taken when the fanout requires resizing (see the size computed inFan.v
).The text was updated successfully, but these errors were encountered: