Skip to content

v2.1.0

Compare
Choose a tag to compare
@umarbutler umarbutler released this 20 Jun 02:50
· 4 commits to main since this release

Fixed

  • Ceased memoizing chunk() (but not token counters) due to the fact that cached outputs of memoized functions are shallow rather than deep copies of original outputs, meaning that if one were to chunk a text and then chunk that same text again and then modify one of the chunks outputted by the first call, the chunks outputted by the second call would also be modified. This behaviour is not expected and therefore undesirable. The memoization of token counters is not impacted as they output immutable objects, namely, integers.