File tree Expand file tree Collapse file tree 2 files changed +19
-1
lines changed
bindings/python/pyonmttok Expand file tree Collapse file tree 2 files changed +19
-1
lines changed Original file line number Diff line number Diff line change @@ -26,6 +26,24 @@ The project follows [semantic versioning 2.0.0](https://semver.org/). The API co
26
26
27
27
### Fixes and improvements
28
28
29
+ ## [ v1.31.0] ( https://github.com/OpenNMT/Tokenizer/releases/tag/v1.31.0 ) (2022-03-07)
30
+
31
+ ### New features
32
+
33
+ * Add utilities to build and use vocabularies:
34
+ * ` pyonmttok.Vocab `
35
+ * ` pyonmttok.build_vocab_from_tokens `
36
+ * ` pyonmttok.build_vocab_from_lines `
37
+ * Define the method ` Tokenizer.__call__ ` to simplify the tokenizer usage when additional features are unused:
38
+
39
+ ``` python
40
+ tokens = tokenizer(text)
41
+ ```
42
+
43
+ ### Fixes and improvements
44
+
45
+ * Update pybind11 to 2.9.1
46
+
29
47
## [ v1.30.1] ( https://github.com/OpenNMT/Tokenizer/releases/tag/v1.30.1 ) (2022-01-25)
30
48
31
49
### Fixes and improvements
Original file line number Diff line number Diff line change 1
1
"""Version information."""
2
2
3
- __version__ = "1.30.1 "
3
+ __version__ = "1.31.0 "
You can’t perform that action at this time.
0 commit comments