Skip to content

Releases: Kensuke-Mitsuzawa/JapaneseTokenizers

Possible to call jumandic and unidic

25 Mar 15:57
3bdfb6b
Compare
Choose a tag to compare

For mecab wrapper class, you could call

  • jumandic. It's alternative dictionary for mecab tokenizer. jumandic has rich morphological information.
  • unidic. It's continuously maintained by NINJA. See more information here, JP only.

And, in this version, some arguments in mecab wrapper class is deleted because it's not consistent anymore.

Cleaned up type hint

21 Jan 07:08
45af698
Compare
Choose a tag to compare
Merge pull request #54 from Kensuke-Mitsuzawa/enhancement/#53

cleaned up type hint

Bug fix for Python3.7 / latest pyknp package

24 Dec 17:13
76e2e5c
Compare
Choose a tag to compare
  • could not install some packages in setup.py because pip.main function is removed. Now setup.py calls subprocess.check_call function instead.
  • pyknp package is updated and Jumanpp module is removed in the latest version. Now, JapaneseTokenizer package calls the latest pyknp

Issue in MacOS

27 Feb 01:57
42789a1
Compare
Choose a tag to compare

The following issue is fixed, which is specific to MacOS, I guess.
#47

1.3.6: Merge pull request #45 from Kensuke-Mitsuzawa/bug/#44

01 Nov 11:04
5122e20
Compare
Choose a tag to compare

improved for using jumanpp

27 Sep 03:29
Compare
Choose a tag to compare

Fixed this issue -> #39

unified py2/py3 modules

21 Sep 08:12
Compare
Choose a tag to compare

fixed this issue #36

Bugs in filtering

11 Sep 09:29
Compare
Choose a tag to compare

The bugs in the filtering process, they are fixed.

Bug fix for specific case

15 Aug 02:31
Compare
Choose a tag to compare

There is a case which mecab + neologd returns a 10th-additional field. That causes value error inside a process of mecab-wrapper module. Detailed info. in #28

Fast call of Jumanpp

29 Jun 10:30
Compare
Choose a tag to compare
  • Jumanpp interface works faster than the previous version.
    • This is only for UNIX distributions. Windows OS is an exception.