nagisa v0.2.11

nagisa 0.2.11 incorporates the following changes:

• Fix the Poetry installation error that @himkt reported #32

This issue was caused by the lack of correct library dependencies in the tar.gz files registered on PyPI. Specifically, in versions prior to 0.2.10, the following information was not included in the PKG-INFO file of the tar.gz.

Requires-Dist: six
Requires-Dist: numpy
Requires-Dist: DyNet38

nagisa-0.2.10.tar.gz does not include the Requires-Dist in PKG-INFO.

The issue was caused by an outdated build environment when creating tar.gz. Therefore, by updating pip, wheel, and build to their latest versions and creating tar.gz, we were able to include the correct dependency information in the tar.gz, resolving this problem.

nagisa-0.2.11.tar.gz includes the Requires-Dist in PKG-INFO.

In Poetry, dependency library information is fetched from https://pypi.org/pypi/nagisa/json, which refers to tar.gz. Therefore, errors occurred in versions prior to 0.2.10.

import requests

version = "0.2.10"

url = f"https://pypi.org/pypi/nagisa/{version}/json"
response = requests.get(url)
data = response.json()
dependencies = data.get("info", {}).get("requires_dist", [])
print(f"Version {version}: {dependencies}")
# Version 0.2.10: ['six', 'numpy', 'DyNet']

With this update, it is now possible to obtain correct information about dependency libraries.

import requests

version = "0.2.11"

url = f"https://pypi.org/pypi/nagisa/{version}/json"
response = requests.get(url)
data = response.json()
dependencies = data.get("info", {}).get("requires_dist", [])
print(f"Version {version}: {dependencies}")
# Version 0.2.11: ['six', 'numpy', 'DyNet38']

• Add Python wheels (3.6, 3.7, 3.8, 3.9, 3.10, 3.11, 3,12) to PyPI for Linux
• Add Python wheels (3.6, 3.7, 3.8, 3.9, 3.10, 3.11) to PyPI for macOS Intel
• Add Python wheels (3.6, 3.7, 3.8) to PyPI for Windows
• Fix the macOS M1/2 installation error #35 #30 (Updated on Jun 15, 2024)
• Add Python wheels (3.9, 3.10, 3.11, 3.12) to PyPI for macOS M1/M2 (Updated on Jun 15, 2024)
• Fix the aarch64 installation error #33 (Updated on Jun 16, 2024)
• Add Python wheels (3.9, 3.10, 3.11, 3.12) to PyPI for aarch64 (Updated on Jun 16, 2024)

nagisa v0.2.10

nagisa 0.2.10 incorporates the following changes:

• Fix hard-coding process for noun id conversion in tagger.py e638343

• Hidden the dynet log that appears when import nagisa is used in Python 3.8, 3.9, 3.10, 3.11, and 3,12 on Linux

• Provide the nagisa-demo page in Hugging Face Spaces

• Provide the stopwords for nagisa in Hugging Face Datasets

• Update read the docs documents

• Compatible with Python 3.12 on Linux

• Add Python wheels (3.6, 3.7, 3.8, 3.9, 3.10, 3.11, 3,12) to PyPI for Linux

• Add Python wheels (3.6, 3.7, 3.8, 3.9, 3.10, 3.11) to PyPI for macOS Intel

• Add Python wheels (3.6, 3.7, 3.8) to PyPI for Windows

nagisa v0.2.9

nagisa 0.2.9 incorporates the following changes:

• Improve the bottleneck in part-of-speech tagging caused by 'list and append', problem resolved by using 'set and add'
  

Until now, there was an issue where the processing time would slow down as the results analyzed by the following code increased in tagger.py.

tids = []
for w in words:
    if w in self._word2postags:
        w2p = self._word2postags[w]
    else:
        w2p = [0] 
    if self.use_noun_heuristic is True:
        if w.isalnum() is True:
            if w2p == [0]:
                w2p = [self._pos2id[u'名詞']] 
            else:
                # bottleneck is here!
                w2p.append(self._pos2id[u'名詞']) 
    w2p = list(set(w2p))
    tids.append(w2p)

By changing to the following code, we have resolved the issue of the processing slowing down.

tids = []
for w in words:
    w2p = set(self._word2postags.get(w, [0]))
    if self.use_noun_heuristic and w.isalnum():
        if 0 in w2p:
            w2p.remove(0)
        w2p.add(2)  # nagisa.tagger._pos2id["名詞"] = 2 
    tids.append(list(w2p))

• Fix dash-separated 'description-file' error in setup.cfg to use 'description_file' in setup.cfg

[metadata]
description_file = README.md

• Add Python wheels (3.6, 3.7, 3.8, 3.9, 3.10, 3.11) to PyPI for Linux
• Add Python wheels (3.6, 3.7, 3.8, 3.9, 3.10) to PyPI for macOS
• Add Python wheels (3.6, 3.7, 3.8) to PyPI for Windows

nagisa v0.2.8

nagisa 0.2.8 incorporates the following changes:

• Fix AttributeError in nagisa_utils.pyx when tokenizing a text containing Latin capital letter I with dot above 'İ'

When tokenizing a text containing 'İ', an AttributeError has occurred. This is because, as the following example shows, lowering 'İ' would have changed to the length of 2, and would not have been extracting features correctly.

>>> text = "İ" # [U+0130]
>>> print(len(text))
1
>>> text = text.lower() # [U+0069] [U+0307]
>>> print(text)
'i̇'
>>> print(len(text))
2

To avoid this error, the following preprocess was added to the source code modification 1, modification 2.

text = text.replace('İ', 'I')

• Add Python wheels (3.6, 3.7, 3.8, 3.9, 3.10, 3.11) to PyPI for Linux
• Add Python wheels (3.6, 3.7, 3.8, 3.9, 3.10) to PyPI for macOS
• Add Python wheels (3.6, 3.7, 3.8) to PyPI for Windows

nagisa v0.2.7

nagisa 0.2.7 incorporates the following changes:

• Fix AttributeError: module 'utils' to rename utils.pyx into nagisa_utils.pyx #14
• Add wheels to PyPI for Linux and Windows users
• Increase test coverage from 92% to 96%
• Fix the problem where min_count (threshold=hp['THRESHOLD']) parameter was not used in train.py

nagisa v0.2.6

nagisa 0.2.6 incorporates the following changes:

• Increase test coverage from 88% to 92%
• Fix readFile(filename) in mecab_system_eval.py for windows users
• Add python3.7 to .travis.yml
• Add a DOI with the data archiving tool Zenodo to README.md
• Add nagisa-0.2.6-cp36-cp36m-win_amd64.whl and nagisa-0.2.6-cp37-cp37m-win_amd64.whl to PyPI to install nagisa without Build Tools for Windows users #23
• Add nagisa-0.2.6-*-manylinux1_i686.whl and nagisa-0.2.6-*-manylinux1_x86_64.whl to PyPI to install nagisa for Linux users

nagisa v0.2.5

nagisa 0.2.5 incorporates the following changes:

• Fix a white space bug in nagisa.decode. This fix resolves an error that occurs when decoding(nagisa.decode) words - contain whitespace.
• Add __version__ to __init__.py
• Add slides link at PyCon JP 2019 to README.md

nagisa v0.2.4

nagisa 0.2.4 incorporates the following changes:

• Add the new tutorial to the document (train a model for Japanese NER).
• Add load_file function to nagisa.utils.
• Fix 'single_word_list' compiler in nagisa.Tagger and support word segmentation using a regular expression.

nagisa v0.2.3

nagisa 0.2.3 incorporates the following changes:

• FIx #11 . By separating tagging into word segmentation and POS tagging in tagger.py, nagisa.tagging reduces wasteful memory and improves the speed in word segmentation.
• Fix typo in README.md

nagisa v0.2.2

nagisa 0.2.2 incorporates the following changes:

• Update the document (e.g, add train a model for Japanese Universal Dependencies).
• Fix log output of nagisa.fit function.
• Fix issues from Codacy (e.g, delete unused codes in train.py).
• Add appveyor.yml for Windows users.