Skip to content

Commit

Permalink
first version
Browse files Browse the repository at this point in the history
  • Loading branch information
alekssamos committed Aug 29, 2020
0 parents commit 4112076
Show file tree
Hide file tree
Showing 13 changed files with 326 additions and 0 deletions.
10 changes: 10 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
tests/*.html
__pycache__
yandexfreetranslate/__pycache__
dist
build
*.egg-info
.tox

yandexfreetranslate/*.pyc
tests/*.pyc
13 changes: 13 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"python.linting.enabled": true,
"python.testing.unittestArgs": [
"-v",
"-s",
"./tests",
"-p",
"test_*.py"
],
"python.testing.pytestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.unittestEnabled": true
}
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
## Installing
`python3 -m pip install git+https://github.com/alekssamos/yandexfreetranslate.git`

or

`python3 -m pip install yandexfreetranslate`
### Using
```python3
from yandexfreetranslate import YandexFreeTranslate
yt = YandexFreeTranslate()

print(yt.translate("en", "ru", "Hello, world!"))```
Or from command line: ```bash
python3 yandexfreetranslate.py en ru "Hello, world!"
```
## Runing tests
```bash
git clone https://github.com/alekssamos/yandexfreetranslate.git
cd yandexfreetranslate
python3 -m pip install -r requirements-dev.txt

python3 -m unittest
# or
python3 -m tox
```
1 change: 1 addition & 0 deletions __init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

20 changes: 20 additions & 0 deletions example/in.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
A Fox once saw a Crow fly off with a piece of cheese in its beak and settle on a branch of a tree.
"That's for me, as I am a Fox," said Master Reynard, and he walked up to the foot of the tree.
"Good day, Mistress Crow," he cried. "How well you are looking today: how glossy your feathers; how bright your eye. I feel sure your voice must surpass that of other birds, just as your figure does; let me hear but one song from you that I may greet you as the Queen of Birds."
The Crow lifted up her head and began to caw her best, but the moment she opened her mouth the piece of cheese fell to the ground, only to be snapped up by Master Fox.
"That will do," said he. "That was all I wanted. In exchange for your cheese I will give you a piece of advice for the future: "Do not trust flatterers."

Once when a Lion was asleep a little Mouse began running up and down upon him; this soon wakened the Lion, who placed his huge paw upon him, and opened his big jaws to swallow him. "Pardon, O King," cried the little Mouse: "forgive me this time, I shall never forget it: who knows but what I may be able to do you a turn some of these days?" The Lion was so tickled at the idea of the Mouse being able to help him, that he lifted up his paw and let him go.
Some time after the Lion was caught in a trap, and the hunters who desired to carry him alive to the King, tied him to a tree while they went in search of a waggon to carry him on. Just then the little Mouse happened to pass by, and seeing the sad plight in which the Lion was, went up to him and soon gnawed away the ropes that bound the King of the Beasts. "Was I not right?" said the little Mouse.

The Hare was once boasting of his speed before the other animals. "I have never yet been beaten," said he, "when I put forth my full speed. I challenge any one here to race with me."
The Tortoise said quietly, "I accept your challenge."
"That is a good joke," said the Hare; "I could dance round you all the way."
"Keep your boasting till you've won," answered the Tortoise. "Shall we race?"
So a course was fixed and a start was made. The Hare darted almost out of sight at once, but soon stopped and, to show his contempt for the Tortoise, lay down to have a nap. The Tortoise plodded on and plodded on, and when the Hare awoke from his nap, he saw the Tortoise just near the winning-post and could not run up in time to save the race.
Then the Tortoise said: "Slow but steady progress wins the race."

The Wind and the Sun were disputing which was the stronger. Suddenly they saw a traveller coming down the road, and the Sun said, "I see a way to decide our dispute. Whichever of us can cause that traveller to take off his cloak shall be regarded as the stronger. You begin." So the Sun retired behind a cloud, and the Wind began to blow as hard as it could upon the traveller. But the harder he blew the more closely did the traveller wrap his cloak round him, till at last the Wind had to give up in despair. Then the Sun came out and shone in all his glory upon the traveller, who soon found it too hot to walk with his cloak on.


http://www.correctenglish.ru/reading/fairy-tales/
26 changes: 26 additions & 0 deletions example/out.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
������� ���� ������, ��� ������ ������� � ������ ���� � ����� � ���� �� ����� ������.
-��� ��� ����, ������ ��� � ���, - ������ ������ ������� � ������� � �������� ������.
- ������ ����, ������� ������! - ���������� ��. -��� ������ �� ��������� �������: ��� ������� ���� �����, ��� ������� ���� �����. � ������, ��� ���� ����� ������ ������������ ������ ������ ����, ��� � ���� ������; ������� ��� �������� �� ���� ������ ���� �����, ����� � ��� �������������� ���� ��� ������ ����."
������ ������� ������
� ��������� ������� ��� ���� ���, �� ���� ��� ������� ���, ��� ����� ���� ���� �� ����� � ��� ��� �� ��������� �������� �����.
-��������, - ������ ��. -��� ���, ���� � �����. � ����� �� ��� ��� � ��� ��� ����� �� �������: "�� ��������� ��������."

�������, ����� ��� ����, ��������� ����� ������ ������ ����� � ���� �� ����; ��� ������ ��������� ����, ������� ������� �� ���� ���� �������� ���� � ������ ���� ������� �����, ����� ���������� ���. - ������, � ����! - ���������� ������.
�������: "������ ���� �� ���� ���, � ������� ����� �� ������: ��� �����, ����� ����, �����-������ � ����� ���� ������? ���� ��� ����������� ����� � ���, ��� ������� ����� ��� ������, ��� �� ������ ���� � �������� ���.
����� ��������� ����� ��� ��� ������ � ������, � ��������, ������� ������ ��������� ��� ����� � ����, ��������� ��� � ������, � ���� ����������� �� ������ �������, ����� ��������� ��� ������. ��� ��� � ��� ����� ������� �������� �������� ���� � ������ ��������� ��������� ���.
� ������� ��������� ���, ������� � ���� � ������ �������� �������, �������� ��� ������ ���� ������. -����� � �� ��� ����?- ������ �������.

������� ���� ��������� ����� ��������� ����� ������� ���������. -���� ��� �� ���� �� ����, - ������ ��, - ����� � �������� �� ���� ����. � ������ ����� ������ �� �������������� �����, ����� ����������� �� ����."
�������� ���� �������: "� �������� ���� �����."
-��� ������� �����, - ������ ����. - � ��� �� ��������� ������ ���� ��� ������."
- ����� ���� ����������� ��� ����, ���� �� ���������, - ������� ��.
��������. - �����, �����������?"
����, ���� ��� ���������, � ����� ��� ������. ���� ����� ����� �� ������� �� ����, �� ������ ����������� �, ����� �������� ���� ��������� � ��������, ������ ����������. �������� ��� ������� � �������, � ����� ���� ������� �� �������, �� ������ �������� ������ ����� � �������� ������� � �� ����� ������� ���������, ����� ������ �����.
����� �������� �������: "���������, �� ���������� �������� ��������� � �����."

����� � ������ �������, ��� �������.
�������� ��� ������� �������, ������� �� ������, � ������ �������: "� ���� ������ ������ ��� ����. ��� �� ���, ��� �������� ����� ������� ����� ����, ����� ��������� ����� �������. �� ���������. ������ �������� �� �������, � ����� ���� ���� ��� ������, ��� ������ ���, �� ���������������. �� ��� ������� �� ���, ��� ������� ������ ����������� � ����, � ������� ����� � �������� ������. ����� ����� ������ � ������� �� ���� ����� �����.
��������������, �������� ������ ����� ������� �����, ����� ���� � �����.


http://www.correctenglish.ru/reading/fairy-tales/
1 change: 1 addition & 0 deletions runtime.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
python-3.7.8
18 changes: 18 additions & 0 deletions setup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
from setuptools import setup, find_packages
from os.path import join, dirname
import sys
if sys.version_info[0] != 3:
print("This script requires Python >= 3")
exit(1)

setup(
name="yandexfreetranslate",
version="1.0",
author="alekssamos",
author_email="[email protected]",
url="https://github.com/alekssamos/yandexfreetranslate/",
packages=find_packages(),
include_package_data=True,
long_description_content_type="text/markdown",
long_description=open(join(dirname(__file__), "README.md"), encoding="UTF8").read(),
)
16 changes: 16 additions & 0 deletions tests/test__sid_to_key.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import sys
import unittest
try:
from yandexfreetranslate import YandexFreeTranslate
except ImportError:
sys.path.insert(0, "..")
from yandexfreetranslate import YandexFreeTranslate
del sys.path[0]

class yt_test_sid_to_key(unittest.TestCase):
def test__sid_to_key(self):
yt = YandexFreeTranslate()
self.assertEqual(yt._sid_to_key("qw.er.ty"), "wq.re.yt-0-0")

if __name__ == "__main__":
unittest.main()
36 changes: 36 additions & 0 deletions tests/test_translate.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import sys
import os
import os.path
import re
import unittest
try:
import yandexfreetranslate
except ImportError:
sys.path.insert(0, "..")
import yandexfreetranslate
del sys.path[0]

class dummy_urlopen(object):
def __init__(self, url, *a, **kw):
if type(url) == yandexfreetranslate.urllibrequest.Request:
url = url.full_url
self.url = url
def read(self):
if "https://translate.yandex.ru/" in self.url:
with open(os.path.join(os.path.dirname(__file__), "page.html"), "rb", encoding="UTF8") as f:
body = f.read()
return body
if "https://translate.yandex.net/api/v1/tr.json/translate?" in self.url+"?":
body='{"text": ["Привет"]}'.encode("UTF8")
return body
raise ValueError("URL!")

yandexfreetranslate.urllibrequest.urlopen = dummy_urlopen

class yt_test_translate(unittest.TestCase):
def test_translate(self):
yt = yandexfreetranslate.YandexFreeTranslate()
self.assertTrue(len(yt.translate("en", "ru", "hello")) > 1)

if __name__ == "__main__":
unittest.main()
19 changes: 19 additions & 0 deletions tox.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# tox (https://tox.readthedocs.io/) is a tool for running tests
# in multiple virtualenvs. This configuration file will run the
# test suite on all supported python versions. To use it, "pip install tox"
# and then run "tox" from this directory.

[tox]
skip_missing_interpreters=true
skipsdist=true
envlist = py36, py37, py38

[testenv]
deps =
urllib3-mock
commands =
python -m unittest discover

[flake8]
ignore = E128,E501,W191
exclude = .git,__pycache__,docs/source/conf.py,old,build,dist
113 changes: 113 additions & 0 deletions yandexfreetranslate/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
#!/usr/bin/env python3
try: from utils import smartsplit
except: from .utils import smartsplit
import json
import os
import os.path
import re
import sys
import time
try:
import urllib.parse as urllibparse
import urllib.request as urllibrequest
except ImportError:
import urllib as urllibrequest
import urllib as urllibparse


class YandexFreeTranslateError(Exception): pass

class YandexFreeTranslate():
siteurl = "https://translate.yandex.ru/"
apibaseurl = "https://translate.yandex.net/api/v1/tr.json/"
ua = r"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:79.0) Gecko/20100101 Firefox/79.0"
key = ""
keysuffix = "-0-0"
keyfilename = os.path.join(os.path.expanduser("~"), ".YandexFreeTranslate.key")
expiretime = 60*60*24*4
def _sid_to_key(self, sid):
splitter = "."
l = []
for item in sid.split(splitter): l.append(item[::-1])
return splitter.join(l)+self.keysuffix
def _parse_sid(self):
req = urllibrequest.Request(self.siteurl)
req.add_header("User-Agent", self.ua)
req.add_header("Accept", r"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 ")
req.add_header("Accept-Language", r"ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3")
req.add_header("DNT", "1")
req.add_header("Accept-Encoding", "gzip, deflate, br")
page = urllibrequest.urlopen(req).read().decode("UTF8")
#open("page.html", "w", encoding="utf8").write(page)
try:
return re.search(r'''SID[\s]?[:][\s]?['"]([^'"]+)['"]''', page).group(1)
except AttributeError:
raise YandexFreeTranslateError("blocked or not found")
def _save_key(self, key):
with open(self.keyfilename, "w", encoding="utf8") as f:
f.write(key)
def _get_key(self):
if os.path.isfile(self.keyfilename) and (time.time() - os.path.getmtime(self.keyfilename)) < self.expiretime:
# print("from file")
with open(self.keyfilename, "r", encoding="utf8") as f:
return f.read()
else:
# print("from internet")
sid = self._parse_sid()
key = self._sid_to_key(sid)
self._save_key(key)
return key
def get_key(self): return self._get_key()
def regenerate_key(self):
if os.path.isfile(self.keyfilename): os.rename(self.keyfilename, self.keyfilename+".back")
return self._get_key()
def __init__(self):
if not os.path.isfile(self.keyfilename) and os.path.isfile(self.keyfilename+".back"):
os.rename(self.keyfilename+".back", self.keyfilename)
def translate(self, source = "auto", target="", text=""):
if self.key == "": self.key = self._get_key()
if source == "auto": source = ""
if len(source) != 0 and len(source) != 2: raise ValueError("source")
if len(target) == 0 or len(target) > 2: raise ValueError("target")
if text == "": raise ValueError("text")
if source==target: return text
if source == "": lang = target
else: lang = source+"-"+target
p=[]
for part in smartsplit(text, 500, 550):
req = urllibrequest.Request(self.apibaseurl+"translate?"+urllibparse.urlencode({
"id":self.key, "srv":"tr-text", "lang":lang, "reason":"paste"
}))
req.add_header("User-Agent", self.ua)
req.add_header("Accept", r"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 ")
req.add_header("Accept-Language", r"ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3")
req.add_header("DNT", "1")
req.add_header("Accept-Encoding", "gzip, deflate, br")
try:
content = urllibrequest.urlopen(req, data = urllibparse.urlencode({
"options": 4, "text":part
}).encode("UTF8")).read().decode("UTF8")
resp = json.loads(content)
except json.JSONDecodeError:
raise YandexFreeTranslateError(content)
if "text" not in resp:
raise YandexFreeTranslateError(content)
p.append(resp["text"][0])
return "\n".join(p)


if __name__ == "__main__":
if len(sys.argv) < 4:
print("usage: "+sys.argv[0]+" \t sourcelang \t targetlang \t text or file.txt \t [output_file_name.txt]")
print('example: {0} "{1}" "{2}" "{3}"'.format(sys.argv[0], "en", "ru", "hello, world!"))
print('example: {0} "{1}" "{2}" "{3}"'.format(sys.argv[0], "en", "ru", "in.txt"))
sys.exit(0)
yt = YandexFreeTranslate()
text = sys.argv[3]
if len(text) < 150 and os.path.isfile(text): text = open(text, "r", encoding="utf8").read()
tr_text = yt.translate(sys.argv[1], sys.argv[2], text)
if len(sys.argv) == 5:
with open(sys.argv[4], "w") as f: f.write(tr_text)
print("text saved in file: "+sys.argv[4])
else:
print(tr_text)
28 changes: 28 additions & 0 deletions yandexfreetranslate/utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
def smartsplit(t, s, e):
"""
smartsplit(text, start_position, end_position)
Splits a string into parts according to the number of characters.
If there is a space between the start and end positions, split before a space, the word will not end in the middle.
If there are no spaces in the specified range, divide as is, by the finite number of characters.
"""
t = t.replace("\r\n", "\n").replace("\r", "\n")
if(e >= len(t)): return [t]
l = []
tmp=""
i=0
for sim in t:
i = i + 1
tmp = tmp + sim
if(i<s): continue
if(i==e):
l.append(tmp)
tmp=""
i=0
continue
if((i > s and i < e) and (sim == chr(160) or sim == chr(9) or sim == chr(10) or sim == chr(32))):
l.append(tmp)
tmp=""
i=0
continue
if(len(tmp)>0): l.append(tmp)
return l

0 comments on commit 4112076

Please sign in to comment.