Skip to content

Commit f49f5da

Browse files
cbleekcbleek
committed
docs: add mkdocs
1 parent e31985b commit f49f5da

File tree

4 files changed

+170
-0
lines changed

4 files changed

+170
-0
lines changed

‎.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,7 @@ behat.yml
44
node_modules/
55
/var/
66
.phpunit.result.cache
7+
8+
# ignore joe's backups
9+
*~
10+
.*~

‎docs/index.md

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# SimpleImport
2+
3+
The SimpleImport module offers a command line tool to import joboffers from feeds using a fix defined structure. It's usefull, if you plan to use
4+
YAWIK as a jobportal and if you would like to import jobs from customers.
5+
6+
The CLI offers the following functionalisites.
7+
8+
```
9+
imple import operations
10+
11+
yawik simpleimport import [--limit] [--name] [--id] Executes a data import for all registered crawlers
12+
yawik simpleimport add-crawler Adds a new import crawler
13+
14+
--limit=INT Number of crawlers to check per run. Default 3. 0 means no limit
15+
--name=STRING The name of a crawler
16+
--id=STRING The Mongo object id of a crawler
17+
--organization==STRING The ID of an organization
18+
--feed-uri=STRING The URI pointing to a data to import
19+
--runDelay=INT The number of minutes the next import run will be proceeded again
20+
--type=STRING The type of an import (e.g. job)
21+
--jobInitialState=STRING The initial state of an imported job
22+
--jobRecoverState=STRING The state a job gets, if it was deleted, but found again in later runs.
23+
24+
25+
yawik simpleimport info Displays a list of all available crawlers.
26+
yawik simpleimport info [--id] <name> Shows information for a crawler
27+
yawik simpleimport update-crawler [--id] <name> [--rename] [--limit] [--organization] [--feed-uri] [--runDelay] [--type] [--jobInitalState] [--jobRecoverState] Updates configuration for a crawler.
28+
yawik simpleimport delete-crawler [--id] <name> Deletes an import crawler
29+
30+
<name> The name of the crawler to delete.
31+
--id Treat <name> as the MongoID of the crawler
32+
--rename=STRING Set a new name for the crawler.
33+
34+
yawik simpleimport guess-language [--limit] Find jobs without language set and pushes a guess-language job into the queue for each.
35+
36+
--limit=INT Maximum number of jobs to fetch. 0 means fetch all.
37+
38+
39+
yawik simpleimport check-classifications <root> <categories> [<query>] Check job classifications.
40+
41+
<root> Root category ("industrues", "professions" or "employmentTypes")
42+
<categories> Required categories, comma separated. E.g. "Fulltime, Internship"
43+
<query> Search query for selecting jobs.
44+
45+
46+
--force Do not ignore already checked jobs.
47+
```
48+
49+
50+
51+
Feeds have to be formatted as defined in :ref:`the scrapy docs<scrapy:format>`.
52+
53+
54+
Example: Add a feed to a an organization
55+
56+
You need an organizationId in order to add a crawler job. So at first you have to create a company in your yawik. The organizationId
57+
appears in an URL, if you try to edit the organization.
58+
59+
```
60+
root@yawik:/var/www/YAWIK# ./vendor/bin/yawik simpleimport add-crawler --name=example-crawler \
61+
--organization=59e4b53e7bb2b553412f9be9 \
62+
--feed-uri=http://ftp.yawik.org/example.json
63+
A new crawler with the ID "59e4b5a87bb2b567468b4567" has been successfully added.
64+
```
65+
66+
This command created a crawler in the mongo collection ``simpleimport.crawler`` and returns the ObjectId.
67+
68+
69+
```
70+
> db.simpleimport.crawler.find({"_id":ObjectId("59e4b5a87bb2b567468b4567")}).pretty();
71+
{
72+
"_id" : ObjectId("59e4b5a87bb2b567468b4567"),
73+
"name" : "example-crawler",
74+
"organization" : ObjectId("59e4b53e7bb2b553412f9be9"),
75+
"type" : "job",
76+
"feedUri" : "http://ftp.yawik.org/example.json",
77+
"runDelay" : NumberLong(1440),
78+
"dateLastRun" : {
79+
"date" : ISODate("1970-01-01T00:00:00Z"),
80+
"tz" : "+00:00"
81+
},
82+
"options" : {
83+
"initialState" : "active",
84+
"_doctrine_class_name" : "SimpleImport\\Entity\\JobOptions"
85+
}
86+
}
87+
```
88+
89+
!!! note
90+
if you execute the command twice, the crawler will be added twice. If you want to remove a crawler, you have to
91+
do so on the mongo cli.

‎docs/schema.json

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-04/schema#",
3+
"type": "object",
4+
"properties": {
5+
"jobs": {
6+
"type": "array",
7+
"required": true
8+
"items": [
9+
{
10+
"type": "object",
11+
"properties": {
12+
"id": {
13+
"type": "string",
14+
"required": true,
15+
"uniqueItems" : true
16+
},
17+
"datePublishStart": {
18+
"type": "string",
19+
"pattern": "^\d{4}(-)(((0)[0-9])|((1)[0-2]))(-)([0-2][0-9]|(3)[0-1])$"
20+
},
21+
"title": {
22+
"type": "string",
23+
"required": true
24+
},
25+
"location": {
26+
"type": "string",
27+
"required" : false
28+
},
29+
"link": {
30+
"type": "string",
31+
"required": true
32+
},
33+
"linkApply": {
34+
"type": "string"
35+
},
36+
"templateValues": {
37+
"type": "object",
38+
"properties": {
39+
"html": {
40+
"type": "string",
41+
"required" : false
42+
}
43+
}
44+
}
45+
}
46+
}
47+
]
48+
}
49+
}
50+
}

‎mkdocs.yml

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
site_name: SimpleIport
2+
site_description: Documentation yawik SimpleImport module
3+
site_url: https://yawik.github.io/SimpleImport/
4+
copyright: 'Copyright &copy; 2021'
5+
repo_name: yawik/simple-import
6+
repo_url: https://github.com/yawik/SimpleImport
7+
site_dir: build/site
8+
theme: material
9+
markdown_extensions:
10+
- codehilite:
11+
guess_lang: true
12+
- toc:
13+
permalink: true
14+
- admonition
15+
- pymdownx.magiclink
16+
- pymdownx.tasklist:
17+
custom_checkbox: true
18+
- pymdownx.mark
19+
- pymdownx.tilde
20+
- pymdownx.extra
21+
- footnotes
22+
plugins:
23+
- section-index
24+
- search
25+

0 commit comments

Comments
 (0)