Merge branch 'main' of https://github.com/peviitor-ro/RaresTestHub

README.md

@@ -1 +1,48 @@
-# RaresTestHub
+
+# Test Job Scrapers for Peviitor.ro
+
+This project is dedicated to testing the accuracy of job data extraction, for details such as job title, city, country, county, and link which are scraped from companies websites from their careers section. Our primary objective is to ensure precise data extraction and uploading of these information to the Peviitor.ro website.
+
+## Features
+
+The project incorporates the following key features:
+
+- GitHub Actions: Enables automated test workflows directly from GitHub repositories, in this context, ensuring regular test reports for the scrapers
+- Pytest: A testing framework for Python that facilitates efficient and scalable testing.
+- API Testing: In-depth testing of APIs to guarantee data accuracy and reliability.
+- Multiple Assertions: Rigorous validation of multiple data points to ensure comprehensive accuracy.
+- Data Validation: The project emphasizes the importance of verifying scraped data against the information available on peviitor.ro.
+- Allure Reporting: Comprehensive reports are generated regularly for detailed insights of the testing process execution
+- Parallel Running: The ability to run multiple scraping processes concurrently for enhanced efficiency.
+
+## API Test Run with parallel running
+
+[https://github.com/peviitor-ro/JobsScrapers/assets/91252395/26dfbf52-d1c0-4213-8331-963ebd1dd94f](https://github-production-user-asset-6210df.s3.amazonaws.com/91252395/278391776-26dfbf52-d1c0-4213-8331-963ebd1dd94f.mp4)
+
+## API Test Run without parallel running
+
+[https://github.com/peviitor-ro/JobsScrapers/assets/91252395/8acaafb9-dff0-4f4f-ad7c-3f138e2daba8](https://github-production-user-asset-6210df.s3.amazonaws.com/91252395/277583302-8acaafb9-dff0-4f4f-ad7c-3f138e2daba8.mp4)
+
+## Allure Report
+
+For detailed insights into the test results, an Allure report is generated, providing a comprehensive overview of the scraping process.
+The report is updated every day when github actions runs and it can be seen live at https://peviitor-ro.github.io/RaresTestHub/api_test_report.html 
+
+![allure result 2](https://github.com/peviitor-ro/JobsScrapers/assets/91252395/fe1565c8-5ddd-481f-9211-c9fb41c02571)
+
+## Setup
+
+To get started with the project, follow these steps:
+
+- Install and Configure Python 3: Ensure you have Python 3 installed and properly configured on your system.
+- Set Up Your IDE: Prepare your preferred Integrated Development Environment (IDE) for working on this project.
+- Import Cloned Repository as Project: Clone the repository and import it into your IDE as a new project.
+- Add Folder Path to PYTHONPATH: Include the path of the project folder in the PYTHONPATH variable in your environment settings.
+- Install Required Packages: Use the following command to install all the necessary packages: 
+```bash
+pip install -r requirements.txt
+```
+- Running Tests: Execute the tests by running the following command:
+```bash
+pytest
+```

docs/allure-results/0015608b-8463-47ed-a6bb-02db7d22b2de-result.json

@@ -0,0 +1 @@
+{"name": "Test job countries from the autonet website against Peviitor API Response", "status": "passed", "steps": [{"name": "Step 1: Get job countries from the scraper", "status": "passed", "start": 1703653809234, "stop": 1703653809234}, {"name": "Step 2: Get job countries from the Peviitor API", "status": "passed", "start": 1703653809234, "stop": 1703653809234}, {"name": "Step 3: Compare job countries from scraper response against Peviitor API Response", "status": "passed", "start": 1703653809234, "stop": 1703653809234}], "attachments": [{"name": "stdout", "source": "da07a6c4-699d-4611-a82c-9d4094f485f7-attachment.txt", "type": "text/plain"}], "parameters": [{"name": "scraper_class", "value": "<class 'sites.autonet.autonetScraper'>"}], "start": 1703653809234, "stop": 1703653809234, "uuid": "daa9cc57-d578-4a11-8dcc-f849f5ba3b00", "historyId": "658ac27e01458c91941f016fb7b583c7", "testCaseId": "220e543539d71e666c936b3b389140f9", "fullName": "tests.test_api.TestScrapers#test_scrapers_country", "labels": [{"name": "tag", "value": "API"}, {"name": "tag", "value": "regression"}, {"name": "parentSuite", "value": "tests"}, {"name": "suite", "value": "test_api"}, {"name": "subSuite", "value": "TestScrapers"}, {"name": "host", "value": "fv-az835-688"}, {"name": "thread", "value": "2277-MainThread"}, {"name": "framework", "value": "pytest"}, {"name": "language", "value": "cpython3"}, {"name": "package", "value": "tests.test_api"}]}

docs/allure-results/00282b32-205d-4879-9ac4-e69cae644bed-container.json

@@ -0,0 +1 @@
+{"uuid": "0403978a-7995-4ea9-b998-1d1b4dd77213", "children": ["d9d8850a-4caf-4256-8554-7f37892a37c4", "b3f2f33d-54af-4237-b872-3b35f4552e1b"], "befores": [{"name": "scraper_class", "status": "passed", "start": 1703625893997, "stop": 1703625893997}], "start": 1703625893997, "stop": 1703625896179}

docs/allure-results/0028e141-b123-4dfe-8ef4-99d808f44cd0-result.json

@@ -0,0 +1 @@
+{"name": "Test job cities from the noodlepack website against Peviitor API Response", "status": "passed", "steps": [{"name": "Step 1: Get job cities from the scraper", "status": "passed", "start": 1703394607595, "stop": 1703394607595}, {"name": "Step 2: Get job cities from the Peviitor API", "status": "passed", "start": 1703394607595, "stop": 1703394607595}, {"name": "Step 3: Compare job cities from scraper response against Peviitor API Response", "status": "passed", "start": 1703394607595, "stop": 1703394607595}], "attachments": [{"name": "stdout", "source": "6a5c3d2e-fa8b-4e48-8033-b6e430a59118-attachment.txt", "type": "text/plain"}], "parameters": [{"name": "scraper_class", "value": "<class 'sites.noodlepack.noodlepackScraper'>"}], "start": 1703394607595, "stop": 1703394607595, "uuid": "7c7e8316-c4ff-4983-a88b-020efc627469", "historyId": "28f23c7ce3da7fbf42b97982f08545ac", "testCaseId": "de858b72cb7dd414b6682dd3b0adb884", "fullName": "tests.test_api.TestScrapers#test_scrapers_city", "labels": [{"name": "tag", "value": "API"}, {"name": "tag", "value": "regression"}, {"name": "parentSuite", "value": "tests"}, {"name": "suite", "value": "test_api"}, {"name": "subSuite", "value": "TestScrapers"}, {"name": "host", "value": "fv-az1053-263"}, {"name": "thread", "value": "1948-MainThread"}, {"name": "framework", "value": "pytest"}, {"name": "language", "value": "cpython3"}, {"name": "package", "value": "tests.test_api"}]}

docs/allure-results/0031338a-471d-4c55-9fc2-f5e0bf38cbec-attachment.txt

@@ -0,0 +1,3 @@
+Expected Results: ['CONSILIER SERVICE', 'CONSILIER VÂNZĂRI DACIA - RENAULT', 'ELECTROMECANIC AUTO', 'Mecanic auto', 'Pregătitor vopsele auto', 'Tinichigiu auto', 'Vopsitor auto']
+
+Actual Results: ['CONSILIER SERVICE', 'CONSILIER VÂNZĂRI DACIA - RENAULT', 'ELECTROMECANIC AUTO', 'Mecanic auto', 'Pregătitor vopsele auto', 'Tinichigiu auto', 'Vopsitor auto']

docs/allure-results/004c0bd8-b0fb-4590-825a-cc1643c2e214-result.json

@@ -0,0 +1 @@
+{"name": "Test job countries from the elytishospital website against Peviitor API Response", "status": "passed", "steps": [{"name": "Step 1: Get job countries from the scraper", "status": "passed", "start": 1703625917236, "stop": 1703625917236}, {"name": "Step 2: Get job countries from the Peviitor API", "status": "passed", "start": 1703625917236, "stop": 1703625917236}, {"name": "Step 3: Compare job countries from scraper response against Peviitor API Response", "status": "passed", "start": 1703625917236, "stop": 1703625917236}], "attachments": [{"name": "stdout", "source": "209dda5c-8683-4964-905f-8e8ed6b8a8f4-attachment.txt", "type": "text/plain"}], "parameters": [{"name": "scraper_class", "value": "<class 'sites.elytishospital.elytishospitalScraper'>"}], "start": 1703625917236, "stop": 1703625917236, "uuid": "937d3b18-6abb-4fcd-8900-dc2b14e40975", "historyId": "acf212a17e1b6053d7a31dfefcc99613", "testCaseId": "220e543539d71e666c936b3b389140f9", "fullName": "tests.test_api.TestScrapers#test_scrapers_country", "labels": [{"name": "tag", "value": "API"}, {"name": "tag", "value": "regression"}, {"name": "parentSuite", "value": "tests"}, {"name": "suite", "value": "test_api"}, {"name": "subSuite", "value": "TestScrapers"}, {"name": "host", "value": "fv-az1531-188"}, {"name": "thread", "value": "1921-MainThread"}, {"name": "framework", "value": "pytest"}, {"name": "language", "value": "cpython3"}, {"name": "package", "value": "tests.test_api"}]}

docs/allure-results/00733a69-7130-4883-92cc-ea8e0080f506-container.json

@@ -0,0 +1 @@
+{"uuid": "54a79a18-bc1b-485a-85ce-15c9bb400ae6", "children": ["fcc90b8c-12e0-4f1f-85d1-6b001e5e7f68"], "befores": [{"name": "scraper_class", "status": "passed", "start": 1703625937400, "stop": 1703625937400}], "start": 1703625937400, "stop": 1703625939411}

docs/allure-results/0078f05f-6ee8-4609-9717-c6492ec50fd1-attachment.txt

@@ -0,0 +1,3 @@
+Expected Results: ['https://www.expertmind.ro/joburi/admin_teren/', 'https://www.expertmind.ro/joburi/call_center/', 'https://www.expertmind.ro/joburi/economist_audit/', 'https://www.expertmind.ro/joburi/economist_contabil/', 'https://www.expertmind.ro/joburi/specialist_hr2/']
+
+Actual Results: ['https://www.expertmind.ro/joburi/admin_teren/', 'https://www.expertmind.ro/joburi/call_center/', 'https://www.expertmind.ro/joburi/economist_audit/', 'https://www.expertmind.ro/joburi/economist_contabil/', 'https://www.expertmind.ro/joburi/specialist_hr2/']

docs/allure-results/008a7360-54d4-4ed1-9b41-666289dfa729-result.json

@@ -0,0 +1 @@
+{"name": "Test job links from the ensemblesoftware website against Peviitor API Response", "status": "passed", "steps": [{"name": "Step 1: Get job links from the scraper", "status": "passed", "start": 1703653780762, "stop": 1703653780762}, {"name": "Step 2: Get job links from the Peviitor API", "status": "passed", "start": 1703653780762, "stop": 1703653780762}, {"name": "Step 3: Compare job links from scraper response against Peviitor API Response", "status": "passed", "start": 1703653780762, "stop": 1703653780762}], "attachments": [{"name": "stdout", "source": "ae2148c8-9511-47c7-82f4-e0f8678e05af-attachment.txt", "type": "text/plain"}], "parameters": [{"name": "scraper_class", "value": "<class 'sites.ensemblesoftware.ensemblesoftwareScraper'>"}], "start": 1703653780762, "stop": 1703653780762, "uuid": "e906a7ef-40dd-408e-9f48-0e013f4044d5", "historyId": "d74371e8fd743d0ba2d5eec48651897f", "testCaseId": "6754344b7f74f1d3db031396b511a41a", "fullName": "tests.test_api.TestScrapers#test_scrapers_link", "labels": [{"name": "tag", "value": "API"}, {"name": "tag", "value": "regression"}, {"name": "parentSuite", "value": "tests"}, {"name": "suite", "value": "test_api"}, {"name": "subSuite", "value": "TestScrapers"}, {"name": "host", "value": "fv-az835-688"}, {"name": "thread", "value": "2268-MainThread"}, {"name": "framework", "value": "pytest"}, {"name": "language", "value": "cpython3"}, {"name": "package", "value": "tests.test_api"}]}

docs/allure-results/00b3eadd-2db0-49ca-a371-2c8fb44924a1-attachment.txt

@@ -0,0 +1,3 @@
+Expected Results: ['https://autosiena.ro/cariera/#1', 'https://autosiena.ro/cariera/#2']
+
+Actual Results: ['https://autosiena.ro/cariera/#1', 'https://autosiena.ro/cariera/#2']

docs/allure-results/00b44f6f-334e-43ce-ad25-e6b1e984783e-result.json

@@ -0,0 +1 @@
+{"name": "Test job cities from the casaautoiasi website against Peviitor API Response", "status": "passed", "steps": [{"name": "Step 1: Get job cities from the scraper", "status": "passed", "start": 1703690021335, "stop": 1703690021335}, {"name": "Step 2: Get job cities from the Peviitor API", "status": "passed", "start": 1703690021335, "stop": 1703690021335}, {"name": "Step 3: Compare job cities from scraper response against Peviitor API Response", "status": "passed", "start": 1703690021336, "stop": 1703690021336}], "attachments": [{"name": "stdout", "source": "ad4012b9-a698-4ccb-9ed7-67201afd730a-attachment.txt", "type": "text/plain"}], "parameters": [{"name": "scraper_class", "value": "<class 'sites.casaautoiasi.casaautoiasiScraper'>"}], "start": 1703690021335, "stop": 1703690021336, "uuid": "4916adf2-d0e0-4c46-82cb-301f9749a9cf", "historyId": "4afc839fe8013dbbe80aa108b02d82c8", "testCaseId": "de858b72cb7dd414b6682dd3b0adb884", "fullName": "tests.test_api.TestScrapers#test_scrapers_city", "labels": [{"name": "tag", "value": "API"}, {"name": "tag", "value": "regression"}, {"name": "parentSuite", "value": "tests"}, {"name": "suite", "value": "test_api"}, {"name": "subSuite", "value": "TestScrapers"}, {"name": "host", "value": "fv-az977-882"}, {"name": "thread", "value": "1944-MainThread"}, {"name": "framework", "value": "pytest"}, {"name": "language", "value": "cpython3"}, {"name": "package", "value": "tests.test_api"}]}

docs/allure-results/00b7bc92-9b10-4dbc-ab23-7df9a3e680bf-container.json

@@ -0,0 +1 @@
+{"uuid": "a9e01c08-5d51-4c8e-ac91-e8113de6cd97", "children": ["574786b0-fd0b-4b34-99e4-f637509d6a6c"], "befores": [{"name": "setup_tests", "status": "passed", "start": 1703690044498, "stop": 1703690046238}], "start": 1703690044498, "stop": 1703690046242}

docs/allure-results/00c377de-aa2a-4e27-8e5c-3b2f7ab5c48e-attachment.txt

@@ -0,0 +1,3 @@
+Expected Results: ['https://www.tcandc.com/company/career/senior-systems-engineer.html', 'https://www.tcandc.com/company/career/software-developer.html', 'https://www.tcandc.com/company/career/software-tester.html', 'https://www.tcandc.com/company/career/systems-engineer.html']
+
+Actual Results: ['https://www.tcandc.com/company/career/senior-systems-engineer.html', 'https://www.tcandc.com/company/career/software-developer.html', 'https://www.tcandc.com/company/career/software-tester.html', 'https://www.tcandc.com/company/career/systems-engineer.html']

docs/allure-results/00c39b2e-5c6a-4943-aefe-21ec8c69c19a-attachment.txt

@@ -0,0 +1,3 @@
+Expected Results: ['https://www.sinaps.ro/wanted-account-manager/', 'https://www.sinaps.ro/wanted-digital-marketing-sales-consultant/', 'https://www.sinaps.ro/wanted-digital-marketing-specialist/', 'https://www.sinaps.ro/wanted-graphic-designer/', 'https://www.sinaps.ro/wanted-junior-ppc-specialist-internship/']
+
+Actual Results: ['https://www.sinaps.ro/wanted-account-manager/', 'https://www.sinaps.ro/wanted-digital-marketing-sales-consultant/', 'https://www.sinaps.ro/wanted-digital-marketing-specialist/', 'https://www.sinaps.ro/wanted-graphic-designer/', 'https://www.sinaps.ro/wanted-junior-ppc-specialist-internship/']

docs/allure-results/00c7ccf1-6ea6-4400-baea-4b5f30a9ddab-attachment.txt

@@ -0,0 +1,3 @@
+Expected Results: ['https://emp.jobylon.com/jobs/184574-basware-application-support-consultant/', 'https://emp.jobylon.com/jobs/192651-basware-lead-database-administrator-oracle/', 'https://emp.jobylon.com/jobs/199904-basware-support-consultant/', 'https://emp.jobylon.com/jobs/201157-basware-associate-support-consultant-with-english-and-french/', 'https://emp.jobylon.com/jobs/210070-basware-customer-support-consultant/', 'https://emp.jobylon.com/jobs/210161-basware-technical-consultant/', 'https://emp.jobylon.com/jobs/210295-basware-project-coordinator/', 'https://emp.jobylon.com/jobs/211161-basware-regional-support-consultant/', 'https://emp.jobylon.com/jobs/211604-basware-associate-support-consultant-with-english-and-german/', 'https://emp.jobylon.com/jobs/212563-basware-bid-management-lead/', 'https://emp.jobylon.com/jobs/213504-basware-senior-technical-consultant/']
+
+Actual Results: ['https://emp.jobylon.com/jobs/184574-basware-application-support-consultant/', 'https://emp.jobylon.com/jobs/192651-basware-lead-database-administrator-oracle/', 'https://emp.jobylon.com/jobs/199904-basware-support-consultant/', 'https://emp.jobylon.com/jobs/201157-basware-associate-support-consultant-with-english-and-french/', 'https://emp.jobylon.com/jobs/210070-basware-customer-support-consultant/', 'https://emp.jobylon.com/jobs/210161-basware-technical-consultant/', 'https://emp.jobylon.com/jobs/210295-basware-project-coordinator/', 'https://emp.jobylon.com/jobs/211161-basware-regional-support-consultant/', 'https://emp.jobylon.com/jobs/211604-basware-associate-support-consultant-with-english-and-german/', 'https://emp.jobylon.com/jobs/212563-basware-bid-management-lead/', 'https://emp.jobylon.com/jobs/213504-basware-senior-technical-consultant/']

docs/allure-results/00c7fe14-662f-4f2d-aa66-4520268612f0-attachment.txt

@@ -0,0 +1,3 @@
+Expected Results: [['Satu Mare'], ['Satu Mare']]
+
+Actual Results: [['Satu Mare'], ['Satu Mare']]

docs/allure-results/00d0ddfd-adec-44f4-a68d-1d78e0347785-result.json

@@ -0,0 +1 @@
+{"name": "Test job links from the kimballelectronics website against Peviitor API Response", "status": "passed", "steps": [{"name": "Step 1: Get job links from the scraper", "status": "passed", "start": 1703653786503, "stop": 1703653786503}, {"name": "Step 2: Get job links from the Peviitor API", "status": "passed", "start": 1703653786503, "stop": 1703653786503}, {"name": "Step 3: Compare job links from scraper response against Peviitor API Response", "status": "passed", "start": 1703653786503, "stop": 1703653786503}], "attachments": [{"name": "stdout", "source": "ea8bdc09-dc56-4f5c-9a91-52759600233f-attachment.txt", "type": "text/plain"}], "parameters": [{"name": "scraper_class", "value": "<class 'sites.kimballelectronics.kimballelectronicsScraper'>"}], "start": 1703653786503, "stop": 1703653786503, "uuid": "6c66617c-5fdc-4f41-a587-c01a15f6f8f6", "historyId": "bf3d7916d5992ec19b1aa749ae1fb25b", "testCaseId": "6754344b7f74f1d3db031396b511a41a", "fullName": "tests.test_api.TestScrapers#test_scrapers_link", "labels": [{"name": "tag", "value": "API"}, {"name": "tag", "value": "regression"}, {"name": "parentSuite", "value": "tests"}, {"name": "suite", "value": "test_api"}, {"name": "subSuite", "value": "TestScrapers"}, {"name": "host", "value": "fv-az835-688"}, {"name": "thread", "value": "2271-MainThread"}, {"name": "framework", "value": "pytest"}, {"name": "language", "value": "cpython3"}, {"name": "package", "value": "tests.test_api"}]}

docs/allure-results/00de35d1-b6be-4b32-8cc7-04595b9b47df-result.json

@@ -0,0 +1 @@
+{"name": "Test job cities from the atpgroup website against Peviitor API Response", "status": "passed", "steps": [{"name": "Step 1: Get job cities from the scraper", "status": "passed", "start": 1703481063192, "stop": 1703481063192}, {"name": "Step 2: Get job cities from the Peviitor API", "status": "passed", "start": 1703481063192, "stop": 1703481063192}, {"name": "Step 3: Compare job cities from scraper response against Peviitor API Response", "status": "passed", "start": 1703481063192, "stop": 1703481063192}], "attachments": [{"name": "stdout", "source": "eb1dedcb-bfc4-420c-8b99-ba28ac8dd602-attachment.txt", "type": "text/plain"}], "parameters": [{"name": "scraper_class", "value": "<class 'sites.atpgroup.atpgroupScraper'>"}], "start": 1703481063192, "stop": 1703481063192, "uuid": "f4bec8a3-1be2-4b27-8b03-ceca4e13f507", "historyId": "38bff7ecd07654b32580b6453a89564a", "testCaseId": "de858b72cb7dd414b6682dd3b0adb884", "fullName": "tests.test_api.TestScrapers#test_scrapers_city", "labels": [{"name": "tag", "value": "API"}, {"name": "tag", "value": "regression"}, {"name": "parentSuite", "value": "tests"}, {"name": "suite", "value": "test_api"}, {"name": "subSuite", "value": "TestScrapers"}, {"name": "host", "value": "fv-az588-803"}, {"name": "thread", "value": "2155-MainThread"}, {"name": "framework", "value": "pytest"}, {"name": "language", "value": "cpython3"}, {"name": "package", "value": "tests.test_api"}]}

docs/allure-results/00decb80-9f17-403c-9eed-583048636b91-result.json

@@ -0,0 +1 @@
+{"name": "Test job titles from the zucchetti website against Peviitor API Response", "status": "passed", "steps": [{"name": "Step 1: Get job titles from the scraper", "status": "passed", "start": 1703394530321, "stop": 1703394530321}, {"name": "Step 2: Get job titles from the Peviitor API", "status": "passed", "start": 1703394530321, "stop": 1703394530321}, {"name": "Step 3: Compare job titles from scraper response against Peviitor API Response", "status": "passed", "start": 1703394530321, "stop": 1703394530321}], "attachments": [{"name": "stdout", "source": "a43ad1ed-8ffa-4859-8b0c-018a0ed34ce7-attachment.txt", "type": "text/plain"}], "parameters": [{"name": "scraper_class", "value": "<class 'sites.zucchetti.zucchettiScraper'>"}], "start": 1703394530321, "stop": 1703394530321, "uuid": "1a995237-0716-41cc-9046-e1fd3b9f8351", "historyId": "38c664b279cff9e7b07cbffd4b8a2dd6", "testCaseId": "c080507e0e859c453fd919081af4356d", "fullName": "tests.test_api.TestScrapers#test_scrapers_title", "labels": [{"name": "tag", "value": "API"}, {"name": "tag", "value": "regression"}, {"name": "parentSuite", "value": "tests"}, {"name": "suite", "value": "test_api"}, {"name": "subSuite", "value": "TestScrapers"}, {"name": "host", "value": "fv-az1053-263"}, {"name": "thread", "value": "1939-MainThread"}, {"name": "framework", "value": "pytest"}, {"name": "language", "value": "cpython3"}, {"name": "package", "value": "tests.test_api"}]}