Skip to content

Commit b35fa4d

Browse files
simonhkswansimonhkswan
authored
Add CI action workflows for testing and building (#21)
* add ci action workflows * Update pytest coverage argument * Add sonarcloud scanning * Update license and classifiers * Use pre-commit.ci instead of action * Fix lint issues
1 parent 8d17059 commit b35fa4d

File tree

12 files changed

+184
-112
lines changed

12 files changed

+184
-112
lines changed

‎.github/workflows/CI.yaml

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
name: CI
3+
4+
on:
5+
pull_request:
6+
types: [opened, synchronize, reopened]
7+
push:
8+
branches:
9+
- main
10+
11+
jobs:
12+
test:
13+
runs-on: ubuntu-latest
14+
continue-on-error: true
15+
16+
steps:
17+
- uses: actions/checkout@v2
18+
with:
19+
ref: ${{ env.GITHUB_SHA }}
20+
fetch-depth: 0
21+
22+
- name: Set up Python
23+
id: setup-py
24+
uses: actions/setup-python@v2
25+
with:
26+
python-version: 3.7
27+
28+
- name: Cache test venv
29+
uses: actions/cache@v2
30+
with:
31+
path: .pip-cache
32+
key: ${{ runner.os }}-pip-tenv-${{ steps.setup-py.outputs.python-version }}-${{ hashFiles('./setup.*') }}
33+
restore-keys: |
34+
${{ runner.os }}-pip-tenv-${{ steps.setup-py.outputs.python-version }}
35+
36+
- name: Create test-venv
37+
run: |
38+
python3 -m venv .tenv
39+
source .tenv/bin/activate
40+
pip install -U pip
41+
42+
- name: Install synthesized-insight
43+
run: |
44+
source .tenv/bin/activate
45+
pip install .[test] --cache-dir .pip-cache
46+
47+
- name: Run unit tests
48+
run: |
49+
source .tenv/bin/activate
50+
pytest -v --junitxml=test-results/junit.xml --cov=src/synthesized_insight --cov-report=xml:coverage-reports/cobertura.xml --cov-branch
51+
sed -ie 's#/home/runner/work/insight/insight#/github/workspace#g' coverage-reports/cobertura.xml
52+
env:
53+
SYNTHESIZED_KEY: ${{ secrets.SYNTHESIZED_KEY }}
54+
55+
- name: SonarCloud Scan
56+
uses: SonarSource/sonarcloud-github-action@master
57+
with:
58+
projectBaseDir: .
59+
env:
60+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any
61+
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
62+
...

‎.github/workflows/lint-test.yaml

Lines changed: 0 additions & 64 deletions
This file was deleted.

‎.github/workflows/publish.yaml

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
name: Publish to PyPI
3+
4+
on:
5+
release:
6+
types: [published]
7+
8+
jobs:
9+
build-and-publish:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- name: Checkout tag
13+
uses: actions/checkout@v2
14+
with:
15+
fetch-depth: 0
16+
ref: ${{ github.event.release.tag_name }}
17+
18+
- name: Set up Python 3.7
19+
uses: actions/setup-python@v1
20+
with:
21+
python-version: 3.7
22+
23+
- name: Install pypa/build
24+
run: python -m pip install build --user
25+
26+
- name: Build wheel and source dist
27+
run: python -m build --sdist --wheel --outdir dist/ .
28+
29+
- name: Publish to PyPI
30+
uses: pypa/gh-action-pypi-publish@release/v1
31+
with:
32+
user: __token__
33+
password: ${{ secrets.PYPI_API_TOKEN }}
34+
...

‎.pre-commit-config.yaml

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,13 +17,16 @@ repos:
1717
hooks:
1818
- id: flake8
1919
- repo: https://github.com/pre-commit/mirrors-mypy
20-
# note that the mypy precommit does not properly check all the code
21-
# it is recommended to run mypy locally in a virtualenv to check properly
2220
rev: v0.910
2321
hooks:
2422
- id: mypy
2523
files: src
24+
additional_dependencies: [pandas, pandas-stubs, numpy]
25+
args: [--install-types, --non-interactive]
26+
# Note that using the --install-types is problematic if running in
27+
# parallel as mutating the pre-commit env at runtime breaks cache.
2628
- repo: https://github.com/pycqa/isort
2729
rev: 5.9.3
2830
hooks:
2931
- id: isort
32+
...

‎LICENSE.md

Lines changed: 25 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,29 @@
1-
MIT License
1+
BSD 3-Clause License
22

3-
Copyright (c) 2021 Synthesized Ltd.
3+
Copyright (c) 2021, Synthesized Ltd.
4+
All rights reserved.
45

5-
Permission is hereby granted, free of charge, to any person obtaining a copy
6-
of this software and associated documentation files (the "Software"), to deal
7-
in the Software without restriction, including without limitation the rights
8-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9-
copies of the Software, and to permit persons to whom the Software is
10-
furnished to do so, subject to the following conditions:
6+
Redistribution and use in source and binary forms, with or without
7+
modification, are permitted provided that the following conditions are met:
118

12-
The above copyright notice and this permission notice shall be included in all
13-
copies or substantial portions of the Software.
9+
1. Redistributions of source code must retain the above copyright notice, this
10+
list of conditions and the following disclaimer.
1411

15-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21-
SOFTWARE.
12+
2. Redistributions in binary form must reproduce the above copyright notice,
13+
this list of conditions and the following disclaimer in the documentation
14+
and/or other materials provided with the distribution.
15+
16+
3. Neither the name of the copyright holder nor the names of its
17+
contributors may be used to endorse or promote products derived from
18+
this software without specific prior written permission.
19+
20+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
21+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
22+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
23+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
24+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
25+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
26+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
27+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
28+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,3 @@
11
# synthesized-insight
2+
3+
[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/synthesized-io/insight/master.svg)](https://results.pre-commit.ci/latest/github/synthesized-io/insight/master)

‎pyproject.toml

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,18 +2,28 @@
22
requires = [
33
"setuptools>=42",
44
"wheel",
5-
"Cython==3.0a7"
5+
"setuptools_scm[toml]>=6.0"
66
]
77
build-backend = "setuptools.build_meta"
88

9+
[tool.setuptools_scm]
10+
write_to = "src/synthesized_insight/version.py"
11+
912
[tool.pytest.ini_options]
10-
addopts = "--strict-markers"
11-
markers = [
12-
"slow"
13-
]
1413
junit_suite_name = "unit"
1514
junit_logging = "all"
1615
junit_log_passing_tests = true
1716
junit_duration_report = "call"
1817
junit_family = "xunit1"
1918
log_level = "INFO"
19+
20+
[tool.mypy]
21+
plugins = "numpy.typing.mypy_plugin"
22+
23+
[[tool.mypy.overrides]]
24+
module = "sklearn.*"
25+
ignore_missing_imports = true
26+
27+
[[tool.mypy.overrides]]
28+
module = "scipy.*"
29+
ignore_missing_imports = true

‎setup.cfg

Lines changed: 18 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,17 @@ author = Synthesized Ltd.
66
author_email = [email protected]
77
license_file = LICENSE.md
88
classifiers =
9-
Programming Language :: Python 3
9+
License :: OSI Approved :: BSD License
10+
Development Status :: 3 - Alpha
11+
Operating System :: Microsoft :: Windows
12+
Operating System :: Unix
13+
Operating System :: MacOS
14+
Intended Audience :: Developers
15+
Intended Audience :: Science/Research
16+
Programming Language :: Python :: 3.6
17+
Programming Language :: Python :: 3.7
18+
Programming Language :: Python :: 3.8
19+
Programming Language :: Python :: 3.9
1020
project_urls =
1121
homepage = https://synthesized.io
1222
documentation = https://docs.synthesized.io/
@@ -16,9 +26,8 @@ project_urls =
1626
packages = find:
1727
package_dir =
1828
= src
19-
zip_safe = False
2029
install_requires =
21-
numpy >= 1.18.4, < 1.20.0
30+
numpy ~= 1.18
2231
pandas ~= 1.1
2332
scikit_learn ~= 0.23
2433
scipy ~= 1.5
@@ -32,21 +41,13 @@ test =
3241
pytest
3342
pytest-cov
3443
doc =
35-
sphinx==3.5.4
36-
pydata-sphinx-theme==0.6.3
37-
sphinx-panels==0.5.2
38-
sphinx-autodoc-typehints==1.12.0
39-
sphinx-inline-tabs==2021.3.28b7
40-
ipython>=7.2.0
44+
sphinx ~= 4.0
45+
pydata-sphinx-theme ~= 0.6
46+
sphinx-panels ~= 0.5
47+
sphinx-autodoc-typehints ~= 1.12
48+
sphinx-inline-tabs == 2021.3.28b7
49+
ipython >= 7.2.0
4150
dev =
42-
typing_extensions; python_version < '3.7'
43-
types-setuptools
44-
types-simplejson
45-
types-PyYAML
46-
types-pkg-resources
47-
flake8
48-
mypy
49-
build
5051
pre-commit
5152

5253
[flake8]

‎sonar-project.properties

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
sonar.organization=synthesized-io
2+
sonar.projectKey=synthesized-io_synthesized-insight
3+
sonar.projectName=synthesized-insight
4+
sonar.sourceEncoding=UTF-8
5+
6+
sonar.sources=src/synthesized_insight/
7+
sonar.tests=tests/
8+
9+
sonar.python.version=3.6, 3.7, 3.8, 3.9
10+
sonar.python.xunit.reportPath=test-results/*.xml
11+
sonar.python.coverage.reportPaths=coverage-reports/*.xml
12+
13+
sonar.pullrequest.provider=github
14+
sonar.pullrequest.github.repository=synthesized-io/synthesized-insight
15+
sonar.pullrequest.github.endpoint=https://api.github.com

‎src/synthesized_insight/check.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
from abc import ABC, abstractmethod
2+
from typing import cast
23

34
import numpy as np
45
import pandas as pd
@@ -66,7 +67,7 @@ def infer_dtype(self, sr: pd.Series) -> pd.Series:
6667

6768
in_dtype = str(col.dtype)
6869
n_nans = col.isna().sum()
69-
n_empty_str = 0 if not pd.api.types.is_string_dtype(col) else (sr == "").sum()
70+
n_empty_str = 0 if not pd.api.types.is_string_dtype(col) else sr.eq("").sum()
7071

7172
# Try to convert it to numeric
7273
if col.dtype.kind not in ("i", "u", "f") and col.dtype.kind != 'M':
@@ -90,7 +91,7 @@ def infer_dtype(self, sr: pd.Series) -> pd.Series:
9091
elif out_dtype in ("i", "u", "f", "f8", "i8", "u8"):
9192
return pd.to_numeric(col, errors="coerce")
9293

93-
return col.astype(out_dtype, errors="ignore")
94+
return cast(pd.Series, col.astype(out_dtype, errors="ignore"))
9495

9596
def continuous(self, sr: pd.Series) -> bool:
9697
"""Checks if the given series is continuous

‎src/synthesized_insight/modelling/preprocessor.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ def fit(self, df: pd.DataFrame):
3838
for cat_col in self._categorical(df):
3939
logger.debug(f"Preprocessing value '{cat_col}'...")
4040
column = df[cat_col]
41-
column = column.fillna('nan').astype(str)
41+
column = cast(pd.Series, column.fillna('nan').astype(str))
4242
if self.target and cat_col == self.target:
4343
x_i = column.to_numpy().reshape(-1, 1)
4444
encoder = LabelEncoder()
@@ -86,7 +86,7 @@ def transform(self, df: pd.DataFrame) -> pd.DataFrame:
8686

8787
for cat_col in self._categorical(df):
8888
column = df[cat_col]
89-
column = column.fillna('nan').astype(str)
89+
column = cast(pd.Series, column.fillna('nan').astype(str))
9090
encoder = self.column_encoders[cat_col]
9191
if self.target and cat_col == self.target:
9292
x_i = encoder.transform(column.values.reshape(-1, 1)).reshape(-1, 1)

0 commit comments

Comments
 (0)