Skip to content

Latest commit

 

History

History
5766 lines (2952 loc) · 379 KB

CHANGELOG.md

File metadata and controls

5766 lines (2952 loc) · 379 KB
Raw

CHANGELOG

v9.1.0 (2024-10-08)

Build System

  • build: Fix installation issues in modern Python 3.8+ (88d6390)

Features

  • feat: Implement ReaderQueryHandler for entity and count endpoints (60c5254)

Unknown

  • Merge pull request #488 from ral-facilities/reader-performance-query

Implement Reader Performance Queries for /datafiles and /datasets (7fba647)

  • Add tests for reader performance queries (e361e2b)

  • Add columns to drop list for SQL dump diff

  • The final diff in the CI will still fail because main doesn't contain these dropped columns, but the initial diff to check that two runs produce identical data should pass (b7b2b07)
  • Upgrade dependencies and ignore vulnerabilities
  • As ever, the vulnerabilities that are being ignored are because we cannot upgrade due to being tied to 3.6 (642ea8b)
  • Code cleanup
  • Add docstrings
  • Remove client_pool kwargs
  • Add type hints
  • Fix any outstanding linting issues (a36b607)
  • Improve exception handling on reader account login
  • If there's an issue when logging in (e.g. due to invalid credentials), the user will now return a 500, not a 403 (74185f9)

  • Move import and remove whitespace (c5325b0)

  • Do config check before ReaderQueryHandler creation (dca658f)

  • Ensure reader client only initialised once (7b4d5af)

  • Remove unused imports (11f4a7b)

  • Move reader_client into ReaderQueryHandler (6a5d1e5)

  • Add entity type lookup when checking user access (6eed052)

  • Use clients from the pool for reader queries (b47391a)

  • Ensure API works when reader config isn't present in config file (292cfef)

  • Add repr for WHERE filter to make logging them more readable (350480b)

  • Add class to handle queries which should be done by reader account (c9449f4)

  • Add config section for reader performance query option (720dbf4)

v9.0.3 (2024-08-01)

Build System

  • build(deps): bump python-semantic-release/python-semantic-release

Bumps python-semantic-release/python-semantic-release from 8.0.8 to 8.3.0.

updated-dependencies:

  • dependency-name: python-semantic-release/python-semantic-release dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] [email protected] (55061c3)

Code Style

  • style: Fix whitespace issues #486 (1b4cfc1)

Continuous Integration

  • ci: Add more ignores on Safety session (a72af44)

Fixes

  • fix: Upgrade dependencies and ignore further vulnerabilities (303529d)

  • fix: Update icat_db_generator.py to properly set datafileCreateTime & datafileModTime #486 (23ac563)

Unknown

  • Merge pull request #487 from ral-facilities/#486-fix-datafileCreateTime-datafileModTime

Update icat_db_generator.py to properly set datafileCreateTime & datafileModTime (7ecf5d9)

  • Merge pull request #469 from ral-facilities/safety-ignore

Add more ignores on Safety session (a9f336c)

  • Merge pull request #465 from ral-facilities/dependabot/github_actions/python-semantic-release/python-semantic-release-8.3.0

build(deps): bump python-semantic-release/python-semantic-release from 8.0.8 to 8.3.0 (5ac810a)

v9.0.2 (2024-01-04)

Continuous Integration

  • ci: Ignore Safety vulnerabilities #467
  • As normal, we can't upgrade because all patched versions have dropped support for Python 3.6 (05b3e13)
  • ci: Add job to test that the API can be installed via Pip #467 (8c2258d)

Fixes

  • fix: Pin Flask to 2.0.3 in pyproject.toml #467
  • This will ensure this version is used when the API is installed via Pip (50dd13c)

Unknown

  • Merge pull request #468 from ral-facilities/pip-install-#467

Fix Pip Install Issues (2d7736f)

v9.0.1 (2023-09-07)

Continuous Integration

  • ci: fix Safety CI job
  • As ever, we're having to ignore these vulnerabilities as patched versions don't support Python 3.6 (04c3eaf)

Fixes

  • fix: Make DataPublicationDate.date to be different to DataPublication.publicationDate #444 (312c466)

  • fix: #444 Generate publication dates between two specific dates

  • This should stop relative generation causing different data to be generated on different days (7908c10)

Unknown

  • Merge pull request #447 from ral-facilities/bugfix/data-publication-dates-#444

fix: #444 Generate publication dates between two specific dates (b21002e)

  • Merge branch 'main' into bugfix/data-publication-dates-#444 (9916813)

v9.0.0 (2023-09-01)

Breaking

  • feat!: Remove code and references to ISIS specific table endpoints #432 (12e0304)

Build System

  • build(deps): bump python-semantic-release/python-semantic-release

Bumps python-semantic-release/python-semantic-release from 7.34.4 to 8.0.8.

updated-dependencies:

  • dependency-name: python-semantic-release/python-semantic-release dependency-type: direct:production update-type: version-update:semver-major ...

Signed-off-by: dependabot[bot] [email protected] (7216beb)

  • build(deps): bump actions/checkout from 3.5.2 to 3.6.0

Bumps actions/checkout from 3.5.2 to 3.6.0.

updated-dependencies:

  • dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] [email protected] (806367d)

  • build(deps): bump actions/setup-java from 3.11.0 to 3.12.0

Bumps actions/setup-java from 3.11.0 to 3.12.0.

updated-dependencies:

  • dependency-name: actions/setup-java dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] [email protected] (291abff)

  • build(deps): bump actions/setup-python from 4.6.0 to 4.7.0

Bumps actions/setup-python from 4.6.0 to 4.7.0.

updated-dependencies:

  • dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] [email protected] (5d0f4eb)

  • build(deps): bump codecov/codecov-action from 3.1.3 to 3.1.4

Bumps codecov/codecov-action from 3.1.3 to 3.1.4.

updated-dependencies:

  • dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] [email protected] (52683d6)

  • build: Update PyYAML to fix 3.10 CI tests (82c3495)

  • build(deps): bump python-semantic-release/python-semantic-release

Bumps python-semantic-release/python-semantic-release from 7.33.2 to 7.34.4.

updated-dependencies:

  • dependency-name: python-semantic-release/python-semantic-release dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] [email protected] (c07bf92)

Continuous Integration

  • ci: Ignore new vulernabilties that cannot be fixed (97cb59e)

  • ci: Relax codecov checks

  • These checks match the checks that happen on SciGateway (65f9a39)

  • ci: fix GitHub Actions error (2319b35)

  • ci: remove reference to setuptools issue across CI

  • This is no longer needed as Python ICAT 1.0 is now used, which doesn't require an older version of setuptools (bd0a1ad)

Unknown

  • Merge pull request #448 from ral-facilities/remove-isis-endpoints-#432

feat!: Remove code and references to ISIS specific table endpoints (2825ea2)

  • Merge branch 'main' into remove-isis-endpoints-#432 (e667c78)

  • Merge pull request #445 from ral-facilities/dependabot/github_actions/python-semantic-release/python-semantic-release-8.0.8

build(deps): bump python-semantic-release/python-semantic-release from 7.34.4 to 8.0.8 (60c539c)

  • Merge pull request #446 from ral-facilities/dependabot/github_actions/actions/checkout-3.6.0

build(deps): bump actions/checkout from 3.5.2 to 3.6.0 (4e83e6b)

  • Merge pull request #441 from ral-facilities/dependabot/github_actions/actions/setup-java-3.12.0

build(deps): bump actions/setup-java from 3.11.0 to 3.12.0 (9761870)

  • Merge pull request #436 from ral-facilities/dependabot/github_actions/actions/setup-python-4.7.0

build(deps): bump actions/setup-python from 4.6.0 to 4.7.0 (b4d3c27)

  • Merge pull request #423 from ral-facilities/dependabot/github_actions/codecov/codecov-action-3.1.4

build(deps): bump codecov/codecov-action from 3.1.3 to 3.1.4 (ea38bbc)

  • Containerize application and configure GitHub Actions to build and push Docker image to Harbor (#426)

  • Set python image #354

  • Upgrade pip #354

  • Install Poetry #354

  • Install a different version of setuptools #354

  • Install Gunicorn #354

  • Install the app dependencies #354

  • Serve the app on a Gunicorn server #354

  • Pin Poetry version #354

  • Define Actions job for building and pushing Docker image to Harbor #355

  • Configure job to only run after other jobs succeed #355

  • Configure job to login to Harbor #355

  • Configure job to extract Docker metadata #355

  • Configure job to build image #355

  • Configure job to push image to Harbor on pushes to k8s-deployment branch #355

  • Add job documentation and TODOs #355

  • Configure dependabot to maintain GH Actions dependencies #355

  • Add TODO for branch name of push events #355

  • Update Dockerfile and add entrypoint script

  • Use specific python and alpine versions in the base image

  • Pin python package versions in Dockerfile

  • Use a cache mount to speed up pip and poetry

  • Comment the RUN step

  • Move things out of the datagateway-api-run directory

  • Remove workaround that is no longer needed

  • log_location value should not be quoted

  • Only copy necessary files to build container

  • Improve readability of RUN instructions

  • Use a temp file instead of sed -i in entrypoint script

  • Create a symlink to the installed python module

  • Address TODOs

  • Change default value of ICAT_CHECK_CERT ENV

  • Upgrade and pin actions to commit SHAs

  • ci(docker): bump actions/checkout to 3.5.3 in docker job

  • Update README

Co-authored-by: Alan Kyffin [email protected] (176417b)

  • Merge pull request #449 from ral-facilities/ci-failures

Fix CI failures (c503f9c)

  • Merge pull request #419 from ral-facilities/relax-codecov-checks

Relax codecov checks (34a2662)

  • Merge pull request #434 from ral-facilities/dependabot/github_actions/python-semantic-release/python-semantic-release-7.34.4 (f4bafa6)

  • Merge pull request #427 from ral-facilities/fix-gh-action-errors

Fix GitHub Actions Errors (ea2db01)

  • Ignore flask vulnerability (623a4b3)

  • Ignore werkzeug vulnerability (be37a71)

  • Merge pull request #416 from RKrahl/doc/readme-setuptools-version

Remove outdated paragraph on constraints on the setuptools version (c1c49b9)

  • Remove outdated paragraph on constraints on the setuptools version (c24d980)

  • Merge pull request #415 from ral-facilities/dependabot/github_actions/actions/checkout-3.5.2

Bump actions/checkout from 3.5.0 to 3.5.2 (8338dd0)

  • Bump actions/checkout from 3.5.0 to 3.5.2

Bumps actions/checkout from 3.5.0 to 3.5.2.

updated-dependencies:

  • dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] [email protected] (f2b3e1d)

  • Merge pull request #417 from ral-facilities/dependabot/github_actions/actions/setup-python-4.6.0

Bump actions/setup-python from 4.5.0 to 4.6.0 (d3ee311)

  • Bump actions/setup-python from 4.5.0 to 4.6.0

Bumps actions/setup-python from 4.5.0 to 4.6.0.

updated-dependencies:

  • dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] [email protected] (457cb3c)

  • Merge pull request #418 from ral-facilities/dependabot/github_actions/codecov/codecov-action-3.1.3

Bump codecov/codecov-action from 3.1.1 to 3.1.3 (26415bf)

  • Bump codecov/codecov-action from 3.1.1 to 3.1.3

Bumps codecov/codecov-action from 3.1.1 to 3.1.3.

updated-dependencies:

  • dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] [email protected] (ea6cd29)

v8.0.0 (2023-04-25)

Breaking

  • fix!: add url_prefix param to fix swagger docs at non-root paths #408 (49632ea)

Continuous Integration

  • ci: pin Python Semantic Release action to commit SHA #397 (53a6993)

  • ci: specify distribution for setup-java action #397 (200d842)

  • ci: address warnings by updating and pinning actions to commit SHAs #397 (2a1a05f)

Unknown

  • Merge pull request #409 from ral-facilities/bugfix/improve-url-prefix-handling-408

Fix swagger docs at non-root paths #408 (d557d04)

  • Add url_prefix to paths in OpenAPI definition (0ed0078)

  • Merge branch 'main' into bugfix/improve-url-prefix-handling-408 (d88d295)

  • Merge pull request #413 from ral-facilities/dependabot/github_actions/actions/setup-java-3.11.0 (5258698)

  • Bump actions/setup-java from 3.10.0 to 3.11.0

Bumps actions/setup-java from 3.10.0 to 3.11.0.

updated-dependencies:

  • dependency-name: actions/setup-java dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] [email protected] (db27fff)

  • Merge pull request #412 from ral-facilities/dependabot/github_actions/actions/checkout-3.5.0 (b072c5c)

  • Bump actions/checkout from 3.3.0 to 3.5.0

Bumps actions/checkout from 3.3.0 to 3.5.0.

updated-dependencies:

  • dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] [email protected] (b31d769)

  • Merge pull request #406 from ral-facilities/fix/address-github-actions-warnings-#397

Address GitHub Actions warnings (5807986)

  • configure Dependabot version updates for actions #397 (7c58f2f)

  • Fix linting (d4f2545)

  • Add integration test for Swagger UI (1daa60d)

  • Fix linting errors & update existing unit tests (fb3d05a)

v7.1.0 (2023-03-23)

Build System

  • build: deal with Safety errors #405 (1f528bd)

Features

  • feat: Add icat 5 entity generation to the data generator script (579e321)

Unknown

  • Merge pull request #407 from ral-facilities/feature/icat-5-entity-generation

Feature/icat 5 entity generation (194afcc)

  • Change entity name to camelCase
  • The previous version works for Python ICAT 1.0, but this is how all the other entities are cased and it means the script is compatible with Python ICAT < 1.0 (07a4e23)

v7.0.0 (2023-02-24)

Documentation

  • docs: add search scoring information to README #398 (a4f833e)

Unknown

  • Merge pull request #399 from ral-facilities/398-adapt-search-query-filter-and-scoring-to-v5

Query filter and scoring (a09748c)

  • Merge branch 'main' into 398-adapt-search-query-filter-and-scoring-to-v5 (d47a45d)

v6.3.2 (2023-02-24)

Fixes

  • fix: add triggers for later stages of consistency test (2d7d677)

Unknown

  • Merge pull request #404 from ral-facilities/fix/add-triggers-consistency-test

fix: add triggers for later stages of consistency test (390ec3c)

  • Remove tmate (3447d9c)

  • Use sudo for adding triggers (d8e185e)

  • tmate before dummy data (8288cc4)

  • Fix database triggers script command (0362d81)

  • Change user to be root (65fa5c4)

v6.3.1 (2023-02-23)

Fixes

  • fix: Remove dataset filesize generator (fb466dd)

Refactoring

  • refactor: remove str_conditions param from get_search method #398 (79fe589)

Unknown

  • Merge pull request #403 from ral-facilities/fix/remove-dataset-filesize

Fix/remove dataset filesize (d530145)

  • Fix test differences (5dce490)

  • remove commented line #398 (5b9db5e)

  • Merge branch 'main' into 398-adapt-search-query-filter-and-scoring-to-v5 (52e1343)

v6.3.0 (2023-02-17)

Breaking

  • refactor(api): modify Search API endpoint URL paths #398

The Search endpoint URL paths have been modified to follow the format that the PaNOSC Federated Search API sets out. These changes were made:

  • The Datasets endpoint URL was modified from /datasets to /Datasets
  • The Documents endpoint URL was modified from /documents to /Documents
  • The Instruments endpoint URL was modified from /instruments to /Instruments

BREAKING CHANGE: modify Search API endpoint URL paths (1dc815b)

  • feat(config): add scoring related values to config #398

BREAKING CHANGE: add scoring related values to config (0472d96)

Features

  • feat: Refactor icat generator to use python-icat (ae6ae0b)

  • feat: add support for scoring results #398 (dd58a8b)

Refactoring

  • refactor: move search scoring logic to a separate module #398 (70cd754)

  • refactor: refactor logic for adding scores to results #398 (c328a0d)

  • refactor: refactor handling of errors when getting scores #398 (4f810fd)

  • refactor: refactor logic for creating SearchAPIScoringFilter #398 (505802a)

  • refactor: refactor implementation of SearchAPIScoringFilter #398 (9277dd3)

Testing

  • test: fix tests following the changes to endpoint URLs #398 (5be35cd)

  • test: rename test_helpers.py to test_search_scoring.py #398 (4ac95b6)

  • test: fix failing integration test #398 (042049f)

  • test: add tests for scoring functionality #398 (22325f4)

  • test: fix failing tests #398 (c40414e)

  • test: fix tests #398 (4637d48)

Unknown

  • Merge pull request #401 from ral-facilities/feature/icat-backend-generator#369

Feature/icat backend generator#369 (31aad3e)

  • move and rename sql script, fix path (f14bccb)

  • Consistency test drop mod/createTime (9e4da17)

  • Add function to remove creationDate from responses (cf7d8a2)

  • Fix lock file (b060ffe)

  • Merge branch 'main' into feature/icat-backend-generator#369

Conflicts: poetry.lock (9616219)

  • Fix search-api tests and linting (d28919e)

  • FIx dg-api tests (1ef9eb2)

  • Changes to filters, endpoints and queries (21b0233)

  • Remove multiprocessing (a4fa2e7)

  • Clean up (a0349ba)

  • Cut number of datacollections (71f3b8b)

  • Less data generation (3f5e6be)

  • Fix datafile parameter generator (9940720)

  • Fix datafileparamter range (25e9c29)

  • Multithread large generators (3d9954d)

  • refresh id for long generators (f89fe2a)

  • Optimisation and cleanup of unused code (23ee841)

  • Working generator (d2c697a)

  • Initial refactor (0dc5a03)

  • raise an error if query filter value is not string #398 (83a3222)

  • Merge branch 'main' into 398-adapt-search-query-filter-and-scoring-to-v5 (4ac42b0)

  • Merge pull request #393 from ral-facilities/tests/reorganise-tests#375

Tests/reorganise tests#375 (b08022a)

  • Update for safety (619d3e4)

  • Merge pull request #396 from ral-facilities/tests/python_icatv1

Tests/python icatv1 (8627dbb)

  • Merge branch 'tests/reorganise-tests#375' into tests/python_icatv1 (fe6b137)

  • Add prep for api for integration tests (35780bb)

  • Remove second checkout (c2e3f77)

  • Poetry install needed for generator (5f11ba2)

  • Readd fix for python 3.10 (f9eb154)

  • Remove poetry install and setuptools install (985aa02)

  • Update readme for new test runners. (aa4350c)

  • Update python icat to full release (a0e9527)

  • Fix linting (24c3bd7)

  • remove get_icat_entity_name_as_camel_case (e7b8794)

  • assert correct values for v1 (86b9a11)

  • update to v1 (ebd60b6)

  • Multiple codecov uploads (a6f5d06)

  • Use latest codecov action (b3d0fc5)

  • Ignore sqlalchemy vulnerability (da66fac)

  • Create config earlier and dependency updates (4e275f6)

  • Checkout api earlier (ebcb7a6)

  • Split unit and integration test runs (c0595ba)

  • Add panosc mapping to unit tests (4ebeb77)

  • reorganise new DB tests (7ffc4b8)

  • Fix linting (aa39a81)

  • Initial reorganisation

Tests have been organised into ones that immediately work with no icat stack and ones that don't. (185e304)

  • add scoring related values to config #398

BREAKING CHANGE: add scoring related values to config (b3fffc2)

This allows to search by a keyword. Example:

/search-api/documents?filter={"query":"diffraction"}

This should returns the document that are related to diffraction (1fee761)

  • Revert change of the case of endpoints, implementation of the query filter (933fcf7)

  • panosc aggregator uses endpoints starting by capital letter (30ebd12)

  • Remove commas after mappings in YAML config #398 (ae05f29)

  • Added scoring configuration parameters (2f6a120)

v6.2.0 (2022-11-23)

Unknown

  • Merge pull request #385 from ral-facilities/tests/db-backend-tests-#370

Tests/db backend tests #370 (41c1586)

  • Merge branch 'main' into tests/db-backend-tests-#370 (b9831bd)

v6.1.1 (2022-11-23)

Fixes

  • fix: do not attempt to create related entities if not included #387 (819e7a1)

Testing

  • test: fix failing tests #387 (fcbfb0f)

Unknown

  • Merge pull request #390 from ral-facilities/bugfix/related-entities-creation-#387

Creation of related entities attempted when not specified to be included (1e6c600)

v6.1.0 (2022-11-23)

Continuous Integration

  • ci: Main branch uses yaml config not json (0b6151b)

Documentation

  • docs: Updated postman collection with new entities (c4b850f)

Features

  • feat: Add new entity endpoints for icat5 (dc14f9f)

Unknown

  • Merge pull request #383 from ral-facilities/feature/icat5-endpoints#373

Add icat5 endpoints (3943ec4)

  • address linter issues (6f10cb3)

  • Test all models and fix issues (63c2faa)

  • generate openapi and fix model references (b7fdbe0)

  • Change dates backref as causing KeyErrors (0e75fef)

  • change nulls and backrefs (8e69dd9)

  • Fix linting (30b8907)

  • Merge pull request #391 from ral-facilities/fix/icat-generator-workflow

ci: Main branch uses yaml config not json (849a4aa)

v6.0.0 (2022-11-22)

Unknown

  • Merge pull request #388 from ral-facilities/feature/use-yaml-config#372

Feature/use yaml config#372 (5060d5e)

  • Generator script workflow changes (5a63e70)

  • Requested changes (c8567ef)

  • Ignore vulnerability and change generator workflow (1e08ed9)

  • fix generator script (f224aca)

  • Add exstension change (3e59c86)

  • Merge branch 'main' into feature/use-yaml-config#372 (8921643)

v5.3.0 (2022-11-16)

Breaking

  • feat: Enable support for yaml configuration files for DatagatewayAPI

BREAKING CHANGE: Remove support for json configuration files (46723de)

Documentation

  • docs: Updated postman collection with new entities (4dd4895)

Features

  • feat: Add warning for no api (3c91635)

  • feat: Add support for yaml configuration (02c3e41)

  • feat: Add new entity endpoints for icat5 (dd18e18)

Unknown

  • Merge pull request #386 from ral-facilities/feature/no-api-warning#380

feat: Add warning for no api (962dd80)

  • Fix linting (7dee9f6)

  • Remove print statement (a24d429)

  • Set example to be configured for datagateway tests (c18b669)

  • Fix example log location (fe1271a)

  • Make user_credentials a dict (7854093)

  • No dg api extension (7246e4b)

  • Change test_user_credentials (a38dc89)

  • Fix log location (196dadc)

  • use / as api url (8ee7f27)

  • Test user credentials to yaml (2be2282)

  • update poetry lock file (bc47a65)

  • generate swagger to false (cff0177)

  • Correct example config (b60b39e)

  • use poetry 1.1 for github actions (1d3059b)

  • Fix linter and safety (5d09b14)

  • Change log file creation in actions to use yq instead of jq (94dc710)

  • Debug github actions (7f26f2d)

  • Update github actions and readme (432b940)

  • add yaml configurations to gitignore (2ae0a20)

  • create example yaml config (be34683)

  • Requested changes from review (9cfa727)

  • linting and safety (6549b3e)

  • Add explanation for why test might fail (2eb6e11)

  • Add login test (e3e2005)

  • figure out strange behaviour, make tests more useful (652d1db)

  • Revert changes to invalid update with id (b311637)

  • Fix tests for icat5 (55574cb)

  • Add tests for where filter (3a5a4a3)

  • Fix sessions and linting (6972dcc)

  • test_update_vallid encounters a datetime faillure (0e7fc3d)

  • test_create_data encountering datetime error (b31d037)

  • Implement more tests (9cc8757)

  • Add tests for multiple updates (451e618)

  • Add test for updating data (5dc78c3)

  • address linter issues (c434218)

  • Test all models and fix issues (df446d6)

  • generate openapi and fix model references (92f8682)

  • Change dates backref as causing KeyErrors (99fb986)

  • change nulls and backrefs (0ce5727)

  • Fix linting (49f31b3)

v5.2.0 (2022-11-03)

Documentation

  • docs: Update the poetry installation documentation (68d4267)

Features

  • feat: Add warning that tests only work with ICAT 5 (d8825fd)

  • feat: Changes to tests so they pass with icat 5 (73f3c77)

Unknown

  • Merge pull request #378 from ral-facilities/docs/new-poetry-installer-#376

docs: Update the poetry installation documentation (2c64b7d)

  • make changes from review (1d43086)

  • Merge pull request #381 from ral-facilities/feature/icat5-tests#374

Changes to tests so they pass with icat 5 (9bb898d)

  • pin flake8-black to 0.2.4

0.2.5 causes flake8 to fail due to unexpected keyword argument 'magic_trailing_comma'. 0.3 and greater require python 3.7 (4d6b066)

v5.1.1 (2022-09-22)

Fixes

  • fix: update to latest dependencies (11a0eeb)

Unknown

  • Merge pull request #382 from ral-facilities/dependency/dparse

fix: update dparse to 0.5.2 (71625f8)

  • Minimise dependencies changed (75646db)

v5.1.0 (2022-05-25)

Features

  • feat: support skip and limit string parameters on search-api (db94b37)

  • feat: support skip/limit string parameters on search-api (d601ba7)

  • feat: support skip and limit string parameters on search-api (eb74970)

  • feat: support skip/limit string parameters on search-api and added testing (8c3dde1)

Testing

  • test: check for a different exception for invalid values (ab3839a)

Unknown

  • Merge pull request #364 from antolinos/issue_363

This fixes the problem with limit when it is not a number (076e805)

  • Added test (a87b2e6)

  • This fixes the problem with limit when it is not a number (562e458)

v5.0.1 (2022-05-16)

Build System

  • build: add Python 3.10 to tests nox session (9fa0b60)

  • build: upgrade pycparser version for Python 3.10 support (e07e6df)

Continuous Integration

  • ci: disable Poetry's new installer for Poetry 1.1.x (caf1861)

  • ci: add Python 3.10 to GitHub Actions (9eb4526)

Documentation

  • docs: find and replace new instances of master #252 (9d6f649)

  • docs: update Swagger Interface section in README (e63adf8)

  • docs: update flask run commands in README (e85887e)

  • docs: update README to include note for issue in Python 3.10 (72181f6)

Fixes

  • fix: fix internal server error when running DG API on its own #359 (b0d3e06)

Unknown

  • Merge pull request #360 from ral-facilities/bugfix/500-internal-server-error-#359

500 Internal server error when running DG API on its own and sending Investigations request (ca638a7)

  • Merge pull request #253 from ral-facilities/rename-master-#252

Rename instances of "master" to "main" (c674205)

  • Merge branch 'master' into rename-master-#252 (ad0176e)

  • #252: Find and replace master for default branch change (f2f7794)

  • Merge pull request #353 from ral-facilities/swagger-readme

Update Swagger Interface README (29eabfd)

  • Merge pull request #351 from ral-facilities/ci/python-3-10

Add Python 3.10 to CI Testing (c737a44)

v5.0.0 (2022-03-29)

Code Style

  • style: fix linting as per CI results #350 (78d76c1)

Continuous Integration

  • ci: remove existing mysql install on GitHub Actions (5c1aa98)

Refactoring

  • refactor: change terminology from 'plugin' to 'mechanism' #350
  • This just keeps the terminology in line with what we use in DataGateway API and more generally within ICAT (4a92ac0)

Testing

  • test: add new config options to mocked config #350
  • This should allow the tests to pass now (75560b3)

Unknown

  • Merge pull request #352 from antolinos/Allow-the-configuration-of-authentication-mechanisms-#350

Allow configuration of the authentication for the search api (1a1ee12)

  • Merge branch 'master' into Allow-the-configuration-of-authentication-mechanisms-#350 (8761800)

  • Merge pull request #349 from ral-facilities/ci/remove-mysql-icat-ansible

ci: remove existing mysql install on GitHub Actions (c6aa15e)

v4.3.0 (2022-03-18)

Breaking

  • feat: add configuration option to set authenticator and its credentials in the search API #350

BREAKING CHANGE: This commits adds a mandatory config option, so is a major change. (1c30f2f)

Continuous Integration

  • ci: move back to ICAT Ansible master branch (b1f46cf)

  • ci: change ICAT Ansible host references to use underscores

  • ci: change ICAT Ansible branch (32554f6)

Features

  • feat: allow multiple datetime formats to be used when filtering in search API #338 (f20ad24)

Fixes

  • fix: output datetime data in the same format as SciCat #338 (4596db9)

Testing

  • test: convert datetime formats on search endpoint tests #338 (939fc77)

  • test: fix datetime formatting on search API tests #338 (01968c0)

Unknown

  • Merge pull request #340 from ral-facilities/feature/search-api-datetime-format-#338

Search API Datetime Format (a5cc190)

  • Merge branch 'master' into feature/search-api-datetime-format-#338 (0fd45d0)

  • Merge pull request #348 from ral-facilities/icat-ansible-dashes

ICAT Ansible Dashes to Underscores (464d754)

  • Change ICAT Ansible branch (3c622d5)

v4.2.0 (2022-02-28)

Documentation

  • docs: add openapi yaml files #281 (d4fc795)

  • docs: Add docs to GET Search API CountFilesEndpoint #281 (ba367a0)

  • docs: Add docs to GET Search API FilesEndpoint #281 (53c2f8c)

  • docs: Add docs to GET Search API CountEndpoint #281 (2e34d7a)

  • docs: Add docs to GET Search API EndpointWithID #281 (f211c57)

  • docs: Add docs to GET Search API Endpoint #281 (d407f0a)

Features

  • feat: create openapi endpoint for Search API #281 (412458c)

Unknown

  • Merge pull request #344 from ral-facilities/add-openapi-docs-for-search-api-#281

Add OpenAPI docs for Search API (aa98c51)

  • create and initialise Search API spec #281 (5e249d9)

  • create swaggerui blueprint for Search API #281 (d358d94)

v4.1.5 (2022-02-28)

Fixes

  • fix: fix AttributeError when running DG API on its own #345 (7479cc3)

Unknown

  • Merge pull request #346 from ral-facilities/bugfix/attribute-error-when-running-DG-API-on-its-own-#345

Fix AttributeError when trying to run DG API on its own (9cc9cdb)

v4.1.4 (2022-02-28)

Fixes

  • fix: improve error handling for search API exceptions #319 (935f22d)

Testing

  • test: improve code coverage for search API error formatting #319 (1e42981)

Unknown

  • Merge pull request #343 from ral-facilities/bugfix/validation-error-code-#319

Improve Search API Error Handling (982518a)

v4.1.3 (2022-02-28)

Fixes

  • fix: fix parameters.value WHERE filter with between operator #270 (3fe8dfe)

Testing

  • test: add missing endpoint tests for example implementation queries #270
  • Some tests rely on ICAT 5 so have been skipped until this is released (2c21bab)

Unknown

  • Merge pull request #342 from ral-facilities/example-query-tests-#270

Add Missing Endpoint Tests & Fix parameters.value bug (6d5edf5)

v4.1.2 (2022-02-28)

Build System

  • build: omit certain lines/files from code coverage #313
  • This commit adds a dev dependency which is used to conditionally exclude lines of code from test coverage reporting. There is a particular line of code in models.py which is only used in Python 3.7+, so for 3.6, this line can be excluded from coverage checks (0587a66)
  • build: set Python ICAT as default backend in example config file #318 (6d92917)

Chores

  • chore: move Postman collection to util/ #320 (4e17c6a)

Documentation

  • docs: update README to include resolution to setuptools issue (4bab813)

  • docs: correct markdown URL #320

Co-authored-by: Viktor Bozhinov [email protected] (0a9c05a)

  • docs: add remaining suggested changes to README #320 (d5d358c)

  • docs: make suggested changes #320

Co-authored-by: Viktor Bozhinov [email protected] (d04447d)

  • docs: update docs for search API #320 (2daf395)

Fixes

  • fix: fix requests which include parameters and filter on them #319 (598bf9f)

Refactoring

  • refactor: change LRUCache import #321 (e1fd0a4)

Testing

  • test: use original test data #313 (20c43b6)

  • test: increase test coverage for search API #313 (d62e45a)

  • test: fix nox tests session on Windows #307 (b65cf0e)

  • test: fix nox tests session on Windows #307 (f6d2520)

Unknown

  • Merge pull request #341 from ral-facilities/bugfix/including-parameters-#319

Fix requests which include parameters and filter on them (6741c8b)

  • Merge pull request #339 from ral-facilities/setuptools-docs

Add Setuptools Issue to README (dd8bc51)

  • Merge pull request #333 from ral-facilities/search-api-docs-#320

Search API Documentation (60a599e)

  • Merge pull request #335 from ral-facilities/test-coverage-search-api

Increase Search API Code Coverage (e9deada)

  • Merge branch 'master' into test-coverage-search-api (6a048c0)

  • Merge branch 'bugfix/wrong-calculation-for-ispublic-fields-#329' into test-coverage-search-api (e1ee64c)

  • Merge pull request #337 from ral-facilities/fix-nox-test-session-on-windows-#307

Fix failing Nox tests session on Windows (8b613da)

  • Merge branch 'fix-nox-test-session-on-windows-#307' of github.com:ral-facilities/datagateway-api into fix-nox-test-session-on-windows-#307 (3416127)

  • Merge pull request #336 from ral-facilities/fix-lrucache-deprecation-warning-#321

Fix cachetools.LRUCache DeprecationWarning (c8d7939)

  • Merge pull request #334 from ral-facilities/change-default-dg-api-backend-#318

Change Default DataGateway API Backend to Python ICAT (8d2264d)

  • Merge branch 'change-default-dg-api-backend-#318' of github.com:ral-facilities/datagateway-api into change-default-dg-api-backend-#318 (f15522f)

  • set Python ICAT as default backend in example config file #318 (23dacb3)

v4.1.1 (2022-02-17)

Fixes

  • fix: ignore filters on isPublic fields #329 (d6c10d5)

  • fix: hardcode isPublic value to True #329 (64f62c2)

Refactoring

  • refactor: return empty list when filtering for non-public data #329 (5f65725)

Testing

  • test: test where filter on isPublic fields #329 (c70b9e9)

  • test: fix failing tests #329 (d8725ab)

  • test: fix failing tests #329 (e8beb57)

  • test: remove test_valid_where_filter tests added in bugfix/filters-on-ispublic-panosc-field-#308 #329 (f8c823e)

  • test: remove ICAT mappings for isPublic fields in mappings fixture #329 (df5bfce)

Unknown

  • Merge pull request #330 from ral-facilities/bugfix/wrong-calculation-for-ispublic-fields-#329

Wrong calculation for isPublic field values (41962c7)

  • remove config value added in make-num-years-determining-public-data-configurable-#312 #329 (05ad0e6)

  • remove ICAT mappings for isPublic fields #329 (d7bba35)

  • remove isPublic field changes from bugfix/filters-on-ispublic-panosc-field-#308 #329 (8d0496d)

v4.1.0 (2022-02-16)

Features

  • feat: add search API error formatting as per specification #296 (3a5a3e8)

Testing

  • test: improve tests for @queries_records #296
  • Previously, asserting the status code was unreachable (dd94c1a)
  • test: add tests for search API error signalling #296 (5193af1)

Unknown

  • Merge pull request #331 from ral-facilities/error-signalling

Search API Error Signalling/Formatting (aefafcc)

v4.0.1 (2022-02-11)

Fixes

  • fix: use alternative ICAT mapping for Technique pid when pid is None #314 (bf1c830)

  • fix: use alternative ICAT mapping for Instrument pid when pid is None #314 (ae7e57a)

  • fix: use alternative ICAT mapping for Document pid when doi is None #314 (736c6bd)

  • fix: use alternative ICAT mapping for Dataset pid when doi is None #314 (b813f3d)

Testing

  • test: fix failing tests #314 (35569fb)

  • test: test logic related to setting of Technique pid #314 (393156a)

  • test: update Technique mappings in mappings fixture #314 (7640e8d)

  • test: test logic related to setting of Instrument pid #314 (29583b1)

  • test: update Instrument mappings in mappings fixture #314 (3385282)

  • test: test logic related to setting of Document pid #314 (07d6a8e)

  • test: update Document mappings in mappings fixture #314 (341ee51)

  • test: test logic related to setting of Dataset pid #314 (559e8b8)

  • test: update Dataset mappings in mappings fixture #314 (6206e9c)

Unknown

  • Merge pull request #324 from ral-facilities/bugfix/validation-error-sample-pid-panosc-field-#314

Validation error for Sample pid field when ICAT value is None (48c1e29)

  • add alternative ICAT mapping field to Technique pid PaNOSC field #314 (2a4a54c)

  • add alternative ICAT mapping field to Instrument pid PaNOSC field #314 (62a7a7f)

  • add alternative ICAT mapping field to Document pid PaNOSC field #314 (3801417)

  • add alternative ICAT mapping field to Dataset pid PaNOSC field #314 (faab544)

  • Merge branch 'master' into bugfix/validation-error-sample-pid-panosc-field-#314 (bed6dec)

v4.0.0 (2022-02-10)

Breaking

  • feat(config)!: add configuration option for determining public data #312 (58e777b)

Testing

  • test: fix failing count with isPublic condition tests #312 (20e3cb1)

Unknown

  • Merge pull request #325 from ral-facilities/make-num-years-determining-public-data-configurable-#312

Make Number of Years to Determine Public Data Configurable (9e2fb9a)

  • determine public data based on config value #312 (1021c80)

v3.6.1 (2022-02-07)

Fixes

  • fix: convert isPublic PaNOSC filter to appropriate ICAT filter #308 (6a40307)

  • fix: make WHERE filter without operator work with int and bool #322 (6988a5a)

Testing

  • test: fix failing isPublic field endpoint tests #308 (c40639d)

  • test: test conversion of isPublic PaNOSC filter #308 (dc85ca4)

Unknown

  • Merge pull request #323 from ral-facilities/bugfix/filters-on-ispublic-panosc-field-#308

Cannot Use Filters on isPublic Field (9ee06a4)

  • raise FilterError when invalid operator is used with bool value #308 (184c894)

v3.6.0 (2022-02-07)

Continuous Integration

  • ci: add anon user to ICAT Server rootUserNames #268 (22a0ea8)

Fixes

  • fix: use alternative ICAT mapping for Sample pid when pid is None #314 (7e211f7)

  • fix: make get by pid endpoints return data in PaNOSC format #266 (0de2b5b)

Refactoring

  • refactor: refactor entity relations code #268 (4343a2c)

  • refactor: apply suggestions from PR review #314

Co-authored-by: Matthew Richards [email protected] (e7fa1b1)

  • refactor: move ICAT relations call to filter handler #268 (8840a6c)

  • refactor: make get_with_pid() more efficient by calling get_search() #266 (73b70bd)

Testing

  • test: correct order of actual and expected results in test #314 (5d6dab9)

  • test: test logic related to setting of Sample pid #314 (c390d73)

  • test: update Sample mappings in mappings fixture #314 (c96a34a)

  • test: unskip endpoint tests and inject data #266, #268 (de26303)

  • test: update pid type on endpoint rules #268 (e428328)

  • test: fix tests after fixing bug #268 (f1539db)

Unknown

  • Merge pull request #302 from ral-facilities/feature/search-endpoints-#268

Implement Search API Endpoints (afad6de)

  • add alternative ICAT mapping field to Sample pid PaNOSC field #314 (58b3850)

  • Merge branch 'master' into feature/search-endpoints-#268 (efdc2a6)

v3.5.3 (2022-02-02)

Fixes

  • fix: retrieve non-related fields that have a list of ICAT relations #265 (2c5edc5)

Refactoring

  • refactor: move logic for including ICAT relations of non-related fields of related entities #265 (49cfe08)

  • refactor: move filter_order_handler.py to common #265 (ff637e7)

  • refactor: update PaNOSC Parameter mappings in example file #265 (524f8a9)

Testing

  • test: test logic for including ICAT relations of non-related fields of related entities #265 (8a33bf3)

  • test: move test_filter_order_handler.py to test dir #265 (e3b28e4)

  • test: remove invalid test case #265

A Document cannot have parameters that have dataset (23200cd)

  • test: update Parameter mappings in mappings fixture #265 (cdacc7b)

Unknown

  • Merge pull request #304 from ral-facilities/include-icat-relations-for-non-related-panosc-fields

Include ICAT relations for non-related PaNOSC fields (fad4c3e)

  • Merge branch 'master' into include-icat-relations-for-non-related-panosc-fields (1faba30)

  • add logging #265 (e1f199d)

  • Merge branch 'filter-input-conversion' into include-icat-relations-for-non-related-panosc-fields (2beb861)

v3.5.2 (2022-02-02)

Fixes

  • fix: make ne and neq operators work with non-numeric values #315 (f5e3b4b)

Testing

  • test: fix failing ne and neq operator tests #315 (f74efc0)

Unknown

  • Merge pull request #316 from ral-facilities/bugfix/non-numeric-values-with-neq-operator-#315

Make ne and neq operators work with non-numeric values (a13e699)

v3.5.1 (2022-01-31)

Code Style

  • style: remove unused import #261 (5da48f8)

Fixes

  • fix: fix nested relations bug #261 (67fcbfe)

  • fix: reference self instead of fixed instance #301

Co-authored-by: Viktor Bozhinov [email protected] (40f5662)

Testing

  • test: remove test that's no longer needed #261 (950b9aa)

Unknown

  • Merge pull request #301 from ral-facilities/feature/search-api-include-filter-#261

Implement Include Filter for Search API (10d1ca8)

  • Merge branch 'master' into feature/search-api-include-filter-#261 (e0b2d95)

  • Merge branch 'master' into feature/search-api-include-filter-#261 (1af6e38)

  • Merge branch 'filter-input-conversion' into feature/search-api-include-filter-#261 (80a5ca7)

  • Merge branch 'master' into feature/search-endpoints-#268 (2fe980d)

v3.5.0 (2022-01-31)

Unknown

  • Merge pull request #300 from ral-facilities/feature/icat-to-panosc-data-model-conversion-#265

ICAT to PaNOSC Data Model Conversion (2609aae)

  • Merge branch 'master' into feature/icat-to-panosc-data-model-conversion-#265 (f17038d)

v3.4.0 (2022-01-31)

Features

  • feat: implement regexp operator #297 (bf3fe0e)

  • feat: implement neq operator #297 (9094bbb)

  • feat: implement nin operator #297 (00dbba5)

  • feat: implement inq operator #297 (fc1cf19)

  • feat: implement between operator #297 (4736888)

Refactoring

  • refactor: merge logic for in and inq operators #297 (3211dfb)

  • refactor: merge logic for ne and neq operators #297

Co-authored-by: Matthew Richards [email protected] (8b344ef)

  • refactor: remove duplicated code for between operator validation #297 (bd15023)

  • refactor: update PaNOSC Parameter name mapping in example file #265 (358ac22)

Testing

  • test: test regexp operator #297 (f25333f)

  • test: test neq operator #297 (5817491)

  • test: test nin operator #297 (8d57256)

  • test: test inq operator #297 (1b14214)

  • test: test between operator #297 (f5525c9)

Unknown

  • Merge pull request #303 from ral-facilities/implement-search-api-where-filter-operators-#297

Implement Search API WHERE Filter Operators (4946d8e)

  • Merge branch 'master' into implement-search-api-where-filter-operators-#297 (92918ac)

  • Merge branch 'implement-search-api-where-filter-operators-#297' of github.com:ral-facilities/datagateway-api into implement-search-api-where-filter-operators-#297 (b8ca890)

  • validate values supplied with nin operator #297 (92b5f36)

  • validate values supplied with inq operator #297 (b0134b8)

  • validate values supplied with between operator #297 (d254642)

  • Merge branch 'master' into feature/icat-to-panosc-data-model-conversion-#265 (fca1a59)

v3.3.0 (2022-01-31)

Build System

  • build: update example mappings (f94d361)

Testing

  • test: update mappings on tests #260 (63d7c21)

Unknown

  • Merge pull request #298 from ral-facilities/filter-input-conversion

Start Defining PaNOSC to ICAT Mapping and Implement WHERE Filter (942b3cf)

  • Merge branch 'master' into filter-input-conversion (a34786c)

v3.2.0 (2022-01-31)

Build System

  • build: upgrade python-icat to 0.21.0 #305 (8e472cf)

  • build: use older version of setuptools when running tests (2b66602)

Code Style

  • style: fix typo #265 (3f1b2d6)

  • style: address linting warnings #265 (a253422)

  • style: general linting fixes #268 (d14c327)

  • style: fix linting issues #260 (8267de0)

  • style: make suggested changes from PR review #260 (3afccd2)

  • style: add logging to WHERE filter work #260 (7dd4fd2)

  • style: fix linting issues #260 (bfad399)

  • style: remove unneeded code #260 (cba6482)

  • style: fix linting issues #260 (9ee5395)

  • style: change some variable names to increase code readability #259

  • filter_input meant multiple things in that file so I've gone and given new names in places (d0b949b)
  • style: add docstrings to functions #259 (ed72451)

Continuous Integration

  • ci: use older version of setuptools to solve Python ICAT build issue (e4cfb38)

  • ci: add steps to make mapping file in GitHub Actions #265 (6845659)

Documentation

  • docs: add new comments and fix existing #265 (3f1b1cf)

  • docs: add docstrings for Flask resource classes #268

  • Also updated the existing ones for DataGateway API (a5aee61)
  • docs: add docstring to static function #260 (618f6b9)

Features

  • feat: implement search API endpoints #266, #267, #268 (dcc332e)

  • feat: implement basic version of SearchAPIIncludeFilter #261 (f2f53c9)

  • feat: add function to get PaNOSC to ICAT mapping for where filter #260 (34b1d81)

Fixes

  • fix: fix example mapping file (3802cc9)

  • fix: fix list type field checking in Python 3.6 #265 (691a59e)

  • fix: add logic to deal with PythonICATIncludeFilter that could be related for ICAT relations for non-related PaNOSC fields #268 (29232c6)

  • fix: correct order of arguments for where filter #266 (1e38eae)

  • fix: update __str__() for WHERE filter to cope with applying filter #260 (8d259d7)

Refactoring

  • refactor: update PaNOSC mappings

Co-authored-by: Viktor Bozhinov [email protected] (c13c0db)

  • refactor: refactor from_icat abstract method #265 (71765a5)

  • refactor: refactor _get_icat_field_value logic #265 (1d730d1)

  • refactor: replace fromisoformat with str_to_datetime_object #265 (0bbcc0d)

  • refactor: update PaNOSC Parameter mappings in example file #265 (601c51b)

  • refactor: add argument for required_related_fields on endpoint #268 (68fa3b3)

  • refactor: make changes for new Python ICAT implementation #259 (f61ed00)

  • refactor: implement new Python ICAT changes for read-only queries #260

  • This removes lots of the complexities of the previous way to build complex WHERE clauses (2478ffa)
  • refactor: update PaNOSC mappings

Co-authored-by: Viktor Bozhinov [email protected] (80c066c)

  • refactor: deal with case when list of ICAT field values are found #265 (98f9157)

  • refactor: deal with case when list of ICAT field names are found #265 (956a902)

  • refactor: make field types non-strict #265 (3cbf11b)

  • refactor: address TODO in models.py #265 (b0983bb)

  • refactor: change expected data type of pid #268

  • This matches the PaNOSC data model (71a0852)
  • refactor: change method signatures of search API endpoint helpers #268
  • The endpoint name won't actually be used in the endpoint code, only the entity name will be
  • Also changed get_with_id() to get_with_pid() as all use cases will be with a persistent identifier (b0f4b13)
  • refactor: Pass kwargs into query class #268
  • This allows the COUNT keyword to be specified during a search API endpoint (34fa7d9)

  • refactor: make query params more efficient to create less include filter objects for multiple and nested related entities #261 (eab7f3d)

  • refactor: move get_icat_mapping() into PaNOSCMappings #261

  • I've moved the code which deals with the parameter value field name list away from this function to make the function more generic/reusable. That code has been moved to applying the WHERE filter, where it'll be used. The mapping will not be reached by an include filter (under a valid API request) because the parameter values are for a field name/attribute, not a related entity (00ee21d)

  • refactor: small refactors based on PR review comments #260 (d368778)

  • refactor: move text operator fields into the entity classes #260 (5b71f09)

  • refactor: make mapping file the example file #265

  • Added filename of the 'actual' mapping file to git ignore (b20c9b4)
  • refactor: remove unused function and place functionality within SearchAPIWhereFilter.apply_filter() #260
  • This commit also has fixes for tests relevant to this change (4f16776)
  • refactor: make filters mandatory #260
  • When called from an endpoint, filters will always be passed in, even if it is an empty list (b256eaf)

  • refactor: change SearchAPIQuery attribute name for better readability #260 (5f2c2be)

  • refactor: rename function #260 (33e92e7)

  • refactor: add improved error message for Python ICAT WHERE filter issues #260 (4e05d64)

  • refactor: set get_condition_values to return tuple #259 (229a6c8)

Testing

  • test: tweak query params tests to avoid Parameter entity #259 (2f0afa0)

  • test: remove code for replacing datetime timezone #265 (5571722)

  • test: fix tests following updates to PaNOSC Parameter mappings #265 (d595ceb)

  • test: update Parameter mappings in mappings fixture #265 (1e8badb)

  • test: correct query params on tests #268 (2fed560)

  • test: add additional isPublic test for endpoints #268 (12bdde5)

  • test: add search API endpoints #268

  • Lots of skips have been added for bugs I've found and need to be fixed (bc2aabd)

  • test: fix existing endpoint test data #268 (7ba059c)

  • test: unit test method for getting ICAT relations for non-related fields of PaNOSC relation #265 (516a9e2)

  • test: unit test method for getting ICAT relations of PaNOSC non-related fields #265 (5fccba7)

  • test: unit test method for getting non related field names of PaNOSC entity #265 (b8178a2)

  • test: unit test logic for nested required related fields #265 (e52d595)

  • test: unit test from_icat raises ValidationError #265 (701eb74)

  • test: unit test from_icat Technique entity creation #265 (fb836c7)

  • test: unit test from_icat Sample entity creation #265 (077c279)

  • test: unit test from_icat Person entity creation #265 (c05a70d)

  • test: unit test from_icat Parameter entity creation #265 (faed594)

  • test: unit test from_icat Member entity creation #265 (f2845e3)

  • test: unit test from_icat Instrument entity creation #265 (0d0a1e4)

  • test: unit test from_icat File entity creation #265 (0554bea)

  • test: unit test from_icat Dataset entity creation #265 (8f2a6c1)

  • test: unit test from_icat Dataset entity creation #265 (3631c61)

  • test: unit test from_icat Affiliation entity creation #265 (07058a0)

  • test: define test data #265 (66f0261)

  • test: add basic valid tests for search API endpoints #266, #267, #268 (a17fb9a)

  • test: add tests for SearchAPIIncludeFilter #261 (fc4556c)

  • test: add tests for PaNOSCMappings #260 (985d433)

  • test: move TestDataGatewayAPIQueryFilterFactory into non-backend specific directory #260 (0944d39)

  • test: fix test caused by lack of ICAT 5.0 #260

  • The mapping file contains ICAT field names which will only be present in the upcoming release of ICAT 5.0, therefore the test gets an error from Python ICAT when trying to use them (cf70161)

  • test: change pytest fixture to SearchAPIQuery #260 (9937c36)

  • test: add more PaNOSC hop test cases #260 (40320f2)

  • test: add test cases to cover ICAT parameter values #260 (08605af)

  • test: fix tests for changes made to the class #260

  • Input to init is now expected to be different, so this has been changed to suit (bcd5ec3)

  • test: rename filter test files and fix them for recent changes made #260 (6b82732)

  • test: make further changes to WHERE filter tests #260 (ebb7528)

  • test: add tests to increase test coverage #259

  • test_get_condition_values() didn't resolve the partial branch coverage but I thought I'd commit it regardless because it might help us later on (e014af6)

  • test: add limit and skip inside scope filter cases #259 (3f4eaa6)

  • test: correct text operator tests #259 (072c3f0)

  • test: correct nested include filter tests #259 (3f6bd7d)

  • test: correct false fail test #259 (cd0a86e)

Unknown

  • Merge pull request #292 from ral-facilities/feature/query-params-search-api-#259

Query Parameters Inputs for Search API (d6506ac)

  • Merge branch 'master' into feature/query-params-search-api-#259 (d913af1)

  • Merge branch 'master' into filter-input-conversion (dbe684e)

  • Merge pull request #306 from ral-facilities/upgrade-python-icat-#305

Upgrade Python ICAT to 0.21.0 (c4bd953)

  • Merge pull request #299 from ral-facilities/ci/setuptools-fix

Fix CI for Python ICAT Build Issue (f3e5c20)

  • Merge branch 'filter-input-conversion' of github.com:ral-facilities/datagateway-api into filter-input-conversion (7cc6c5a)

  • Merge branch 'feature/icat-to-panosc-data-model-conversion-#265' of github.com:ral-facilities/datagateway-api into feature/icat-to-panosc-data-model-conversion-#265 (3f463fb)

  • Apply suggestions from code review #265

Co-authored-by: Matthew Richards [email protected] (299cbcd)

  • Merge branch 'include-icat-relations-for-non-related-panosc-fields' into feature/search-endpoints-#268 (003bf40)

  • include all ICAT relations of non-related fields when querying ICAT #265 (27866f9)

  • implement method for getting ICAT relations for non-related fields of PaNOSC related entities #265 (d888053)

  • implement method for getting ICAT relations for non-related fields of PaNOSC entity #265 (a314849)

  • implement method for getting non related field names of PaNOSC entity #265 (4ced97a)

  • Merge branch 'feature/icat-to-panosc-data-model-conversion-#265' into feature/search-endpoints-#268 (4648c5a)

  • Merge branch 'filter-input-conversion' into feature/icat-to-panosc-data-model-conversion-#265 (7f4cd96)

  • Merge branch 'feature/query-params-search-api-#259' into filter-input-conversion (d7e2d81)

  • add logic for dealing with nested required related fields #265 (d606d6b)

  • update Parameter mappings in example file #265 (e691f24)

  • work out if dataset or document is public #265 (eca0a46)

  • disable dataset and document validator in Parameter entity #265 (94dcfba)

  • feature: deal with required related entity cases #265 (3ceb365)

  • set plural entity fields to default to empty list #265 (ce9e2af)

  • set signular related entity fields to default to None #265 (ea29f47)

  • set non-related optional entity fields to default to None #265 (c183a85)

  • Merge branch 'feature/search-api-include-filter-#261' into feature/icat-to-panosc-data-model-conversion-#265 (3d8447e)

  • Merge branch 'feature/search-api-include-filter-#261' into feature/search-endpoints-#268 (e0b54ea)

  • Merge branch 'filter-input-conversion' into feature/search-api-include-filter-#261 (95319a8)

  • conversion proof of concept #265 (85c4bfc)

  • resolve TODOs for PaNOSC to ICAT mapping file #265 (5aa4510)

  • add functionality for applying WHERE filter #260 (694f7f2)

  • fix various edge cases for SearchAPIQueryFilterFactory #260 (e820c31)

  • change text operator fields to entity name format #260

  • This change is prompted because the entity_name will always be in entity name format, rather than field name format (c5f2e74)

  • add init code to deal with self.search_api_query #260 (9d95948)

  • add function to fetch entity name from a related field name #260 (9a9e957)

  • add TODOs for additional operators that need to be implemented #260 (f91a4ef)

  • add code to search endpoints to handle filters #260

  • This code has been added to manually test how the filters are working looking at an entire request from start to finish (82ae018)

  • make nested WHERE filters more supportive of applying their data to Python ICAT #260 (eb96b09)

  • add related entity mappings #265

  • I've added TODOs where there are things I'm still slightly unsure about (29ddbf4)

  • Merge branch 'feature/query-params-search-api-#259' into filter-input-conversion (e1bd6cd)

  • Merge branch 'master' into feature/query-params-search-api-#259 (cafabc5)

  • raise FilterError if scope filter has a limit or skip filter #259 (bc4d3ba)

  • use like operation when creating where filter from text operator #259 (e2850d0)

  • only create a single include filter object per nested include #259 (d2b3826)

  • prefix all where filters inside include filter with entity name #259 (c6d7ab5)

  • Merge branch 'master' into feature/query-params-search-api-#259 (7ae6e77)

  • Merge branch 'master' into filter-input-conversion (8b4c216)

v3.1.1 (2021-12-15)

Chores

  • chore: add pattern to gitignore
  • I've got a few different config files as I'm switching between the old and new config style, so this change keeps them from Git worrying about them (b851f96)

Code Style

  • style: remove irrelevant comment #264 (b43db8a)

  • style: remove comments #264 (46c15cc)

  • style: fix linting issues #259 (b0a4c47)

  • style: make get_query_filter() less complex #259

  • I've moved most of the functionality into separate functions, so flake8 doesn't complain that get_query_filter() is too complex (c09c982)

  • style: add logging to satisfy linter #259 (f7d9829)

  • style: clean up comments #259 (94af185)

Features

  • feat: convert PaNOSC to ICAT for where filter fields #260 (ff9595d)

  • feat: add class to represent nested conditions #259 (583cbf2)

  • feat: add first pass of query param implementation #259 (ee668e3)

Fixes

  • fix: correct reference to class name #264 (fc4c180)

Refactoring

  • refactor: correct models following updates to data model guide #264 (6b18d63)

  • refactor: use alternative query for search API filters #265 (45a972a)

  • refactor: pass in endpoint and entity name into helper functions #265

  • Entity name will be the format used in the mapping json file (d2eed60)

  • refactor: remove to_icat() functions #264 (2753e6e)

  • refactor: reference not-yet constructed models using string #264 (f2f0826)

  • refactor: edit model to follow deliverable document more #264

  • Where the deliverable document hasn't given sufficient detail, we have fallen back on the data models markdown. (84116af)

  • refactor: make use of pydantic in data models #264 (54196da)

  • refactor: remove boolean_operator from SearchAPIWhereFilter #259

  • This is no longer used, boolean operators are managed by NestedWhereFilters (60cc5d1)

  • refactor: clean up SearchAPIQueryFilterFactory #259 (4134a16)

  • refactor: make fetching filters from request more generic #259

  • The function will now work for DataGateway API and Search API variants (600ddf7)
  • refactor: add in SearchAPIQueryFilterFactory #259
  • Added in a generic QueryFilterFactory object which inherits from the search API and DataGateway API versions
  • Also fixed imports of the DataGateway API specific implementation (bbc9412)
  • refactor: make QueryFilterFactory return a list of filters #259
  • This is in preparation to add a SearchAPIQueryFilterFactory where a single query parameter will have multiple filters
  • This commit also fixes the tests which impact this change (04d92f7)

Testing

  • test: add tests for applying WHERE filters #260 (f326e02)

  • test: move filter tests to specific directory #260 (31e9e16)

  • test: refactor more query params tests #259 (605315c)

  • test: refactor tests for where filter in include #259 (d6c62ea)

  • test: remove test cases on things we're not going to support #259 (38d17a5)

  • test: refactor TestSearchAPIQueryFilterFactory #259 (1abaf66)

  • test: fix text operator tests for query params #259 (9e404ed)

  • test: fix some query param tests #259

  • It's a big ol' file this! (2106e99)

  • test: fix tests for NestedWhereFilters #259 (46cb5c3)

  • test: add test cases for filter input with all filters #259 (60c3bad)

  • test: add more test cases for include filter with scope #259 (0f06917)

  • test: add more test cases for where filters with boolean operators #259 (8f90ac5)

  • test: add initial tests for NestedWhereFilters #259

  • I will likely add more tests to this class in the future, particularly to cover a mixture of SearchAPIWhereFilter and NestedWhereFilters (530f024)

  • test: correct unit tests #259 (c39795c)

  • test: fix failing tests and add more text operator test cases #259 (68b85ef)

  • test: add remaining test cases for query parameter inputs #259 (d211278)

  • test: add test for OR operator on WHERE filter #259 (f56465c)

  • test: add boolean operator assertion #259 (7b3d039)

  • test: add some tests for search API query params #259 (9a5e02b)

  • test: fix imports #262 #263 (7310707)

Unknown

  • Merge pull request #289 from ral-facilities/add-panosc-search-api-data-model-#264

Add PaNOSC Search API Data Model (206cea8)

  • Merge remote-tracking branch 'origin/add-panosc-search-api-data-model-#264' into filter-input-conversion (ff745ec)

  • add basic implementation of SearchAPIQuery #265 (8479d23)

  • add ISIS mappings for PaNOSC to ICAT models #265 (a081f4c)

  • add class to load PaNOSC mappings into the code #265 (f9bb6de)

  • add blank mapping file #265 (978a8bd)

  • Merge branch 'add-panosc-search-api-data-model-#264' into filter-input-conversion (9473782)

  • Merge branch 'feature/query-params-search-api-#259' into filter-input-conversion (75f8a91)

  • make LHS and RHS support lists of WHERE filters #259

  • Relevant for request filter inputs that come with 3 or more
  • Test cases have been added to cover this as well as edge cases such as lists with a single filter in them (31c07ab)
  • add functionality to deal with an empty RHS #259
  • Relevant when a filter input has a boolean operator but there's only one condition in that list. Makes the boolean operator redundant but is still an edge case we should support
  • Additional test cases have been added to cover this (ea5deb0)

  • add __repr__ functions for easier testing #259 (df323fe)

  • add string representation function to SearchAPIWhereFilter #259

  • To be used by NestedWhereFilters when LHS or RHS has an input of this type (490470e)
  • This means when the input is of type SearchAPIWhereFilter, it won't execute Python ICAT code until the filter needs to be applied/processed (0c4f9ac)

  • pass in entity name to get_query_filter() to be used by text operator #259 (1d24021)

  • use OR boolean operator for WHERE filters in text operator #259 (865bf97)

  • add support for nested AND/ OR #259 (a590514)

  • move text operator logic to WHERE filter #259 (cdb30ff)

  • Merge branch 'master' into feature/query-params-search-api-#259 (434aeeb)

  • add support for text operator in query params #259 (19ff841)

  • add support for WHERE filters coming from count endpoints #259 (f9d1e57)

  • add boolean operator to WHERE filter creation #259 (6882b9f)

  • Merge branch 'master' into feature/query-params-search-api-#259 (cab9f3f)

  • add fetch filters to search API endpoints #259 (22e8a73)

  • add boolean operator to WHERE filter #259 (12c680f)

  • Merge branch 'feature/search-api-endpoint-definition-#257' into feature/query-params-search-api-#259 (78ec0fc)

  • Merge pull request #295 from ral-facilities/implement-limit-and-skip-filters

Implement Limit and Skip filters (dd23cce)

  • Merge branch 'master' into implement-limit-and-skip-filters (a01072c)

v3.1.0 (2021-12-06)

Code Style

  • style: make changes to satisfy flake8 #258 (a261774)

Features

  • feat: implement session/client handling for search API #258 (46a1539)

Refactoring

  • refactor: move config to its own class #258
  • This change has been made so the config can be mocked in tests (1cf73e4)
  • refactor: allow ICATClient to use search API config where appropriate #258
  • This change will not impact the existing use of the class in DataGateway API (8dba0e1)

  • refactor: rename PythonICATSkipFilter use variable #263 (a474cfa)

  • refactor: allow PythonICATSkipFilter to use Search API config #263 (5ab60de)

Testing

  • test: remove print statement

Co-authored-by: Viktor Bozhinov [email protected] (67a119b)

  • test: fix imports #258 (8508af0)

  • test: use alternate fixture

Co-authored-by: Viktor Bozhinov [email protected] (00c76b9)

  • test: add tests for search API session/client handling #258 (b7dea73)

  • test: move test config fixtures so they can be accessed by search API tests #258 (75bb983)

  • test: change output of endpoint helper function for load testing #258

  • Testing done with Apache JMeter (71cc98a)

  • test: add tests for SearchAPISkipFilter #263 (d8572ed)

  • test: add tests for TestSearchAPILimitFilter #262 (7257672)

  • test: create ICAT related test fixtures #262 #263 (2eb14cf)

Unknown

  • Merge pull request #294 from ral-facilities/feature/session-handling-#258

Search API Session/Client Handling (93905e3)

  • remove unused config items from example file #258 (67c308d)

  • add helper functions for each endpoint type for search API #258 (1049a5f)

v3.0.1 (2021-11-24)

Fixes

  • fix: allow blank extensions and slash extension to be valid (70ddb7a)

Testing

  • test: correct test data

Co-authored-by: Viktor Bozhinov [email protected] (f4339f2)

  • test: add tests for extension validation logic (36b42c0)

Unknown

  • Merge pull request #293 from ral-facilities/bugfix/api-extension-validation

Extension Validation Bugfix (1a10fd4)

v3.0.0 (2021-11-23)

Code Style

  • style: fix formatting and remove TODO #283 (9129288)

Unknown

  • Merge pull request #290 from ral-facilities/respect-config-for-search-api-endpoints-#283

Respect Config for Search API Endpoints (20c46a1)

  • Merge branch 'respect-config-for-search-api-endpoints-#283' of github.com:ral-facilities/datagateway-api into respect-config-for-search-api-endpoints-#283 (c029aae)

  • Merge branch 'master' into respect-config-for-search-api-endpoints-#283 (0edfb2e)

v2.0.0 (2021-11-22)

Breaking

  • docs: adjust versioning documentation

BREAKING CHANGE: Adding breaking change to correct the version bump which didn't happen when merging #285 (44c48e8)

  • feat: configure end part of endpoint urls to contain api extension #283

BREAKING CHANGE: modify endpoint urls to use relevant api extension (5bdd72e)

Testing

  • test: fix failing tests #283 (6aac97b)

  • test: unit test api extension validations #256 (c068bf3)

Unknown

  • Merge branch 'master' into respect-config-for-search-api-endpoints-#283 (a5ce9fa)

  • Merge branch 'master' into respect-config-for-search-api-endpoints-#283 (1ba2b5e)

  • Merge pull request #285 from ral-facilities/add-datagateway-and-panosc-modes-#256

Add DataGateway and PaNOSC Modes (5fd00c4)

  • validate api extensions #256 (8774537)

  • Merge branch 'master' into add-datagateway-and-panosc-modes-#256 (72201a3)

v1.1.0 (2021-11-19)

Code Style

  • style: change import order #257 (896d3b8)

Documentation

  • docs: rebuild openapi docs #257 (de15357)

Features

  • feat: add unimplemented endpoint definitions for search API #257 (d0e52d9)

Testing

  • test: add tests for search API endpoints #257 (bc1d09d)

Unknown

  • Merge pull request #284 from ral-facilities/feature/search-api-endpoint-definition-#257

Define Endpoints for Search API (9e137c6)

  • Merge branch 'expands-search-api-structure' into feature/search-api-endpoint-definition-#257 (39f7c4b)

  • Merge branch 'expands-search-api-structure' into feature/search-api-endpoint-definition-#257 (754136a)

  • Merge branch 'expands-search-api-structure' into feature/search-api-endpoint-definition-#257 (8ea6e2d)

  • add explicit endpoint names to avoid collisions #257

  • Collisions were occurring between /datasets between DataGateway API and the Search API despite the /search_api extension (47d3c47)
  • change endpoint function names to avoid import collisions #257
  • These functions had the same function names as the ones defined for DataGateway API (63ba953)
  • Merge pull request #274 from ral-facilities/expands-search-api-structure

Search API Structure (14dfc2a)

  • Merge branch 'master' into expands-search-api-structure (a2de545)

v1.0.1 (2021-11-15)

Build System

  • build: fix poetry.lock issue #256 (59eca6e)

Code Style

  • style: reorder import statements to satisfy flake8 (c0acd33)

Continuous Integration

  • ci: specify Python 3.9.7 to fix issue found with 3.9.8

Details of error encountered can be seen at: https://www.mail-archive.com/[email protected]/msg1829077.html (f257c3c)

Documentation

  • docs: add study PID to swagger docs #287 (89a9e27)

Fixes

  • fix: add PID field for study in DB backend #287 (18379be)

  • fix: add generation of study.pid #287 (f6a8ebc)

Refactoring

  • refactor: move PID specific Faker instance into relevant generator class #287 (4f282dc)

  • refactor: change imports for new directory structure (4be29a5)

  • refactor: fix typo #256 (615f4d5)

  • refactor: make non-mandatory production config options optional (7d68501)

Testing

  • test: fix failing tests #283 (e69af78)

  • test: fix failing tests #256 (f64a94c)

  • test: have DataGateway API and search API separated tests (4c32759)

Unknown

  • Merge pull request #288 from ral-facilities/bugfix/study-pids-generator-#287

Add Study PIDs to Generator Script (a9a6d13)

  • use new Faker instance to generate study PIDs #287 (cf7c40b)

  • Merge pull request #286 from ral-facilities/ci/use-python-3.9.8

ci: specify Python 3.9.7 to fix issue found with 3.9.8 (254249f)

  • configure end part of endpoint urls to contain api extension #283

BREAKING CHANGE: modify endpoint urls to use relevant api extension (81fa469)

  • create api endpoints based on config objects #283 (cbdaed6)

  • Merge branch 'feature/search-api-endpoint-definition-#257' into respect-config-for-search-api-endpoints-#283 (0b685e8)

  • Merge branch 'expands-search-api-structure' into add-datagateway-and-panosc-modes-#256 (c23b0e5)

  • Merge branch 'expands-search-api-structure' of github.com:ral-facilities/datagateway-api into expands-search-api-structure (dc6dba4)

  • change directory structure of API (bf35193)

  • add configuration option for datagateway and panosc mode #256

BREAKING CHANGE: extend configuration to allow for different API modes (be64faf)

  • Merge branch 'expands-search-api-structure' into add-datagateway-and-panosc-modes-#256 (b57c127)

  • Merge branch 'expands-search-api-structure' of github.com:ral-facilities/datagateway-api into expands-search-api-structure (f51a7b3)

  • Merge branch 'expands-search-api-structure' into add-datagateway-and-panosc-modes-#256 (fb738dd)

  • Merge branch 'expands-search-api-structure' of github.com:ral-facilities/datagateway-api into expands-search-api-structure (01d86fd)

  • Merge branch 'master' into expands-search-api-structure (c87b6cd)

v1.0.0 (2021-11-03)

Breaking

  • docs: Add documentation to explain versioning on this repo #242

BREAKING CHANGE: As the API will be approaching production use soon, this seems like a good opportunity to bump the version to 1.0.0. This also serves as a good test that the introduction of automatic versioning actually works (ccf6d29)

Build System

  • build: Add dependency to help with versioning releases #242 (1afbd71)

  • build: Add config for semantic_release #242

  • Used so the tool can make a release via our CI (ceef3be)

  • build: add pydantic dependency #256 (d7f7bef)

  • build: set min python version to 3.6.1 #256 (fd4e70c)

Continuous Integration

  • ci: use admin personal access token to automate releases (fd76315)

  • ci: Add file to create releases and do automatic versioning #242 (4358f68)

  • ci: Add config file for semantic-pull-requests #242 (7223be1)

Documentation

  • docs: follow Angular commit message capitalisation #242

Co-authored-by: Viktor Bozhinov [email protected] (d53d85a)

Refactoring

  • refactor: #257: Add unimplemented endpoint classes
  • Matching DataGateway API's class structure, with relevant TODOs to add in code when endpoints are defined and implemented (f2f5edc)
  • refactor: use pydantic for configuration handling #256 (0cca696)

Testing

  • test: unit test config logic #256

The test that was creating a temporary file was failing (even without the config changes) due to Windows file permission issue. Though a unit test should run in isolation and should not touch the file system at all therefore I instead mocked the builtins.open to fake with open. (fab9ce7)

  • test: fix failing unit tests #256 (ca9d600)

Unknown

  • Merge pull request #282 from ral-facilities/bugfix/releases-permissions

Fix Automated Releases (12d3fc9)

  • Merge pull request #279 from ral-facilities/feature-versioning-#242

Semantic Versioning (a24b330)

  • Merge branch 'master' into feature-versioning-#242 (fe53174)

  • #243: Upgrade Python ICAT version (b59abba)

  • #243: Add tests for ilike and nilike operators (bb8098f)

  • #243: Add ilike and nilike operators (f594790)

  • #243: Fix tests for changes in Python ICAT 0.20.0 (c4ca364)

  • Upgrade Flask-RESTful

  • Upgrade version of werkzeug via Flask (b0d8b4d)

  • Add flake8 ignore comments to long imports (5431c95)

  • Fix spacing in PR template (bcb84b1)

  • Use Python 3.9 on CI jobs (12a45ba)

  • Fix imports for new directory structure

  • Directory structure due to implementation of search API (bca8d22)

  • Change ignore paths for new directory structure (8caccfd)

  • #256: Revert back to old example config format

  • Reverting back so the tests pass until I re-implement the config (f563380)
  • #256: Change directory structure to suit search API
  • Since this repository will support DataGateway API and the Search API, there needs to be some changes in directory structure to split the different files up. Essentially this means adding datagateway_api/ and search_api/ in common/ and src/resources/
  • The imports will be fixed in a future commit (b57b92d)

  • Merge branch 'master' into expands-search-api-structure (0e35684)

  • Merge pull request #273 from ral-facilities/feature/ilike-operator-#243

Add Insensitive Like Operator to WHERE Filter (822408b)

  • Merge branch 'bugfix/default-python-version' into feature/ilike-operator-#243 (550de36)

  • #243: Upgrade Python ICAT version (39424fa)

  • #243: Fix spacing on PR template (f8a3a9b)

  • Merge branch 'master' into feature/ilike-operator-#243 (f69f136)

  • #243: Add tests for ilike and nilike operators (19c48a5)

  • #243: Add ilike and nilike operators (e352812)

  • #243: Fix tests for changes in Python ICAT 0.20.0 (a6156e3)

  • Merge pull request #275 from ral-facilities/bugfix/default-python-version

Use Python 3.9 on CI jobs (06e28dd)

  • Merge pull request #278 from ral-facilities/upgrade-werkzeug

Upgrade werkzeug for security reasons (9ef4f8e)

  • Upgrade Flask-RESTful

  • Upgrade version of werkzeug via Flask (4ce184f)

  • Add flake8 ignore comments to long imports (b60f532)

  • Merge branch 'bugfix/default-python-version' into expands-search-api-structure (0b7c9fd)

  • Fix spacing in PR template (42c29be)

  • Use Python 3.9 on CI jobs (5da8994)

  • Fix imports for new directory structure

  • Directory structure due to implementation of search API (ea68ce3)

  • Change ignore paths for new directory structure (ef5973e)

  • #256: Revert back to old example config format

  • Reverting back so the tests pass until I re-implement the config (b99cd4a)
  • #256: Change directory structure to suit search API
  • Since this repository will support DataGateway API and the Search API, there needs to be some changes in directory structure to split the different files up. Essentially this means adding datagateway_api/ and search_api/ in common/ and src/resources/
  • The imports will be fixed in a future commit (568af6d)
  • #257: Add unimplemented endpoint classes
  • Matching DataGateway API's class structure, with relevant TODOs to add in code when endpoints are defined and implemented (4dd84e0)
  • #258: Set out file for search API session handling
  • Unimplemented idea to have a singleton which controls a Python ICAT client with an anon user session ID which will be monitored and refreshed as needed, perhaps using a decorator similar to DataGateway API which checks for a valid session ID at the start of processing an incoming request (32a42c9)
  • #260: Add class definitions for search API filters
  • This just inherits from the Python ICAT versions for now as they're unimplemented for the time being (5a9aba9)
  • #264, #265: Add unimplemented search API data models
  • Comments represent mappings to ICAT data model
  • This commit also adds a base class for a search API entity (PaNOSCAttribute) which has two abstract methods, which will be used to convert between data models when implemented (01b2a25)
  • #256: Change example config for search API
  • This format of config has not been implemented at time of commit (858fcbd)
  • Merge pull request #272 from ral-facilities/icat-ansible-ci-#271

Change ICAT Ansible back to master (a426ec7)

  • #271: Update version of Poetry used
  • Hopefully this will fix the 'Link' object has no attribute 'is_absolute' error seen on GitHub Actions when installing the API's dependencies (0b02151)

  • #271: Only run unit tests on matrix version (621df10)

  • #271: Change ICAT Ansible back to master (a7da072)

  • Merge pull request #254 from ral-facilities/ubuntu-20-ci-#239

Use Ubuntu 20 on CI (4167fbf)

  • #239: Correct OS version (0439c0d)

  • #239: Use ICAT Ansible branch compatible with Ubuntu 20 (0d642a9)

  • #239: Change CI to use Ubuntu 20 (f3e77c5)

  • Merge pull request #251 from ral-facilities/increase-test-coverage

Increase Test Coverage (87fbbeb)

  • #250: Add test class for ICATClient
  • Class used to create object pools of Python ICAT's client class (9e0977d)

  • #250: Add test to mock Python ICAT data update failure (01cb60c)

  • #250: Add test to mock invalid backend on query filter getter (f6e8ebd)

  • #250: Add tests for OpenAPI type conversion (b946467)

  • #250: Add test to mock session with < 0 mins remaining (4a3cf7e)

  • #250: Add test to mock TypeError when setting query limits

  • All possible through the use of mocking! (e0cb2e0)
  • Merge pull request #249 from ral-facilities/feature/ping-endpoint-#241

Ping Endpoint (f48b471)

  • #241: Remove print statement (c902750)

  • #241: Add tests for ping endpoint (7e89e29)

  • #241: Add /ping to generated swagger docs (0439247)

  • #241: Implement /ping on ICAT backend (05fdc70)

  • #241: Implement /ping on DB backend (7384a88)

  • #241: Add non-backend specific infrastructure for /ping (1cf4972)

  • #241: Remove init from ICAT backend

  • This is no longer used (was used for client handling previously before the object-pool-cache design was implemented) (0c57614)
  • Merge pull request #247 from ral-facilities/feature/ci-generator-script-compare

Add CI Generator Script Comparison (a393483)

  • #246: Add continue-on-error keyword to generator script CI job (d66512a)

  • #246: Add notes to make potential non-issue failures clear (4c41b71)

  • #246: Remove debugging (4957dd3)

  • #246: Add setup for API's master version (d62e9b7)

  • #246: Add second test to CI generator script job

  • This is to compare data from master and the branch where the CI job was started from (86eeaeb)
  • Merge pull request #245 from ral-facilities/documentation/generator-script-timezones

Add timezone workaround to docs (97d74b6)

  • #244: Add note about inconsistent timezones for generator script (1130f4e)

  • Merge pull request #240 from ral-facilities/bugfix/join-specs-order-#238

Allow One-Many Related Orderings to not miss out blank results (34a8c42)

  • #238: Update README with info about generator script (33f895f)

  • #238: Use default generator args for CI

  • This will line up the CI's data with what I have locally and what is on SciGateway preprod's icatdb
  • This means the failing query test from a commit or two ago will work on CI (724acdf)

  • #238: Update Python ICAT to 0.19.0 (9bb24f3)

  • #238: Add test cases for one-to-many related ordering (1b17a29)

  • #238: Update comment for accuracy

  • 1-many related ordering is now supported as of 0.19.0 of Python ICAT (ad3f52d)
  • #238: Edit expected data on query test
  • Not too sure why this is happening, perhaps the generator script fixes? Would've thought it would've brought this up in the relevant PR though but I'll monitor this via CI on this branch (6cd0d8d)
  • #238: Implement join specs on queries with one-to-many ordering
  • This feature is currently still in PR (therefore unreleased) so I haven't updated the dependency version, I will do so when Python ICAT is released for 0.19.0 (cbef7b9)
  • Merge pull request #237 from ral-facilities/bugfix/icatdb-generator-data-#236

Make Generator Script Consistent (369e38c)

  • #236: Move InvestigationParameter method into its generator class
  • No functionality change, just to make this function styled like all the others (f211b44)

  • #236: Change seeding for Faker (28fd5eb)

  • #236: Update Faker to latest version

  • Version kept constant to help the generator script stay consistent over time (ede12bf)

  • #236: Add skip comments option to satisfy diff (04c033a)

  • #236: Change credentials for mysql commands (80afd12)

  • #236: Choose Python version on new Actions job (61d8b7f)

  • #236: Add job to CI workflow to test generator script (c44506d)

  • #236: Use faker to generate foriegn key data (7671a7d)

  • #236: Use faker to generate non-foreign key integer data

  • This appears to give consistent data, unlike randrange() (327557f)
  • #236: Remove multiprocessing pool implementation
  • Removing this and just using for loops fixes the inconsistent ordering of data, referred to as IDs being juggled in the original issue (556faa0)
  • Merge pull request #235 from ral-facilities/bugfix/nested-include-filter-inputs-#234

Fix complex include filters (d72e473)

  • #234: Fix missing brackets on include filter logic
  • join() only accepts iterables, not multiple strings
  • This commit also adds a test case to cover this (2961b18)

  • #234: Add tests for bugfix (b18da3b)

  • #234: Fix bug with include filters on ICAT backend (58056db)

  • Merge pull request #233 from ral-facilities/feature/distinct-filter-related-entities-#223

Allow related entities on DB distinct filter (205f155)

  • #223: Remove irrelevant docstring (da118af)

  • #223: Remove underscore prefixes

  • _get_field() is the only function in that class that's used internally, so I've remvoed the underscores from the other functions as I'm not sure I've used them correctly (0a202d3)

  • #223: Add test cases for the bug fixed in previous commit (f16cb15)

  • #223: Fix bug with related distinct field with no unrelated fields

  • This fix replaces the distinct flag and as a result, that flag has been removed. Quite happy with this fix, much better than what I achieved on Friday (9021fc8)
  • #223: Remove commented code
  • This was the best solution I could find, particularly as the two init's have different method signatures (c9aa20b)
  • #225: Fix timezone-related issues found by merging master
  • These issues were a result of writing new code that involved timezones before master got merged in, which had the original fixes for timezones on DB backend (dc583a8)

  • Merge branch 'master' into feature/distinct-filter-related-entities-#223 (bfa8c84)

  • Merge pull request #229 from ral-facilities/bugfix/optional-config-options-#228

Make certain config options optional (5901e03)

  • #228: Make certain config options optional
  • Explained in the docstring (302042f)
  • Merge pull request #232 from ral-facilities/refactor/datehandler-db-backend-#225

Add Timezone Data to DB Backend (827454b)

  • #225: Fix tests involving timezone data (9b2dc7d)

  • #225: Add timezone info to session details on ICAT backend (26b7bd6)

  • #225: Add timezone info to datetimes on DB backend (f30caa6)

  • Merge pull request #231 from ral-facilities/update-dependencies-#230

Update dependencies to prevent Poetry warning (f8b1136)

  • #230: Update dependencies to prevent Poetry warning (3ef5f3a)

  • #223: Add tests for distinct filter with related entities (f261f7b)

  • #223: Improve assertion on distinct test case (2623b18)

  • #223: Fix bug where single related distinct field was given

  • This fixes requests such as: /investigations?distinct=["investigationtype.createTime"] (0dfa7a3)

  • #223: Add tests for _get_entity_model_for_filter() (3916a56)

  • #223: Separate out _add_query_join()

  • This function used to have two purposes - the returning of the entity model/field has been moved to another function
  • Separating out these two jobs means the SQLAlchemy warning has been fixed/no longer appears (a2538cc)

  • #223: Add tests for DatabaseFilterUtilities (bc96351)

  • #223: Fix linting issues (4a79940)

  • #223: Move tests for distinct attr mapping (d21bd67)

  • #223: Finish detail on docstring (d3bda6e)

  • #223: Allow data from related entities from distinct filters to be nested correctly

  • This is using the functions moved to common.helpers in the previous commit (dec86b7)
  • #223: Move map distinct attrs to results to common.helpers
  • These two functions will now also be used in the DB backend, so should be moved to a more common location (e67ee26)

  • #223: Allow related entities to be given in ICAT schema form (4bc637a)

  • #223: Implement DatabaseFilterUtilities into DatabaseWhereFilter

  • Similar implemenation as DatabaseDistinctFieldFilter (11f75a2)
  • #223: Allow DatabaseDistinctFilter to recognise related entity inputs
  • This takes existing code from the DatabaseWhereFilter and makes it generic. Future commits will get the WhereFilter to also use this generic version in the same way as the DistinctFilter (1b7f132)
  • Merge pull request #222 from ral-facilities/refactor/config-backend-specific-#210

Refactor config to be aware of the backend in use (0c68549)

  • Merge branch 'master' into refactor/config-backend-specific-#210 (5d83b92)

  • Merge pull request #216 from ral-facilities/bugfix/client-cache-#209

Improve Client Handling (99450bc)

  • Merge branch 'master' into bugfix/client-cache-#209 (a643145)

  • Merge pull request #219 from ral-facilities/bugfix/fix-distinct-filter-#141

Fix Distinct Filter on ICAT Backend (c3c607f)

  • Merge branch 'master' into bugfix/fix-distinct-filter-#141 (68abaff)

  • #141: Update Python ICAT to 0.18.1 (fdd04b0)

  • #141: Change data structures to match behaviour of Python ICAT 0.18.1 (c7789bd)

  • #141: Fix linting issues (891f520)

  • #141: Remove previous implementation for distinct fields (677424d)

  • #141: Fix issue with count test cases (4057126)

  • #141: Fix bug when a count endpoint with distinct filter should get 0 results but caused an IndexError

  • Test case to cover this edge case also added in this commit (44b707a)

  • #141: Add test cases for query exeuction (1b7cb60)

  • #141: Fix GET requests w/ distinct filters (8504fd1)

  • #141: Add flag to mark ISIS endpoints

  • This flag is needed to distinguish ISIS endpoints (that use DISTINCT, but don't select multiple fields) from queries that use a distinct filter (DISTINCT and select multiple fields) (504e94f)

  • #141: Add test cases for distinct attribute mapping (ede08f2)

  • #141: Make distinct requests work with related entities (3401027)

  • #141: Add tests for new functions in query.py (916e3d1)

  • #141: Edit test data to make sense

  • Behaviour of 404s was changed, but the test data wasn't, so it could be confusing why test data saying a 404 should occur then isn't tested that happens (which it shouldn't now) (552f743)

  • #141: Add test case for distinct filter on count endpoint (1d63692)

  • #141: Remove unused functions (0322a99)

  • #141: Extend test cases for ICATQuery init (9a6d7bc)

  • #141: Remove irrelevant ICATQuery test (63c4506)

  • #141: Fix tests for ICAT distinct filter

  • Also adds a test for manual_count (9d14566)
  • #141: Remove checks for related entities on a distinct filter
  • Python ICAT is smart enough to add JOINs to the JPQL queries in the situation where a related entity is used in a distinct filter (1530ca6)

  • #141: Fix manual count flag (f5c3bbe)

  • #141: Change function to fetch distinct attributes

  • I haven't deleted the original function yet, but it's very likely I will at the end of this work as there are no other calls to that function (42dea33)
  • #141: Add manual_count flag
  • Flag used for a count request that has a distinct filter (abb31bd)

  • #141: Add function to map distinct attrs to results (5f11db2)

  • #141: Make use of setAttributes() for distinct filter

  • Also moved the log.debug() to the start of the function. When it was at the end, it would not be logged out if there was an exception, the situation where you want that debug statement (e3169e1)
  • #141: Update python-icat
  • The new version specified contains the changes for setAttributes() (c0f5fde)

  • Add heading to contents (85e52f0)

  • #209: Remove irrelevant TODO (4ed8061)

  • #209: Fix LRU cache test (c6e34e8)

  • #209: Increase test coverage for extended cache (bdc72fb)

  • #209: Add tests for custom LRU cache (cad5677)

  • #209: Remove create client function

  • Client creation is handled by ICATClient (6acfd48)
  • #209: Remove executor
  • This was never used and won't be needed for the solution to client handling (1a1064c)
  • #209: Remove client defensiveness
  • This is no longer needed due to the new solution for client handling (a9600e9)

  • #209: Fix linting issues (7a885d2)

  • Merge branch 'master' into bugfix/client-cache-#209 (904ded8)

  • #209: Add documentation for client handling (400b6b5)

  • #209: Flush session ID before putting client back into pool (e0fff60)

  • #209: Fix API on DB backend

  • The client pool design wasn't working on DB backend, but now it does (b0350c9)
  • #209: Change 'max capacity' to 'max size'
  • This change should just help make things a bit clearer due to 'init size', make the terminology more similar (33bdcd1)

  • #209: Make client handling values configurable (e4abe88)

  • #209: Add comments and move code to a more logical location (416288c)

  • #119: Implement client object pool using LRU cache to hold 'in use' clients

  • LRU cache allows recently used client objects to be kept around. This means there's a 1 client object to 1 session ID ratio, more resource sensitive than a 1 client object for every request ratio
  • Workflow: client pool is created at startup, incoming request fetches client from LRU cache, which pulls a client from the pool. Client is kept in the cache until it becomes least recently used, at which point it's then put back into the pool
  • POST to /sessions uses the same workflow, passing a (pretend) session ID of None to get the same client from the cache each time
  • The first time a new session ID is passed in the request headers, there is no slowdown in the request like before (where a client was being created for that ID). The client is fetched from the pool
  • I've ran this commit against some e2e tests on the frontend and the performance from the API was good, similar to the branch which uses a single client object, passed around using kwargs. I have no concerns regarding a 'slow API' with the pool and cache
  • This is just a rough proof of concept, there's lots of cleaning up to do, including making the resource stats on the pool accurate and not just passing the default ones each time. One potential solution is to make something similar to the Executor class in the pool library. I experiemented with this (think the class I mocked up is in this commit?) but I wanted to get a basic example working before worrying about the stats (which the API doesn't make use of, but it might be useful to keep accurate stats if they ever need to be logged out. All part of the cleanup process
  • This doesn't use the context manager as this wouldn't allow me to implement the LRU cache in the way I have (74f1edd)

  • #209: Add client cache size as a configurable option (fb20095)

  • #209: Make login() use client cache

  • Extra attention should be paid to the flushing of session ID on the client object to previous users being logged out the next time backend.login() is called (3c202a8)

  • #209: Add infomration of how credentials should be structured in request body of POST /sessions (10fceff)

  • #209: Change ICATSessionError exception to output exception message

  • When running the API, it was difficult to distinguish between the AuthenticationError affected in this commit, and the one a couple of lines above it (a81c145)
  • #209: Add client caching function
  • This uses an LRU caching algorithm to determine which client objects should remain cached when the cache is full (d1559a6)

  • #210: Apply suggested change from PR (150f647)

  • #210: Apply suggestions from code review

Co-authored-by: Viktor Bozhinov [email protected] (2ffe5a9)

  • #210: Fix linting issues (d5c536f)

  • #210: Remove client handling config options

  • These aren't in the current branch, they'll need to be added back when the client handling PR gets merged in (01ab8e1)

  • #210: Add leading underscore to instance variables (1cb1003)

  • #210: Update test_config.py to reflect refactor (5f2b33e)

  • #210: Implement config enum class (76bcb7d)

  • #210: Add enum class for config options

  • Similar to what now happens in SciGateway Auth (171dbdf)

  • #210: Add function to check existence of config options (cab681b)

  • #210: Replace old config getters with calls to generic

  • This commit also removes the old getters, as they're no longer being used (9ad2f57)

  • #210: Make all config keys use same casing (c1a02e5)

  • #210: Add generic config getter

  • Following how SciGateway Auth does this, should reduce code in this file (3c40ccb)
  • #210: Remove config calls from being added as a constant
  • No real reason for this to happen, once the JSON is loaded, the values aren't changed until API restart and not modified elsewhere currently
  • It's probably a good thing the request behind ICAT_PROPERTIES actually gets called on demand in case the values get changed before the API is restarted
  • Ultimately, this is the cause of the issue (requiring DB_URL despite using ICAT backend) (79d00e8)
  • Merge pull request #221 from ral-facilities/bugfix/fix-ci-#220

Fix Broken CI (5d99cd3)

  • #220: Force hostname to localhost (ab09e13)

  • Merge pull request #218 from ral-facilities/refactor/increase-test-coverage

Increase Test Coverage (1b7944c)

  • #217: Fix linting issue (e6aee90)

  • #217: Add tests for rollback functionality on update and create endpoints (63b3537)

  • #217: Add test case for invalid backend name (af49cd1)

  • #217: Add tests for custom API exceptions (0aec6da)

  • #217: Add tests for camelCase ICAT entity name function (285362a)

  • #217: Add tests for get_entity_object_from_name() (b9fc454)

  • #217: Update a date attribute on update by ID endpoint (550bd41)

  • #217: Add test case for empty list where filter (7e723be)

  • #217: Add tests for QueryFilter abstract class

  • Similar testing principle to testing the abstract backend class (fb57cee)

  • #217: Correct config test function call (20e4b68)

  • #217: Improve comments (6eedca5)

  • Merge pull request #215 from ral-facilities/bugfix/configurable-reloader-#214

Make Flask Reloader Configurable (7626a92)

  • #214: Add documentation for configurable flask reloader
  • I've mentioned the new solution for client handling which isn't currently in this branch, but when those changes are merged, the documentation will make full sense (1ed467d)

  • #214: Add config option for code reloading (089ace3)

  • Merge pull request #205 from ral-facilities/feature/camel-case-db-backend-#119

Make DB Backend use camelCase (0601fb2)

  • #119: Add comments to related entity logic (a0c966e)

  • Merge branch 'master' into feature/camel-case-db-backend-#119 (ae63c89)

  • #119: Fix tests for change to no results behaviour

  • A couple of commits ago, the ICAT backend was changed so that 404s no longer occur when no results are found. This is because no results being found is correct behaviour, so should give a 200 (268dac3)

  • #119: Make icatdb generator script use camelCase foreign keys (1ee7961)

  • #119: Fix linting issues (47e430d)

  • #119: Fix tests for entities referencing foreign keys (7912f96)

  • #119: Change foreign keys to use camelCase (85b76b6)

  • #119: Remove irrelevant 404s when no data is found (c1fc105)

  • #119: Add NULL to an empty list when using the in operator of a where filter

  • This will allow the ICAT backend to return a 200 when something like the following is sent as a where filter - {"id": {"in": []}} (6662f86)

  • #119: Change attribute names to camelCase (6377ad5)

  • #119: Fix OpenAPI docs for related entities (346731c)

  • #119: Fix entity_helper tests (fc73642)

  • #119: Fix ISIS endpoints on DB backend

  • They became broken due to the change to camelCase within the backend (3393f58)
  • #119: Set response body keys based on field name rather than table name
  • This change also allows related entities to be retrieved on with plural field name
  • Related entities with a singular field name also typically have a foreign key attribute in the class. When an include filter is used in this situation, the included entity will be returned in the response body, rather than the foreign key. This had a better outcome than I thought it would've done :) (ce5e763)
  • #119: Allow relationships to be accessed from camelCase input
  • The addition of singularfieldname and pluralfieldname will be used in the following commit, where these will be used to set JSON response body keys (so they're camelCase, not SNAKE_CASE) (b9e52d9)
  • #119: Allow entity objects to be fetched from plural field names
  • This feature will be needed when there's camelCase field names for related entities, e.g. for user input on include filters
  • This change also moves the location of get_entity_object_from_name() to prevent any issues with circular imports. Since the function no longer makes use of globals(), there's no requirement for the function to be in models.py (f543007)
  • #119: Move endpoints dict to a separate file
  • This change is to prevent a circular import when get_entity_object_from_name is modified
  • Also removes a TODO comment that has previously addressed - it was regarding camelCasing of modId etc. (5b6f055)

  • #119: Ensure rotated log files are ignored by git (170baa4)

  • #119: Fix foreign keys not being added to dictionary conversions

  • The edits to test data return the tests back to how they should be, where they were edited to test what was going on (42739e4)

  • #119: Correct return class in docstring (1f05f6c)

  • #119: Fix OpenAPI YAML to use camelCase attribute names (3c086a6)

  • #119: Add flake8 file ignore due to DB backend camelCase attribute names (edba864)

  • #119: Add missing trailing commas (382de45)

  • #119: Edit entity conversion to work with most camelCase attribute names

  • This doesn't work with foreign key attribute keys yet - e.g. "DATACOLLECTION_ID" will be changed to "datacollectionid" and this doesn't match the variable names in the entities (0d0c5d0)
  • #119: Edit DB backend tests to use camelCase
  • Most DB tests pass now, but a decision needs to be made regarding foreign keys to allow the tests in test_entity_helper.py to pass (279681a)

  • #119: Change attribute name usage to camelCase (58ae900)

  • #119: Edit attribute names to be camelCase (28cfc72)

  • Merge pull request #212 from ral-facilities/bugfix/include-distinct-requests-#211

Bugfix/include distinct requests #211 (671b33d)

  • #211: Increase test coverage for POST /sessions
  • Unrelated to the issue tagged in the commit, but after my previous commit I noticed this would be an easy win for test coverage percentages so quickly modified the tests to get the increased test coverage (02778a2)

  • #211: Update dependencies as identified by safety (622eebb)

  • #211: Add unit tests for distinct attribute mapping

  • This should give a nice increase in test coverage, hopefully codecov agrees! (d3980e2)
  • #211: Fix bug with nested-included distinct attributes
  • Also expands the unit test now those nested input values are possible to occur in the API (8b31f5d)

  • #211: Fix distinct/include bug and add test for it (623a494)

  • Merge pull request #207 from ral-facilities/feature/wsgi

Add WSGI script and fix errors with pip install (04eafd5)

  • Add missing python-dateutil dependency (1572c0c)

  • Merge branch 'master' into feature/wsgi

Conflicts: poetry.lock pyproject.toml (521a1ab)

  • Ignore linter errors in wsgi.py (8f3ffc6)

  • Add trailing comma to appease linter (2b87c76)

  • Add requests as a dependency (668d0e9)

  • Add wsgi.py (8afe25a)

  • Move config.json to datagateway_api directory (6b8e1e9)

  • Don't write to openapi.yaml when generate_swagger set to false (611c227)

  • Merge pull request #204 from ral-facilities/bugfix/improve-db-sessions

Bugfix/improve db sessions (856c31e)

  • Use old session management for icat_db_generator script (9e8bed2)

  • Fix tests by manually pushing app context (582817a)

  • Fix SQLALCHEMY_TRACK_MODIFICATIONS warning (825c2dd)

  • Fix import typo (5f1c2b2)

  • Fix styling issues (9d3ddc6)

  • Move session init code to more sensible places (38dc28e)

  • Merge branch 'master' into bugfix/improve-db-sessions

Conflicts: datagateway_api/src/main.py poetry.lock (5f9c318)

  • Fix session handling for db backend using Flask-SQLAlchemy (f67ad95)

  • Merge pull request #206 from ral-facilities/bugfix/codecov-path-fixing

Fix 404s on Codecov File Views (339174f)

  • #203: Apply path fixing solution to other workflow jobs
  • This commit has no effect on codecov path fixing, but it makes the entire workflow consistent in how the API is cloned onto the Actions runner (f6a4853)
  • #203: Stop API being cloned into a separate directory
  • This should help fix path issues with codecov (1d64040)

  • #203: Test codecov path fixing (f3498a8)

  • Merge pull request #202 from ral-facilities/bugfix/fix-teardown-icat-create-tests-#201

Add teardown for POST entity endpoint tests (460b680)

  • #201: Add teardown for POST entity endpoint tests
  • This replaces the data deletion loop that was after the assert statement, which wouldn't run if the an AssertionError was raised, resulting in undeleted test data (71a65b3)
  • Merge pull request #200 from ral-facilities/feature/remove-sql-dependency-from-backends-#154

Recreate SQL Dependency PR (89b9d19)

  • #154: Fix broken tests (c84eccf)

  • Merge branch 'master' into feature/remove-sql-dependency-from-backends-#154 (deb1539)

  • Merge pull request #175 from ral-facilities/feature/improve-logging-icat-backend-#164

Improve Logging Throughout Python ICAT Backend (d5ea7ba)

  • Merge pull request #171 from ral-facilities/feature/remaining-icat-endpoints-#145

Implement Remaining Standard Endpoints for Python ICAT Backend (e9a9173)

  • #145: Improve the GET /sessions helper function
  • Remove rounding on the expireDateTime value and ensure it always returns a consistent, accurate result
  • Change the keys to camelCase, in line with the rest of the API (d6bff57)
  • #145: Improve datetime conversion
  • The 'accepted_date_format' constant is removed since it's no longer used. Datetime to string conversion will convert to ISO format and string to datetime can be converted using the same format (allowing for easy-tripping of datetimes for API users). (e8c13e9)
  • Merge pull request #172 from ral-facilities/feature/isis-specific-endpoints-icat-#146

Implment ISIS Specific Endpoints for Python ICAT Backend (fc2c48c)

  • #145: Correct mistake made in merge conflict resolution (ed3fe4f)

  • Merge pull request #196 from ral-facilities/feature/implement-ci-#163

Implement GitHub Actions Workflow (fae9695)

  • Merge pull request #197 from ral-facilities/feature/code-coverage-#167

Add Code Coverage Reports (bccce12)

  • #167: Remove codecov as a dependency
  • This is no longer needed, I've opted to use the GitHub Actions codecov plugin (9542fc8)
  • #167: Remove coverage nox session
  • This was not used, with the codecov GitHub Actions plugin used instead (680a5e7)

  • #167: Remove current branch to launch CI build (ce6f2fc)

  • #167: Remove coverage workflow (3c8ba50)

  • #167: Add coverage steps in build and test (b5c22a2)

  • #167: Add arg for noxfile.py (16e434b)

  • #167: Add coverage Actions workflow (55ecf7a)

  • #167: Add status badge for codecov (e43b39e)

  • #167: Add coverage nox session (a938083)

  • #167: Add codecov dependency (dd223c5)

  • #163: Update PR template (d012130)

  • #163: Allow CI to be triggered manually (689c2a1)

  • #163: Disable current branch from running CI when pushed (5281922)

  • #163: Change conditional for when CI runs (d0c3a0a)

  • #162: Disable fail-fast on test jobs

  • It can be useful to let all versions complete to give a better picture of the issues (d5a40a5)

  • #163: Remove unneeded nox sessions from tests job (7117e61)

  • #163: Add separate jobs for each nox session (9a19073)

  • #163: Remove Python 3.5 support

  • Some dependencies require Python >=3.6 (b04abbd)

  • #163: Allow Poetry to use Python 3.5 (5ab6c93)

  • #163: Remove Python 3.4 from CI build (9ebc58c)

  • #163: Remove non-working Python versions (8391a5c)

  • #163: Update setup-python Action (e1341cd)

  • #163: Remove specific python version on non-specific nox sessions (2792e7d)

  • #163: Add more versions of Python to build against (0a50a69)

  • #163: Update pymysql (31e3002)

  • #163: Remove parallel restriction (a7d5193)

  • #163: Update SQLAlchemy

  • See if this fixes the Python 3.8 build (d672986)

  • #163: Add comments to workflow (a3c6963)

  • #163: Remove debugging steps (23e5c77)

  • #163: Change debug to localhost (6079255)

  • #163: Correct sed command (9ac5ec0)

  • #163: Remove blank step (c084d98)

  • #163: Replace default payara user (ec72663)

  • #163: Change inventory file (585afbf)

  • #163: Change icat-ansible to use non-forked version (64ab5ba)

  • #163: Comment out Ansible Actions usage (71e70e8)

  • #163: Debugging for Actions workflow (d4c1163)

  • #163: Update py dependency (9299cf7)

  • #163: Change test_valid_db_url to new DB_URL (83714a1)

  • #163: Edit dummy data cmd (7b43ead)

  • #163: Change dummy data cmd (ed470b3)

  • #163: Add step to use icat_db_generator (22d987e)

  • #163: Change example ICAT_URL to localhost (7d1d0b4)

  • #163: Change debug to localhost (cfa20d8)

  • #163: Correct sed command (5c78a76)

  • #163: Remove blank step (5d246d8)

  • #163: Replace default payara user (05cd039)

  • #163: Change inventory file (71ad07b)

  • #163: Update example config to use localhost values (c2391c2)

  • #163: Specify noxfile (30ec28f)

  • #163: Change icat-ansible to use non-forked version (d6ac7b2)

  • #163: Comment out Ansible Actions usage (4672e03)

  • #163: Lots of debugging (c9d290d)

  • #163: Make path explict (1e7a9ce)

  • #163: Add missing uses line (cdf2b1c)

  • #163: Add steps to build a local instance of ICAT (05b2337)

  • #163: Force each Python version build to run sequentially

  • With using preprod as the ICAT URL, there's issues with the 3 jobs (3 python versions) running tests at the same time meaning that they fail. This is a workaround for the time being, though I think a move to icat-ansible and using a local ICAT instance could prevent this from being a problem because jobs run on different hosted runners, so each job would use a different 'local' ICAT instance (d8b618d)
  • #163: Set log location for exampple config file
  • This will prevent tests related to testing log location from failing (33b7531)

  • #163: Add GH Actions status badge (2500207)

  • #163: Use jq to edit log location (4e4fac1)

  • #163: Actually add the value and file envs (617aff1)

  • #163: Change key to lowercase, to match the actual key

  • Also test if jq exists on the machines, as an alternative to json-edit-action (bd7bbc0)

  • #163: Separate log file location and log_location config change (feaedf4)

  • #163: Add step to create log file for testing (0823e12)

  • #163: Change example ICAT_URL to SciGateway preprod

  • Perhaps a temporary change just to get some automated tests running on GH Actions (8a4fcd0)
  • #163: Add conditionals to nox sessions
  • This is to get all nox sessions to run even if one has failed - nox sessions do not depend on each other (47d1c01)
  • #163: Copy the example config to config.json
  • This commit also adds 'name' tags to everything to make things more understandable (81acc92)

  • #163: Add baseline GitHub Actions workflow (863d1f2)

  • #154: Fix issue with the API when served via WSGI (abae518)

  • #154: Make EntityHelper into an abstract class (f6181d3)

  • Merge pull request #194 from ral-facilities/feature/add-not-like-filter-#193

Add NOT Like Operator to WHERE Filter (a6c35ae)

  • #193: Add NOT like operatior for WHERE filter to DB backend (9455685)

  • #193: Add NOT like operation to WHERE filter on ICAT backend (2d33c0c)

  • Merge pull request #191 from ral-facilities/feature/test-multiple-backends-#150

Add Tests for ICAT Backend (b11001d)

  • Merge pull request #192 from ral-facilities/feature/add-icat-backend-documentation-#190

Add ICAT Backend Documentation (5301cfd)

  • #190: Add --without-hashes options to poetry export cmd
  • This commit also adds the "tests" session to the list of Nox sessions (fb3900b)

  • #190: Add link to icat.manual repo for tutorials on setting up an ICAT instance (73bb72d)

  • #190: Make requested changes from PR linked to this issue (f320600)

  • #165: Remove details regarding tmpdir cmd option

  • This has been removed in a different branch so I've removed the documented details about it in this branch and adjusted it according to the new solution found (135965c)

  • #190: Make change to project tree based on previous commit (93d0980)

  • #190: Rename query_filter_factory.py (60c56f6)

  • #165: Add a note to remind users to have user's Python added to PATH (f2f7f55)

  • #190: Update README.md for ICAT backend (8b24725)

  • #190: Make small changes to existing docs (96f5aec)

  • Merge branch 'feature/test-multiple-backends-#150' of github.com:ral-facilities/datagateway-api into feature/test-multiple-backends-#150 (6dea07e)

  • Update datagateway_api/common/backends.py

Co-authored-by: Viktor Bozhinov [email protected] (baff80d)

  • #150: Make requested changes on PR review (e826e20)

  • Merge branch 'feature/fix-code-linting-#184' into feature/test-multiple-backends-#150 (01449cc)

  • Merge branch 'feature/fix-code-linting-#184' into feature/test-multiple-backends-#150 (91f90a1)

  • Merge branch 'feature/fix-code-linting-#184' into feature/test-multiple-backends-#150 (5f10ecc)

  • Merge branch 'feature/fix-code-linting-#184' into feature/test-multiple-backends-#150 (3b6251c)

  • #150: Add documentation for running the tests (4039233)

  • #150: Add ISIS endpoint testing for DB backend (0c59b58)

  • #150: Actually fix openapi YAML creation

  • This is essentially an undo of 8c1941dfbda5d77092a19a61c2e14a2a300f5786 as I've realised the backend must get parsed into the endpoint classes for testing reasons (c5a867c)
  • #150: Add findone tests for DB backend
  • This commit also includes a fixes a bug found on the 404 of that test class (3eec346)
  • #150: Fix openapi YAML creation
  • I've reverted the setup for the ISIS and session endpoints back to way it was done at the start of this branch, there's no need to parse backend now the test backend is set in the config object, just create it locally on each endpoint file (8c1941d)

  • #150: Add get with filter tests for DB backend (1b91217)

  • #150: Add remaining data creation fixtures for DB backend tests

  • These fixtures haven't been tested, they will when I write the remaining endpoint tests for the DB backend (b2af27d)
  • #150: Create localised conftest files
  • Local to each backend, with a shared file in test/ (42359fc)
  • #150: Ensure filenames are unique across test suite
  • pytest requires that all filenames are unique, even if they're stored in different directories (578f897)
  • #150: Move endpoint rules tests
  • This aren't backend-specific (6baf55d)

  • #150: Add tests for /count for DB backend (bbca31c)

  • #150: Add get by ID tests for DB backend (b6c7c66)

  • #150: Add test to ensure all required abstract methods are present for Backend (f5c4558)

  • #150: Rewrite queries_records tests (97bf1a4)

  • #150: Rewrite get_filter_from_query_string tests (d6f6ab8)

  • #150: Complete TODOs from 'backend can be set for tests' work (74215cd)

  • #150: Rewrite session ID from header tests (e8f1139)

  • #150: Rename valid ICAT creds header

  • This has been renamed due to the addition of a valid credentials header for the DB backend (89d58d6)

  • #150: Rewrite DB session decorator tests (a07f0db)

  • #150: Rewrite JSON validity tests (b97b19b)

  • #150: Rewrite database helper tests

  • The file has been renamed to reflect these tests actually test the QueryFilterFactory (cf89d7e)

  • #150: Rewrite DB entity helper tests in pytest (cff2ce4)

  • #150: Restructure test files

  • Before this commit, I kept the pytest tests I've created in this branch separate from the unittest tests that were there beforehand to keep them isolated. This restructure means that non-backend specific tests are together, and backend specific tests are put in the correctly named directories (b8324db)
  • #150: General cleanup of recent work
  • Sort out linting and remove a few lines of code used for logging etc. (c866be3)
  • #150: Fix FlaskAppTest based on changes to main.py
  • This is a temporary fix before rewriting the original tests in pytest (bcea7ec)

  • #150: Remove unneeded import (dce47c1)

  • #150: Fix logging issues when dealing with session IDs which are strings, not numbers

  • When dealing with session IDs, you cannot assume they are numbers, since they're in the format of UUIDs, so %s is more suitable (c29f68e)

  • #150: Fix imports on test helpers (c676f46)

  • #150: Correct import for QueryFilterFactory (dd9a92a)

  • #150: Fix imports on database helpers (adc48d1)

  • #150: Fix failing endpoint rule tests

  • The tests were looking at the flask app from src.main which is only setup if running the file as per README.md. So a generic flask app fixture has been created and that's used to check endpoint rules (db97d8a)
  • #150: Add separate flask app fixtures for each backend
  • The DB app will be used when I rewrite the old tests into pytest, tests which will remain for testing the DB backend (3890c55)
  • #150: Allow backend type to be set when a flask test fixture runs
  • This will allow tests to run on a backend set by the fixture instead of relying on the contents of config.json. This means both backends can be tested in one test session and the contents of the config file doesn't matter (70dc393)
  • #150: Move QueryFilterFactory to its own file
  • It used to be in the database helpers file, but this class is used across both backends
  • The imports have been moved inside the staticmethod because the Flask TEST_BACKEND config option won't have been set by the time QueryFilterFactory has been imported, so the imports which decide which set of filters to import must be put in some code. (7a6a8ef)
  • #150: Expand flask test app so the backend can be configured
  • This doesn't fully work yet because filters are still created based on contents of config.json (6902921)

  • #150: Checkpoint in making backend configurable for testing (9532d91)

  • #150: Move Flask/API setup into functions

  • This will help to specify whether a unit test should use the db or python_icat backend, to be fully implemented in future commits (64f9981)

  • #150: Remove planning comments made at start of branch (2c37baa)

  • #150: Solve linting issues (e85f378)

  • #150: Fix order filter test failures

  • test_direction_is_uppercase() wasn't being destroyed correctly (90f14a1)

  • #150: Increase coverage of PythonICATIncludeFilter (49ef7f8)

  • #150: Add valid tests for table endpoints (5910404)

  • #150: Fix test failures due to refactor (9ed2694)

  • #150: Refactor creation of investigation test data (7aae8c8)

  • #150: Use one-off session ID for logout test

  • Once this test had completed, any tests afterwards using the icat_client fixture (or valid_credentials_header) would fail because those credentials had been deleted from icatdb (275cb43)

  • #150: Simplify icat_query fixture (25f2282)

  • #150: Add invalid tests for ISIS specific endpoints (1470b22)

  • #150: Add session handling tests for ICAT backend (861b4f8)

  • #150: Add tests to check all endpoints exist and have correct HTTP methods on them (79bdbbe)

  • #150: Move endpoint tests to their own files/classes (ba59926)

  • #150: Add test to create single investigation

  • Also remove assertion for data deletion in these tests - I've changed the name of the data used in these tests so they don't have an impact on later tests, so there's no need to highlight any failures in data deletion (b16098e)

  • #150: Add invalid tests for data creation (e1f07b7)

  • #150: Add valid test for data creation (e358ec7)

  • #150: Fix data creation (ce048da)

  • #150: Add pytest plugin to improve assertion diffs (9edd0e9)

  • #150: Add tests for DELETE endpoints (03c7c8f)

  • #150: Add tests for no result requests

  • This was trying to cover 404s but I'm not sure how I'd get a count query to 404.. (a2fd644)

  • #150: Add more invalid tests for update endpoint (bead8cf)

  • #150: Change ICATValidationError to raise a BadRequestError

  • An exception raised as an alternative to the database going into an invalid state will usually be due to invalid user input
  • This fixes the failing test created in the previous commit (ba62e44)
  • #150: Add invalid update by ID test
  • Currently fails, request gets a 500, not a 400 despite the user input actually being the issue (451e4d7)
  • #150: Modify existing test to actually point to an update by ID endpoint
  • I got the function names mixed up when originally doing that function... (6477491)

  • #150: Add invalid test for data updates (510943e)

  • #150: Add tests for PATCH endpoints

  • Single and multiple update tests (fc7f3a1)

  • #150: Add test for skip & limit filter merge (fdafb7a)

  • #150: Add fixture to inject multiple investigation results

  • Add test to utilise said fixture
  • Add invalid test (9c3ece5)
  • #150: Add update test
  • Added some logging to the API to make logging a bit better (9e0c56b)
  • #150: Add tests for valid GET requests
  • Test to see if all endpoints have been added to the API failed, hence the block comment around it (833f8e9)

  • #150: Add fixture for flask test app (00923bd)

  • #150: Add fixture to return auth header (08b8fb6)

  • #150: Add test to break a query (6de60c7)

  • #150: Add function to remove ICAT meta attributes and new test (367eebb)

  • #150: Correct argument case (df48005)

  • #150: Rename pytest fixture for ICAT data injection (a3c16e1)

  • #150: Fix failing filter tests

  • This adds validation on skip and limit values on the respective filters (ebf8b2e)

  • #150: Add skeleton classes for remaining ICAT backend tests (07e42ec)

  • #150: Add beginning of tests for ICATQuery

  • This commit also includes a skeleton for the remaining tests for this class (d7b88dd)
  • #150: Add way of injecting data into ICAT for testing endpoint and ICATQuery
  • This fixture also removes the data from ICAT at the end of the test (5444bd5)
  • #150: Ensure list flatten always returns the list in the same order
  • That function would return the elements of a list in different order each time (despite identical inputs each time) so this could prove more difficult to test (7bf1141)

  • #150: Add tests for filter handler (e621f42)

  • Merge branch 'master' into feature/test-multiple-backends-#150 (9625cc1)

  • #150: Add test for backend creation (55397e5)

  • #150: Add tests for ICAT limit filters (f4ef19f)

  • #150: Add tests for ICAT skip filters (34d46ba)

  • #150: Add tests for ICAT include filters (7fe62ff)

  • #150: Add tests for ICAT distinct filters (90f3b4b)

  • #150: Add tests for ICAT order filters (3c350ee)

  • #150: Add tests for ICAT where filters (d5b189b)

  • #150: Add pytest fixtures for filter testing

  • Since they're placed in conftest.py, pytest will automatically pick these up, they don't need to be imported into the places I use them in (71720d0)
  • #150: Combine config test username and password
  • This will make it more convenient to use through the tests in the repo (572e89c)

  • #150: Add tests for test configuration options (7455c48)

  • #150: Add configuration options required for repo's tests

  • This commit also adds getters in config.py (57fcd68)
  • #150: Add pytest-cov to the repo
  • nox -s tests -- --cov is a good starting point for usage
  • I'm not sure how much I trust the output - there's around half coverage for both DB & ICAT backend's helper files, even though I've currently got the ICAT backend configured... (257082e)

  • #150: Add tests for config (d1b045f)

  • #150: Make config file path configurable

  • This will make testing the configuration easier since you cannot guarantee what the contents of config.json will be
  • Default is what it was before (230eaca)

  • #150: Add tests for DateHandler (343f8d4)

  • Merge branch 'feature/fix-code-linting-#184' into feature/test-multiple-backends-#150 (b655de1)

  • #150: Add nox 'tests' session

  • This will run the discovered unit tests in multiple versions of Python (unless nox is specified with -p [version_num] which is probably what I'll do for the majority of the time, just useful to test multi-version compatability) (29f8111)
  • #150: Add pytest to dev dependencies
  • This also updates a couple of dependencies as marked by safety, which didn't get merged in from a recent git merge (96b3728)

  • Merge branch 'feature/configure-log-file-location-#182' into feature/test-multiple-backends-#150 (04f25f1)

  • Merge branch 'feature/remove-sql-dependency-from-backends-#154' into feature/test-multiple-backends-#150 (79e6e88)

  • #150: Allow backends to be created for testing purposes

  • Currently getting a 403 on these changes but I'm not sure the code is at fault... (2dee8f0)
  • #150: Work out how the existing tests will work with the new test structure
  • Just a few comments to help me work out what to do with the existing tests for DB backend. These will be moved to work with Pytest once the new test structure has been defined (db43d3d)
  • Merge pull request #185 from ral-facilities/feature/add-code-linting-#165

Add Code Linting and Implementation of Hypermodern Python (ca0efc6)

  • Merge pull request #186 from ral-facilities/feature/fix-code-linting-#184

Apply Fixes suggested by Linting Tools (9b4907c)

  • Merge branch 'feature/add-code-linting-#165' into feature/fix-code-linting-#184 (c4b63eb)

  • Merge branch 'feature/remove-sql-dependency-from-backends-#154' into feature/add-code-linting-#165 (821e8b9)

  • Merge branch 'feature/improve-logging-icat-backend-#164' into feature/remove-sql-dependency-from-backends-#154 (ec1dd64)

  • Merge pull request #177 from ral-facilities/feature/add-postman-collection-#162

Add Postman Collection to Repo (0578120)

  • #162: Make changes requested to collection
  • Make variables work for the collection, not the environment
  • Set default port to 5000, matching the default port on the API (493ac65)

  • Merge branch 'feature/improve-logging-icat-backend-#164' into feature/remove-sql-dependency-from-backends-#154 (adc1f40)

  • Merge branch 'feature/isis-specific-endpoints-icat-#146' into feature/improve-logging-icat-backend-#164 (4acfc92)

  • #146: Correct mistake made in merge conflict resolution (1a16451)

  • Merge branch 'feature/remaining-icat-endpoints-#145' into feature/isis-specific-endpoints-icat-#146 (7532606)

  • Merge branch 'master' into feature/remaining-icat-endpoints-#145 (4d00fb2)

  • Merge pull request #170 from ral-facilities/feature/distinct-filter-included-columns-#148

Update the distinct filter to accept 'included' fields (7b72dc9)

  • #148: Make suggested changes to distinct attribute mapping (84164c9)

  • Merge pull request #159 from ral-facilities/feature/add-issue-templates-#133

Adding issue/PR templates (7fffd18)

  • #133: Add template for which issue a PR will automatically close (de159f8)

  • Adding issue/PR templates (921c9d9)

  • Merge branch 'master' into feature/remaining-icat-endpoints-#145 (69f96bd)

  • Merge pull request #169 from ral-facilities/feature/where-filter-included-columns-#144

Allow WHERE Filter to use Included Fields for Python ICAT Backend (8e3e161)

  • Merge pull request #168 from ral-facilities/feature/icat-include-filter-#143

Implement Include Filter for Python ICAT Backend (f4e3e6c)

  • Merge pull request #156 from ral-facilities/feature/fix-session-handling-#135

Fix session handling for ICAT backend (bf64e9b)

  • Merge branch 'master' into feature/fix-session-handling-#135 (f1fd838)

  • #135: Simplify assignment of client in ICAT backend functions (074d4da)

  • Merge branch 'master' into feature/fix-session-handling-#135 (f4a06ce)

  • #135: Make client objects bound to an endpoint, not a backend

  • This will prevent the same client object being used across different endpoints, which could be a security concern. This means that for each request, the client object is built with the session ID assigned each time, so each request checks that the user has a valid, active session ID (853b0a1)
  • #135: Fix failing unit test
  • Unit test was failing due to null session data being entered to the database (e662c48)
  • #135: Fix session handling
  • This will mean a valid session ID can be used straight away, instead of having to use the login endpoint to create a new session ID and use the API using that. This will help when prod users use session IDs which have come from other auth services (662fc3c)
  • #145: Add attempt to restore old data if an update goes wrong
  • This is a solution to a comment made on the PR for this branch
  • This solution doesn't need to be applied to update by ID because if an error occurs there, there's no other data that could possibly be updated (3224a27)
  • #145: Ensure the database remains in a similar state to what it started in if POST returns an error
  • Similar purpose to the previous commit but expanded so changes are 'rolled back' if an exception occurs on the .create() (b6395e5)
  • #145: Ensure updated data is pushed at the end of the request
  • This will prevent an issue where you're updating multiple pieces of data and the request returns a 400, you don't know which pieces of data have been updated (ed562e8)
  • Merge pull request #174 from ral-facilities/feature/make-swagger-yaml-stay-same-on-startup

Make swagger YAML stay the same on startup (58f995b)

  • #164: Move logging for consistency (d0104eb)

  • Merge pull request #176 from ral-facilities/test-existing-bug-issues

Make Distinct Field Filters Apply Correctly on ISIS Endpoints (eadd9fc)

  • Merge branch 'feature/add-code-linting-#165' into feature/fix-code-linting-#184 (36dfeb0)

  • #165: Disable auto file deletion for NamedTemporaryFile

  • This fixes a PermissionError that was found when using these Nox sessions on Windows
  • This is a replacement solution for the tmp_dir option/fix, so that's now been removed (ed448ca)

  • Merge branch 'feature/add-code-linting-#165' into feature/fix-code-linting-#184 (1013e82)

  • #165: Allow tmp directory to be configured for nox sessions (0366d08)

  • Merge branch 'feature/add-code-linting-#165' of github.com:ral-facilities/datagateway-api into feature/add-code-linting-#165 (3682165)

  • Update README.md

Co-authored-by: Viktor Bozhinov [email protected] (a4ca2df)

  • #165: Update repo tree structure
  • Also added a command to generate this tree in the future (I just used to update it manually). (b80347f)
  • #184: Make response descriptions have consistent syntax
  • The commit also includes a rebuilt openapi.yaml (fe88196)
  • #184: Fix linting issues as brought up in PR
  • Most likely caused when I merged other branches after I created the PR (b01eaf5)

  • Merge branch 'feature/add-code-linting-#165' into feature/fix-code-linting-#184 (116ca2a)

  • Merge branch 'feature/remove-sql-dependency-from-backends-#154' into feature/add-code-linting-#165 (9e34729)

  • Merge pull request #178 from ral-facilities/feature/fix-swagger-data-parsing-#166

Update Swagger/OpenAPI Docs for New Backend (864e020)

  • #166: Add modified OpenAPI YAML from recent modifications (588158c)

  • #166: Add examples to match all possible WHERE filter operations

  • Also modified description to aid users if they're having issues getting example values to work (2724911)
  • #166: Add not equal to DB where filter
  • This will match the functionality seen in the Python ICAT version of the WHERE filter, so I can put an example of this operation in the Swagger docs (6666f29)

  • #166: Adapt example filter inputs for ICAT backend (7d0205c)

  • #166: Fix Swagger data parsing for entity by ID endpoints (fe9623b)

  • Merge pull request #180 from ral-facilities/feature/fix-exception-handling-#147

Fix Exception Handling for Non-Debug Mode (b12e7cd)

  • #147: Override handle_error on extended Api class (511a8ef)

  • #147: Misc. code formatting changes (80943ad)

  • Merge branch 'feature/remove-sql-dependency-from-backends-#154' into feature/add-code-linting-#165 (c977be0)

  • Merge pull request #179 from ral-facilities/feature/update-swagger-genaration-in-readme-#134

Update Swagger Generation Section in README.md (5125106)

  • #134: Misc. changes of existing sections in README
  • Also updated requirements.in (with a freshly compiled requirements.txt) to reflect the requirement of Python ICAT (da21adb)

  • #134: Edit Swagger doc section of README to reflect changes in generation (c2e8099)

  • #134: Make file adhere to Black's 88 character/line rule

  • Since the Python files in the repo adhere to this limit, it seems sensible that the README file also following this formatting rule where possible (efb732c)
  • Merge pull request #187 from ral-facilities/feature/disable-openapi-yaml-write-#183

Disable openapi.yaml Generation (2caae79)

  • Merge pull request #188 from ral-facilities/feature/configure-log-file-location-#182

Allow Log File Location to be Configurable (463e3db)

  • #182: Allow log file to be configured
  • In production, this will be used to store the API's logs in /var/log/
  • Also remove an unused variable that I noticed (45b5896)

  • #182: Add log location configuration option (7723d69)

  • #183: Allow openapi.yaml generation to be disabled

  • This will mean this configurable parameter can be disabled when running the API in production, thereby avoiding any issues with read-only directories (1596f73)

  • #184: Fix previous changes to log.exception() lines (b752c85)

  • #184: Update dependencies with security issues found by safety (82aa40d)

  • #184: Make certain function less complex

  • I've had to increase the max complexity in .flake8 because there's not much you can do with a couple of the functions. Still made some improvements though! (a694129)
  • #184: Fix G200 from flake8-logging-format
  • This linting status code is regarding passing an exception object directly into log.exception() (af816bf)

  • #184: Ensure all test classes names use PascalCase (de6255d)

  • #184: Fix misc. linting issues (ed8aa34)

  • #184: Add file specific ignore for 'random' usage

  • S311 (from Bandit) states "Standard pseudo-random generators are not suitable for security/cryptographic purposes.". The generator script doesn't generate data that would be vulnerable to security (no keys or anything of that nature is created using the random library) hence this status code is ignored (33193de)

  • #184: Rebuild openapi.yaml file to reflect recent changes (93e1e3b)

  • #184: Edit docs to reflect Nox session renaming (4846a3d)

  • #184: Ensure every line meets 88 character length

  • This is the suggested line length defined by Black, which flake8 is configured to also follow (81671f4)

  • #184: Avoid catching bare exceptions (9b85ea7)

  • #184: Remove reassignment of Python builtins (51fabee)

  • #184: Reorder import statements (ffe23fe)

  • #184: Fix local import names

  • This is so flake8 can correctly detect which import statements are bringing in local code, so said statements can be ordered in a consistent style (3b208a0)

  • #184: Add trailing commas to meet flake8 output (a9f3b08)

  • #165: Change README's project structure (0f232dd)

  • #165: Add dev environment creation summary (e32543b)

  • #165: Make small formatting changes to README (b4aa172)

  • #165: Add documentation to the new dev environment created by the changes on this branch (4077f10)

  • #165: Remove unused requirements files

  • These files have been replaced by the use of Poetry to store the API's dependencies (be6e229)

  • #165: Add flake8 plugins to Poetry and nox session (b28446c)

  • #165: Update version of standard pre commit hooks used (21dcee2)

  • #165: Add additional pre-commit hooks (14276a4)

  • #165: Apply pre-commit hooks to all files

  • Done by pre-commit run --all-files (7e7154c)

  • #165: Add pre-commit hook config (348acf1)

  • #165: Allow dependencies to be installed according to Poetry's dependencies

  • This will help keep consistent versions of flake8 etc being installed between developers, to prevent incosistent linting/safety outputs (b890d98)

  • #165: Add flake8 config changes needed from additional plugins used (a478ab1)

  • #165: Add .python-version to gitignore (90badd6)

  • #165: Add Poetry configuration (baa1768)

  • #165: Edit target for config file due to previous commit (5761dc6)

  • #165: Make this repo pip installable

  • As suggested by Alan, a top level directory is required to make this repo ready for production use. This is also required for Poetry (9796ddf)

  • #165: Install Black before running the command (d4911c5)

  • #165: Add skeleton linting functionality

  • As per Hypermodern Python guide (fb3f320)

  • #154: Add rebuilt openapi.yaml due to changes made for this issue (66d0ab1)

  • #154: Convert get_python_icat_entity_name() to a camel case only returning function

  • Due to removing the SQL dependency, there is no use for the primary aim of this function. As a result, it's now only used once in this repo. It has been renamed (and modified) since the function only returns a camelCase version of its input (08ee87d)
  • #154: Adapt Swagger docs for entity_type
  • This commit re-enables the Swagger docs and adapts them for the non-backend specific contents of entity_type
  • This also fixes some issues where example values wouldn't show in PATCH and POST endpoints (ac095e1)

  • #154: Misc. code formatting changes made by Black (c7cc4c1)

  • #154: Make ICAT backend entity_type non-backend specific

  • Same aim as the previous commit, but for the Python ICAT backend this time (bc17502)
  • #154: Make DB backend entity_type non-backend specific
  • Instead of parsing an instance of something like common.database.models.USER, the field name is parsed into the DB backend function instead, and an instance of that entity is created in the DB backend functions, to make that data parsing non-backend specific (06522be)
  • #154: Disable most of the Swagger generation for dev purposes
  • This will be modified later on (f0ff0a1)
  • #154: Change endpoints dict to contain field name
  • This is a change from containng an instance of the entity model, to make entities parsed round the API non-backend specific (669a558)

  • #154: Add function to get instance of DB model entity from a name (17f9eec)

  • #154: Move session_manager to common.database

  • This is code for the database backend, so it seems fitting to move it there
  • This commit also includes the changes in imports and updates to documentation (c2e7497)
  • #154: Move common.models to common.database
  • The only model in common.models was for the database backend (there's no model for the Python ICAT backend), hence the change
  • This commit also includes the changes required to correctly import the file from the new location
  • Updated the file tree in the README (56b7606)
  • #162: Add Postman Collection compiled for ICAT backend
  • This is a Postman Collection I've built up over the past few months, containing 320 requests. The requests are aimed at the Python ICAT backend (with query params created for that backend) but can be used for the DB backend by changing parameters to SNAKE_CASE where needed. The collection has been exported as v2.1 and is ready to be imported into Postman (cf83b3f)

  • #114: Remove logging used for development purposes (7aa794e)

  • #114: Remove isis_endpoint attribute from ICATQuery

  • Due to the previous commit there is now no need for this parameter (c9e80c0)

  • #114: Allow distinct filters to correct apply to ISIS endpoints (5e19893)

  • #164: Add logging throughout Python ICAT backend (4aa1331)

  • #173: Add OpenAPI YAML file ordered alphabetically (537cafc)

  • #173: Order OpenAPI YAML request types alphabetically

  • Things like get, patch, post, delete etc. weren't alphabetically ordered as changing the order of the YAML file on startup (88c623a)

  • #146: Add to ICATQuery docstring (adcb7a6)

  • #146: Add logging for ISIS endpoints (1652048)

  • #146: Add DISTINCT aggregate to all ISIS specific queries

  • Testing on dev ISIS ICAT determined this was needed (b52634a)

  • #146: Implement second ISIS specific endpoint (e28c39b)

  • #146: Apply count query flag logic to second ISIS endpoint

  • It should be noted that the affected endpoint doesn't currently do anything (8d1dba2)
  • #146: Add count functionality for 1st ISIS endpoint
  • Makes use of the existing function but adds a flag to detect whether the function should be used as a count query or not (68bd282)

  • #146: Fix missing logging import (5944932)

  • #146: Correct facility cycle ISIS endpoint

  • This ensures the correct DB tables are joined together, to match the output of the request in the DB backend. This has been compared with a query from TopCat and also gives an identical output for my test data (be35d4b)
  • #146: Fix issue with RHS of where filters being a reference
  • This fixed is used for the ISIS endpoints (397602d)
  • #146: Add flag for ISIS endpoints in ICATQuery
  • ISIS endpoints require use of the DISTINCT aggregate, which is different to the DISTINCT filter in this API (10679bb)
  • #146: Add skeleton to an ISIS endpoint
  • This endpoint does not work currently, there are issues with speech marks in WHERE filter of the startDate conditions and a DISTINCT aggregate needs to be added (with a flag in ICATQuery to mark an ISIS endpoint so the aggregate doesn't behave like a distinct filter - these are two different things) (b6333e9)

  • Merge branch 'feature/remaining-icat-endpoints-#145' into feature/isis-specific-endpoints-icat-#146 (eb2382d)

  • Merge branch 'feature/distinct-filter-included-columns-#148' into feature/remaining-icat-endpoints-#145 (3702123)

  • #148: Fix ImportError (88908f4)

  • Merge branch 'feature/distinct-filter-included-columns-#148' into feature/remaining-icat-endpoints-#145 (c1e5480)

  • #146: Change where include filter data is referenced (a089db4)

  • #146: Add docstrings to each of the ISIS endpoint helper functions (2c8842c)

  • #146: Create skeleton functions for ISIS endpoints for ICAT backend (1ee4f0e)

  • #146: Fix ISIS endpoints for DB backends

  • The function calls were to functions that didn't exist in the backend files, so I've corrected them so they do exist and also slightly renamed them during this change (90a2e28)

  • Merge branch 'feature/distinct-filter-included-columns-#148' into feature/remaining-icat-endpoints-#145 (6de7c73)

  • Merge branch 'feature/icat-include-filter-#143' into feature/distinct-filter-included-columns-#148 (8b6cfad)

  • Merge branch 'master' into feature/icat-include-filter-#143 (a023678)

  • Merge pull request #161 from ral-facilities/feature/icat-distinct-filter-#141

Implement Distinct Filter for Python ICAT Backend (e419d8c)

  • #141: Set LIKE operator on WHERE filter to do wildcard searches
  • This should allow these types of searches to provide more accurate results, where previously none could be found (acc9672)

  • Merge branch 'feature/icat-limit-skip-filters-#139' into feature/icat-distinct-filter-#141 (e58cbae)

  • #141: Rename ICATQuery and move it to a separate file (4f84e52)

  • Merge pull request #153 from ral-facilities/feature/python-icat-where-filter-#142

Change structure of Filters and Implement Basic WHERE Filter for Python ICAT backend (767aef1)

  • Merge pull request #160 from ral-facilities/feature/icat-limit-skip-filters-#139

Implement Skip/Limit Filters for Python ICAT Backend (04db3d5)

  • #139: Send a request for ICAT properties at start-up only (7256627)

  • Merge pull request #158 from ral-facilities/feature/icat-order-filter-#140

Implement Order Filter for Python ICAT Backend (a014b11)

  • Merge pull request #151 from ral-facilities/feature/python-icat-entity-id-endpoints-#136

Entity by ID Methods (0600a0a)

  • Merge branch 'master' into feature/python-icat-entity-id-endpoints-#136 (6c1dd4c)

  • Update LICENSE (412cf1b)

  • Merge branch 'feature/python-icat-get-with-filters-#137' into feature/python-icat-entity-id-endpoints-#136 (cc75ec0)

  • #145: Remove logging statement from development work (8e046f6)

  • #145: Move filter-specific functions to FilterOrderHandler

  • This will help clean up common.icat.helpers and move filter-specific functions to the relevant file
  • This commit also adds a function that calls other functions in its class to reduce code duplication
  • This is a 'leave the code in a better state than you found it' kind of commit (8c8a241)
  • #145: Remove return_first_value_only flag
  • This functionality has been replaced with a limit filter as per the previous commit (2d2f4f1)
  • #145: Add LIMIT filter into /findone endpoints
  • This helps to make these types of requests more snappy, without impacting the output (264c130)

  • #145: Make small changes to date handler (b8534e4)

  • #145: Add logging (1f866f1)

  • #145: Move date-related functions to a separate class

  • In the future, these could be used by other backends so I've moved these 3 functions into their own utility class. This change helps clean up common.icat.helpers a bit which is another reason this change has been made (2a88aae)

  • #145: Add exception to be caught when creating ICAT data (d7741f7)

  • #145: Remove unused code

  • Recursing through related objects is no longer required (be3c797)
  • #145: Deal with attributes that have MANY relationships
  • This code needs to be recursed over, since multiple 'levels' of mandatory MANY relationships don't happen currently and this results in a database related error (85860ff)
  • #145: Remove parameter from str_to_datetime_object()
  • I've also made use of this in create_entities() which made me notice that said parameter wasn't used (5eed66e)
  • #145: Allow Python ICAT names to be fetched as camelCase
  • This is required for created ICAT objects in Python ICAT (5cb5068)
  • #145: Add first implementation of POST request (create) for entities
  • This has only been tested on /investigations so far, so further changes may be needed when I test on the rest of the entities (944b877)

  • #145: Add docstring for ICAT PATCH helper function (b33de8a)

  • #145: Prevent related entities from displaying in response of a PATCH request

  • Related entities (i.e. includes="1") must be retrieved when updating data to avoid an IcatException which highlights that a related entity field is trying to be set to null which is invalid (2970dde)
  • #145: Add basic implementation of updating multiple entities in one request
  • This works some of the time, but not all as of this commit (4f549bc)

  • #145: Fix bug where GET requests without distinct filter sometimes return a 400 (d3a0f0b)

  • #145: Remove to_dict() conversion on PATCH entity endpoints

  • Python ICAT backend doesn't need this conversion, so this has been moved into the relevant DB backend helper function (a4baba3)

  • #145: Implement /findone for all entities (0747c5b)

  • #145: Fix bug on non-count queries (692ac0f)

  • #145: Allow /count to work with distinct filters (86c6f27)

  • #145: Basic implementation of /count

  • Currently breaks when you add a distinct filter so that needs fixing (5fd2622)

  • #148: Fix bug where no distinct filter is present (f6df659)

  • #148: Allow distinct fields to work correctly with included columns of >1 depth (0760556)

  • #148: Move checking WHERE filters for distinctiveness to its own function

  • This commit also moves the separation and flattening of the included field sets into its own function
  • Various bits of documentation adding too
  • Replace icat_query.attribute_names with a non instance variable (20a886d)
  • #148: Check that included-distinct fields have included entities in the include filter
  • The appended docstring in this commit explains the change far better than the commit message.. (571f720)

  • #148: Add docstrings and general comments (2a300d5)

  • #148: Move duplicated code to its own function

  • This involves moving the distinct fields belonging to the entity that's going to be sent into entity_to_dict() (via recursion) to be moved to the base so it can be checked when that recursion call executes (f4b89db)

  • #148: Allow included fields to be specified as distinct fields (4470944)

  • #148: Map distinct fields to the entity they belong to

  • These fields can be picked out so they are the only ones returned in a query result
  • This is put into a separate function unlike before, where similar functionality was in entity_to_dict() (96814f1)
  • #148: Map distinct entities to attribute names
  • This data will be used later on in the function to filter the distinct fields from the rest of the data (e55fbeb)

  • #144: Add documentation and style changes (9a1b060)

  • #144: Move extraction of filter fields to specific backend

  • This makes the WHERE filter interact well with included fields for the Python ICAT backend - e.g. {'investigationUsers.id': "= '6'"} (d3b08e4)

  • #143: Remove unused import (6fa0d9a)

  • #143: Minor docstring/logging changes (1e7572a)

  • #143: Improve structure of execute_query()

  • This commit will also add some info logging to track what happens to the query during its lifecycle (26f303a)

  • #143: Make entity_to_dict() more readable (2210fdb)

  • #143: Allow dates to be converted to strings in nested structures

  • Nice reduction in lines :) (3381e82)
  • #143: Allow included fields to be added to response to n levels deep
  • A nice recursive function allows dictionary-based include filters to be correctly added to the request's response
  • Requests are still currently broken because of date conversion still not happening correctly, but the log message which shows data before being returned shows the data structure is correct, near identical to the way its assembled in the database backend (bba6a23)
  • #143: Checkpoint for making data nested
  • Written a specific solution for the data I've been looking at.
  • This now uncovered another issue, with the way I search dict_result to convert datetime objects to strings (0064ac8)
  • #143: Checkpoint for adding included data to response
  • Data is now added when dictionaries are used within an include filter, however it's not nested correctly (b89baea)
  • #143: Deal with included fields which are joined by dots
  • Also add some comments in execute_query() (c32dd5e)

  • #143: Convert lists and dicts into a form compatible with Python ICAT (3d0f59c)

  • #143: Solve warnings by VS Code

  • Warning were caused by missing import statements (3257204)
  • #143: Checkpoint for dealing with include filter input
  • Input for PythonICATIncludeFilter (9d52037)
  • #143: Start to improve how include filter input is adapted for Python ICAT
  • This deals with strings and lists very well, work is needed on handling dictionaries (da3ac2c)
  • #143: Allow dictionary of include filter to be converted to Python ICAT format
  • This change doesn't make dictionary include filter work, but they're now converted to a notation that Python ICAT accepts (9b8e08e)
  • #143: Add basic implementation of include filter
  • This filter currently accepts a single entity name, or a list of entity names. Dictionaries are not yet supported (c724f72)

  • Merge branch 'feature/icat-limit-skip-filters-#139' into feature/icat-distinct-filter-#141 (32e6927)

  • Merge branch 'feature/icat-order-filter-#140' into feature/icat-limit-skip-filters-#139 (61b63a2)

  • #140: Make suggested changes as per PR comments (8c1dcd7)

  • #140: Add docstrings (182d054)

  • #140: Allow multiple order filters to be used in conjunction (aec81f1)

  • #141: Fix bug on IN operation of WHERE filter with single element value

  • This change fixes a bug where if a value used for a WHERE filter with an IN operation had only a single element in a list, it would cause a JPQL error. This happened because the list was converted into a tuple (to satisfy JPQL formatting of array based data). With single element tuples, Python adds a trailing comma, something which is invalid in JPQL. The value is now converted into a string, with the square brackets being replaced with normal brackets before being sent off to JPQL.
  • Also added type checking on the value if an IN operation is used to ensure the value is a list (2aea37c)

  • #141: Fix bug where JSON serialisable dates wouldn't be applied when using distinct filter (2d7eeff)

  • #141: Fix where distinct filter doesn't show field because of overlapping WHERE filter

  • This change means that if there's a distinct filter of an attribute, and a WHERE filter specifying a condition of the same attribute, the distinct filter will act as intended (where before the data of that attribute wasn't added to the response) (92d6b01)
  • #141: Refactor ICAT creation and execution
  • Turned the two functions into a class, since these are related operations
  • This change has been made so I can add another function to this class later on (0029978)
  • #141: Refactor the distinct filter aspects of execute_icat_query()
  • Combine some of the logic with the existing logic where return_json_formattable is True - this will ensure that datetimes are converted to an appropriate format when using distinct filters
  • This commit also adds checking of the query's conditions to only select attribute names of those which are "!= null" as set by the distinct filter (7031f59)
  • #141: Allow multiple fields to be used in a distinct filter
  • Inside a PythonICATDistinctFieldFilter, a where filter is created for each field in the request. These are then searched for in execute_icat_query() and the data is compiled to only include data from those fields (26b7893)
  • #141: First trial of dealing with distinct filters using conditions from helper side
  • This requires the addition of creating WHERE filters within a distinct filter to deal with multiple fields within a single distinct filter (eedc803)
  • #141: Add 'not equal' operation to ICAT WHERE filter
  • The primary used for this operation will be to add WHERE filters when a distinct filter is in a request, so distinct filters can accomodate multiple fields (bda22f4)

  • #141: Add distinct aspect of ICAT distinct filter (d41359f)

  • #141: Implement distinct filter

  • This changes the output of the data to a list of values, rather than the traditional list of dictionaries, containing attribute name and its value (20c7120)

  • #139: Code formatting changes made by Black (0888686)

  • #139: Fix independent use of skip filter

  • Grabs max num of entities from ICAT properties and applies that to the count element of the tuple - the currently configured number is 10000, which should be more than enough for any applicable use cases of this API (39ab528)
  • #139: Allow a request to have both limit and skip filters
  • The skip filter will be merged into the limit filter, with the skip filter being removed from the filter handler
  • There is a bug when using a skip filter on it's own, as it'll always return a 404 (93eb468)
  • #140: Move create_condition() to PythonICATWhereFilter
  • This is to remove a circular dependency that I found while implementing an order filter
  • Also added a couple of init.py files which didn't previously exist (51924f8)

  • #140: Remove unused imports (19a286b)

  • #140: Basic implementation of ICAT order filter

  • If there's more than one order filter in a request, the others get overwritten due to the way setOrder() works in Python ICAT (578ac21)
  • #140: Move FilterOrderHandler into its own file
  • This will remove circular imports in a later change
  • Also helps to tidy up the code base by separating the filter handler from the abstract classes of the filters (78ef98e)

  • #140: Fix DB order filter from an import error (4bee04b)

  • #142: Rebuild Swagger docs

  • They've changed a little since they've last been committed (6d225d6)

  • #142: Make changes as requested in PR review (537d4c6)

  • Merge branch 'feature/python-icat-where-filter-#142' of github.com:ral-facilities/datagateway-api into feature/python-icat-where-filter-#142

  • Caused by not doing a git pull before a made changes... (a70ee2b)
  • Merge pull request #157 from ral-facilities/feature/linter-black-#155

Implement Black (code formatter) into repo's development cycle (358b74b)

  • Merge branch 'feature/python-icat-where-filter-#142' into feature/linter-black-#155 (aeb05dc)

  • #155: Add documentation regarding code formatter (d7f726d)

  • #155: Prevent internal ICAT exceptions from occurring

  • Bug introduced when I implemented the WHERE filter (a22e8f4)
  • #155: Put longer strings onto multiple lines to keep with 88 character line length
  • A couple have been left alone as they're only 1/2 characters over 88 (as defined by Black). It seemed better to keep them on a single line for readability reasons (0f67fc7)
  • #155: Make things missed by Black mostly abide by 88 characters
  • Black mostly missed docstrings and some really long strings, but that's understandable. Black doesn't directly edit strings for fear of editing their contents (4aff02d)
  • #155: Changes made by Black code formatter
  • This is directly after doing black . on the root directory. It'll be interesting to see if any corrections are needed (93b4d85)
  • #142: Change 'BadFilterError' to 'FilterError'
  • Part of PR review for this branch (e27c887)
  • #142: Add 'gt' & 'lt' operations to WHERE filter
  • This change affects both database and Python ICAT backends (4707be6)

  • Merge branch 'feature/python-icat-get-with-filters-#137' into feature/python-icat-where-filter-#142 (26b8033)

  • Merge branch 'feature/python-icat-entity-id-endpoints-#136' into feature/python-icat-get-with-filters-#137

Conflicts: common/python_icat_helpers.py (80b89e9)

  • #136: Allow round-tripping of datetime data
  • Added an ICAT exception where data modification was stopped to prevent icatdb from being put into an invalid state (511655e)

  • #136: Changes made as requested in pull request (9611a11)

  • #136: Make improvements as per PR for this issue (bc1cdf9)

  • #137: Make improvements as per PR for this issue (f6ac645)

  • #142: Change project structure in README

  • This has changed as a result of having directories for each type of backend: database and python_icat (3290f08)

  • #142: Implement like and in expressions for WHERE filter (95e8c37)

  • #142: Implement aspects of WHERE filter for ICAT backend

  • 'like' and 'in' operations don't work yet (9d2b4b5)

  • #142: Remove duplicated constructor for DB filters (b6224db)

  • #142: Move ICAT files and skeleton for filters (8ac7efc)

  • #142: Move DB filters to their own directory

  • This change also moves the database implementation of the backend and the helper functions to a specific database folder. The same will be done with the python_icat versions of these
  • Unit tests still pass when using the database backend (ea45b66)
  • #142: Move skeleton filters implementation to separate files
  • This adds filters.py to be able to implement filters using Python ICAT (01bd4bb)

  • #137: Pass error messages from Python ICAT to PythonICATError (e0180d3)

  • #137: Basic implementation of GET endpoint for entities

  • Implementation for the Python ICAT backend
  • This implementation ignores any filter parameters in the request as this haven't been implemented for the Python ICAT backend. As a result, this endpoint just gets all the records for a specified entity (cc4358e)
  • #136: Catch exception when an attribute cannot be found
  • API now returns a 400 with a message to the user (cac1003)
  • #136: Add Python ICAT exception and docstrings across helpers
  • Python ICAT exception returns a 500 to users when something Python ICAT related may have gone wrong (e.g. creating a Query object). This has been implemented for that example
  • Added docstrings to the session related functions to match with the ones for entity by ID helper function
  • Removed an empty queries_records function inside the ICAT helper file as it's not needed (785adde)
  • #136: Resolve ICAT error where related entity were trying to be set to null
  • The code to convert request body values into dates has been tested and is fully working. (4b15a05)

  • #136: Add support for delete entity by ID on new backend (f072c1d)

  • #136: Allow relevant updated data to be converted into datetime objects

  • Code is currently untested due to issues I'm having with Python ICAT. The code will be tested before review. (8a1728b)
  • #136: Add basic implementation for PATCH entity by ID
  • Basic implementation, work needs doing to allow dates and other unusual types to be updated (53acd2a)
  • #136: Allow get_entity_by_id() to return data in different formats
  • A flag is used to determine whether the data will be returned in a manner ready to be converted to JSON, or whether to leave it in a Python ICAT state ready to be edited/changed (8627e8b)
  • #136: Refactor get entity by ID helper function and associated infrastructure
  • This change is mostly about moving code into their own functions, but also making the id_condition more generic so it can be used throughout the helper functions. A design decision needs to be made about how to best structure the code creating conditions
  • Added docstrings throughout, documenting the accepted parameters and their types (f747bae)

  • #136: Add framework for PATCH entity by ID endpoints (9bfefcf)

  • #136: Allow any entity by ID request to correctly get data

  • Added docstrings to the functions related to this work. Some of them are incomplete but have TODO marked against them where that's the case
  • Removed the hardcoded 'User' entity so any entity can be used. This compares the table name to each of the entity names from Python ICAT and passes the Python ICAT entity name to the query builder. This is due to Python ICAT being case sensitive. A 400 is thrown if an entity cannot be found within Python ICAT
  • Moved query execution into a generic function for use elsewhere in the codebase
  • Changed a blanket string conversion just to converting datetimes to strings. This was causing IDs to be converted to strings which isn't correct behaviour. Datetimes must be converted to strings so they become JSON serializable (0b0bc20)
  • #136: Add functionality to get a user by their ID
  • The 'User' entity is currently hardcoded into get_entity_by_id, due to facing an issue with case sensitivity on Python ICAT's side. In the future, any entity should be able to use this code (3cc1301)
  • Merge pull request #149 from ral-facilities/feature/python-icat-backend-login-endpoints-#135

Python ICAT Backend: Login endpoints & session validation (ddca280)

  • #135: Allow session IDs to not be overwritten in the database
  • This is done by creating a new client object each time users login. If the same object is used for every login, the session ID will just be overwritten, even if logging in as a different user (5195edb)

  • #135: Add mechanism parameter to login endpoint for Swagger docs (851166c)

  • #135: Implement refresh session logic for Python ICAT backend (95bcb7b)

  • #135: Allow users to logout from Python ICAT backend (8ca17eb)

  • #135: Further tweaks for Python ICAT session decorator

  • This also moves some of the DB helper code into the relevant file
  • These changes cause one of the tests in test_helper.py to fail when using the Python ICAT backend. This is because @requires_session_id only really tests if a session ID is valid when it's actually doing an API call. I'm not 100% happy with this but the feature does function (dbe8ba1)

  • #135: Add Python ICAT decorator to ensure valid session ID (168ba94)

  • #135: Add config option for whether certs for ICAT

  • Moved the client object into the constructor so it can be accessed by all functions in the class, though this might mean a session ID becomes useless? (93d2305)
  • #135: Implement login function for python ICAT backend
  • Because this backend can use different mechanisms, I've changed the DB backend so it can use a mechanism as per the request body
  • Any installed mechanism can be used, though if one isn't supplied, it defaults to simple (9755e0b)

  • #135: Add skeleton outline for new python-icat backend (c2e09bf)

  • Merge pull request #127 from ral-facilities/feature/configurable-backend-#125

Enable backend to be configurable (e4db23b)

  • Merge branch 'master' into feature/configurable-backend-#125 (108256f)

  • Merge branch 'id-endpoints-404-#129'

Conflicts: src/swagger/openapi.yaml (016ce7b)

  • #129: Add rebuilt openapi.yaml for Swagger docs (821680d)

  • #129: Correct other customer user input parameters so they work in Swagger (8f1c409)

  • #129: Change name of ID parameters to correct case for Swagger docs

  • Swagger is case sensitive when using parameters
  • This change means endpoints with an ID parameter (e.g. /users/{id}) will now work using the Swagger UI user input for said ID (203707f)
  • Merge branch 'include-filter-count-endpoints-swagger-#128'

Conflicts: src/swagger/openapi.yaml (d330972)

  • #128: Add rebuilt openapi.yaml for Swagger docs (8a7a47f)

  • #128: Make INCLUDE filters visible to all count endpoints for Swagger web UI (d31ac7a)

  • Merge pull request #130 from ral-facilities/remove-userinvestigations-query-#126

Remove unused users investigation endpoints (948b4dd)

  • #126: Add rebuilt openapi.yaml for Swagger docs (215c0dc)

  • Remove unused users investigation endpoints

  • Removing /users/{id}/investigations and /users/{id}/investigations/count
  • These endpoints are being removed because equivalent behaviour can be achieved by the respective endpoints in /investigations by using WHERE and INCLUDE filters (9bba91c)

  • #125 - extract out backends into seperate files (8a2f6a2)

  • #125 - use non-decorator version of error handler registerer (54e50ce)

  • #125 - fix duplicated to_dict in get_id_endpoint (755e85f)

  • Merge branch 'master' into feature/configurable-backend-#125 (3862d11)

  • Merge pull request #121 from ral-facilities/feature/set-session-info-#120

#120 - set username + expiredatetime on POST /session (05ad009)

  • set username to simple/root to give root access on default install (206a78f)

  • #120 - set username + expiredatetime on POST /session (f7f9b74)

  • Merge pull request #124 from ral-facilities/feature/improve-openpi-spec-gen-#123

Improve OpenAPI spec generation (130a736)

  • #123 - complete OpenAPI spec documentation (03d768f)

  • #123 - document using apispec

  • initialise spec
  • adds models
  • filters documented
  • base entity endpoint fully documented
  • sessions endpoint fully documented (dfebf55)
  • #125 - add abstract "backend" class and move database specific code in endpoints to there.

Also, an change in that errors are no longer caught, and are translated by flask into error codes. (5ae2cc8)

  • Revert to bb164ae (committed to wrong branch) (1d84c39)

  • Include 'findone' endpoints in generated openapi.yaml (bc55c74)

  • Merge pull request #117 from ral-facilities/feature/schema-references-#116

Feature/schema references #116 (bb164ae)

  • Updated openapi.yaml with entries in insertion order (351f7e8)

  • Retain insertion order when dumping to YAML (889e84a)

  • Updating openapi.yaml to version with named schemas (97cbb72)

  • Update swagger generator to reference named schemas instead of inlining them. Fixes #116 (762f490)

  • Merge pull request #112 from ral-facilities/111_create_count_queries_for_table

111 create count queries for table (274c0c1)

  • #111: Create count queries (b9293c8)

  • #111: Match style guides (5721181)

  • Merge pull request #91 from ral-facilities/87_improve_readme

Improve README (7140077)

  • #87: Split class diagram into 2 diagrams (9e7a5ee)

  • #87: Add Query Docstring (c4b46fb)

  • #87: Update example (0b45c2a)

  • Merge branch 'master' into 87_improve_readme (95a1601)

  • #87: Fix typo (f427cb2)

  • #87: Add authentication (6bde478)

  • #87: Update main.py (b7d473f)

  • #87: Add swagger to contents (791c993)

  • #87: Add querying and filtering (f96f152)

  • #87:Add swagger generation (58faa46)

  • #87: Remove debugging line (2763929)

  • Merge pull request #110 from ral-facilities/109_add_example_responses

Add example responses to swagger spec (5df83d9)

  • #109: Regenerate swagger (7aa7cd0)

  • #109: Add examples to endpoints (8052f44)

  • #109: Create dictionary to store examples (e9d37c7)

  • #109: Add function to map types (0df043e)

  • Merge pull request #104 from ral-facilities/93_add_tests

Test EntityHelper (ce639cf)

  • #93: Test dictionary include (22b1838)

  • #93: Add investigation (1f55522)

  • #93: Test EntityHelper (5271333)

  • Merge pull request #106 from ral-facilities/105_problem_with_returning_decimals

Problem with returning decimals (1c99cac)

  • #105: Use method (b78d251)

  • #105: Create serializer method (d294df4)

  • Merge pull request #108 from ral-facilities/107_add_distinct_count_to_swagger

107 add distinct count to swagger (4f15e86)

  • #107: Regenerate swagger (b8e40c7)

  • #107: Add distinct parameter to count in generator (70e8674)

  • Merge pull request #103 from ral-facilities/102_add_bracket

#102: Add bracket (69b7939)

  • #102: Add bracket (0974012)

  • Merge pull request #101 from ral-facilities/100_generate_2_datasets

Generate 2 datasets (78799d5)

  • #100: Style changes (ae9d342)

  • #100: Reduce the number of datafiles (6a52ea9)

  • #100: Generate datasets per investigation (4d163b2)

  • Merge pull request #94 from ral-facilities/92_refactor_endpoints

Proposed refactoring for 92 (72248a8)

  • Merge branch 'master' into 92_refactor_endpoints (2c6a862)

  • Merge pull request #99 from ral-facilities/83_add_dev_requirements

Add dev requirements (0478a45)

  • #83: Remove pyyaml (a68f1f5)

  • #83: Actually fix it this time (9920385)

  • #83: Fix line spacing (3b37b96)

  • #83: Update readme (07fbc76)

  • #83: Update and split requirements (1a78b60)

  • Merge pull request #98 from ral-facilities/80_debug_mode_is_not_applied

Debug mode is not applied (0a269e3)

  • #80: Update readme (0386de2)

  • #80: move debug option (611dd2a)

  • Merge pull request #97 from ral-facilities/95_logging_nothing

Remove empty log line (9e624a1)

  • #95: Remove line that was logging nothing (c936c2e)

  • Merge pull request #96 from ral-facilities/42_add_filtering_to_includes

Add way to filter included entities (8191110)

  • #42: Add attributes to constructor (cb7fbf2)

  • #42: Add level 2 include filtering (e410825)

  • #42: Add private method to set fields and tables (22d0be1)

  • #42: Add way to filter included entities (f1e6e8a)

  • #92: Add docstrings (18832dc)

  • #92: Update readme (7d4b1b0)

  • #92: Add findone and count (067873c)

  • #92: Fix typo (77d36ce)

  • #92: Update swagger generator (b5e4a6b)

  • #92: Update main to use generated endpoints (3b1e5f7)

  • #92: Add entity map (12c4353)

  • #92: Create way to generate endpoints at startup (5e2a25d)

  • #92: Delete old endpoint modules (8540d39)

  • Merge pull request #84 from ral-facilities/70_add_db_creation_script

Add db creation script (c74c0cf)

  • #70: rename function (d6d550a)

  • #70: Update running instructions (bc34eb9)

  • #70: Update readme (5bb9cf2)

  • #70: Add argument passing (15693c1)

  • Merge branch 'master' into 70_add_db_creation_script (77a2c6f)

  • #70: add datafile location (c9b41ec)

  • #70: Update requirements (5a6a348)

  • #70: Add db creation script (eeb0de3)

  • #70: Add util package (22a30bd)

  • Merge pull request #90 from ral-facilities/89_use_json

Correct response types (5b1f878)

  • #89: Update column types (d5f0e31)

  • #89: Only convert to string for datetime (a99cc3a)

  • Merge pull request #86 from ral-facilities/85_add_in_filter

#85: Add in filter (6ae5616)

  • #85: Add test (f37081f)

  • #85: Regenerate swagger (d555156)

  • #85: Add in filter (5caed7f)

  • Merge pull request #77 from ral-facilities/75_fix_paths

#75: Change paths (71393af)

  • #75: Use file to get relative paths (6d1f323)

  • Merge branch 'master' into 75_fix_paths (620a248)

  • Merge pull request #82 from ral-facilities/81_enable_cors

Enable Cors (1cf7dd9)

  • #81: Use cors (24c782a)

  • #81: Update requirements for flask-cors (2c36559)

  • Merge pull request #76 from ral-facilities/72_add_requirements_file

Add requirements file (87e7bbb)

  • #72: Add versions and remove pip-tools (da0622f)

  • #72: Generate requirements.txt (96441b3)

  • #72: Add requirements.in (b7bfc48)

  • #72: Update README.md (780dee6)

  • Merge pull request #79 from ral-facilities/49_fix_post_on_entities

Fix post on entities (beff0de)

  • #49: Update endpoints (846607b)

  • #49: Allow insertion of multiple rows (f752db5)

  • #49: Make create_row return the inserted row (0be6159)

  • #49: Add refresh to inserted entities (4d81ab9)

  • Merge pull request #69 from ral-facilities/68_specify_host_and_port_in_config

Specify host and port in config (e69981d)

  • #68: Specify port and host in main.py (6cbb1ae)

  • #68: Update config.py (3a34c3b)

  • #68: Update example config (5fb2bd6)

  • Merge pull request #66 from ral-facilities/39_only_one_include_filter

Allow only one include (0d97b57)

  • #39: Regenerate swagger (3702328)

  • #39: Update swagger (c7031f2)

  • Merge branch 'master' into 39_only_one_include_filter (2e8e41d)

  • #39: Handle exception and send response (b755af3)

  • #39: Raise exception if multiple includes applied (31e5b6e)

  • #39: Add MultipleIncludeError (13ac77e)

  • Update README.md (a3b9551)

  • Merge pull request #74 from ral-facilities/21_improve_testing

Improve testing (991210a)

  • #21: Test Pascal to normal case (db4c1b9)

  • #21: Test helpers.py (3494989)

  • #21: Test FilterFactory (edad675)

  • #21: Create Base FlaskAppTest (1fdf883)

  • #21: Create helper tests (48000de)

  • #21: Fix is_valid_json (d031885)

  • #21: Remove redundant tests (f537c5d)

  • Merge branch 'master' into 75_fix_paths (03c015b)

  • Merge pull request #73 from ral-facilities/71_change_file_paths

Use pathlib for config and logger (e2cf13d)

  • #71: use pathlib (c2eb214)

  • #75: Change paths (36ec26c)

  • Update README.md (ac62218)

  • Merge pull request #67 from ral-facilities/48_change_session_handling

Change handling of session (c6ddda4)

  • #48: Move to using with (f6de8e2)

  • #48: Remove old closing of sessions (77aeb54)

  • #48: Allow use of with keyword with Queries (ee9265a)

  • Merge pull request #65 from ral-facilities/50_update_swagger

Update swagger generation (444342a)

  • Merge branch 'master' into 50_update_swagger (9d5962c)

  • #50: Regenerate openapi.yaml (b2acf40)

  • #50: Use pyyaml to write to file (0f759ba)

  • #50: Create SwaggerSpec class (e2c2c8f)

  • #50: Create an entity class (412a895)

  • #50: Create a parameter class (d34cef8)

  • #50: Use pyyaml instead of f strings (abe4252)

  • #50: Remove unused import (bb2f913)

  • Create LICENSE (663b68c)

  • Update README.md (3a0cf98)

  • Merge pull request #64 from ral-facilities/61_return_401_for_no_credentials

Handle no credentials (40a3463)

  • Merge branch 'master' into 61_return_401_for_no_credentials (8e46366)

  • Merge pull request #57 from ral-facilities/51_refactoring_filters

Refactor getting filters (bd206b1)

  • #51: Update _get_results_with_include (1b2a389)

  • Merge branch 'master' into 51_refactoring_filters (01c0a1c)

  • #51: Use updated include filter (ce6beb7)

  • #51: Change include filter (958da33)

  • #51: Use add filters method (147ec49)

  • #51: Add method to add a list of filters to filter handler (9c6cfb1)

  • #51: Make Querystring filter method return filter objects (33ea947)

  • Merge pull request #63 from ral-facilities/59_handle_trailing_slashes

Handle trailing slashes (c3eb99f)

  • #59: Handle trailing slashses (3b6d8c6)

  • Merge pull request #62 from ral-facilities/60_remove_config

Remove config from repo (ca0a466)

  • #60: Update gitignore (f912dd8)

  • #60: Delete config (857263c)

  • #61: Handle no credentials (e1cda06)

  • #61: Add missing credentials exception (ccef03e)

  • Merge pull request #58 from ral-facilities/54_use_enum_for_parameter_type

Use enum for parameter type (db5330d)

  • #54: Use EnumAsInteger Column type (ba49d1e)

  • #54: Fix docstring (10f974e)

  • #54: Extend TypeDecorator (4cec6df)

  • #54: Create ValueTypeEnum (9360b8b)

  • #54: Add DatabaseError exception (0eb5d9d)

  • Merge pull request #56 from ral-facilities/41_add_ability_to_get_distinct_values

Add ability to get distinct values (5695e3f)

  • #41: Whitespace changes (12adcab)

  • Merge branch 'master' into 41_add_ability_to_get_distinct_values (d8a8499)

  • Merge pull request #55 from ral-facilities/52_fix_isis_queries

#52 - fix accidentally pluralised entities (bb0a2a3)

  • #52 - fix accidentally pluralised entities (9a3c8c9)

  • Merge pull request #44 from ral-facilities/43_allow_many_to_many_includes

Allow many to many includes (82e7dcb)

  • Merge branch 'master' into 43_allow_many_to_many_includes (7ec4f3f)

  • #43: Move included entities to nested array (100d004)

  • #43: Refactor for readability (f88b6bc)

  • #43: Allow many to many includes (107e7c6)

  • Merge pull request #53 from ral-facilities/52_fix_isis_queries

Fix ISIS facility cycles & ISIS investigations queries (fb16374)

  • #52: fix ISIS facility cycles & ISIS investigations queries (609ce2a)

  • Merge pull request #38 from ral-facilities/34_add_table_specific_endpoints

34 add table specific endpoints (43638e2)

  • #34: Fix indent (5511c5c)

  • #34: Remove table method from where filter (8b81ea4)

  • #34: Use joins on investigation user queries (266c644)

  • #34: Minor refactor (957cf3a)

  • #34: Use joins for cycle investigations query (0c9046a)

  • #34: Use joins (9c1393b)

  • #34: Add way to close all sessions in queries (697c042)

  • #34: Change how filters are set (6bbabda)

  • #34: Make method non static (cd95508)

  • #34: Override get_all_results (ae5aad6)

  • #34: Update function (957542f)

  • #34: Add new method of getting facility cycles (a8c68cb)

  • #34: Change to only filter the start dates (e0ea0c4)

  • #34: Change class name (2d51bc0)

  • #34: Change class name (fb2666e)

  • #34: Add missing filter params (1f6a915)

  • #34: Update docstring (cbec9d2)

  • #34: Add missing filter param (d8156fb)

  • #34: Update endpoints (c7026b3)

  • #34: Update count function (7b32a1f)

  • #34: Use InstrumentCycleInvestigation query (98121f8)

  • #34: Create instrumentInCycleInvestigationsQuery (30af58c)

  • #34: Update endpoint (c68d930)

  • #34: Update docstring (83754b4)

  • #34: Add filters (b7648e2)

  • #34: Refactor and allow filters (013adbf)

  • #34: Create InstrumentFacilityCyclesQuery (fc9ce04)

  • #34: Refactor get_investigations_for_user (2676563)

  • #34: Refactor get rows_by_filter (452ce6d)

  • #34: Rename to UserInvestigations (bf29c0b)

  • #34: Change case (2613ed3)

  • #34: Add resources to api (3ffa3fa)

  • #34: Add count (127e6be)

  • #34: Add final resources (cfc9b8d)

  • #34: Add investigations for instruments in cycle (c709dd6)

  • #34: Temporary fix of 48 (225f182)

  • Merge branch 'master' into 34_add_table_specific_endpoints (bd06b89)

  • #34: Update resource class for count (de74984)

  • #34: Add cycles count function (2a08152)

  • #34: Update endpoint resource (8256c1d)

  • #34: Add get cycles for instrument (e5180cd)

  • #34: Use get investigations count in resource (ccdec3b)

  • #34: Update _get_table_to_filter (3261184)

  • #34: Add get investigation count function (9e8e0e5)

  • #34: Add ISISInvestigationsCountQuery (31677c3)

  • #34: Add Missing docstring (455bfdd)

  • #34: Add ISIS User Investigations endpoint (ccfb01e)

  • #34: create get_investigations_for_user (885ba6f)

  • #34: Change how table is selected for filtering (f0dd894)

  • Merge branch 'master' into 34_add_table_specific_endpoints (9016789)

  • #34: Add _get_table_to_filter method (ac2b923)

  • #34: Create ISISInvestigationsQuery class (c8e85c4)

  • #34: Change type of endpoint id to int (f55879c)

  • #34: Add resources to api (8fbbaeb)

  • #34: Add table specific resource classes (7f13768)

  • #41: Add and use function for distinct fields (2f85368)

  • #41: Extract include into function (eebf29a)

  • #41: Update Factory (1b76f38)

  • #41: Add implementation to filter (2a496a0)

  • #41: Add is_distinct bool to ReadQuery (cc2459a)

  • #41: Add Distinct field filter and update precedence (cb67f07)

  • #41: Make precedence a required property (a217699)

  • Merge pull request #45 from ral-facilities/23_improve_logging

Improve logging (26b15dc)

  • #23: Change to show traceback (e763ebb)

  • Merge pull request #47 from ral-facilities/46_change_to_int_on_endpoint_id

Change endpoint ID type to int (38c8d4a)

  • #46: Update README.md (968aa0c)

  • #46: Change to int (cea1d05)

  • Merge pull request #40 from ral-facilities/32_improve_include_filtering

Improve include filtering (60c668d)

  • #32: User ternary (b60e2d1)

  • #32: Change to_nested_dict (cdd1133)

  • #32: Add get_related_entity (6e8c8c1)

  • Merge pull request #37 from ral-facilities/18_add_more_where_filtering_types

Add Like, less than and greater than where filtering. (7b6adef)

  • #18: Fix typo (b2d2750)

  • #18: Change to lte and gte (a9fb2fa)

  • #18: Implement greater than and less than filters (03f603c)

  • #18: Correct entity handling on update (086478c)

  • #18: Update get_row_by_id (171ad98)

  • #18: Update filter factory (f7335c3)

  • #18: Add like filtering (1382e9b)

  • Merge pull request #36 from ral-facilities/31_change_format_of_query_filters

Change format of accepted query strings (fa374f7)

  • #31: Change format of accepted query strings (63aa305)

  • Merge pull request #35 from ral-facilities/30_use_order_of_operations

Use order of operations for filtering (2f59010)

  • #30: Remove is_limited (38b9ec0)

  • #30: Use filter handler (288b21b)

  • #30: Add precedence to filters (31fbe83)

  • #30: Create order handler (bac726e)

  • Merge pull request #29 from ral-facilities/use_classes_for_queries_and_filters

Use classes for queries and filters, and create SessionManager (c6f0b87)

  • Merge branch 'master' into use_classes_for_queries_and_filters (70844ce)

  • Merge pull request #27 from ral-facilities/15_generate_swagger

Generation of openapi.yaml (22bfd1e)

  • #15: Use config to control generation (735eda4)

  • #15: Add is_generate_swagger to config class (cd1668d)

  • #15: Add swagger generation to config files (468bf73)

  • #15: Fix merge conflict (fb9972e)

  • Merge pull request #28 from ral-facilities/13_setup_config_file

Use a config file (251d2bd)

  • #13: Add example config (fb96925)

  • #13: Exit on missing config values (eec6821)

  • #13: Update README.md (ff81709)

  • #13: Use config class (4b6f0e9)

  • #13: Create config class (c6bcff6)

  • #13: Add config file (8e414f6)

  • #15: Fix crash when not generating (6dcdc7b)

  • #15: Regenerate Swagger (0e4a7c0)

  • #15: Remove limit filter from findOne (5a50472)

  • #15: Remove unused filters from count (56fdfe9)

  • #15: Allow generator to be disabled (33ef610)

  • #15: Change path (e6e418a)

  • #15: Output of generator (b6e2f66)

  • #15: Import generator and write to file (8c6b525)

  • #15: Apply decorator to resources (166fcdd)

  • #15: Create method to write to the file (a6df922)

  • #15: Create method to write paths (3687e82)

  • #15: Create method to write top part (5b0c8a0)

  • #15: Create method for converting from PascalCase (a6c90ce)

  • #15: Add filepath for openapi.yaml (6a7c575)

  • #15: Create class wrapper to collect endpoints (c955db0)

  • #15: Create init and endpoints list for SwaggerGen (134e74f)

  • #15: Create SwaggerGenerator class (3d4abfa)

  • Remove try added in error (3728d89)

  • Fix Indentation (37b28be)

  • Move session closing (eff2931)

  • Move the closing of the session (1f38a3c)

  • Fix indentation (1d1092f)

  • Make sure session is closed correctly (075f476)

  • Add pool_size and only create 1 engine/factory (beb9df8)

  • Increase pool size (85304fc)

  • Change filtered row count function (e6f53da)

  • Remove duplicated queries (b246805)

  • Create a count query class (b6897fb)

  • Use the new scoped_session SessionManager (12affc7)

  • Move the session manager and use scoped_session (3106203)

  • Fix order filter (7841559)

  • Use SessionManager (719aed7)

  • Create SessionManager (0e7dfa4)

  • update get_rows_by_filter (27a977c)

  • Update update_row_from_id (d874231)

  • Update delete_row_by_id (62eb47e)

  • Update get_row_by_id (630f683)

  • Update insert_row_into_table (60b3c98)

  • Update Create Row from JSON (58e4dba)

  • Create QueryFilterFactory (a2d8050)

  • Create SkipFilter (f9600ec)

  • Create IncludeFilter (7e855ce)

  • Create LimitFilter (a0b18a8)

  • Create OrderFilter (67bf964)

  • Create WhereFilter (202c009)

  • Create QueryFilter (40af194)

  • Create DeleteQuery (af62c14)

  • Create UpdateQuery (0c63010)

  • Create CreateQuery (e3bf3b3)

  • Create Read Query (d8f5ab8)

  • Create abstract base query class (b6b92b5)

  • Merge pull request #16 from ral-facilities/2_add_include_and_order_filters

Add filtering (27cfed0)

  • #2: Remove repeated arg (ed6ebd8)

  • #2: Add docstring to method (a9c83d7)

  • Merge branch 'master' into 2_add_include_and_order_filters (3415ce5)

  • Merge pull request #20 from ral-facilities/17_allow_empty_filters

Allow empty filters (457630e)

  • #17: Allow empty filters (2862b7c)

  • #17: Whitspace (c97b70a)

  • #17: throw exception if filter is empty (76d756b)

  • #2: Remove print statement (d9716af)

  • #2: Use nested dict method (fb0fb90)

  • #2: Add nested dict method (875fe03)

  • #2: Remove function (be24106)

  • #2: Rename shadow named vars (af5c0c2)

  • #2: Change name (e72561d)

  • #2: Pull out including to func for readability (e305449)

  • #2: Allow includes from dictionary (d0a2313)

  • #2: Allow including multiple relations (e7485a6)

  • #2: Allow a single include (21a6e5f)

  • #2: Update backref on db models (bf00f23)

  • #2: Add limit and order filters (8f776cd)

  • #2: Change how exceptions are logged (c45075a)

  • #2: Allow ordering before limit and vise versa (36352dc)

  • Merge branch 'master' into 2_add_include_and_order_filters (7970165)

  • Merge pull request #14 from ral-facilities/11_add_logging

Add Logging (47d33c8)

  • #11: Add logging (766221e)

  • #11: Set up logger (d14d14c)

  • #11: Create logger setup (0a079c2)

  • #11: Update git ignore (b9d9585)

  • #11: Update gitignore (40461f2)

  • #11: Remove unused functions (463ebf8)

  • Merge pull request #12 from ral-facilities/1_add_patch_http_method_for_non_id_endpoints

Add patch method to entity endpoints (2cfd027)

  • #1: Fix type check and get_row_id args (9870458)

  • #1: Catch BadRequestError in query wrapper (3986258)

  • #1: Allow single dictionary in patch function (07d29d4)

  • #1: Add bad request exception (fc6b12e)

  • #1: Add patch to endpoints (02e44d2)

  • #1: Add patch function (9b08dc6)

  • Merge pull request #8 from ral-facilities/4_post_sessions_take_data_in_body

#4: Move credentials checking to post body (e58f162)

  • #4: Whitespace changes (d2fa4fc)

  • #4: Update session post tests. (f00d46b)

  • #4: Change how POST checks are done. (0cc96d2)

  • #4: Move credentials checking to post body (08fc3e6)

  • Update README.md (542e86f)

  • Merge pull request #10 from ral-facilities/3_imporve_instructions_on_running_app

Update instructions to run api in README.md (fab7fab)

  • #3: Update README.md with better instructions for running the api (9a59889)

  • #2: Add order filtering (b243897)

  • #2: Fix typo (8599c5e)

  • Merge pull request #9 from ral-facilities/5_add_bearer_authentication_type_to_auth_header

Use bearer type in authorisation header (2ec57a9)

  • #5: Move to using Bearer type Authorization header (b3440ed)

  • #5: Create AuthenticationError Exception (fcc5ec7)

  • #5: Update test Constant (de8969a)

  • Merge pull request #7 from ral-facilities/6_entities_endpoints_dont_work

Fixes broken get method on entities endpoints (647dde6)