Skip to content

Releases: xchem/fragalysis-backend

Mint incremental release

04 Nov 13:04
@mwinokan mwinokan
2024.11.1
f346b79
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Mint incremental release Latest
Latest

What's Changed

  • Fix ComputedMolecule name format (issue 1524) by @kaliif in #675
  • Fixed (probably) data loss on metadata.csv uploads (issue 1535) by @kaliif in #676
  • feat: versioned auto-assigned upload tags by @kaliif in #677
  • fix: correct source paths by @kaliif in #678
  • Allow uploading same target to multiple projects (issue 1514) by @kaliif in #679
  • fix: add code path to handle old uploads (with task_id) (issue 1514) by @kaliif in #680
  • fix: a bug in migration script (issue 1514) by @kaliif in #681
  • RHS data download (issue 1527, partial) by @kaliif in #682
  • Another fix to data structure migration script. by @kaliif in #683
  • Added timestamp to auto-upload tag (issue 1517) by @kaliif in #684
  • Single sdf in downloads has correct structure separators (issue 1555) by @kaliif in #685
  • Merge staging changes into production for 2024.11.1 by @mwinokan in #686

Full Changelog: 2024.09.4...2024.11.1

Contributors

kaliif and mwinokan
Assets 2
Loading

Fixes for API CSS (1500) and metadata upload (1530)

30 Sep 09:28
@mwinokan mwinokan
2024.09.4
4d5f6c0
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Fixes for API CSS (1500) and metadata upload (1530)

Full Changelog: 2024.09.3...2024.09.4

What's Changed

  • Fix missing CSS in REST API (issue 1500) by @kaliif in #672
  • Fixes issue 1530 (metadata loading) and remainder of 1496 (missing columns) by @kaliif in #673
  • Promote 1500 (backend CSS) and 1530 (metadata upload fix) to production by @mwinokan in #674

Full Changelog: 2024.09.3...2024.09.4

Contributors

kaliif and mwinokan
Assets 2
Loading

Bulk tag upload 1263

23 Sep 09:58
@mwinokan mwinokan
2024.09.3
fbde0d7
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Bulk tag upload 1263

What's Changed

Full Changelog: 2024.09.2...2024.09.3

Contributors

kaliif, dependabot, and alanbchristie
Assets 2
Loading

Add missing pose to CSV

13 Sep 11:42
@alanbchristie alanbchristie
2024.09.2
d6e5289
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Add missing pose to CSV 
Missing Pose column (#665)

* build(deps): bump django from 3.2.24 to 3.2.25

Bumps [django](https://github.com/django/django) from 3.2.24 to 3.2.25.
- [Commits](https://github.com/django/django/compare/3.2.24...3.2.25)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* stashing

* Target loader now accepts experiments marked as 'manual'

* build(deps): bump pillow from 10.2.0 to 10.3.0

Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.2.0 to 10.3.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/10.2.0...10.3.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Attempt to reduce pop-up "flicker" (1403) (#569)

* fix: Attempt to debug timeout errors

* fix: More logging on service_query

* fix: Varioustimeout adjustments

* fix: Removed exception during timeout

* fix: Explit log on SSH connection error

* fix: Retry attempts for MySQL connections

* fix: Service timeout now 28 (was 17)

* fix: Add pymysql read and write timeouts

* fix: Quiter (expected) connection failure handling

* fix: TIMEOUT now DEGRADED

* fix: Fix while loop exit conditions

* fix: Better loop logic

* style: services logging reduced and back to debug

* fix: SSHTunnel logging now ERROR (was DEBUG)

* fix: Quieter securty

* fix: More failures permitted (and debug tweaks)

* fix: Leaner logging

* fix: Leaner logging (only report when we're having topruble)

* fix: Better constant name

* fix: Reduced service logging

* docs: Doc tweak

* fix: Minor log tweak

* fix: Fixed duplicate log content

---------

Co-authored-by: Alan Christie <[email protected]>

* feat: endpoint to download first reference pdb from assemblies.yaml

* stashing

* stashing

Changes so far:
- removed endpoint FirstAssemblyview
- moved the functionality to template_protein field in
TargetSerializer
- removed TargetMoleculesserializer
- removed sequences field from TargetSerializer

This is a result of Boris' comment in
github (https://github.com/m2ms/fragalysis-frontend/issues/1373#issuecomment-2047417958)
where he said the field isn't used and template_protein field is not
used. Looking at the code where this may be used, revealed that
Targetmoleculesserializer can be removed as well

NB! they're not removed-removed right now, only commented in. This
commit can be used to restore the code.

* fix: removed code mentioned in previous commit

* basic functionality

TODO: add and test PATCH method on PoseSerializer

* stashing

* build(deps): bump idna from 3.6 to 3.7

Bumps [idna](https://github.com/kjd/idna) from 3.6 to 3.7.
- [Release notes](https://github.com/kjd/idna/releases)
- [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst)
- [Commits](https://github.com/kjd/idna/compare/v3.6...v3.7)

---
updated-dependencies:
- dependency-name: idna
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* stashing

* Added 'hidden' property to viewer.Tag model

Also, added exclude directives to .pre-commit-config.yaml to not touch
django migration files

* Better user/proposal cache (#576)

* refactor: Quieter ISPyB cache

* refactor: Security cache log now quieter

* fix: Better cache logic - and reduced log

---------

Co-authored-by: Alan Christie <[email protected]>

* stashing

Nested serializer for pose, but I don't think I can use them

* feat: added compound_code to pose serializer

* fix: deeper nesting level to site observaton in meta_aligner.yaml

Data loads successfully but actual v2 upload has not been tested

* fix: renamed panddas_event_files to ligand_binding_events in meta_al

* fix: static resources are now loaded again

Fixes bare html API pages

* fix: fixed nginx config

* Revert "Fix static ressources not being loaded"

* stashing

Added sorting keys for versioned key. V1 data loading, waiting for
fixes in conf site presentation to continue with v2

* fix: more robust update method

Allows sending incomplete requests (no idea why seralizer isn't
populating fields).
Also fixed a bug where field updates on poses with multiple
observations where blocked.

* feat: added pose tags

* stashing

Working on allowing incomplete requests payloads. Turns out to be
quite tricky, may have to go back on this.

* feat: fully functional versioned data

Reads and processes upload_2+ data where version numbers are given in suffix

* fix: removed some dead code

* stashing

* build(deps): bump tqdm from 4.66.1 to 4.66.3

Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.1 to 4.66.3.
- [Release notes](https://github.com/tqdm/tqdm/releases)
- [Commits](https://github.com/tqdm/tqdm/compare/v4.66.1...v4.66.3)

---
updated-dependencies:
- dependency-name: tqdm
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* feat: only non-superseded sites available from the api

* build(deps-dev): bump black from 23.12.0 to 24.3.0

Bumps [black](https://github.com/psf/black) from 23.12.0 to 24.3.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/23.12.0...24.3.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>

* chore: merged conflicting migrations from issue 1311 branch

* fix: pose instance attributes updating successfully

* fix: better check for already uploaded data

* fix: merge conflicting migrations

Not sure why the previous merge didn't work

* Adds basic metrics (#588)

* feat: Experiment with django prometheus

* feat: Fix build

* fix: Fix build (locking drf)

* feat: Fix lock file

* feat: Update to non-slim Python 3.11.9

* feat: Back to slim image

* fix: Some basic internal metrics

* fix: Removed rogue line

* fix: Fix lint issues

* fix: Removed custom metrics

---------

Co-authored-by: Alan Christie <[email protected]>

* Attempt to add prometheus to DB (#589)

Co-authored-by: Alan Christie <[email protected]>

* feat: service state queries to backend

Instead of a frontend call triggering a query to each service in every
30 seconds or so, ping the services in the backend, store the results
in a table and serve this up in an endpoint. Seems that when each
user's frontend was testing ISPyB, it was enough to grind it to a
halt.

TODO:
- user friendly names of services
- separate compose and settings file to enable launching celery and
  beat separately (and make sure backend still works when not)
- delete old service query stuff
- most serious one - creating services on startup in apps.py, weird
  issue with service table supposedly not existing in db

* feat: service state queries to backend

Refactored a bit and fixed an issue with automatically starting
services on startup. External service queries fixed.

TODO:
- separate compose file
- delete old services stuff

* Add custom (security) metrics (#593)

* feat: Re-attempt to add custom metrics

* feat: New metric (ISpyB connection attempts)

* feat: Add cache hit/miss metrics

* feat: Leaner metrics (and initial dashboard)

* feat: Metrics now initialised

* docs: Updated dashboard

* fix: Attempt to fix hit/miss metrics

* docs: New dashboard

* docs: Dashboard tweak

---------

Co-authored-by: Alan Christie <[email protected]>

* feat: service state queries to backend

Separate compose file

TODO:
- remove old services

* fix: Fixed connection retry metric (#594)

Co-authored-by: Alan Christie <[email protected]>

* feat: service state queries to backend

Completed, updated readme and deleted old services.

TODO:
- follow-up issue 1359, better ISPyB query

* fix: missing migration and pylint ignore

* fix: remove auto-generating pose tags

* fix: Fix for tunnel connection failure metrics (#597)

Co-authored-by: Alan Christie <[email protected]>

* fix: Adjust metrics (tunnels and cache) (#598)

Co-authored-by: Alan Christie <[email protected]>

* fix: service test functions now scheduled correctly

Minor tweaks to test functions themselves
Start/stop management command fully functional

* fix: changed canon site autogenerated tag name

* docs: Updated grafana dashboard

* feat: filename attribute to RHS download serializer

* fix: update observation's longcode template

* Adjust logging for cache debug (#603)

* refactor: Adds extra logging in proposal extraction

* style: Initial cache now a set rather than empty list

---------

Co-authored-by: Alan Christie <[email protected]>

* Fix connection failure logic (#604)

* refactor: Adds extra logging in proposal extraction

* style: Initial cache now a set rather than empty list

* fix: Fix cache collection failure logic

---------

Co-authored-by: Alan Christie <[email protected]>

* Reduced security logging (#605)

* refactor: Adds extra logging in proposal extraction

* style: Initial cache now a set rather than empty list

* fix: Fix cache collection failure logic

* fix: Reduced logging for proposal cache

---------

Co-authored-by: Alan Christie <[email protected]>

* feat: add management commands to save and restore curated tags

In app container either
curated_tags --dump tags.json
or
curated_tags --load tags.json

* feat: incremental RHS upload (issue 1394)

Allows upload. Currently adds unnecessary compound sets and overwrites
old ones when it doesn't need to

* feat: incremental RHS uploads

Fully working, upload, delete, overwrite, etc.
NB! migrations!

* fix: set ConformerSites and CrystalformSites tags to hidden by default

* fix: code cleanup for merge

* fix: updated crystalform site tag generation scheme

* Initial 'security' changes - Target PATCH now "locked down" (#612)

* fix: Adds extra log to Targets view

* fix: More target logging

* fix: safe query set now derived from ModelViewSet

* feat: Experiment with IsProposalMember class

* fix: Another permission tweak

* fix: Another permission tweak

* fix: Typo

* fix: Fixed silly typo

* fix: Display view filter_permissions

* fix: Experimental new has_permission

* fix: Add user_is_member_of_any_given_proposals to security

* fix: Change to filter class name

* fix: Better has_object_permission

* feat: Leaner has_object_permission

* docs: Minor doc tweak

* fix: Experiment with SafeQuerySet

* fix: Experiment with ModelViewSet in ISPYB

* fix: Back to read-only viewset (and reduced log)

* fix: User must be authenticated

* docs: Doc tweak

* feat: Experiment with filter class for Target view

* fix: Attempt to fix build errors

* fix: Switch to filter_class

* fix: Attempt to fix filter logging

* fix: Another tweak of filters

* feat: Fix filerset typo

* fix: Back to built-in (genric) views

* feat: Attempt to fix has no attribute 'get_extra_actions'

* fix: Silly typo

* fix: Back to ispybsafequeryset

* fix: Restore queryset

* fix: Experiment with mixins

* docs: Doc tweaks

* docs: Doc tweak

* fix: Switch to update() from patch()

* fix: Back to patch()

* refactor: Minor refactor

* feat: Better permissions class for proposals

* Align 1247 with latest staging (#611)

* feat: incremental RHS upload (issue 1394)

Allows upload. Currently adds unnecessary compound sets and overwrites
old ones when it doesn't need to

* feat: incremental RHS uploads

Fully working, upload, delete, overwrite, etc.
NB! migrations!

* fix: code cleanup for merge

---------

Co-authored-by: Kalev Takkis <[email protected]>

---------

Co-authored-by: Alan Christie <[email protected]>
Co-authored-by: Kalev Takkis <[email protected]>

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

* fix: updates to tag generation

Changed how some of the tags are generated as per the comment here: https://github.com/m2ms/fragalysis-frontend/issues/1482#issuecomment-2248079966

* feat: added centroid_res field to CanonSite model

Also, removed fetching centroid_res from CANON_SITES_FILE. Seems that
now it's being added to meta_aligner.yaml, so reading an additional
file is not necessary. I hope...

* feat: added new fields to metadata.csv

Experiment code and centroid res

* feat: added tag aliases to metadata.csv

* Copies Target proposals to new (RHS) Compounds (#629)

* fix: Branch for project reference fix

* fix: Projects copied from Target (during RHS cset-upload)

* fix: Add save before copying projects

* fix: Remove unnecessary save()

* ci: Attempt to fix docker-compose problem

* ci: Fix staging and production builds (docker compose)

---------

Co-authored-by: Alan Christie <[email protected]>

* fix: Fix typo accessing Target projects (#632)

* fix: Add support for target_warning_message context variable (#634)

Co-authored-by: Alan Christie <[email protected]>

* API authentication and security changes (#635)

* style: Reorder functions in the module

* refactor: Moved non-view functions to viewer.utils

* fix: Removed unused save_pdb_zip and minor refactoring

* feat: Removed circular import

* feat: Fix get_open_targets (also get_open_proposals now not _private_)

* feat: Fix get_open_proposals reference

* refactor: ISpyB -> ISPyB

* docs: Updated for use of mixins

* feat: More API security migrations

* feat: More security migrations

* feat: Security migrations for hotspots and hypothesis

* feat: More security fixes

* feat: More security changes

* feat: More security changes (and get_params -> get_img_from_smiles with default w/h)

* fix: Attempt to fix calls to /xcdb/fragspect/ 500 errors

* feat: Another attempot to fix ISPyB

* feat: Use of new user_is_member_of_target()

* feat: Experiment with validator

* feat: Better serializer log

* feat: Even more work on the serializer

* feat: Minor error message tweak

* feat: Add support for TEST_RESTRICTED_TAS_LIST (#614)

Co-authored-by: Alan Christie <[email protected]>

* target permission validation mixin pattern implemented for Pose

* feat: Fix restricted logic

* most endpoints secured with VaildateTargetMixin

* fix: Removed unused endpoint

* fix: secure SessionActions serializer

* fix: Removed pset_download

* fix: Design set upload now unsupported (404)

* fix: Snapshots now open again

* fix: CompoundIdentifierTypeView & TagCategoryView now read-only views

* fix: Discourse POST now requires login

* feat: User now needs to be a member of CSET target to download it

* fix: secured TaskStatus endpoint

* feat: Removal of unsed xcdb app

* feat: Add log to use of dicttocsv

* feat: More secure DictToCsv

* feat: More consistent use of _ISPYB_SAFE_QUERY_SET

* feat: Stricter UploadCSet class inheritance

* feat: Fix isort issues

* feat: Fix ListAPIView

* feat: Remove references to xcdb

* fix: secure UploadTaskView and ValidateTaskView

TODO: secure UpdateTaskView (if used)

* Align 1247 with latest staging code (#616)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

---------

Co-authored-by: Kalev Takkis <[email protected]>

* Align 1247 from staging (#619)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

---------

Co-authored-by: Kalev Takkis <[email protected]>

* Align 1247 with latest staging (#620)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

---------

Co-authored-by: Kalev Takkis <[email protected]>

* feat: Rstore CSetUpload post()

* feat: Revert UploadCSet inheritance

* fix: Another attempt to fix UploadCSet

* fix: Another attmept to fix the view

* fix: Anotehr attempt to get UploadCSet

* feat: Fix UploadCSet view

* feat: Fix JobRequest GET (restrict to members of the project)

* feat: Enhanced logging for membership check failures

* docs: Improve docs relating to security

* docs: Minor typo

* fix: Remove TEST_RESTRICTED_TAS_LIST feature

* Align 1247 with staging (#621)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

---------

Co-authored-by: Kalev Takkis <[email protected]>

* Align 1247 with latest staging (#623)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

---------

Co-authored-by: Kalev Takkis <[email protected]>

* fix: Attempt to fix PoseView (now Pose)

* fix: Attempt to debug Pose failure

* fix: Another patch to Pose

* fix: Fix log typo

* fix: Attempt to fix permission on create

* fix: Fix for ValidateTargetMixin?

* fix: Better Mixin (renamed and copes with shortest filter string)

* fix: Fix some project mixin views (includes some renaming)

* refactor: View name consistency

* fix: Fix for targetdownload mixin (and extra log)

* fix: Better file handling

* fix: Now serches ExpUpload for first matching record

* fix: Better experiment download (use of only ExpUpload record)

* fix: ExpDownload now inspects Project

* fix: More naming consistency changes

* fix: Attempt to fix  'ManyRelatedManager' is not iterable

* fix: Use of correct download path

* Align 1247 with staging (#624)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

---------

Co-authored-by: Kalev Takkis <[email protected]>

* Align 1247 with staging (#627)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

---------

Co-authored-by: Kalev Takkis <[email protected]>

* docs: Tweak messages

* fix: Better file handling

* docs: Doc tweak

* Merge compound fix to 1247 (#628)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

* fix: Branch for project reference fix

* fix: Projects copied from Target (during RHS cset-upload)

---------

Co-authored-by: Kalev Takkis <[email protected]>
Co-authored-by: Alan Christie <[email protected]>

* refactor: restrict_to_membership now restrict_pubic_to_membership

* fix: ValidateProjectMixin does not insist on public proposal membership for GET

* fix: Apply conflict from staging

* Align 1247 with staging (#631)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

* fix: updates to tag generation

Changed how some of the tags are generated as per the comment here: https://github.com/m2ms/fragalysis-frontend/issues/1482#issuecomment-2248079966

* feat: added centroid_res field to CanonSite model

Also, removed fetching centroid_res from CANON_SITES_FILE. Seems that
now it's being added to meta_aligner.yaml, so reading an additional
file is not necessary. I hope...

* feat: added new fields to metadata.csv

Experiment code and centroid res

* feat: added tag aliases to metadata.csv

* Copies Target proposals to new (RHS) Compounds (#629)

* fix: Branch for project reference fix

* fix: Projects copied from Target (during RHS cset-upload)

* fix: Add save before copying projects

* fix: Remove unnecessary save()

* ci: Attempt to fix docker-compose problem

* ci: Fix staging and production builds (docker compose)

---------

Co-authored-by: Alan Christie <[email protected]>

---------

Co-authored-by: Kalev Takkis <[email protected]>
Co-authored-by: Alan Christie <[email protected]>

* fix: Fix project_id type (aligns with staging)

* Align 1247 with staging (#633)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

* fix: updates to tag generation

Changed how some of the tags are generated as per the comment here: https://github.com/m2ms/fragalysis-frontend/issues/1482#issuecomment-2248079966

* feat: added centroid_res field to CanonSite model

Also, removed fetching centroid_res from CANON_SITES_FILE. Seems that
now it's being added to meta_aligner.yaml, so reading an additional
file is not necessary. I hope...

* feat: added new fields to metadata.csv

Experiment code and centroid res

* feat: added tag aliases to metadata.csv

* Copies Target proposals to new (RHS) Compounds (#629)

* fix: Branch for project reference fix

* fix: Projects copied from Target (during RHS cset-upload)

* fix: Add save before copying projects

* fix: Remove unnecessary save()

* ci: Attempt to fix docker-compose problem

* ci: Fix staging and production builds (docker compose)

---------

Co-authored-by: Alan Christie <[email protected]>

* fix: Fix typo accessing Target projects (#632)

---------

Co-authored-by: Kalev Takkis <[email protected]>
Co-authored-by: Alan Christie <[email protected]>

---------

Co-authored-by: Alan Christie <[email protected]>
Co-authored-by: Kalev Takkis <[email protected]>

* fix: Adds variable to context

* fix: Fix variable order

* style: Add note and display template and its context (#637)

Co-authored-by: Alan Christie <[email protected]>

* fix: fixed tag loading management command

* fix: fixed tag loader management command some more

* feat: Add support for RESTRICTED_TAS_USERS (#640)

Co-authored-by: Alan Christie <[email protected]>

* Better handling of project path AttributeError (#641)

* fix: Better handling of project path

* fix: Now also dumps vars(base_start_object)

---------

Co-authored-by: Alan Christie <[email protected]>

* fix: Protect permissions from bad objects (#642)

Co-authored-by: Alan Christie <[email protected]>

* fix: Fix merge issue (#643)

Co-authored-by: Alan Christie <[email protected]>

* Relaxes authentication for SessionProject (#644)

* refactor: Explicit error is there are no Projects

* chore: Removed commented-out code

* fix: Relaxed suth for SessionProject

---------

Co-authored-by: Alan Christie <[email protected]>

* refactor: Now displays object class name (#645)

Co-authored-by: Alan Christie <[email protected]>

* Add additional context to the API error (#646)

* style: Additional context debug (view name etc.)

* style: Add traceback

---------

Co-authored-by: Alan Christie <[email protected]>

* stashing

* Relaxed auth for SessionActionsView & SnapshotActionsView (#647)

* fix: Relaxed auth for SessionActionsView & SnapshotActionsView

* fix: Removed ValidateProjectMixin from serializers

---------

Co-authored-by: Alan Christie <[email protected]>

* fix: fix subsequent upload file path overwriting bug (1492)

Introduced with a fix to 1311, all subsequent uploads overwrote
previous version paths with their own root directory and resulting
paths didn't point to any actual file. Simplified LHS data structure
so the update mechanism remains in place (as it should, 1311 had it's
function) but the root directory for all versions stays the same.

TODO: fix path lookups for downloads and RHS uploads

* fix: Relax auth on SessionProjectTag & SiteObservationTag (#648)

Co-authored-by: Alan Christie <[email protected]>

* fix: Can now download public RHS without being logged int (#649)

Co-authored-by: Alan Christie <[email protected]>

* feat: updated LHS downloads to new data schema

* fix: log compound names when alignment is not found (issue 1495)

Currently breaks on first fail, does not process the full list and
report back all offending molecules.

* stashing

* fix: sort canon sites by number of site observations

* stashing

* fix: fixed issue 1504, missing yaml files

* fix: shortened crystalform sites tags

* feat: added code prefix to experiment model

* feat: updated both short- and longcode

Shortcode dropped the 'x' character, longcode was made shorter with
'v' as an version indicator

* fix: fixed tagging issues

Introduced by canon site sorting, issue 1498

* fix: fixed remaining tag issues

* fix: the return of the ribbon

Ribbon was missing because the 'template_protein' attribute was
missing, because the path was compiled incorrectly and file was not
found. Template protein file is not explicitly given in metadata, so
trial-error method is used.

* stashing

* fix: a bug in target loader's create_objects func

catch block was wrongly positioned and may have been masking error messages

* feat: changed tag name format

/ instead of + and _

* fix: restore pose column to metadata.csv (#663)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kalev Takkis <[email protected]>
Co-authored-by: Alan Christie <[email protected]>
Co-authored-by: Kalev Takkis <[email protected]>
Co-authored-by: Warren Thompson <[email protected]>
Assets 2
Loading

Initial production release candidate

13 Sep 11:17
@alanbchristie alanbchristie
2024.09.1
e911269
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Initial production release candidate 
Restore pose column in metadata.csv (#664)

* build(deps): bump django from 3.2.24 to 3.2.25

Bumps [django](https://github.com/django/django) from 3.2.24 to 3.2.25.
- [Commits](https://github.com/django/django/compare/3.2.24...3.2.25)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* stashing

* Target loader now accepts experiments marked as 'manual'

* build(deps): bump pillow from 10.2.0 to 10.3.0

Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.2.0 to 10.3.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/10.2.0...10.3.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Attempt to reduce pop-up "flicker" (1403) (#569)

* fix: Attempt to debug timeout errors

* fix: More logging on service_query

* fix: Varioustimeout adjustments

* fix: Removed exception during timeout

* fix: Explit log on SSH connection error

* fix: Retry attempts for MySQL connections

* fix: Service timeout now 28 (was 17)

* fix: Add pymysql read and write timeouts

* fix: Quiter (expected) connection failure handling

* fix: TIMEOUT now DEGRADED

* fix: Fix while loop exit conditions

* fix: Better loop logic

* style: services logging reduced and back to debug

* fix: SSHTunnel logging now ERROR (was DEBUG)

* fix: Quieter securty

* fix: More failures permitted (and debug tweaks)

* fix: Leaner logging

* fix: Leaner logging (only report when we're having topruble)

* fix: Better constant name

* fix: Reduced service logging

* docs: Doc tweak

* fix: Minor log tweak

* fix: Fixed duplicate log content

---------

Co-authored-by: Alan Christie <[email protected]>

* feat: endpoint to download first reference pdb from assemblies.yaml

* stashing

* stashing

Changes so far:
- removed endpoint FirstAssemblyview
- moved the functionality to template_protein field in
TargetSerializer
- removed TargetMoleculesserializer
- removed sequences field from TargetSerializer

This is a result of Boris' comment in
github (https://github.com/m2ms/fragalysis-frontend/issues/1373#issuecomment-2047417958)
where he said the field isn't used and template_protein field is not
used. Looking at the code where this may be used, revealed that
Targetmoleculesserializer can be removed as well

NB! they're not removed-removed right now, only commented in. This
commit can be used to restore the code.

* fix: removed code mentioned in previous commit

* basic functionality

TODO: add and test PATCH method on PoseSerializer

* stashing

* build(deps): bump idna from 3.6 to 3.7

Bumps [idna](https://github.com/kjd/idna) from 3.6 to 3.7.
- [Release notes](https://github.com/kjd/idna/releases)
- [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst)
- [Commits](https://github.com/kjd/idna/compare/v3.6...v3.7)

---
updated-dependencies:
- dependency-name: idna
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* stashing

* Added 'hidden' property to viewer.Tag model

Also, added exclude directives to .pre-commit-config.yaml to not touch
django migration files

* Better user/proposal cache (#576)

* refactor: Quieter ISPyB cache

* refactor: Security cache log now quieter

* fix: Better cache logic - and reduced log

---------

Co-authored-by: Alan Christie <[email protected]>

* stashing

Nested serializer for pose, but I don't think I can use them

* feat: added compound_code to pose serializer

* fix: deeper nesting level to site observaton in meta_aligner.yaml

Data loads successfully but actual v2 upload has not been tested

* fix: renamed panddas_event_files to ligand_binding_events in meta_al

* fix: static resources are now loaded again

Fixes bare html API pages

* fix: fixed nginx config

* Revert "Fix static ressources not being loaded"

* stashing

Added sorting keys for versioned key. V1 data loading, waiting for
fixes in conf site presentation to continue with v2

* fix: more robust update method

Allows sending incomplete requests (no idea why seralizer isn't
populating fields).
Also fixed a bug where field updates on poses with multiple
observations where blocked.

* feat: added pose tags

* stashing

Working on allowing incomplete requests payloads. Turns out to be
quite tricky, may have to go back on this.

* feat: fully functional versioned data

Reads and processes upload_2+ data where version numbers are given in suffix

* fix: removed some dead code

* stashing

* build(deps): bump tqdm from 4.66.1 to 4.66.3

Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.1 to 4.66.3.
- [Release notes](https://github.com/tqdm/tqdm/releases)
- [Commits](https://github.com/tqdm/tqdm/compare/v4.66.1...v4.66.3)

---
updated-dependencies:
- dependency-name: tqdm
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* feat: only non-superseded sites available from the api

* build(deps-dev): bump black from 23.12.0 to 24.3.0

Bumps [black](https://github.com/psf/black) from 23.12.0 to 24.3.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/23.12.0...24.3.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>

* chore: merged conflicting migrations from issue 1311 branch

* fix: pose instance attributes updating successfully

* fix: better check for already uploaded data

* fix: merge conflicting migrations

Not sure why the previous merge didn't work

* Adds basic metrics (#588)

* feat: Experiment with django prometheus

* feat: Fix build

* fix: Fix build (locking drf)

* feat: Fix lock file

* feat: Update to non-slim Python 3.11.9

* feat: Back to slim image

* fix: Some basic internal metrics

* fix: Removed rogue line

* fix: Fix lint issues

* fix: Removed custom metrics

---------

Co-authored-by: Alan Christie <[email protected]>

* Attempt to add prometheus to DB (#589)

Co-authored-by: Alan Christie <[email protected]>

* feat: service state queries to backend

Instead of a frontend call triggering a query to each service in every
30 seconds or so, ping the services in the backend, store the results
in a table and serve this up in an endpoint. Seems that when each
user's frontend was testing ISPyB, it was enough to grind it to a
halt.

TODO:
- user friendly names of services
- separate compose and settings file to enable launching celery and
  beat separately (and make sure backend still works when not)
- delete old service query stuff
- most serious one - creating services on startup in apps.py, weird
  issue with service table supposedly not existing in db

* feat: service state queries to backend

Refactored a bit and fixed an issue with automatically starting
services on startup. External service queries fixed.

TODO:
- separate compose file
- delete old services stuff

* Add custom (security) metrics (#593)

* feat: Re-attempt to add custom metrics

* feat: New metric (ISpyB connection attempts)

* feat: Add cache hit/miss metrics

* feat: Leaner metrics (and initial dashboard)

* feat: Metrics now initialised

* docs: Updated dashboard

* fix: Attempt to fix hit/miss metrics

* docs: New dashboard

* docs: Dashboard tweak

---------

Co-authored-by: Alan Christie <[email protected]>

* feat: service state queries to backend

Separate compose file

TODO:
- remove old services

* fix: Fixed connection retry metric (#594)

Co-authored-by: Alan Christie <[email protected]>

* feat: service state queries to backend

Completed, updated readme and deleted old services.

TODO:
- follow-up issue 1359, better ISPyB query

* fix: missing migration and pylint ignore

* fix: remove auto-generating pose tags

* fix: Fix for tunnel connection failure metrics (#597)

Co-authored-by: Alan Christie <[email protected]>

* fix: Adjust metrics (tunnels and cache) (#598)

Co-authored-by: Alan Christie <[email protected]>

* fix: service test functions now scheduled correctly

Minor tweaks to test functions themselves
Start/stop management command fully functional

* fix: changed canon site autogenerated tag name

* docs: Updated grafana dashboard

* feat: filename attribute to RHS download serializer

* fix: update observation's longcode template

* Adjust logging for cache debug (#603)

* refactor: Adds extra logging in proposal extraction

* style: Initial cache now a set rather than empty list

---------

Co-authored-by: Alan Christie <[email protected]>

* Fix connection failure logic (#604)

* refactor: Adds extra logging in proposal extraction

* style: Initial cache now a set rather than empty list

* fix: Fix cache collection failure logic

---------

Co-authored-by: Alan Christie <[email protected]>

* Reduced security logging (#605)

* refactor: Adds extra logging in proposal extraction

* style: Initial cache now a set rather than empty list

* fix: Fix cache collection failure logic

* fix: Reduced logging for proposal cache

---------

Co-authored-by: Alan Christie <[email protected]>

* feat: add management commands to save and restore curated tags

In app container either
curated_tags --dump tags.json
or
curated_tags --load tags.json

* feat: incremental RHS upload (issue 1394)

Allows upload. Currently adds unnecessary compound sets and overwrites
old ones when it doesn't need to

* feat: incremental RHS uploads

Fully working, upload, delete, overwrite, etc.
NB! migrations!

* fix: set ConformerSites and CrystalformSites tags to hidden by default

* fix: code cleanup for merge

* fix: updated crystalform site tag generation scheme

* Initial 'security' changes - Target PATCH now "locked down" (#612)

* fix: Adds extra log to Targets view

* fix: More target logging

* fix: safe query set now derived from ModelViewSet

* feat: Experiment with IsProposalMember class

* fix: Another permission tweak

* fix: Another permission tweak

* fix: Typo

* fix: Fixed silly typo

* fix: Display view filter_permissions

* fix: Experimental new has_permission

* fix: Add user_is_member_of_any_given_proposals to security

* fix: Change to filter class name

* fix: Better has_object_permission

* feat: Leaner has_object_permission

* docs: Minor doc tweak

* fix: Experiment with SafeQuerySet

* fix: Experiment with ModelViewSet in ISPYB

* fix: Back to read-only viewset (and reduced log)

* fix: User must be authenticated

* docs: Doc tweak

* feat: Experiment with filter class for Target view

* fix: Attempt to fix build errors

* fix: Switch to filter_class

* fix: Attempt to fix filter logging

* fix: Another tweak of filters

* feat: Fix filerset typo

* fix: Back to built-in (genric) views

* feat: Attempt to fix has no attribute 'get_extra_actions'

* fix: Silly typo

* fix: Back to ispybsafequeryset

* fix: Restore queryset

* fix: Experiment with mixins

* docs: Doc tweaks

* docs: Doc tweak

* fix: Switch to update() from patch()

* fix: Back to patch()

* refactor: Minor refactor

* feat: Better permissions class for proposals

* Align 1247 with latest staging (#611)

* feat: incremental RHS upload (issue 1394)

Allows upload. Currently adds unnecessary compound sets and overwrites
old ones when it doesn't need to

* feat: incremental RHS uploads

Fully working, upload, delete, overwrite, etc.
NB! migrations!

* fix: code cleanup for merge

---------

Co-authored-by: Kalev Takkis <[email protected]>

---------

Co-authored-by: Alan Christie <[email protected]>
Co-authored-by: Kalev Takkis <[email protected]>

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

* fix: updates to tag generation

Changed how some of the tags are generated as per the comment here: https://github.com/m2ms/fragalysis-frontend/issues/1482#issuecomment-2248079966

* feat: added centroid_res field to CanonSite model

Also, removed fetching centroid_res from CANON_SITES_FILE. Seems that
now it's being added to meta_aligner.yaml, so reading an additional
file is not necessary. I hope...

* feat: added new fields to metadata.csv

Experiment code and centroid res

* feat: added tag aliases to metadata.csv

* Copies Target proposals to new (RHS) Compounds (#629)

* fix: Branch for project reference fix

* fix: Projects copied from Target (during RHS cset-upload)

* fix: Add save before copying projects

* fix: Remove unnecessary save()

* ci: Attempt to fix docker-compose problem

* ci: Fix staging and production builds (docker compose)

---------

Co-authored-by: Alan Christie <[email protected]>

* fix: Fix typo accessing Target projects (#632)

* fix: Add support for target_warning_message context variable (#634)

Co-authored-by: Alan Christie <[email protected]>

* API authentication and security changes (#635)

* style: Reorder functions in the module

* refactor: Moved non-view functions to viewer.utils

* fix: Removed unused save_pdb_zip and minor refactoring

* feat: Removed circular import

* feat: Fix get_open_targets (also get_open_proposals now not _private_)

* feat: Fix get_open_proposals reference

* refactor: ISpyB -> ISPyB

* docs: Updated for use of mixins

* feat: More API security migrations

* feat: More security migrations

* feat: Security migrations for hotspots and hypothesis

* feat: More security fixes

* feat: More security changes

* feat: More security changes (and get_params -> get_img_from_smiles with default w/h)

* fix: Attempt to fix calls to /xcdb/fragspect/ 500 errors

* feat: Another attempot to fix ISPyB

* feat: Use of new user_is_member_of_target()

* feat: Experiment with validator

* feat: Better serializer log

* feat: Even more work on the serializer

* feat: Minor error message tweak

* feat: Add support for TEST_RESTRICTED_TAS_LIST (#614)

Co-authored-by: Alan Christie <[email protected]>

* target permission validation mixin pattern implemented for Pose

* feat: Fix restricted logic

* most endpoints secured with VaildateTargetMixin

* fix: Removed unused endpoint

* fix: secure SessionActions serializer

* fix: Removed pset_download

* fix: Design set upload now unsupported (404)

* fix: Snapshots now open again

* fix: CompoundIdentifierTypeView & TagCategoryView now read-only views

* fix: Discourse POST now requires login

* feat: User now needs to be a member of CSET target to download it

* fix: secured TaskStatus endpoint

* feat: Removal of unsed xcdb app

* feat: Add log to use of dicttocsv

* feat: More secure DictToCsv

* feat: More consistent use of _ISPYB_SAFE_QUERY_SET

* feat: Stricter UploadCSet class inheritance

* feat: Fix isort issues

* feat: Fix ListAPIView

* feat: Remove references to xcdb

* fix: secure UploadTaskView and ValidateTaskView

TODO: secure UpdateTaskView (if used)

* Align 1247 with latest staging code (#616)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

---------

Co-authored-by: Kalev Takkis <[email protected]>

* Align 1247 from staging (#619)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

---------

Co-authored-by: Kalev Takkis <[email protected]>

* Align 1247 with latest staging (#620)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

---------

Co-authored-by: Kalev Takkis <[email protected]>

* feat: Rstore CSetUpload post()

* feat: Revert UploadCSet inheritance

* fix: Another attempt to fix UploadCSet

* fix: Another attmept to fix the view

* fix: Anotehr attempt to get UploadCSet

* feat: Fix UploadCSet view

* feat: Fix JobRequest GET (restrict to members of the project)

* feat: Enhanced logging for membership check failures

* docs: Improve docs relating to security

* docs: Minor typo

* fix: Remove TEST_RESTRICTED_TAS_LIST feature

* Align 1247 with staging (#621)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

---------

Co-authored-by: Kalev Takkis <[email protected]>

* Align 1247 with latest staging (#623)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

---------

Co-authored-by: Kalev Takkis <[email protected]>

* fix: Attempt to fix PoseView (now Pose)

* fix: Attempt to debug Pose failure

* fix: Another patch to Pose

* fix: Fix log typo

* fix: Attempt to fix permission on create

* fix: Fix for ValidateTargetMixin?

* fix: Better Mixin (renamed and copes with shortest filter string)

* fix: Fix some project mixin views (includes some renaming)

* refactor: View name consistency

* fix: Fix for targetdownload mixin (and extra log)

* fix: Better file handling

* fix: Now serches ExpUpload for first matching record

* fix: Better experiment download (use of only ExpUpload record)

* fix: ExpDownload now inspects Project

* fix: More naming consistency changes

* fix: Attempt to fix  'ManyRelatedManager' is not iterable

* fix: Use of correct download path

* Align 1247 with staging (#624)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

---------

Co-authored-by: Kalev Takkis <[email protected]>

* Align 1247 with staging (#627)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

---------

Co-authored-by: Kalev Takkis <[email protected]>

* docs: Tweak messages

* fix: Better file handling

* docs: Doc tweak

* Merge compound fix to 1247 (#628)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

* fix: Branch for project reference fix

* fix: Projects copied from Target (during RHS cset-upload)

---------

Co-authored-by: Kalev Takkis <[email protected]>
Co-authored-by: Alan Christie <[email protected]>

* refactor: restrict_to_membership now restrict_pubic_to_membership

* fix: ValidateProjectMixin does not insist on public proposal membership for GET

* fix: Apply conflict from staging

* Align 1247 with staging (#631)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

* fix: updates to tag generation

Changed how some of the tags are generated as per the comment here: https://github.com/m2ms/fragalysis-frontend/issues/1482#issuecomment-2248079966

* feat: added centroid_res field to CanonSite model

Also, removed fetching centroid_res from CANON_SITES_FILE. Seems that
now it's being added to meta_aligner.yaml, so reading an additional
file is not necessary. I hope...

* feat: added new fields to metadata.csv

Experiment code and centroid res

* feat: added tag aliases to metadata.csv

* Copies Target proposals to new (RHS) Compounds (#629)

* fix: Branch for project reference fix

* fix: Projects copied from Target (during RHS cset-upload)

* fix: Add save before copying projects

* fix: Remove unnecessary save()

* ci: Attempt to fix docker-compose problem

* ci: Fix staging and production builds (docker compose)

---------

Co-authored-by: Alan Christie <[email protected]>

---------

Co-authored-by: Kalev Takkis <[email protected]>
Co-authored-by: Alan Christie <[email protected]>

* fix: Fix project_id type (aligns with staging)

* Align 1247 with staging (#633)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

* fix: updates to tag generation

Changed how some of the tags are generated as per the comment here: https://github.com/m2ms/fragalysis-frontend/issues/1482#issuecomment-2248079966

* feat: added centroid_res field to CanonSite model

Also, removed fetching centroid_res from CANON_SITES_FILE. Seems that
now it's being added to meta_aligner.yaml, so reading an additional
file is not necessary. I hope...

* feat: added new fields to metadata.csv

Experiment code and centroid res

* feat: added tag aliases to metadata.csv

* Copies Target proposals to new (RHS) Compounds (#629)

* fix: Branch for project reference fix

* fix: Projects copied from Target (during RHS cset-upload)

* fix: Add save before copying projects

* fix: Remove unnecessary save()

* ci: Attempt to fix docker-compose problem

* ci: Fix staging and production builds (docker compose)

---------

Co-authored-by: Alan Christie <[email protected]>

* fix: Fix typo accessing Target projects (#632)

---------

Co-authored-by: Kalev Takkis <[email protected]>
Co-authored-by: Alan Christie <[email protected]>

---------

Co-authored-by: Alan Christie <[email protected]>
Co-authored-by: Kalev Takkis <[email protected]>

* fix: Adds variable to context

* fix: Fix variable order

* style: Add note and display template and its context (#637)

Co-authored-by: Alan Christie <[email protected]>

* fix: fixed tag loading management command

* fix: fixed tag loader management command some more

* feat: Add support for RESTRICTED_TAS_USERS (#640)

Co-authored-by: Alan Christie <[email protected]>

* Better handling of project path AttributeError (#641)

* fix: Better handling of project path

* fix: Now also dumps vars(base_start_object)

---------

Co-authored-by: Alan Christie <[email protected]>

* fix: Protect permissions from bad objects (#642)

Co-authored-by: Alan Christie <[email protected]>

* fix: Fix merge issue (#643)

Co-authored-by: Alan Christie <[email protected]>

* Relaxes authentication for SessionProject (#644)

* refactor: Explicit error is there are no Projects

* chore: Removed commented-out code

* fix: Relaxed suth for SessionProject

---------

Co-authored-by: Alan Christie <[email protected]>

* refactor: Now displays object class name (#645)

Co-authored-by: Alan Christie <[email protected]>

* Add additional context to the API error (#646)

* style: Additional context debug (view name etc.)

* style: Add traceback

---------

Co-authored-by: Alan Christie <[email protected]>

* stashing

* Relaxed auth for SessionActionsView & SnapshotActionsView (#647)

* fix: Relaxed auth for SessionActionsView & SnapshotActionsView

* fix: Removed ValidateProjectMixin from serializers

---------

Co-authored-by: Alan Christie <[email protected]>

* fix: fix subsequent upload file path overwriting bug (1492)

Introduced with a fix to 1311, all subsequent uploads overwrote
previous version paths with their own root directory and resulting
paths didn't point to any actual file. Simplified LHS data structure
so the update mechanism remains in place (as it should, 1311 had it's
function) but the root directory for all versions stays the same.

TODO: fix path lookups for downloads and RHS uploads

* fix: Relax auth on SessionProjectTag & SiteObservationTag (#648)

Co-authored-by: Alan Christie <[email protected]>

* fix: Can now download public RHS without being logged int (#649)

Co-authored-by: Alan Christie <[email protected]>

* feat: updated LHS downloads to new data schema

* fix: log compound names when alignment is not found (issue 1495)

Currently breaks on first fail, does not process the full list and
report back all offending molecules.

* stashing

* fix: sort canon sites by number of site observations

* stashing

* fix: fixed issue 1504, missing yaml files

* fix: shortened crystalform sites tags

* feat: added code prefix to experiment model

* feat: updated both short- and longcode

Shortcode dropped the 'x' character, longcode was made shorter with
'v' as an version indicator

* fix: fixed tagging issues

Introduced by canon site sorting, issue 1498

* fix: fixed remaining tag issues

* fix: the return of the ribbon

Ribbon was missing because the 'template_protein' attribute was
missing, because the path was compiled incorrectly and file was not
found. Template protein file is not explicitly given in metadata, so
trial-error method is used.

* stashing

* fix: a bug in target loader's create_objects func

catch block was wrongly positioned and may have been masking error messages

* feat: changed tag name format

/ instead of + and _

* fix: restore pose column to metadata.csv (#663)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kalev Takkis <[email protected]>
Co-authored-by: Alan Christie <[email protected]>
Co-authored-by: Kalev Takkis <[email protected]>
Co-authored-by: Warren Thompson <[email protected]>
Assets 2
Loading

Significant bug-fix, feature, and metrics changes

21 May 13:17
@alanbchristie alanbchristie
2024.05.1
d4681de
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Significant bug-fix, feature, and metrics changes

Alkigns production from the latest staging branch

Assets 2
Loading

Add support for private key SSH tunnels

24 Apr 10:16
@alanbchristie alanbchristie
2023.09.1-1
e71c89d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Add support for private key SSH tunnels
  • Ports logic back from the V2 production code
Assets 2
Loading

Improved SSHTunnel & MySQL handling

05 Apr 16:42
@alanbchristie alanbchristie
2024.04.1
3f59569
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Improved SSHTunnel & MySQL handling
  • Improved timeouts
  • Implementation of a MySQL connection retry
  • Adds MySQL read timeout
  • Improved logging (now recording, succinctly, MySQL connection issues)
  • No more TIMEOUT state, services are either OK or DEGRADED
Assets 2
Loading

A replacement for 2023.03.4

22 Mar 15:06
@alanbchristie alanbchristie
2024.03.5
ee5fd26
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
A replacement for 2023.03.4
  • 2023.03.4 was a tag on staging, not production
Assets 2
Loading

2024.03.4 (Kalev's metadata fix)

22 Mar 14:24
@mwinokan mwinokan
2024.03.4
45f38e0
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
2024.03.4 (Kalev's metadata fix)

What's Changed

  • Some changes to cset_upload.py to allow site observation short codes by @kaliif in #527
  • Short code prefix and tooltip read from yaml file (issue 1334) by @kaliif in #528
  • New format to download zip (issue 1326) by @kaliif in #530
  • cset_upload.py: lhs_pdb renamed to ref_pdb (issue 1327) by @kaliif in #531
  • Renamed canon- and conf site tags by @kaliif in #532
  • Adds support for key-based SSH connections by @alanbchristie in #534
  • build(deps): bump cryptography from 42.0.0 to 42.0.2 by @dependabot in #533
  • Updates documentation for 1328 by @alanbchristie in #536
  • build(deps): bump django from 3.2.20 to 3.2.24 by @dependabot in #535
  • Reverting unnecessary changes re RHS upload by @kaliif in #538
  • Download zip; ligand sdfs now in aligned_files not in missing_sdfs by @kaliif in #540
  • RHS upload backend side functional by @kaliif in #541
  • Increased loglevel to error on unexpected exceptions block by @kaliif in #542
  • Discourse service check now checks API key by @alanbchristie in #544
  • build(deps): bump cryptography from 42.0.2 to 42.0.4 by @dependabot in #539
  • Changes metadata.csv format in download and adds 'upload_name' attribute to tag model (issues 1343 and 1355) by @kaliif in #546
  • Updates to api/download_structures endpoint (issue 1326) by @kaliif in #547
  • Added parsing directives to DownloadStructuresserializer by @kaliif in #548
  • Consecutive numbering of observations under canon site by @kaliif in #549
  • Issue 1326, crystallographic_files folders in download now sans suffix by @kaliif in #550
  • SiteObservatdion.tag split to tag and tag_prefix (1361) by @kaliif in #551
  • Tag names understand prefix in download's metadata.csv by @kaliif in #552
  • Return all proteins listed in api/download_structure (issue 1375) by @kaliif in #553
  • Fixed 'All structures' option not working in download dialog by @kaliif in #554
  • Restore 'upload_name' field in SiteObservationTags to prefix-tag format by @kaliif in #556
  • Serve all uploaded files in download by @kaliif in #557
  • Allow empty code_prefix and tooltip by @kaliif in #558
  • Add force_error_display to connection functions (default False) by @alanbchristie in #559
  • Align staging with production by @alanbchristie in #560
  • Add DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP by @alanbchristie in #561
  • Metadata.csv in download now showing correct tags by @kaliif in #563
  • Fixed tag creation process for multiple uploads by @kaliif in #565

Full Changelog: 2024.03.3...2024.03.4

Contributors

kaliif, dependabot, and alanbchristie
Assets 2
Loading