Refactor TreeRankRecord to allow null tree id

[sharadsw]

Fixes #6321

The cause for the bug was that TreeRankRecord always expected a treeId and in cases where no treeId was found it would determine the treedef to upload to in a very arbitrary way

specify7/specifyweb/workbench/upload/treerecord.py

Lines 124 to 133 in 166bad8

	if treedef_id is None:
	TreeNodeModel = getattr(models, tree)
	treedefitem_ids = treedefitems.filter(name=rank_name).values_list('id', flat=True)
	treedefitem_id = sorted(
	treedefitem_ids,
	key=lambda x: TreeNodeModel.objects.filter(definitionitem_id=x).count(),
	reverse=True,
	)[0]
	ranks = treedefitems.filter(id=treedefitem_id)
	return ranks

After this PR, TreeRankRecord will allow a null treeid and upload will default to the collection's default treedef after the TreeRecord has been scoped. Additionally, the frontend will pass a treeid for all tree tables and not just for Taxon.

The PR should handle all the following cases:

• New datasets will always have a treeId
• Datasets after 7.10 and before 7.10.2 would have faced this issue only for non-Taxon trees but this PR will ensure uploads happen to the default tree of the collection
• Datasets before 7.10 that don't use TreeRankRecord in the upload plan will also have nodes uploaded to the default tree

Checklist

• Self-review the PR after opening it to make sure the changes look good and
  self-explanatory (or properly documented)
• Add automated tests
• Add relevant issue to release milestone
• Add relevant documentation (Tester - Dev)
• Add a reverse migration if a migration is present in the PR

Testing instructions

Tests with single Taxon tree in the db

• Create a WB dataset with Taxon base table

• Map a single rank to a column

• Enter data, verify upload and rollback

• Map multiple ranks in the same dataset

• Enter data in all/some ranks, verify upload and rollback

  • Ensure the entire 'path' gets uploaded correctly
    

• Repeat above steps with a non Taxon base table (Geography, Storage, geo/paleo trees)

• Create a WB dataset with CollectionObject base table

• Map a column to Taxon and to a non-Taxon table in the same dataset

  • Eg: CO -> Determination -> Taxon and CO -> CE -> Locality -> Geography

• Enter data, verify upload and rollback

• Use multiple ranks in the dataset for all trees

• Enter data in all/some ranks, verify upload and rollback

  • Ensure the entire 'path' gets uploaded/matched correctly
  • NOTE: 'Matched' refers to cases where a path already exists in the tree. Eg: North and Central America -> United States -> South Carolina -> Dorchester already exists in the Geography tree and doesn't get highlighted

Tests with multiple Taxon trees in the db

• Add a Taxon tree to the db and repeat all above steps
• Create a COType
• Create a dataset with CO as base table
• Map columns to COType, Determination -> Taxon, CE -> Locality -> Geography
• Enter data in all/some ranks, verify upload and rollback
• Verify there is a validation error when a COType does not belong to the same tree as the Taxon rank
• Enter data to ranks that belong to different trees in the same row
  • Verify there is a validation error for Multiple tree definitions in row
    

Test old datasets

• Re-run tests on datasets that were created before 7.10.1 (saiab_2025_03_12 has some old datasets that can be used)

[sharadsw]

TODO: Need to handle old datasets with 'incorrect' upload plans

[sharadsw]

A gist of the relevant changes since the PR refactors a lot of things

[emenslin]

• Enter data, verify upload and rollback

• Enter data in all/some ranks, verify upload and rollback

  • Ensure the entire 'path' gets uploaded correctly

• Repeat above steps with a non Taxon base table (Geography, Storage, geo/paleo trees)

• Enter data, verify upload and rollback

• Enter data in all/some ranks, verify upload and rollback

  • Ensure the entire 'path' gets uploaded/matched correctly

• Add a Taxon tree to the db and repeat all above steps

• Enter data in all/some ranks, verify upload and rollback

• Verify there is a validation error when a COType does not belong to the same tree as the Taxon rank

  • Verify there is a validation error for Multiple tree definitions in row

• Re-run tests on datasets that were created before 7.10.1 (saiab_2025_03_12 has some old datasets that can be used)

All trees except for taxon are being unmapped after save

03-18_08.54.mp4

https://ojsmnh20250314-issue-6321.test.specifysystems.org/specify/workbench/plan/132/

[sharadsw]

@emenslin Should be fixed now

[emenslin]

• Enter data, verify upload and rollback

• Enter data in all/some ranks, verify upload and rollback

  • Ensure the entire 'path' gets uploaded correctly

• Repeat above steps with a non Taxon base table (Geography, Storage, geo/paleo trees)

• Enter data, verify upload and rollback

• Enter data in all/some ranks, verify upload and rollback

  • Ensure the entire 'path' gets uploaded/matched correctly

• Add a Taxon tree to the db and repeat all above steps

• Enter data in all/some ranks, verify upload and rollback

• Verify there is a validation error when a COType does not belong to the same tree as the Taxon rank

  • Verify there is a validation error for Multiple tree definitions in row

• Re-run tests on datasets that were created before 7.10.1 (saiab_2025_03_12 has some old datasets that can be used)

Looks good now, no trees are getting unmapped!

[combs-a]

Tests with single Taxon tree in the db

• Enter data, verify upload and rollback (Taxon)
• Enter data in all/some ranks, verify upload and rollback (Taxon)
  • Ensure the entire 'path' gets uploaded correctly (Taxon)
• Repeat above steps with a non Taxon base table (Geography, Storage, geo/paleo trees)
• Enter data, verify upload and rollback (CO)
• Enter data in all/some ranks, verify upload and rollback (CO)
  • Ensure the entire 'path' gets uploaded/matched correctly (CO)

Tests with multiple Taxon trees in the db

• Add a Taxon tree to the db and repeat all above steps
• Enter data in all/some ranks, verify upload and rollback (w/ COType)
• Verify there is a validation error when a COType does not belong to the same tree as the Taxon rank
  • Verify there is a validation error for Multiple tree definitions in row

Test old datasets

• Re-run tests on datasets that were created before 7.10.1 (saiab_2025_03_12 has some old datasets that can be used)

---

Didn't run into any major issues, the errors were showing up when they should. Good work!