Fix versification crash and missing versification table clean up

[pmachapman]

Fixes sillsdev/serval#830

---

This change is 

[codecov-commenter]

Codecov Report

❌ Patch coverage is 62.96296% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.30%. Comparing base (c271990) to head (5536c65).

Files with missing lines	Patch %	Lines
src/SIL.Machine/Corpora/UsxTextAlignmentCorpus.cs	0.00%	5 Missing ⚠️
...chine/Corpora/ParatextProjectSettingsParserBase.cs	73.33%	3 Missing and 1 partial ⚠️
src/SIL.Machine/Corpora/UsxFileTextCorpus.cs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #437      +/-   ##
==========================================
- Coverage   73.31%   73.30%   -0.01%     
==========================================
  Files         443      443              
  Lines       37190    37203      +13     
  Branches     5110     5110              
==========================================
+ Hits        27264    27272       +8     
- Misses       8800     8805       +5     
  Partials     1126     1126              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ddaspit]

@ddaspit reviewed 4 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on Enkidu93 and pmachapman).

---

src/SIL.Machine/Corpora/ParatextProjectSettingsParserBase.cs line 63 at r1 (raw file):

                else
                {
                    using (Lock.Lock())

Why do we only lock in this case?

[Enkidu93]

Thanks for cleaning this up! Any ideas what particular arrangement was causing our continued problems? I'm just wondering if it's a symptom of another problem - e.g. we're getting multiple copies of a corpus or project settings and we don't have to be? Or does this only happen when USFM is fetched multiple times at the same time for projects with the same custom versification (maybe the same source projects 🤔? or we have test projects with duplicate GUIDs?)?

@Enkidu93 reviewed 4 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on pmachapman).

[pmachapman]

Any ideas what particular arrangement was causing our continued problems?

The Versification.Table class maintains a static dictionary of unknown versfications. When we load a custom versification, this dictionary is modified, and so the Dictionary.Add() fails. This PR attempts to solve that by always clearing that dictionary after a versification is loaded, and to make sure that an out lock is used in ParatextProjectSettingsParserBase so that the logic that adds/removes form that dictionary is not executed concurrently.

@pmachapman made 2 comments.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on ddaspit).

---

src/SIL.Machine/Corpora/ParatextProjectSettingsParserBase.cs line 63 at r1 (raw file):

Previously, ddaspit (Damien Daspit) wrote…

Why do we only lock in this case?

As far as I can tell, this is the only place the code is called from Serval, and so may in certain circumstances be run concurrently. To effectively lock the other locations, we would need a global lock (Versification.Table.Implementation.Load() already has a lock within it), and I thought that might be overkill.

If you think we need a global lock, please let me know of a suitable class that you think would work for it (I was unsure of this).

[Enkidu93]

Yes, but only if it's a duplicate custom versification, right? As in the having the same project GUID, correct? I'm just wondering why we're loading the same custom versification more than once so frequently. I think we should put in these protections anyways, but I was just curious what circumstance was causing this. I'm guessing it's fetching the USFM for the same project that has a custom.vrs for different books?

It's a shame that they don't just call Exists within their own Load method while it's locked - what's the point in having a lock then 😅. Part of me does wonder if we're just causing unnecessary overhead/reloading with Versification.Table.Implementation.RemoveAllUnknownVersifications(). To Damien's point, couldn't we just lock the whole Exists-Load block and then not have to call this function at all?

@Enkidu93 reviewed 1 file and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on ddaspit).

[ddaspit]

@ddaspit reviewed 1 file and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on pmachapman).

---

src/SIL.Machine/Corpora/ParatextProjectSettingsParserBase.cs line 63 at r1 (raw file):

Previously, pmachapman (Peter Chapman) wrote…

As far as I can tell, this is the only place the code is called from Serval, and so may in certain circumstances be run concurrently. To effectively lock the other locations, we would need a global lock (Versification.Table.Implementation.Load() already has a lock within it), and I thought that might be overkill.

If you think we need a global lock, please let me know of a suitable class that you think would work for it (I was unsure of this).

I think we should lock across all places in Machine where we load a versification, so we need some type of shared lock. Also, the entire if/else should go in the lock.

[pmachapman]

art of me does wonder if we're just causing unnecessary overhead/reloading with Versification.Table.Implementation.RemoveAllUnknownVersifications(). To Damien's point, couldn't we just lock the whole Exists-Load block and then not have to call this function at all?

This method still needs to be called to clear the static Dictionary used as a cache internally to libpalaso. We clear it after use as we do not (and should not) be rely on or use the cache at all, given the multithreaded environment.

@pmachapman made 2 comments.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on ddaspit).

---

src/SIL.Machine/Corpora/ParatextProjectSettingsParserBase.cs line 63 at r1 (raw file):

Previously, ddaspit (Damien Daspit) wrote…

I think we should lock across all places in Machine where we load a versification, so we need some type of shared lock. Also, the entire if/else should go in the lock.

I have placed the lock in CorporaUtils. Let me know if you would prefer another class, or perhaps a dedicated internal static class LockManager or similar. I've also implemented use of the lock in unit tests for consistency sake, rather than to fix any actual found bugs.

[ddaspit]

@ddaspit reviewed 6 files and all commit messages, made 1 comment, and resolved 1 discussion.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on pmachapman).

[Enkidu93]

@Enkidu93 reviewed 6 files and all commit messages, and made 1 comment.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on pmachapman).