Skip to content

Conversation

@myieye
Copy link
Collaborator

@myieye myieye commented Oct 22, 2025

With this change users will be able to find data from FieldWorks regardless of whether they type in NFD or NFC. Previously they would have had to type in NFD. That's the vast majority of data in a project, so that's a big win.

Unfortunately, users will now not be able to find NFC data that was created in FieldWorks Lite (and hasn't yet been round-tripped through a sync), because we actually save it in NFC. (See #2065)

I also:

  • refactored tests a bit, so that they generally use the normalization wrapper api by default.
  • Added some tests to demonstrate normalization (with some todos pointing at Normalize all saved text to NFD #2065)

The normalization wrapper was added a long time ago, but we overlooked actually using it 🙃.

The wrapper should maybe be more deeply embedded in our IoC container, but...I'm not sure how much effort that would be.
I think this is good for now.

@github-actions github-actions bot added the 💻 FW Lite issues related to the fw lite application, not miniLcm or crdt related label Oct 22, 2025
@coderabbitai
Copy link

coderabbitai bot commented Oct 22, 2025

📝 Walkthrough

Walkthrough

This PR introduces string normalization into the MiniLcm API pipeline by creating a new normalization wrapper and refactoring test infrastructure. Tests now use a BaseApi field (unwrapped API) and an Api field (wrapped with normalization). The wrapper is positioned first in the composition chain before validation and notification wrappers.

Changes

Cohort / File(s) Summary
Test API cast refactoring
backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/BasicApiTests.cs, ComplexFormComponentTests.cs, CreateEntryTests.cs, MediaTests.cs, PartOfSpeechTests.cs, QueryEntryTests.cs, UpdateEntryTests.cs
Replaced Api casts with BaseApi casts across multiple test methods to access the unwrapped API instance for repository operations. Pattern applied consistently across 7 test files.
API wrapper composition
backend/FwLite/FwLiteShared/Services/MiniLcmJsInvokable.cs, backend/FwLite/FwLiteProjectSync/CrdtFwdataProjectSyncService.cs
Updated constructor signatures and wrapper ordering: Added MiniLcmApiStringNormalizationWrapperFactory parameter to MiniLcmJsInvokable; reordered composition to prioritize normalization wrapper before validation and notification wrappers. Added normalization using directive to CrdtFwdataProjectSyncService.
Test infrastructure
backend/FwLite/MiniLcm.Tests/MiniLcmTestBase.cs
Introduced BaseApi field and wrapping logic: BaseApi holds the unwrapped API from NewApi(), then wrapped with MiniLcmApiStringNormalizationWrapperFactory to produce Api. Added using directives for MiniLcm.Normalization and MiniLcm.Wrappers.
Normalization tests
backend/FwLite/FwLiteProjectSync.Tests/EntrySyncTests.cs, backend/FwLite/MiniLcm.Tests/QueryEntryTestsBase.cs
Added NormalizesStringsToNFD() test method to EntrySyncTestsBase; updated SuccessfulMatches test theory signature to include bool identical parameter with new InlineData entries covering Cyrillic and diacritics normalization scenarios.
Namespace and import updates
backend/FwLite/MiniLcm/Normalization/MiniLcmApiStringNormalizationWrapper.cs, backend/FwLite/MiniLcm.Tests/NormalizationTests.cs, backend/FwLite/MiniLcm/Validators/MiniLcmValidators.cs
Moved MiniLcmApiStringNormalizationWrapper from Validators to Normalization namespace; updated using directives to reference new namespace; added MiniLcm.Normalization import to MiniLcmValidators.cs.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~35 minutes

The PR involves systematic refactoring across many test files (homogeneous pattern of Api→BaseApi replacements), balanced against meaningful architectural changes to wrapper composition order, constructor signature updates, and new test logic for normalization coverage. Mixed complexity across API wiring changes and test infrastructure updates.

Possibly related PRs

Suggested reviewers

  • rmunn
  • hahn-kev

Poem

🐰 Strings now normalize with grace so true,
BaseApi and Api dance in perfect queue—
Wrappers stack like lettuce layers green,
The cleanest composition ever seen! 🥬
String forms bloom in FormD delight,
Our rabbit tests pass with all their might! ✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The title "Use normalization wrapper in js-invokable" directly aligns with the primary change in the changeset—enabling the normalization wrapper in the MiniLcmJsInvokable class by adding a new MiniLcmApiStringNormalizationWrapperFactory parameter to its constructor and reordering the wrapper composition to include the normalization wrapper first. While the changeset also includes supporting changes such as test refactoring (switching from Api to BaseApi) and new normalization tests, the title captures the main point from the developer's perspective. The title is concise, specific, and clear enough for a teammate to understand the primary change when scanning history.
Description Check ✅ Passed The pull request description directly and accurately reflects the changeset. The author describes enabling the normalization wrapper in js-invokable (evidenced by MiniLcmJsInvokable.cs changes), refactoring tests to use the normalization wrapper API by default (evidenced by multiple test files switching from Api to BaseApi), and adding new normalization tests (evidenced by the NormalizesStringsToNFD test method). The description provides meaningful context about the problem being solved and explains both the benefits and known limitations of this change. All claims in the description have corresponding changes in the provided raw summary.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch 1860-search-values-are-not-normalized

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@argos-ci
Copy link

argos-ci bot commented Oct 22, 2025

The latest updates on your projects. Learn more about Argos notifications ↗︎

Build Status Details Updated (UTC)
default (Inspect) ✅ No changes detected - Oct 22, 2025, 2:13 PM

@github-actions
Copy link

UI unit Tests

  1 files  ±0   45 suites  ±0   29s ⏱️ -1s
111 tests ±0  111 ✅ ±0  0 💤 ±0  0 ❌ ±0 
160 runs  ±0  160 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit a5139e7. ± Comparison against base commit 19cc689.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
backend/FwLite/MiniLcm/Normalization/MiniLcmApiStringNormalizationWrapper.cs (1)

42-45: Bug: wrapper breaks disposal chain (leak).

IDisposable.Dispose() is empty, so disposing the outer wrapped API won’t dispose inner wrappers or the base API. Fix by forwarding disposal; also offer IAsyncDisposable.

Apply:

+public partial class MiniLcmApiStringNormalizationWrapper(
+    IMiniLcmApi api) : IMiniLcmApi, IAsyncDisposable
+{
@@
-    void IDisposable.Dispose()
-    {
-    }
+    public async ValueTask DisposeAsync()
+    {
+        if (_api is IAsyncDisposable asyncDisposable)
+            await asyncDisposable.DisposeAsync();
+        else if (_api is IDisposable disposable)
+            disposable.Dispose();
+    }
+
+    void IDisposable.Dispose()
+    {
+        if (_api is IAsyncDisposable asyncDisposable)
+            asyncDisposable.DisposeAsync().AsTask().GetAwaiter().GetResult();
+        else if (_api is IDisposable disposable)
+            disposable.Dispose();
+    }
+}
🧹 Nitpick comments (2)
backend/FwLite/MiniLcm/Normalization/MiniLcmApiStringNormalizationWrapper.cs (1)

27-40: Broaden normalization coverage.

Currently normalizes SearchEntries/CountEntries/GetEntries and options. Consider also normalizing inputs for create/update paths and other query surfaces to ensure end-to-end consistency (e.g., CreateEntry/UpdateEntry, Create/UpdateSense, PartOfSpeech/ComplexFormType names, semantic domain search, media metadata if applicable).

backend/FwLite/MiniLcm.Tests/MiniLcmTestBase.cs (1)

18-21: Avoid passing null project to WrapWith.

Use a test stub implementing IProjectIdentifier (e.g., “TestProject” with DataFormat appropriate to the suite) instead of null to de-risk future factories that read project data.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 19cc689 and a5139e7.

📒 Files selected for processing (15)
  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/BasicApiTests.cs (1 hunks)
  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/ComplexFormComponentTests.cs (3 hunks)
  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/CreateEntryTests.cs (1 hunks)
  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/MediaTests.cs (1 hunks)
  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/PartOfSpeechTests.cs (2 hunks)
  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/QueryEntryTests.cs (1 hunks)
  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/UpdateEntryTests.cs (4 hunks)
  • backend/FwLite/FwLiteProjectSync.Tests/EntrySyncTests.cs (2 hunks)
  • backend/FwLite/FwLiteProjectSync/CrdtFwdataProjectSyncService.cs (1 hunks)
  • backend/FwLite/FwLiteShared/Services/MiniLcmJsInvokable.cs (2 hunks)
  • backend/FwLite/MiniLcm.Tests/MiniLcmTestBase.cs (1 hunks)
  • backend/FwLite/MiniLcm.Tests/NormalizationTests.cs (1 hunks)
  • backend/FwLite/MiniLcm.Tests/QueryEntryTestsBase.cs (1 hunks)
  • backend/FwLite/MiniLcm/Normalization/MiniLcmApiStringNormalizationWrapper.cs (1 hunks)
  • backend/FwLite/MiniLcm/Validators/MiniLcmValidators.cs (1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-07-29T07:10:53.388Z
Learnt from: hahn-kev
PR: sillsdev/languageforge-lexbox#1836
File: backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/MediaTests.cs:28-33
Timestamp: 2025-07-29T07:10:53.388Z
Learning: In test code for FwData-specific functionality (like MediaTests in FwDataMiniLcmBridge.Tests), direct casting to FwDataMiniLcmApi is preferred over safe casting because it serves as an assertion that the test setup is correct. If the cast fails, it indicates a test configuration bug that should be caught immediately rather than silently ignored.

Applied to files:

  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/CreateEntryTests.cs
  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/BasicApiTests.cs
  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/PartOfSpeechTests.cs
  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/UpdateEntryTests.cs
  • backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/MediaTests.cs
🧬 Code graph analysis (10)
backend/FwLite/MiniLcm.Tests/QueryEntryTestsBase.cs (2)
backend/FwLite/LcmCrdt/Data/MiniLcmRepository.cs (10)
  • Task (26-29)
  • Task (95-103)
  • Task (105-112)
  • Task (114-118)
  • Task (120-125)
  • Task (157-195)
  • Task (215-233)
  • Task (235-241)
  • Task (243-249)
  • Task (251-256)
backend/FwLite/LcmCrdt/FullTextSearch/EntrySearchService.cs (1)
  • LexemeForm (76-82)
backend/FwLite/FwLiteShared/Services/MiniLcmJsInvokable.cs (4)
backend/FwLite/MiniLcm/Validators/MiniLcmApiValidationWrapper.cs (3)
  • MiniLcmApiValidationWrapperFactory (8-16)
  • IMiniLcmApi (10-10)
  • IMiniLcmApi (12-15)
backend/FwLite/MiniLcm/Normalization/MiniLcmApiStringNormalizationWrapper.cs (3)
  • MiniLcmApiStringNormalizationWrapperFactory (7-15)
  • IMiniLcmApi (9-9)
  • IMiniLcmApi (11-14)
backend/FwLite/FwLiteShared/Services/MiniLcmApiNotifyWrapper.cs (1)
  • IMiniLcmApi (12-15)
backend/FwLite/MiniLcm/Wrappers/MiniLcmWrappers.cs (2)
  • IMiniLcmApi (7-7)
  • IMiniLcmApi (12-20)
backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/CreateEntryTests.cs (2)
backend/FwLite/FwDataMiniLcmBridge/Api/FwDataMiniLcmApi.cs (1)
  • FwDataMiniLcmApi (26-1764)
backend/FwLite/FwDataMiniLcmBridge.Tests/Fixtures/ProjectLoaderFixture.cs (1)
  • FwDataMiniLcmApi (28-34)
backend/FwLite/MiniLcm.Tests/MiniLcmTestBase.cs (1)
backend/FwLite/MiniLcm/Normalization/MiniLcmApiStringNormalizationWrapper.cs (1)
  • MiniLcmApiStringNormalizationWrapperFactory (7-15)
backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/BasicApiTests.cs (2)
backend/FwLite/FwDataMiniLcmBridge/Api/FwDataMiniLcmApi.cs (1)
  • FwDataMiniLcmApi (26-1764)
backend/FwLite/FwDataMiniLcmBridge.Tests/Fixtures/ProjectLoaderFixture.cs (1)
  • FwDataMiniLcmApi (28-34)
backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/PartOfSpeechTests.cs (2)
backend/FwLite/FwDataMiniLcmBridge/Api/FwDataMiniLcmApi.cs (1)
  • FwDataMiniLcmApi (26-1764)
backend/FwLite/FwDataMiniLcmBridge.Tests/Fixtures/ProjectLoaderFixture.cs (1)
  • FwDataMiniLcmApi (28-34)
backend/FwLite/FwLiteProjectSync.Tests/EntrySyncTests.cs (1)
backend/FwLite/MiniLcm/SyncHelpers/EntrySync.cs (1)
  • EntrySync (8-327)
backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/UpdateEntryTests.cs (1)
backend/FwLite/FwDataMiniLcmBridge/Api/FwDataMiniLcmApi.cs (1)
  • FwDataMiniLcmApi (26-1764)
backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/QueryEntryTests.cs (2)
backend/FwLite/FwDataMiniLcmBridge/FwDataFactory.cs (1)
  • FwDataMiniLcmApi (42-45)
backend/FwLite/FwDataMiniLcmBridge/Api/FwDataMiniLcmApi.cs (1)
  • FwDataMiniLcmApi (26-1764)
backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/MediaTests.cs (3)
backend/FwLite/FwDataMiniLcmBridge/Api/FwDataMiniLcmApi.cs (1)
  • FwDataMiniLcmApi (26-1764)
backend/FwLite/FwDataMiniLcmBridge.Tests/Fixtures/ProjectLoaderFixture.cs (1)
  • FwDataMiniLcmApi (28-34)
backend/FwLite/MiniLcm.Tests/MiniLcmTestBase.cs (2)
  • Task (14-14)
  • Task (16-22)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: Build FwHeadless / publish-fw-headless
  • GitHub Check: frontend-component-unit-tests
  • GitHub Check: frontend
  • GitHub Check: Build FW Lite and run tests
  • GitHub Check: Analyze (csharp)
🔇 Additional comments (17)
backend/FwLite/FwLiteProjectSync.Tests/EntrySyncTests.cs (2)

1-1: LGTM!

The using System.Text; directive is necessary for the NormalizationForm enum used in the new test.


70-101: Good test demonstrating normalization behavior.

The test effectively verifies that string normalization to NFD works during sync operations. The test structure is clear with well-defined arrange/act/assert sections, and the assertions on lines 77-80 properly validate the test setup.

The commented-out assertions for entry2 with the reference to issue #2065 appropriately document a known limitation, which aligns with the PR objectives.

One minor suggestion: consider adding a brief comment before line 83 explaining the test pattern where entry1 is created empty in the database, but the sync "before" state (line 84) represents what the external source thinks exists. This pattern might not be immediately obvious to readers unfamiliar with sync testing.

backend/FwLite/MiniLcm.Tests/QueryEntryTestsBase.cs (1)

333-362: LGTM! Good normalization test coverage.

The addition of the identical parameter improves test readability when cases differ only in normalization form. The Cyrillic test cases (lines 344-351) provide valuable coverage for both FormC and FormD normalization variants.

The TODO comment on line 356 appropriately references issue #2065 for future cleanup.

backend/FwLite/FwLiteProjectSync/CrdtFwdataProjectSyncService.cs (1)

10-10: LGTM! Using directive addition aligns with normalization namespace.

This addition supports the normalization wrapper usage introduced in this PR.

backend/FwLite/MiniLcm.Tests/NormalizationTests.cs (1)

1-1: LGTM! Namespace update reflects normalization code relocation.

The using directive correctly references the new MiniLcm.Normalization namespace.

backend/FwLite/MiniLcm/Validators/MiniLcmValidators.cs (1)

4-4: LGTM! Using directive supports normalization integration.

This addition enables access to normalization wrapper types in this file.

backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/CreateEntryTests.cs (1)

20-20: LGTM! BaseApi access pattern is correct.

The change from Api to BaseApi correctly accesses the unwrapped FwDataMiniLcmApi instance for direct repository access. This aligns with the test architecture where BaseApi provides the underlying API before normalization wrapping.

backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/QueryEntryTests.cs (1)

16-16: LGTM! BaseApi access pattern is correct.

The change from Api to BaseApi correctly accesses the unwrapped FwDataMiniLcmApi instance for direct repository access.

backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/UpdateEntryTests.cs (1)

29-29: LGTM! BaseApi access pattern consistently applied.

All four instances correctly use BaseApi instead of Api to access the unwrapped FwDataMiniLcmApi for direct repository and cache operations. This aligns with the test architecture where repository-level operations require the unwrapped API instance.

Also applies to: 78-78, 118-118, 181-181

backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/MediaTests.cs (1)

24-24: LGTM! BaseApi access pattern correctly applied in test lifecycle methods.

Both InitializeAsync and DisposeAsync correctly use BaseApi to access the unwrapped FwDataMiniLcmApi instance for cache and project folder operations.

Also applies to: 30-30

backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/ComplexFormComponentTests.cs (1)

30-33: Good switch to BaseApi for FwData internals.

Direct casting to FwDataMiniLcmApi from BaseApi is the right assertion in FwData-specific tests and matches the new wrapper layering. LGTM.

Based on learnings

Also applies to: 72-76, 110-114

backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/BasicApiTests.cs (1)

25-31: LGTM: use BaseApi for repository/UoW access.

Switching to (FwDataMiniLcmApi)BaseApi keeps tests precise and asserts correct setup.

Based on learnings

backend/FwLite/MiniLcm/Normalization/MiniLcmApiStringNormalizationWrapper.cs (1)

20-21: Confirm normalization form (FormD) is intentional.

Verify FormD matches how storage, search index, and JS client normalize to avoid mismatches with composed forms (NFC).

backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/PartOfSpeechTests.cs (1)

29-35: LGTM: BaseApi cast for FwData-specific setup.

Matches the suite-wide pattern and asserts test configuration early.

Based on learnings

Also applies to: 64-70

backend/FwLite/MiniLcm.Tests/MiniLcmTestBase.cs (1)

12-13: BaseApi exposure is appropriate.

Exporting BaseApi alongside wrapped Api cleanly separates raw FwData access from normalized behavior in tests. LGTM.

backend/FwLite/FwLiteShared/Services/MiniLcmJsInvokable.cs (2)

25-26: Wrapper order is correct (normalize → validate → notify).

Given WrapWith reverses factories, this yields Normalization as the outermost wrapper. Good.

Please confirm DI registration now supplies MiniLcmApiStringNormalizationWrapperFactory to this constructor across all hosting environments.


21-24: Constructor DI surface change.

Adding normalizationWrapperFactory is fine; ensure all composition roots (server, tests, any tools) are updated to avoid activation failures.

@myieye myieye merged commit 388d4c0 into develop Oct 27, 2025
19 checks passed
@myieye myieye deleted the 1860-search-values-are-not-normalized branch October 27, 2025 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

💻 FW Lite issues related to the fw lite application, not miniLcm or crdt related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants