[Bug][SubscriptionBilling]: Performance: Billing document creation does not scale to large datasets (4M+ subscription lines)#8553
Open
miljance wants to merge 1 commit into
Conversation
…ing runs (microsoft#7690) Introduces a suite of code-level and database-index optimizations to make billing-proposal creation, document creation, and usage-data processing scale to millions of subscription lines. Progress & UX - New "Progress Tracker" codeunit (8035): reusable ETA/throughput dialog, GuiAllowed-guarded, time-throttled, safe for Job Queue runs. - Progress dialogs added to: CreateBillingDocuments, BillingProposal (create/clear/delete), GenericConnectorProcessing, ProcessUsageDataBilling. - Recurring Billing page: cached per-row contract/partner name lookups (eliminates 10k+ uncached Gets per render); removed 9 full temp-table rebuilds from pure-navigation drilldowns; targeted group refresh for document drilldowns, Delete Billing Line, and Change Billing To Date actions. Transaction boundaries - BillingProposal: per-N-contract commit checkpoint (N=10) for large automated runs; test suite updated to match new commit semantics. - CreateBillingDocuments: per-document commit checkpoint on the non-posting path. - ProcessUsageDataBilling: removed per-commitment Commit() inside loop. Code-level optimizations - BillingPriceCalcSkip (new CU 8036): manual-binding subscriber that skips redundant unit-price/cost engine calls when inserting billing document lines, eliminating price-list lookups for every line. - CreateBillingDocuments: Dictionary-based temp-billing-line aggregation replacing O(n) FindFirst per line; per-contract TrimTempBillingLines using shared-table copy to keep temp table in memory. - BillingProposal: SetLoadFields on contract/subscription header Gets in the recursive UpdateBillingLineFromServiceCommitment path. - FilterBillingLinesOnServiceCommitment: added Subscription Header No. filter so proposal loop seeks on existing SK1 index. - GenericConnectorProcessing: hoisted loop-invariant GenericImportSettings Get out of the per-row loop. - UsageBasedContrSubscribers: removed 3 event subscribers whose logic now lives directly in BillingLine.OnDelete, SalesDocuments, and PurchaseDocuments - eliminating event dispatch overhead on hot paths. - CheckOnlyOneServicePartnerType: replaced full-scan loop with two IsEmpty() checks. Database indexes - BillingLine: SK7 ("Subscription Line Entry No."), TableRelation + DropDown fieldgroup added. - BillingLineArchive: DropDown fieldgroup added. - SubscriptionLine: Key4 ("Supplier Reference Entry No.", "Subscription Line Start Date"). - UsageDataBilling: key2-key4 reordered/extended; key5 ("Billing Line Entry No."); TableRelation for Document Type = None added so the field navigates correctly from the list. - UsageDataGenericImport: Key2 ("Usage Data Import Entry No.", "Processing Status"). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces a suite of code-level and database-index optimizations to make billing-proposal creation, document creation, and usage-data processing scale to millions of subscription lines.
Progress & UX
Transaction boundaries
Code-level optimizations
Database indexes
What & why
Linked work
Fixes #7690
How I validated this
What I tested and the outcome (required — be specific: scenarios, commands, screenshots for UI changes)
RecurringBillingTest,UsageBasedBillingTest, contract and UBB suites); all pass.Document No.with correct group subtotals and no stale rows.Risk & compatibility
CreateBillingProposal: a checkpoint is committed every 10 contracts. Any caller that holds an open page or record reference across aCreateBillingProposalcall on a dataset larger than 10 contracts will see a stale-rowversion error on the next edit (BC's standard concurrency message). The two affected standard tests were updated. Third-party extensions keeping a subpage open across a proposal run would need the same treatment.BillingPriceCalcSkip(CU 8036): while bound during sales/purchase line initialisation,OnBeforeUpdateUnitPrice,OnBeforeGetUnitCost,OnBeforeUpdateQuantityFromUOMCode(Sales Line) andOnBeforeUpdateUnitCost(Purchase Line) are handled and skipped. A third-party subscriber using those events to apply a surcharge or override during billing document creation would be bypassed. The subscriber is bound/unbound surgically around field initialisation only; all other validation still runs.DeleteBillingProposalnow uses per-lineDelete(true)instead ofDeleteAll(true):OnDeletetriggers still fire identically; the only difference is that partial progress is durable on interruption. No behavioral change.BillingLine.OnDeletedirectly updatesUsageDataBilling(previously via event subscriber withIsTemporaryguard): ALOnDeletenever fires for temporary records so the guard was redundant — no behavioral change.BillingLine SK7,UsageDataBilling key5, others) are purely additive; no data migration or upgrade codeunit needed.