enhance: Optimize datacoord meta mutex

[bigsheeper]

1. Use a separate collection mutex.
2. Avoid holding locks for extended periods, such as in GetQuotaInfo. (TODO)

issue: #40551

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: bigsheeper
To complete the pull request process, please assign liliu-z after the PR has been reviewed.
You can assign the PR to them by writing /assign @liliu-z in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• internal/datacoord/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

Attention: Patch coverage is 92.30769% with 7 lines in your changes missing coverage. Please review.

Project coverage is 80.52%. Comparing base (3899b0f) to head (c74b558).
Report is 5 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/datacoord/meta.go	93.25%	6 Missing ⚠️
internal/datacoord/services.go	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #40552      +/-   ##
==========================================
- Coverage   82.19%   80.52%   -1.67%     
==========================================
  Files        1165     1474     +309     
  Lines      179833   207607   +27774     
==========================================
+ Hits       147817   167183   +19366     
- Misses      25965    34348    +8383     
- Partials     6051     6076      +25     

Components	Coverage Δ
Client	79.74% <ø> (ø)
Core	70.00% <ø> (∅)
Go	82.27% <92.30%> (-0.05%)	⬇️

Files with missing lines	Coverage Δ
internal/datacoord/services.go	84.16% <50.00%> (ø)
internal/datacoord/meta.go	89.09% <93.25%> (+<0.01%)	⬆️

... and 342 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[bigsheeper]

related to #37630