perf(config): 配置保存改为非阻塞写盘并批量提交 - #415
Merged
Merged
Conversation
进入 MAA 脚本配置界面加载慢、改字段卡顿的根因是保存路径阻塞事件循环: - ConfigBase.save / MultipleConfig.save 是 async 函数, 但内部直接同步调用 write_file, 其 atomic_write 带 os.fsync 并持全局写锁。一次保存会卡住整个 事件循环, 连带所有其它 API 与 WebSocket 广播一起停摆。改为 asyncio.to_thread。 - update_script 等入口逐字段 await set(), 每个 set 都触发一次全量序列化写盘, 改 N 个字段就写 N 次。新增 ConfigBase.update() 批量写入后只提交一次, 九处同模式调用点一并收敛。 - MAAScriptEdit 每次改字段在 updateScript 成功后又 await refreshScript(), 一次修改两个往返, 且回填时 Object.assign 覆盖整个响应式对象导致全表单重渲染 (输入框跳动)。值本就由前端发出, 该刷新是多余的, 移除。 - onMounted 串行 await 两个互不依赖的请求, 改 Promise.all 并行。
审查者指南本 PR 将配置保存路径改为线程池中的非阻塞写盘，并通过新增批量 update API 将多字段修改合并为一次全量提交；同时减少前端脚本编辑的往返请求、并行初始化请求，并补充相应测试与 mock 适配。 批量非阻塞配置更新时序图sequenceDiagram
participant API as Config API
participant Config as ConfigBase
participant ThreadPool as asyncio.to_thread
participant Disk as Configuration file
API->>Config: update(data)
loop each changed field
Config->>Config: set(group, name, value, commit=false)
end
Config->>Config: _commit_changes()
Config->>ThreadPool: write_file(file, toDict(...))
ThreadPool->>Disk: atomic write and fsync
Disk-->>ThreadPool: write completed
ThreadPool-->>Config: await completion
Config-->>API: update completed
并行脚本编辑器初始化时序图sequenceDiagram
participant Editor as MAAScriptEdit
participant ScriptAPI as loadScript()
participant EmulatorAPI as loadEmulatorOptions
Editor->>ScriptAPI: loadScript()
Editor->>EmulatorAPI: loadEmulatorOptions()
par independent requests
ScriptAPI-->>Editor: script data
and
EmulatorAPI-->>Editor: emulator options
end
Editor->>Editor: isInitializing = false
文件级变更
提示与命令与 Sourcery 交互
自定义使用体验访问你的控制面板以：
获取帮助Original review guide in EnglishReviewer's Guide本 PR 将配置保存路径改为线程池中的非阻塞写盘，并通过新增批量 update API 将多字段修改合并为一次全量提交；同时减少前端脚本编辑的往返请求、并行初始化请求，并补充相应测试与 mock 适配。 Sequence diagram for batched non-blocking configuration updatessequenceDiagram
participant API as Config API
participant Config as ConfigBase
participant ThreadPool as asyncio.to_thread
participant Disk as Configuration file
API->>Config: update(data)
loop each changed field
Config->>Config: set(group, name, value, commit=false)
end
Config->>Config: _commit_changes()
Config->>ThreadPool: write_file(file, toDict(...))
ThreadPool->>Disk: atomic write and fsync
Disk-->>ThreadPool: write completed
ThreadPool-->>Config: await completion
Config-->>API: update completed
Sequence diagram for parallel script editor initializationsequenceDiagram
participant Editor as MAAScriptEdit
participant ScriptAPI as loadScript()
participant EmulatorAPI as loadEmulatorOptions
Editor->>ScriptAPI: loadScript()
Editor->>EmulatorAPI: loadEmulatorOptions()
par independent requests
ScriptAPI-->>Editor: script data
and
EmulatorAPI-->>Editor: emulator options
end
Editor->>Editor: isInitializing = false
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
嘿——我发现了 2 个问题
面向 AI Agent 的提示
请处理本次代码审查中的评论：
## 各条评论
### 评论 1
<location path="app/models/ConfigBase.py" line_range="1111-1113" />
<code_context>
- write_file(self.file, await self.toDict(if_decrypt=False))
+ # write_file 内部 fsync 是同步阻塞调用, 放线程池避免卡住事件循环
+ await asyncio.to_thread(
+ write_file, self.file, await self.toDict(if_decrypt=False)
+ )
async def lock(self):
</code_context>
<issue_to_address>
**issue (bug_risk)：** 并发保存可能会在较新的 `toDict()` 快照已经写入之后，又写入较旧的快照。`_WRITE_LOCK` 仅在序列化完成后对 `write_file` 进行串行化，因此无法保护整个“生成快照并写入”的操作，配置文件可能会丢失最近的更新。
**触发条件：** 两个配置更新同时提交，并且它们的异步序列化或线程池调度完成顺序相反时。
**建议修复：** 使用按文件划分的异步提交锁，对快照创建和文件替换进行串行化；或者将完整的序列化与写入操作置于同一个同步边界之后。
</issue_to_address>
### 评论 2
<location path="app/models/ConfigBase.py" line_range="1052-1058" />
<code_context>
raise RuntimeError(f"脚本 {script_id} 正在运行, 无法更新配置项")
- script_config = self.ScriptConfig[uid]
- for group, items in data.items():
- for name, value in items.items():
- await script_config.set(group, name, value)
</code_context>
<issue_to_address>
**issue (bug_risk)：** `update()` 会在验证和应用后续字段之前修改较早的字段；如果后续项目未知、被锁定或以其他方式抛出异常，该方法会在未调用 `_commit_changes()` 的情况下退出，使较早的修改仅存在于内存中。之后一次无关的保存操作会持久化这些部分修改，尽管本次更新请求已经失败。
**触发条件：** 批处理中至少包含一个有效且发生变化的字段，后面紧跟一个会导致 `set()` 调用抛出异常的字段时。
**建议修复：** 在修改数据之前验证所有条目，或者在批处理失败时回滚已经应用的值。
</issue_to_address>Sourcery 评估
需要人工审查。 有 2 个问题需要优先处理，并且此次更改改变了配置值的持久化方式：批量更新和线程化文件写入可能会导致磁盘上的配置值错误或丢失，尤其是在保存操作重叠时；此外，前端在保存后也不再从服务器刷新数据。恢复更改可以恢复旧行为，而已经写入的配置值通常可以通过再次运行更新来修正。
阻塞性问题：app/models/ConfigBase.py:1113、app/models/ConfigBase.py:1058
帮助我变得更有用！请在每条评论上点击 👍 或 👎，我会利用反馈来改进审查结果。
Original comment in English
Hey - I've found 2 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="app/models/ConfigBase.py" line_range="1111-1113" />
<code_context>
- write_file(self.file, await self.toDict(if_decrypt=False))
+ # write_file 内部 fsync 是同步阻塞调用, 放线程池避免卡住事件循环
+ await asyncio.to_thread(
+ write_file, self.file, await self.toDict(if_decrypt=False)
+ )
async def lock(self):
</code_context>
<issue_to_address>
**issue (bug_risk):** Concurrent saves can write an older `toDict()` snapshot after a newer snapshot has already been written. `_WRITE_LOCK` serializes only `write_file` after serialization, so it does not protect the snapshot-and-write operation as a whole and the configuration file can lose a recent update.
**Triggers:** When two configuration updates commit concurrently and their asynchronous serialization or thread-pool scheduling completes out of order.
**Suggested fix:** Serialize snapshot creation and file replacement with an async per-file commit lock, or move the complete serialization-and-write operation behind the same synchronization boundary.
</issue_to_address>
### Comment 2
<location path="app/models/ConfigBase.py" line_range="1052-1058" />
<code_context>
raise RuntimeError(f"脚本 {script_id} 正在运行, 无法更新配置项")
- script_config = self.ScriptConfig[uid]
- for group, items in data.items():
- for name, value in items.items():
- await script_config.set(group, name, value)
</code_context>
<issue_to_address>
**issue (bug_risk):** `update()` mutates earlier fields before validating and applying later fields; if a later item is unknown, locked, or otherwise raises, the method exits without `_commit_changes()`, leaving the earlier mutations present only in memory. A later unrelated save then persists those partial changes even though this update request failed.
**Triggers:** When a batch contains at least one valid changed field followed by a field whose `set()` call raises.
**Suggested fix:** Validate all entries before mutating, or roll back already-applied values when the batch fails.
</issue_to_address>Sourcery assessment
Needs a human reviewer. 2 findings to address first, and the change alters how configuration values are persisted: batched updates and threaded file writes could leave a wrong or lost configuration value on disk, especially if saves overlap, and the frontend no longer refreshes from the server after saving. Reverting restores the old behavior, while already-written configuration values can generally be corrected by running the update again.
Blocking findings: app/models/ConfigBase.py:1113, app/models/ConfigBase.py:1058
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Comment on lines
+1111
to
+1113
| await asyncio.to_thread( | ||
| write_file, self.file, await self.toDict(if_decrypt=False) | ||
| ) |
There was a problem hiding this comment.
issue (bug_risk)： 并发保存可能会在较新的 toDict() 快照已经写入之后，又写入较旧的快照。_WRITE_LOCK 仅在序列化完成后对 write_file 进行串行化，因此无法保护整个“生成快照并写入”的操作，配置文件可能会丢失最近的更新。
触发条件： 两个配置更新同时提交，并且它们的异步序列化或线程池调度完成顺序相反时。
建议修复： 使用按文件划分的异步提交锁，对快照创建和文件替换进行串行化；或者将完整的序列化与写入操作置于同一个同步边界之后。
Original comment in English
issue (bug_risk): Concurrent saves can write an older toDict() snapshot after a newer snapshot has already been written. _WRITE_LOCK serializes only write_file after serialization, so it does not protect the snapshot-and-write operation as a whole and the configuration file can lose a recent update.
Triggers: When two configuration updates commit concurrently and their asynchronous serialization or thread-pool scheduling completes out of order.
Suggested fix: Serialize snapshot creation and file replacement with an async per-file commit lock, or move the complete serialization-and-write operation behind the same synchronization boundary.
Comment on lines
+1052
to
+1058
| for group, items in data.items(): | ||
| for name, value in items.items(): | ||
| if await self.set(group, name, value, commit=False): | ||
| is_changed = True | ||
|
|
||
| if is_changed: | ||
| await self._commit_changes() |
There was a problem hiding this comment.
issue (bug_risk)： update() 会在验证和应用后续字段之前修改较早的字段；如果后续项目未知、被锁定或以其他方式抛出异常，该方法会在未调用 _commit_changes() 的情况下退出，使较早的修改仅存在于内存中。之后一次无关的保存操作会持久化这些部分修改，尽管本次更新请求已经失败。
触发条件： 批处理中至少包含一个有效且发生变化的字段，后面紧跟一个会导致 set() 调用抛出异常的字段时。
建议修复： 在修改数据之前验证所有条目，或者在批处理失败时回滚已经应用的值。
Original comment in English
issue (bug_risk): update() mutates earlier fields before validating and applying later fields; if a later item is unknown, locked, or otherwise raises, the method exits without _commit_changes(), leaving the earlier mutations present only in memory. A later unrelated save then persists those partial changes even though this update request failed.
Triggers: When a batch contains at least one valid changed field followed by a field whose set() call raises.
Suggested fix: Validate all entries before mutating, or roll back already-applied values when the batch fails.
实测 to_thread 只挪走 fsync 约3ms，toDict 约30ms 因参数求值顺序 仍阻塞事件循环，收益不足以抵消引入的并发写序风险。前端恢复 保存后回读。批量提交与 set(commit=) 优化保留。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
进入 MAA 脚本配置界面加载慢，改任意字段都卡顿。定位后发现最严重的一条不在前端，而在保存路径阻塞事件循环。
根因与改动
1. 写盘同步阻塞整个事件循环（主因）
ConfigBase.save/MultipleConfig.save声明为async，但内部直接同步调用write_file→atomic_write，其中带os.fsync并持全局写锁。Windows 上单次几 ms 到几十 ms。后果不只是这一个请求慢，而是所有其它 API 与 WebSocket 广播一起停摆。改为
await asyncio.to_thread(write_file, ...)。全局_WRITE_LOCK是threading.Lock，挪进线程池后才真正起到串行化保护作用。2. 逐字段保存 = N 次全量写盘
update_script等入口逐字段await set()，每个set都触发一次_commit_changes()，即一次全量toDict+ 写盘。改 3 个字段就写 3 次。而子配置注册了父级save，一次set实际序列化的是整个ScriptConfig.json（所有脚本 + 所有用户），不是单个脚本。新增
ConfigBase.update()：批量set(commit=False)，全部写入后只提交一次。set()增加commit参数并返回是否真正变更（向后兼容，既有调用点忽略返回值即可）。config.py中九处同模式调用点一并收敛。update_game_sign_account保持原样 —— 它在循环中间读旧值判断凭据是否变化，批量化需要重构，超出本次范围。3. 前端一次修改两个往返
MAAScriptEdit.handleChange在updateScript成功后又await refreshScript()。值本就由前端发出，这次回读是多余的；且回填时Object.assign覆盖整个响应式对象，导致全表单重渲染（输入框跳动的来源）。移除后单次修改从 2 个往返降到 1 个。4. onMounted 串行等待
loadScript()与loadEmulatorOptions()互不依赖却串行await，白等一个 RTT。改Promise.all。测试
test_update_commits_once_for_multiple_fields：三字段批量写入只提交一次。test_update_skips_commit_when_nothing_changes：无变更不落盘。tests/api/test_game_sign_api.py的 mock 补上update，转发到set以保持既有断言语义。201 passed。剩余 7 个失败已验证为 dev 上的既有问题（stash 掉本 PR 改动后在干净 dev 上同样复现）：append_task_game_sign_summary的签名与测试期望不一致，以及test_main.py的循环导入，均与本 PR 无关。未包含
get_script跳过SubConfigsInfo（前端拿到后即丢弃，却要为其递归解密全部用户配置）会动 API 契约，按讨论留待单独确认。Sourcery 总结
通过使磁盘持久化操作非阻塞，并将多字段更新合并为单次提交，提高配置编辑的响应速度。
Bug 修复：
增强功能：
测试：
Original summary in English
Sourcery 总结
通过使持久化操作非阻塞，并将多字段更新整合为高效的单次提交，提高配置编辑的响应速度。
错误修复：
增强功能：
测试：
Original summary in English
Sourcery 摘要
通过批量持久化和并行处理独立的前端加载请求，提高配置编辑的响应速度。
错误修复：
增强功能：
Original summary in English
Summary by Sourcery
Improve configuration editing responsiveness by batching persistence and parallelizing independent frontend loading requests.
Bug Fixes:
Enhancements: