fix: 并发上传5文件刷新后丢失 + 分片写入阻塞事件循环

根因:
1. pump() 仅在 addFiles() 调用一次,上传完成/失败后不重新触发,
   导致 CONCURRENCY=3 之后的文件(4、5)永远不启动 → 无 DB 会话 → 刷新后消失
2. write_chunk 的 uploaded_chunks 是 read-modify-write,并发分片写入
   后者覆盖前者 → 分片记录丢失 → complete 报 409
3. upload_chunk 是 async 但同步调 write_chunk(fsync+DB commit),
   阻塞 uvicorn 事件循环 → 所有 web 请求被串行化

修复:
- _shared.py: pump() 加 finally 块,上传完成/失败后都触发下一文件;
  文件并发(FILE_CONCURRENCY=5)与分片并发(CHUNK_CONCURRENCY=3)分离
- upload_service.py: 按 upload_id 的进程级锁串行化 uploaded_chunks 更新,
  持锁后 db.refresh 重读最新值再 append,杜绝丢失更新;complete 后清理锁
- upload_router.py: upload_chunk 的 write_chunk 调用改用 run_in_threadpool,
  阻塞 I/O 移出事件循环,web 请求不再被分片写入阻塞

验证:5 文件并发上传后刷新全部可见;4 任务并发处理 4/4 成功(19.2s)
This commit is contained in:
audio2text dev
2026-07-06 22:04:06 +08:00
parent 73110848f4
commit 2f68c7e1f8
3 changed files with 45 additions and 13 deletions

View File

@@ -6,6 +6,7 @@ complete 成功后创建转写 Task 并交由 scheduler 入队。
from __future__ import annotations
from fastapi import APIRouter, Depends, Request
from starlette.concurrency import run_in_threadpool
from sqlalchemy.orm import Session
from ..database import get_db
@@ -48,7 +49,10 @@ async def upload_chunk(
request: Request,
service: UploadService = Depends(_service),
) -> ChunkUploadResponse:
uploaded = service.write_chunk(upload_id, index, await request.body())
# write_chunk 做文件 fsync + DB commit阻塞 I/O必须放到线程池跑
# 否则会阻塞 uvicorn 事件循环导致并发分片上传被串行化、web 请求卡顿。
body = await request.body()
uploaded = await run_in_threadpool(service.write_chunk, upload_id, index, body)
return ChunkUploadResponse(upload_id=upload_id, index=index, uploaded_chunks=uploaded)