Compare commits

..

19 Commits

Author SHA1 Message Date
7c193ca46e 前端整理: 合并文件管理页 + 新增导航页
1. 合并 files-page 与 pdf-admin 为统一文件管理页 /api/files-page:
   - 新增 GET /api/admin/files/with-pdf 接口,以 uploaded_files 为基础,
     用 pdf_jobs.source_file_id / output_file_id 内存匹配,为关联文件标注
     转换状态、用户软删标记与角色(源epub/产物PDF)
   - PdfJobOut 补 source_file_id / output_file_id 字段
   - file_browser 新增 PDF 任务列(状态徽标/软删标记/硬删任务按钮)
   - 删除 /api/pdf-admin 路由及 static/pdf_admin.* 三个文件

2. 新增导航页 /api/index,卡片式收集所有页面入口;
   各子页(upload/whiteboard/system_status/files-page)脚注加返回导航链接

3. 更新 docs/routes.md、docs/configuration.md 同步说明

测试: pytest tests/test_pdf_service.py 7 passed; 手动校验各页面路由与合并接口响应
2026-07-28 14:04:12 +08:00
9af28f41b4 refactor: 清理死代码/提前失败/日志/高内聚低耦合
死代码移除:
- whiteboard_hub.py: 移除未引用的 reset_hub 单例重置函数
- tunnel_service.py: 移除未引用的 is_port_allowed (逻辑已在 sftp_server 内联)
- tunnel_session_dao.py: 移除未引用的 get_active_by_port
- pdf_job_dao.py: 移除未用 datetime 导入
- pdf_converter.py: 移除未用 shutil 导入
- pdf_service.py: 移除未用 PdfSubmitResponse 导入 + _do_convert 内未用 hashlib 导入
- upload_html.py: 移除未用 escape 导入 (JS 侧自有 escapeHtml)
- pdf_controller.py: 移除 _resolve_cookie 内未用 cfg 局部变量

提前失败/分层修复:
- database.py init_db_schema: 建表后用 inspector 校验既有表列与模型一致,
  缺列即抛 RuntimeError (fail-fast on schema drift), 避免运行期才暴露
- whiteboard_dao.get_or_create: 仅 IntegrityError 才回滚重读, 其他异常向上抛
  (原 except Exception 会掩盖 schema/连接等真实故障)
- pdf_service.admin_delete/_safe_delete_file: 改用 PdfJobDAO.delete /
  UploadedFileDAO.delete, 不再直接操作 job_dao.db / file_dao.db (修复分层契约:
  DAO 头注释声明 service 不直接操作 session)
- PdfJobDAO 新增 delete(job) 方法

日志补全 (8 处 silent catch):
- whiteboard_hub.py disconnect/close_board 关闭 ws: logger.debug
- whiteboard_controller _safe_send/_safe_close: logger.debug
- sftp_server _close_tunnel_dao/读用户名: logger.debug
- sftp_server validate_public_key: logger.warning (auth 路径, 避免静默失败)

文档:
- 新增 docs/routes.md, docs/configuration.md, docs/error-handling.md
- README.md 精简为简介/结构/外部依赖/apache2 配置/Ubuntu 安装/docs 链接
2026-07-28 11:34:35 +08:00
1c0f776571 feat(pdf-admin): 支持全选/多选批量硬删并去掉删除二次确认
- 工具栏新增「批量硬删」按钮,实时显示选中数,无选中时禁用
- 表头复选框全选/反选,行复选框多选,行级选中状态同步全选框
- 批量删除并发执行 DELETE,逐项淡出移除并汇总结果 toast
- 单条与批量删除均直接执行,移除 confirm() 二次确认
- 新增 .col-check 复选框列样式(窄宽居中)
2026-07-28 09:39:39 +08:00
e7b4a3d5e3 feat: 移除冗余 PDF 用户服务端页,前后端分离由 zMainPage 组件接管
PDF 用户侧 UI 改由 zMainPage 的 zPDF_package 组件提供(构建期 import,非 iframe),
zTools2 仅保留 /api/pdf/* REST API 与 /api/pdf-admin 管理页。cookie 认证不受影响:
zk_pdf 仍在首次 POST /api/pdf/jobs 时种下,与已删的 /api/pdf 页面路由无关。

- app/main.py:删除 GET /api/pdf 页面路由(pdf_page)
- static/pdf.html、static/pdf.js、static/pdf.css:删除用户页 HTML/JS/CSS
  (功能已被 zPDF_package 的 App.vue/Uploader.vue/JobList.vue 取代)
- static/pdf_admin.css:合并原 pdf.css 中管理页依赖的样式,成为自包含样式表
- static/pdf_admin.html:移除指向已删 /api/pdf 的页脚死链与 pdf.css 引用
- README.md:功能一览/访问入口表去 /api/pdf 用户页条目,路由约定去 iframe 表述
2026-07-27 17:14:33 +08:00
ff2ad3fcb3 feat: 所有入口统一到 /api/ 前缀
将 zTools2 托管的页面/静态/探针/WebSocket 路由全部从顶级路径迁移到 /api/ 下:
- 页面:/pdf -> /api/pdf、/pdf-admin -> /api/pdf-admin、/upload -> /api/upload、
  /files -> /api/files-page、/wb/{id} -> /api/wb-page/{id}、/wb-admin -> /api/wb-admin
  (页面类加 -page 后缀以规避同名 REST API /api/files、/api/wb/{id})
- 静态资源:/static -> /api/static
- 探针:/health -> /api/health
- WebSocket:/ws/wb/{id} -> /api/ws/wb/{id}
- 前端 HTML 壳与 JS 中的资源/页间链接/WS URL 同步更新
- 手动测试脚本 BASE_WS 同步

这样反代与 vite proxy 只需一条 /api/ 规则即可转发全部入口;
前端 iframe 用同源相对路径 /api/pdf,与环境无关,不再误打到其它环境域名。
README 新增「路由约定」说明。
2026-07-27 15:42:47 +08:00
729c77e98b feat: 隐藏 PDF 用户页的管理页入口,凭据改为 账号a / 66511315
- pdf.html 移除页脚"管理页"链接:管理页入口对普通用户隐藏,
  仅管理员知晓 /pdf-admin 直达路径(权限同 /files 文件浏览页,
  复用 require_docs_auth Basic Auth)
- test_pdf_service.py 同步测试凭据为 a/66511315

注:实际凭据存放于 config.yaml 的 docs 段(git-ignored,仅本机 root 持有),
本次仅隐藏入口;管理页鉴权机制(require_docs_auth)此前已与
文件浏览页 /files 完全一致,无需改动。
2026-07-27 14:17:16 +08:00
29c5462734 feat: 新增 systemd 持久化部署(deploy/ztools2.service + install-systemd.sh)
systemd 管理 zTools2 后端实现开机自启 + 崩溃自动重启:
- deploy/ztools2.service 模板(Restart=on-failure,after mysql)
- deploy/install-systemd.sh 自动检测 .venv 或系统 uvicorn,填充路径安装
- README 补持久化部署章节
2026-07-27 13:52:23 +08:00
aa08a1cf60 feat: 新增 PDF 转换服务(epub->pdf,cookie 用户隔离,软删/硬删两级删除)
纯 Python 转换(ebooklib 解析 epub + weasyprint 渲染 HTML/CSS 为 PDF),
无需 calibre/xvfb 系统依赖。复用 UploadedFile 存储(原始文件与产物 PDF)。

- model/dao/schema:PdfJob 记录转换任务(owner_cookie/source/output/进度/状态)
- pdf_converter:epub->PDF 转换器(按 spine 顺序拼 HTML,OPF 目录解析相对资源)
- pdf_service:上传落盘 + asyncio 后台转换(独立 Session)+ 进度追踪 + 两级删除
  · 用户软删(user_deleted,管理页仍可见)· 管理员硬删(真正删磁盘+记录)
- pdf_controller:/api/pdf/*(用户,cookie)+ /api/admin/pdf/*(Basic Auth)
- /pdf、/pdf-admin 页面(原生 JS,复用 common.css/js)
- 配置 pdf 段(max_size 250MB、转换超时、cookie)
- pytest 7 项(SQLite→MySQL 隔离测试库):上传/转换/下载/软删/硬删/超限/越权
2026-07-27 11:33:43 +08:00
fa980e74e0 fix: 白板页强制浅色,避免被 iframe 嵌入到浅色站点时出现黑色背景
- whiteboard.html 加 <meta name="color-scheme" content="light">
- whiteboard.css 在 :root 钉死 color-scheme: light 与浅色变量,
  覆盖 common.css 的 prefers-color-scheme: dark
2026-07-22 02:44:03 +00:00
zikai
09705a8843 feat: SFTP 上传的文件经磁盘扫描补录后可在文件浏览页展示
SFTP 上传的文件直接落盘到 upload_root(chroot 根),不经 HTTP 路径,无 DB 记录,
文件浏览页(读 uploaded_file 表)看不到。新增磁盘扫描机制:

- UploadService.scan_sftp_files(force=False):遍历 uploads/ 下所有文件(排除 .work/
  与 .part 残品),与 DB 已有 storage_path 比对,磁盘有 DB 无的补录(source="sftp",
  流式算 size + sha256)。
- 频率限制:模块级 _last_scan_time,两次扫描间隔 < 3s 则跳过返回 0(只读 DB 缓存),
  force=True 强制扫。
- file_admin_controller 的 list 接口返回前触发 scan_sftp_files(),文件浏览页打开/
  刷新即自动同步 SFTP 文件(受 3s 频率限制,不会每次翻页都扫磁盘)。
- UploadedFileDAO 加 list_storage_paths() 批量查已记录路径集合供比对。

SFTP 文件补录后可正常下载/删除(与 HTTP 上传文件一致)。
2026-07-22 01:25:41 +00:00
zikai
30a263ed50 fix: 代码审查修复 + 精简重写 README
后端修复:
- 白板删除踢人失效:delete_whiteboard 改 async def,删除后直接 await
  hub.close_board()。原实现用 asyncio.get_running_loop() 在同步 REST handler
  (threadpool)里调用必抛 RuntimeError 被 except 吞掉,close_board 从不执行。
  同时移除 service 的 hub 依赖(close_board 改由 controller 调用,service 只管 DB)。
- delete_file 去重复查询:原先 get_out_with_disk_path + get_by_id 查两次,
  合并为一次;磁盘 unlink 失败加 logger.warning(原静默吞掉致磁盘泄漏无记录)。
- get_hub 单例加 threading.Lock 双重检查(防 REST threadpool 与 WS 事件循环
  并发首访各建一个 hub)。
- file_controller 公开 /api/files list 加 Query(ge=1, le=10000) 约束(原无上限可 DoS)。

前端修复:
- applyRemoteUpdate 有未发送编辑时重发:合并远端更新后若本地有 pending 编辑
  (editor.value !== lastSentText)重新 scheduleSend,避免被 lastSentText 短路丢弃。
- init 不覆盖未发送编辑:断线重连后若本地有未发送内容,作为新版本发上去而非被 init 覆盖。
- applyRemoteUpdate 仅在编辑器已有焦点时恢复焦点,避免抢按钮焦点。
- api() 401 时 location.reload() 触发浏览器 Basic Auth 弹窗(原只 toast 卡死)。

README:
- 精简重写,补全 Ubuntu 从 0 安装、Apache 反代(含 WS)、配置项表格、防火墙说明。
2026-07-22 01:03:48 +00:00
zikai
4de2648178 fix: 记事本修改需手动刷新才同步(同浏览器多 tab clientId 共享导致 update 被跳过)
根因:clientId 存 localStorage,同浏览器多个 tab 共享同一 clientId。
前端 onMessage 的 update 分支用 `if (msg.client_id === clientId) break` 跳过
「自己的」更新,但服务端 exclude=conn 是按连接对象排除,非按 client_id。
于是 tab B 编辑后,tab A 收到的 update client_id 等于自己的 clientId,
被误判为「自己的」跳过 -> 不同步,只有手动刷新(重新 init)才看到内容。

修复:
- clientId 改用 sessionStorage(每 tab 独立),不再跨 tab 共享。
- 移除 update 分支的 client_id 跳过逻辑(服务端 exclude=conn 已排除发送者,
  前端跳过是多余且会误杀同 client_id 的其他 tab)。
2026-07-22 00:36:44 +00:00
zikai
1030801712 docs: 补 Apache WebSocket 反代配置说明(/ws/ 须 proxy_wstunnel 透传) 2026-07-21 15:12:15 +00:00
zikai
da7d2fb4b4 security: 白板 WS 加单帧大小限制与单 board 连接数上限
安全审计发现:
- WS receive_text 无应用层大小限制,恶意客户端可发超大帧(uvicorn 默认 16MB
  才拦截)占内存。加 512KB 单帧检查(与 content 256KB 限制对齐留余量)。
- 单 board 无连接数上限,恶意脚本可开海量连接耗尽资源。config 加
  max_connections_per_board(默认 50),hub.register 超限返回 False,
  controller 回 error 帧并关闭连接。

审计结论(无需修复):
- SQL:全部 SQLAlchemy ORM 参数化,无注入。
- 路径穿越:storage_path 服务端生成(uuid+basename(ext)),用户不可控分隔符。
- 鉴权:管理类操作均有 require_docs_auth,公开写接口符合设计。
- 下载 filename CRLF:Starlette FileResponse 用 quote() 编码,无响应拆分。
- XSS:前端 el() 用 createTextNode,textarea 纯文本,innerHTML 仅用于静态文案。
2026-07-21 15:09:13 +00:00
zikai
374c3d150f fix: docs 分组与 summary 统一 + 记事本页加「复制链接」按钮
docs 乱:
- main.py 页面路由(/upload /files /wb-admin /wb/{id} /docs /redoc / /health)
  加 tags(pages/docs/meta)+ summary/description,Swagger UI 按 tag 分组显示。
- whiteboard_controller 残留「白板/笔画/修改次数」改为「记事本/文本/编辑次数」。

复制链接:
- 记事本页加「复制链接」按钮,复制 {origin}/wb/{id} 分享链接(保留「复制文本」)。
2026-07-21 15:04:02 +00:00
zikai
655e039aad refactor: 白板从画笔画板改为文本记事本
原实现是 Canvas 画笔画板,与「文字白板/记事本」需求不符。重做为纯文本实时协作:
- model:strokes JSON -> content TEXT + version INT(乐观锁)+ edit_count;
  schema/dao/service 同步重构,append_stroke/replace_strokes -> update_content。
- WS 协议:stroke -> edit(发完整文本,debounce 400ms);init 下发 content/version。
  update 帧广播给他人,cleared 广播给所有人。
- 前端:canvas -> textarea;收到远端 update 用最长公共前后缀算变更区间,
  仅替换该区间并保留光标(区间前不动/后平移/内移末尾);清空/复制文本按钮。
- schema.sql 更新 whiteboard 表 DDL;DB 旧表 DROP 重建(开发环境)。
- 测试脚本与 README 同步更新。
2026-07-21 15:00:25 +00:00
zikai
7bacc33679 fix: fmtBytes 大文件显示成 PiB(累除逻辑错误)+ 改用 KB/MB/GB 单位
原实现循环里 n = x / 1024 每次都用原始 x 除一次,未累除,导致大文件
一路跌到 PiB 分支显示错误值。改为 while 循环每次 x /= 1000 累除,
单位改为 B/KB/MB/GB/TB/PB(1000 进制),B 显示整数其余保留 1 位小数。
2026-07-21 14:52:44 +00:00
zikai
7188c62d3a refactor: 白板路由改 /wb 前缀,消除 HTML 与 REST 同路径冲突
问题:GET /whiteboard/{board_id} 同时被 REST(返回 JSON)与 main.py 的 HTML 页面
注册,FastAPI 按注册顺序匹配到 REST,导致浏览器访问拿到 JSON 而非前端页面。

改为按职责分命名空间,避免冲突:
- HTML 页面:/wb/{id}(main.py)、/wb-admin(main.py,Basic Auth)
- 公开 REST:GET /api/wb/{id}(前端 init 拉取初始笔画)
- WS:/ws/wb/{id}(实时同步 + 心跳)
- 管理 REST:GET /api/admin/wb、DELETE /api/admin/wb/{id}(Basic Auth)

前端 whiteboard.js / whiteboard_admin.js、测试脚本、README 路径同步更新。
旧 /whiteboard/* 路径不再注册(404)。
2026-07-21 14:51:33 +00:00
zikai
3284269399 fix: 复选框初始状态错误(el 对布尔属性用 setAttribute 导致 checked:false 仍勾选)
common.js 的 el() 对 checked/disabled/hidden 等布尔属性调用 setAttribute(k, v),
而 setAttribute("checked", "false") 仍会令 checkbox 呈勾选态(属性存在即为真)。
改为维护 BOOL_PROPS 集合,布尔属性用 node[k] = !!v 赋值;同时 v === false 时跳过 setAttribute。
修复后文件浏览页复选框初始未选中时正确显示为未勾选。
2026-07-21 14:41:10 +00:00
50 changed files with 2204 additions and 603 deletions

269
README.md
View File

@@ -1,213 +1,124 @@
# zikai file service # zTools2 - 个人 Web 服务后端
`f.zikai.wang` 的 Python Web 服务FastAPI提供主机监控、大文件上传、共享白板与 基于 FastAPI 的个人 Web 服务后端,提供文件上传/浏览/下载、共享记事本实时协作、主机监控、SFTP 暂存、反向隧道、PDF 转换。采用 Spring 风格分层架构controller -> service -> dao -> ORM model -> MySQL自带 API 文档。
文件浏览:**HTTP整文件 + 分片/断点续传)**,并内置 **SFTP 服务器** 用于原始文件暂存。
采用 Spring 风格分层架构(`controllers` -> `services` -> `dao`,外加 `models`
`schemas`),自带自动生成的 API 文档,全部运行在自包含的 `.venv` 中。
## 功能 所有入口统一挂在 `/api/` 前缀下,前端用同源相对路径调用,无需区分 dev/prod。前端子项目[timeTableFix](https://git.zikai.wang/zikai/timeTableFix)、[zPDF_package](https://git.zikai.wang/zikai/zPDF_package)、[zWhiteBoard](https://git.zikai.wang/zikai/zWhiteBoard))经 [zMainPage](https://git.zikai.wang/zikai/zMainPage) 构建期组件 import 集成,生产由 apache2 静态托管 + `/api` 反代到本服务。
- `GET /api/system/status` - CPU、内存、各磁盘使用率via `psutil`)。 ## 项目结构
- `POST /api/files/upload` - **流式** multipart 上传(内存恒定,支持多 GB落盘时算 SHA-256。
- `GET /upload` - 拖拽上传页面:多文件、**分片4 MiB**、**断点续传**、sha256 去重。
- `POST /api/files/chunk-uploads/*` - 支撑 `/upload` 的分片上传 API建会话 / 查状态 / 传分片 / 完成)。
- `GET /api/files``GET /api/files/{id}``GET /api/files/{id}/download`
- **文件浏览页** `GET /files`Basic Auth同 docs列出/下载/**硬删除**已上传文件;删除后不再显示。
管理 API`GET /api/admin/files``GET /api/admin/files/{id}``GET /api/admin/files/{id}/download`
`DELETE /api/admin/files/{id}`(均 Basic Auth
- **共享白板** `GET /whiteboard/{id}`公开不存在则新建Canvas 实时协作 + **清空 / 复制链接**,兼容移动端。
实时同步走 `WS /ws/whiteboard/{id}`**心跳 3s连续 5 次丢失判失活并移除**)。
- **白板管理页** `GET /whiteboard-admin`Basic Auth同 docs查看创建时间/修改次数/上次修改时间/删除。
管理 API`GET /api/admin/whiteboards``DELETE /api/admin/whiteboards/{id}`(均 Basic Auth
- **反向隧道反代**`ALL /api/userPort/{userName}` -- 把请求经 SSH 反向隧道转发到该 user 的本机服务。
- **内置 SFTP/SSH 服务器**asyncssh支持 **密码 + 公钥** 鉴权,同时承载 SFTP 文件暂存与反向隧道。
- `/docs`Swagger UI`/redoc` - 交互式文档,自动列出所有 API。
- 元数据持久化在 **独立的 MySQL 数据库**`zikai_filesvc`)。
- `start.sh` / `stop.sh` 生命周期管理;`setup.sh` 一次性初始化。
## 架构Spring 风格分层)
``` ```
app/ zTools2/
├── controllers/ # FastAPI 路由 -- HTTP 边界(类似 @RestController ├── app/
├── services/ # 业务逻辑SystemService, UploadService, ChunkUploadService, │ ├── main.py # FastAPI 应用工厂、路由注册、生命周期reaper
# WhiteboardService, WhiteboardHub, SFTP 服务 ├── config.py # 从 config.yaml 加载的类型化 Settingspydantic-settings
├── dao/ # 数据访问对象 -- 唯一发出 SQL/ORM 的层 ├── database.py # SQLAlchemy 引擎/Session/Base/get_db 依赖 + schema 校验
├── models/ # SQLAlchemy ORM 实体UploadedFile, UploadSession, Whiteboard, ... │ ├── security.py # Basic Authrequire_docs_auth常量时间比较
├── schemas/ # pydantic DTO请求/响应校验) │ ├── controllers/ # 路由层file/system/chunk/tunnel/whiteboard/pdf/admin
├── views/ # 服务端渲染的 HTML 页面(系统状态、上传页) │ ├── services/ # 业务层Upload/ChunkUpload/System/Whiteboard/Tunnel/Pdf/sftp
├── static/ # 前端静态资源(文件浏览/白板/白板管理的 HTML+JS+CSS经 StaticFiles 挂载 │ ├── dao/ # 数据访问层:唯一发 SQL 的层SQLAlchemy ORM 参数化
├── database.py # 引擎、Session、Base、get_db() 依赖 │ ├── models/ # ORM 实体UploadedFile/UploadSession/Whiteboard/TunnelSession/PdfJob
├── config.py # 从 config.yaml 加载的类型化 Settings │ ├── schemas/ # pydantic 请求/响应 DTO
└── scripts/ # init_db.py -- 数据库初始化 │ ├── views/ # 服务端渲染 HTML系统状态页、上传页
│ ├── static/ # 前端静态资源
│ └── scripts/init_db.py # 数据库初始化(建库建账、随机密码写回 config.yaml
├── sql/schema.sql # 建表 DDL参考实际由 ORM 自动建表)
├── config.example.yaml # 配置模板(含注释)
├── config.yaml # 实际配置git-ignored含密码
├── requirements.txt
├── setup.sh # 一次性初始化venv + 依赖 + 建库 + SFTP 密钥
├── start.sh / stop.sh # 启停 HTTP127.0.0.1:6867+ SFTP2022
└── deploy/ # systemd 持久化部署
``` ```
请求流程**controller** -> **service** -> **dao** -> **ORM model** -> MySQL。 **请求流程**`controller -> service -> dao -> ORM model -> MySQL`。DB Session 由 `get_db` 依赖注入
DB Session 由 FastAPI 的 `get_db` 依赖注入并向下传递。前端三套页面走「独立静态文件 +
StaticFiles 挂载」的前后端分离模式HTML 壳由具名路由返回(便于各自挂 Basic Auth
JS 调用同源 `/api/...`
## 快速开始 ## 外部依赖
| 项 | 说明 |
|----|------|
| Python | ≥ 3.11(用 `.venv` |
| MySQL | 8.x独立库 `zikai_filesvc`,由 `setup.sh` 建账) |
| Apache2 | 反向代理对外提供 HTTPS服务本身只绑 `127.0.0.1:6867` |
| 系统库 | `libpango/cairo`weasyprint PDF 转换)、`build-essential`bcrypt/asyncssh 编译) |
### Apache2 反向代理配置
服务只绑 `127.0.0.1:6867`,通过 Apache 对外提供 HTTPS。安装模块并配置 vhost
```bash ```bash
cd /root/zikai a2enmod ssl proxy proxy_http proxy_wstunnel rewrite headers
./setup.sh # 一次性venv、依赖、建库建账、SFTP 主机密钥
./start.sh # 启动 HTTP127.0.0.1:6867+ SFTP0.0.0.0:2022
./stop.sh # 停止两者
``` ```
`setup.sh` 可重复执行。它会创建 `.venv`、安装 `requirements.txt`、复制 创建 `/etc/apache2/sites-available/f.zikai.wang.conf`(关键部分):
`config.example.yaml``config.yaml`(若不存在)、通过本机 root socket 建一个
**全新的独立 MySQL 数据库与应用账户**,并生成 SFTP 主机密钥。
## 访问方式 ```apache
<VirtualHost *:443>
ServerName f.zikai.wang
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/f.zikai.wang/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/f.zikai.wang/privkey.pem
| 入口 | URL | ProxyPreserveHost On
|------|-----| ProxyPass /fdata !
| 状态页HTML | https://f.zikai.wang/api/system/status | # 所有 zTools2 入口(页面/静态/探针/WS/API统一在 /api/ 下,一条规则即可;
| 状态页JSON | https://f.zikai.wang/api/system/status?format=json`Accept: application/json` | # WebSocket 走 /api/ws/wb/{id},靠 proxy_wstunnel 透传 Upgrade 头
| API 文档Swagger | https://f.zikai.wang/docs **HTTP Basic Auth -- 见 config.yaml 的 `docs:` 段)** | ProxyPass /api/ http://127.0.0.1:6867/api/
| API 文档ReDoc | https://f.zikai.wang/redoc同样鉴权 | ProxyPassReverse /api/ http://127.0.0.1:6867/api/
| 上传页(拖拽、分片、断点续传) | https://f.zikai.wang/upload | ProxyTimeout 300
| 文件浏览页(列出/下载/删除) | https://f.zikai.wang/files **Basic Auth同 docs** | </VirtualHost>
| 共享白板(实时协作) | https://f.zikai.wang/whiteboard/{id}(公开,`{id}``[a-zA-Z0-9_-]{1,64}`,不存在则新建) |
| 白板管理页 | https://f.zikai.wang/whiteboard-admin **Basic Auth同 docs** |
| 上传curl | `curl -F file=@big.iso https://f.zikai.wang/api/files/upload` |
| SFTP | `sftp -P 2022 uploader@f.zikai.wang` |
`/api/system/status` 做内容协商:浏览器(`Accept: text/html`)拿到带进度条的可读页面;
API 客户端拿到 JSON。可用 `?format=html``?format=json` 强制指定。
`/docs``/redoc``/openapi.json` 需要 HTTP Basic Auth —— 浏览器会弹出登录框。用户名与
明文密码写在 `config.yaml``docs:` 段。`/health``/` 保持公开。
Apache`/etc/apache2/sites-available/f.zikai.wang-le-ssl.conf`)把 `f.zikai.wang` 反代到
`127.0.0.1:6867``ProxyPreserveHost On`),因此服务只绑 loopback。
> **大文件/慢速 HTTP 上传:** Apache 代理段继承全局 `Timeout 300`。多 GB 慢链路传输建议走
> **分片上传**`/upload` 页面或 `/api/files/chunk-uploads`,单片 4 MiB 在超时内可传完)或
> **SFTP**(完全绕过 HTTP 代理)。要提高 HTTP 上限可在 Apache vhost 加 `ProxyTimeout`/`Timeout`。
## 配置
所有运行时配置都在 **`config.yaml`**git-ignored。完整 schema 见 `config.example.yaml`
关键配置项:
- `server` — 绑定 host/port保持 `127.0.0.1:6867` 以对齐 Apache
- `database` — host/port/user/password/database。密码由 `setup.sh`/`init_db.py` 自动生成并写回。
- `storage.upload_dir``storage.chunk_bytes`(默认 1 MiB 流式分片)、`storage.chunk_session_dir`
(分片会话暂存目录)、`storage.chunk_session_ttl_seconds`(被放弃会话的存活秒数,默认 300
- `sftp` — enabled、host/port、host key + authorized_keys 路径、`users`
- `whiteboard` - `heartbeat_interval_seconds`(默认 3`heartbeat_miss_threshold`(默认 5`max_board_id_length`(默认 64`list_limit`(默认 100
### 设置 /docs 管理密码
直接编辑 `config.yaml`,无需哈希:
```yaml
docs:
enabled: true
username: admin
password: "your-plaintext-password"
realm: "zikai docs"
``` ```
然后 `./stop.sh && ./start.sh`。该文件 root 持有且仅在本机;比较使用常量时间
`secrets.compare_digest`)。
### 设置 SFTP 凭据
**密码鉴权** —— 生成 bcrypt hash 写入 `config.yaml`
```bash ```bash
.venv/bin/python -c "import bcrypt;print(bcrypt.hashpw(b'yourpass',bcrypt.gensalt()).decode())" a2ensite f.zikai.wang
# 输出粘贴到 sftp.users[].password_hash然后 ./stop.sh && ./start.sh systemctl reload apache2
``` ```
**公钥鉴权** —— 把每个客户端的公钥OpenSSH 格式)追加到 `keys/authorized_keys`(每行一个)。 > **防火墙**:放开 443HTTPS与 2022SFTP。6867 不对外(仅 loopback)。
`sftp.users[]` 中的用户随后可用任一方式登录 > **大文件上传**Apache 全局 `Timeout 300`,慢链路建议走分片上传(`/api/upload`)或 SFTP
### 重新生成数据库密码 ## 从零安装Ubuntu 22.04+
### 1. 安装系统依赖
```bash ```bash
.venv/bin/python -m app.scripts.init_db # 生成新随机密码 apt update
KEEP_DB_PASSWORD=1 .venv/bin/python -m app.scripts.init_db # 保留现有密码 apt install -y python3-venv python3-pip mysql-server apache2 \
libssl-dev build-essential # build-essential 给 bcrypt/asyncssh 编译
# PDF 转换依赖 weasyprint需 pango/cairo 系统库:
apt install -y libpango-1.0-0 libpangoft2-1.0-0 libcairo2 libgdk-pixbuf-2.0-0
``` ```
## SFTP 说明 ### 2. 获取代码并初始化
- SFTP 服务无法穿透 Apache 的 HTTP 代理,因此直接绑 `0.0.0.0:2022`。**请在防火墙放开 ```bash
2022 端口** 供外部客户端FileZilla/WinSCP/scp连接。 git clone <repo> /root/zikai
- 会话 chroot 到上传根目录(`uploads/`),与 HTTP 共用存储。 cd /root/zikai/zTools2
-`sftp.users` 中列出的用户可连接;只允许 SFTP无 shell/exec ./setup.sh # 创建 .venv + 装依赖 + 复制 config.yaml + 建 MySQL 库账 + 生成 SFTP 密钥
- SFTP 服务器作为文件暂存通道;不再提供 HTTP 登记接口。 ```
## 反向隧道 ### 3. 配置凭据
SSH 服务器2022同时承载 SFTP 文件暂存与反向隧道。隧道 user 在 `config.yaml` 编辑 `config.yaml`(详见 [`docs/configuration.md`](./docs/configuration.md)`docs.username/password`(管理页 Basic Auth`sftp.users[].password_hash`bcrypt、可选 `tunnel.users[]`
`tunnel.users[]` 独立配置(与 `sftp.users[]` 分开):
- user 端跑 `user/tunnel.py`,连 2022 请求 remote port forward 绑 `tunnel_port` ### 4. 启动
- `ZikaiSSHServer.server_requested` 校验该 user 是否允许绑该端口,记一条 `tunnel_session`
到 DBuser IP、local_port、tunnel_port、起止时间
- `GET /api/userPort/{userName}` 查 DB 该 user 活跃隧道的端口,反代到 `127.0.0.1:tunnel_port`
(经隧道回指 user 本地服务)。无活跃隧道返回 502。
- user 断开时 `connection_lost` 回调关闭 DB 会话(记 `ended_at`);另有启动 reaper 兜底
清理进程异常重启后的孤儿记录。
配置示例见 `config.example.yaml``tunnel:` 段。生成 bcrypt hash 的方式同 SFTP。 ```bash
./start.sh # 启动 HTTP(127.0.0.1:6867) + SFTP(0.0.0.0:2022)
./stop.sh # 停止
```
## 文件浏览页 ### 5. 持久化部署systemd
- `GET /files`Basic Auth同 docs渲染 `static/file_browser.html`JS 调同源管理 API。 ```bash
- 管理 API均 Basic Auth ./deploy/install-systemd.sh # 安装并启动 ztools2 服务(开机自启)
- `GET /api/admin/files?limit=&offset=` -> `{total, items:[UploadedFileOut]}` systemctl status ztools2
- `GET /api/admin/files/{id}` -> `UploadedFileOut` journalctl -u ztools2 -f
- `GET /api/admin/files/{id}/download` -> 文件流(磁盘缺失返回 410 ```
- `DELETE /api/admin/files/{id}` -> 硬删除:删 DB 行 + 删磁盘文件(`unlink missing_ok`)。
- **删除后不再显示**:列表每次进入或删除后重新 fetch前端不缓存DB 行已删,列表自然不含。
- 公开 `/api/files` 系列user.py 依赖的查重/查询/下载)保留不变。
## 共享白板 > 与 zMainPage 整体部署配合systemd 管后端,`zMainPage/deploy.sh --no-restart` 部署前端。
白板无鉴权,任何人凭 `/whiteboard/{id}` 即可访问并实时协作;`{id}` 须匹配 ## 了解更多
`[a-zA-Z0-9_-]{1,64}`,非法返回 400。访问不存在的 id 自动新建空板。白板长期留存
(存 MySQL `whiteboard` 表),进程重启后内容仍在。
### 实时同步与心跳 - [路由与访问入口](./docs/routes.md)
- [配置说明](./docs/configuration.md)
- 连接:`WS /ws/whiteboard/{id}`公开。JSON 文本帧协议: - [错误处理与日志约定](./docs/error-handling.md)
- client -> server`{"type":"hello","client_id":"..."}`(首帧,可选)、
`{"type":"ping"}`(心跳)、`{"type":"stroke","stroke":{points,color,width}}``{"type":"clear"}`
- server -> client`{"type":"init","strokes":[...],"stroke_count":n}``{"type":"pong"}`
`{"type":"stroke","stroke":{...},"client_id":"..."}`(广播给他人,不含发送者)、
`{"type":"cleared","client_id":"..."}`(广播给所有人)、`{"type":"error","msg":"..."}`
- **心跳**:客户端每 `whiteboard.heartbeat_interval_seconds`(默认 3s发一次 `ping`,服务端回 `pong`
并刷新计时。后台 reaper 每秒扫描,连续 `heartbeat_miss_threshold`(默认 5次未收到心跳
(即 15s判失活**关闭该连接并从 hub 移除**。
- **内存安全**`WhiteboardHub` 维护 `{board_id: set[Connection]}`
- `disconnect` 幂等,空 set 从 dict 删除(防 board 键无限增长);
- WS 主循环 `try/finally` 必调 `disconnect`,异常/断连均清理;
- `broadcast` 对单连接发送失败立即 `disconnect`,不影响其他连接;
- 删除白板时 `close_board` 关闭并清理该 board 的全部连接。
- **多 worker 限制**hub 是进程内存,多 uvicorn worker 下不同进程的连接不互通。生产部署需
保持 `server.workers: 1`,或后续接 Redis pub/sub 跨进程广播。
### 白板管理
- `GET /whiteboard-admin`Basic Auth同 docs渲染 `static/whiteboard_admin.html`
- 管理 API均 Basic Auth
- `GET /api/admin/whiteboards?limit=&offset=` -> `{total, items:[{board_id, stroke_count, created_at, updated_at}]}`
- `DELETE /api/admin/whiteboards/{id}` -> 删 DB 行 + 关闭该 board 所有在线 WS 连接。
## 临时文件清理
- `complete` 成功(含去重命中)后,会话目录 `uploads/.work/<upload_id>/` 立即删除。
- 被放弃的上传(`pending` 状态且超过 `chunk_session_ttl_seconds` 无活动,默认 5 分钟)由
**后台 reaper** 清理:每 60 秒扫一次,删 `.work/<upload_id>/` 目录 + DB 会话行。
- `start.sh` 启动时仍会兜底清掉残留的 `.work/``*.part`(进程异常退出时的半成品)。
## 日志与 pidfile
- HTTP 日志 → `logs/app.log`SFTP 日志 → `logs/sftp.log`
- pidfile`app.pid``sftp.pid``stop.sh` 使用)。

View File

@@ -102,10 +102,30 @@ class WhiteboardConfig(BaseModel):
heartbeat_miss_threshold: int = 5 heartbeat_miss_threshold: int = 5
# board_id 合法字符集与长度上限,防路径/注入 # board_id 合法字符集与长度上限,防路径/注入
max_board_id_length: int = 64 max_board_id_length: int = 64
# 单 board 并发连接上限,防资源耗尽(同 board 同时在线人数)
max_connections_per_board: int = 50
# 列表/管理页分页默认值 # 列表/管理页分页默认值
list_limit: int = 100 list_limit: int = 100
class PdfConfig(BaseModel):
"""PDF 转换服务配置。
用户侧(上传/查看/下载/软删)凭 httpOnly cookie 标识;管理侧(列表/硬删)
走 docs 同款 Basic Auth。原始文件与产物 PDF 复用 storage.upload_dir 落盘。
"""
# 单文件大小上限字节。250 MiB。
max_size_bytes: int = 250 * 1024 * 1024
# 转换超时(秒):超大/复杂文件兜底,避免长期占用 worker。
convert_timeout_seconds: int = 600
# 列表分页默认值
list_limit: int = 100
# 用户 cookie 名与有效期
cookie_name: str = "zk_pdf"
cookie_max_age_seconds: int = 365 * 24 * 3600
class Settings(BaseModel): class Settings(BaseModel):
server: ServerConfig = ServerConfig() server: ServerConfig = ServerConfig()
database: DatabaseConfig = DatabaseConfig() database: DatabaseConfig = DatabaseConfig()
@@ -114,6 +134,7 @@ class Settings(BaseModel):
docs: DocsConfig = DocsConfig() docs: DocsConfig = DocsConfig()
tunnel: TunnelConfig = TunnelConfig() tunnel: TunnelConfig = TunnelConfig()
whiteboard: WhiteboardConfig = WhiteboardConfig() whiteboard: WhiteboardConfig = WhiteboardConfig()
pdf: PdfConfig = PdfConfig()
def db_url(self) -> str: def db_url(self) -> str:
c = self.database c = self.database

View File

@@ -3,6 +3,7 @@
from .chunk_upload_controller import router as chunk_upload_router from .chunk_upload_controller import router as chunk_upload_router
from .file_admin_controller import router as file_admin_router from .file_admin_controller import router as file_admin_router
from .file_controller import router as file_router from .file_controller import router as file_router
from .pdf_controller import router as pdf_router
from .system_controller import router as system_router from .system_controller import router as system_router
from .tunnel_controller import router as tunnel_router from .tunnel_controller import router as tunnel_router
from .whiteboard_controller import router as whiteboard_router from .whiteboard_controller import router as whiteboard_router
@@ -11,6 +12,7 @@ __all__ = [
"chunk_upload_router", "chunk_upload_router",
"file_admin_router", "file_admin_router",
"file_router", "file_router",
"pdf_router",
"system_router", "system_router",
"tunnel_router", "tunnel_router",
"whiteboard_router", "whiteboard_router",

View File

@@ -14,8 +14,15 @@ from pydantic import BaseModel, Field
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from ..database import get_db from ..database import get_db
from ..dao.pdf_job_dao import PdfJobDAO
from ..dao.uploaded_file_dao import UploadedFileDAO from ..dao.uploaded_file_dao import UploadedFileDAO
from ..schemas.file import FileListResponse, UploadedFileOut from ..schemas.file import (
FileListResponse,
FileWithPdfListResponse,
PdfJobBrief,
UploadedFileOut,
UploadedFileWithPdfOut,
)
from ..security import require_docs_auth from ..security import require_docs_auth
from ..services.upload_service import UploadService from ..services.upload_service import UploadService
@@ -43,7 +50,8 @@ class BatchDeleteResult(BaseModel):
"", "",
response_model=FileListResponse, response_model=FileListResponse,
summary="列出已上传文件(需鉴权)", summary="列出已上传文件(需鉴权)",
description="供文件浏览页使用;字段同公开 /api/files但要求 docs Basic Auth。支持分页。", description="供文件浏览页使用;字段同公开 /api/files但要求 docs Basic Auth。支持分页。"
"返回前会触发磁盘扫描(频率限制 3s把 SFTP 上传但未入库的文件补录进来。",
) )
def list_files( def list_files(
limit: int = Query(100, ge=1, le=10000, description="每页条数1-10000"), limit: int = Query(100, ge=1, le=10000, description="每页条数1-10000"),
@@ -51,10 +59,58 @@ def list_files(
service: UploadService = Depends(_service), service: UploadService = Depends(_service),
_: str = Depends(require_docs_auth), _: str = Depends(require_docs_auth),
) -> FileListResponse: ) -> FileListResponse:
service.scan_sftp_files() # 频率限制内置3s 内重复只返回 DB 缓存
total, items = service.list_files(limit=limit, offset=offset) total, items = service.list_files(limit=limit, offset=offset)
return FileListResponse(total=total, items=items) return FileListResponse(total=total, items=items)
@router.get(
"/with-pdf",
response_model=FileWithPdfListResponse,
summary="列出已上传文件并附带 PDF 转换任务关联(需鉴权)",
description=(
"合并管理页使用:以 uploaded_files 为基础分页拉取,再内存匹配 pdf_jobs"
"为每个文件附带它作为 PDF 任务「源文件(epub)」或「产物(PDF)」时的状态、"
"进度与用户软删标记。匹配键为 pdf_job.source_file_id / output_file_id -> uploaded_file.id。"
"返回前会触发磁盘扫描(频率限制 3s把 SFTP 上传但未入库的文件补录进来。"
),
)
def list_files_with_pdf(
limit: int = Query(100, ge=1, le=10000, description="每页条数1-10000"),
offset: int = Query(0, ge=0, description="偏移量"),
service: UploadService = Depends(_service),
db: Session = Depends(get_db),
_: str = Depends(require_docs_auth),
) -> FileWithPdfListResponse:
service.scan_sftp_files() # 频率限制内置3s 内重复只返回 DB 缓存
total, items = service.list_files(limit=limit, offset=offset)
# 拉全部 pdf_jobs数据量小不分页建反查 map
jobs = PdfJobDAO(db).list_all(limit=10000, offset=0)
by_source: dict[int, list] = {}
by_output: dict[int, list] = {}
for j in jobs:
by_source.setdefault(j.source_file_id, []).append(j)
if j.output_file_id is not None:
by_output.setdefault(j.output_file_id, []).append(j)
out_items: list[UploadedFileWithPdfOut] = []
for f in items:
briefs: list[PdfJobBrief] = []
for j in by_source.get(f.id, []):
briefs.append(PdfJobBrief(
job_id=j.id, role="source", status=j.status,
progress=j.progress, user_deleted=j.user_deleted, deleted_at=j.deleted_at,
))
for j in by_output.get(f.id, []):
briefs.append(PdfJobBrief(
job_id=j.id, role="output", status=j.status,
progress=j.progress, user_deleted=j.user_deleted, deleted_at=j.deleted_at,
))
out_items.append(UploadedFileWithPdfOut(**f.model_dump(), pdf_jobs=briefs))
return FileWithPdfListResponse(total=total, items=out_items)
@router.post( @router.post(
"/batch-delete", "/batch-delete",
response_model=BatchDeleteResult, response_model=BatchDeleteResult,

View File

@@ -2,7 +2,7 @@
from __future__ import annotations from __future__ import annotations
from fastapi import APIRouter, Depends, File, HTTPException, UploadFile from fastapi import APIRouter, Depends, File, HTTPException, Query, UploadFile
from fastapi.responses import FileResponse from fastapi.responses import FileResponse
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -43,8 +43,8 @@ async def upload_file(
@router.get("", response_model=FileListResponse, summary="列出已上传的文件") @router.get("", response_model=FileListResponse, summary="列出已上传的文件")
def list_files( def list_files(
limit: int = 100, limit: int = Query(100, ge=1, le=10000),
offset: int = 0, offset: int = Query(0, ge=0),
service: UploadService = Depends(_service), service: UploadService = Depends(_service),
) -> FileListResponse: ) -> FileListResponse:
total, items = service.list_files(limit=limit, offset=offset) total, items = service.list_files(limit=limit, offset=offset)

View File

@@ -0,0 +1,181 @@
"""PDF 转换接口用户侧cookie 标识)+ 管理侧Basic Auth
路由:
POST /api/pdf/jobs 用户上传文件并提交转换(首次无 cookie 则下发)
GET /api/pdf/jobs 当前用户任务列表(仅未软删)
GET /api/pdf/jobs/{id} 单任务状态(轮询进度)
GET /api/pdf/jobs/{id}/download 下载产物 PDF
DELETE /api/pdf/jobs/{id} 用户软删(不再对用户展示,管理页仍可见)
GET /api/admin/pdf/jobs 管理页列表(全部,含已软删标记)
DELETE /api/admin/pdf/jobs/{id} 管理员硬删(真正删除磁盘与 DB
HTML 页面 /pdf用户与 /pdf-admin管理由 main.py 返回静态文件,
不在此 controller 注册,避免与 REST 同路径冲突。
"""
from __future__ import annotations
from fastapi import APIRouter, Cookie, Depends, File, HTTPException, Query, Request, UploadFile
from fastapi.responses import FileResponse, JSONResponse
from sqlalchemy.orm import Session
from ..config import get_settings
from ..database import get_db
from ..dao.pdf_job_dao import PdfJobDAO
from ..dao.uploaded_file_dao import UploadedFileDAO
from ..schemas.pdf import DeleteResult, PdfJobListResponse, PdfJobOut, PdfSubmitResponse
from ..security import require_docs_auth
from ..services.pdf_service import PdfService, new_owner_cookie
router = APIRouter(tags=["pdf"])
def _service(db: Session = Depends(get_db)) -> PdfService:
return PdfService(PdfJobDAO(db), UploadedFileDAO(db))
def _resolve_cookie(request: Request, zk_pdf: str | None = Cookie(default=None)) -> str:
"""解析用户 cookie无则生成新值由 controller 写入响应头)。
cookie 缺失时把新值挂到 request.state供响应阶段 set_cookie。
"""
if zk_pdf and len(zk_pdf) == 32:
return zk_pdf
new = new_owner_cookie()
request.state.new_pdf_cookie = new
return new
@router.post(
"/api/pdf/jobs",
response_model=PdfSubmitResponse,
summary="上传文件并提交 PDF 转换",
description=(
"multipart/form-data 上传 epub 文件≤250MB流式落盘后创建 pending 转换任务,"
"后台异步转换。用户凭 zk_pdf cookie 标识;首次无 cookie 时响应下发新 cookie。"
),
)
async def submit_job(
request: Request,
file: UploadFile = File(..., description="要转换的 epub 文件"),
service: PdfService = Depends(_service),
owner_cookie: str = Depends(_resolve_cookie),
) -> PdfSubmitResponse:
job, _source_id = service.submit(file, owner_cookie)
# 提交成功后触发后台转换
service.schedule_convert(job.id)
resp = PdfSubmitResponse(job=job, set_cookie=False)
new_cookie = getattr(request.state, "new_pdf_cookie", None)
if new_cookie:
resp.set_cookie = True
# 用 JSONResponse 显式 set_cookie 后返回模型体
cfg = get_settings().pdf
data = resp.model_dump(mode="json")
response = JSONResponse(data)
response.set_cookie(
key=cfg.cookie_name,
value=new_cookie,
max_age=cfg.cookie_max_age_seconds,
httponly=True,
samesite="lax",
path="/",
)
return response
return resp
@router.get(
"/api/pdf/jobs",
response_model=PdfJobListResponse,
summary="当前用户的任务列表",
description="按 zk_pdf cookie 返回该用户未软删的任务,按创建时间倒序。",
)
def list_jobs(
service: PdfService = Depends(_service),
owner_cookie: str = Depends(_resolve_cookie),
limit: int = Query(100, ge=1, le=500),
offset: int = Query(0, ge=0),
) -> PdfJobListResponse:
return service.list_for_user(owner_cookie, limit=limit, offset=offset)
@router.get(
"/api/pdf/jobs/{job_id}",
response_model=PdfJobOut,
summary="查询单任务状态(轮询进度)",
)
def get_job(
job_id: int,
service: PdfService = Depends(_service),
owner_cookie: str = Depends(_resolve_cookie),
) -> PdfJobOut:
return service.get_job(job_id, owner_cookie)
@router.get(
"/api/pdf/jobs/{job_id}/download",
summary="下载转换后的 PDF",
description="仅任务状态为 done 且归属本人未软删时可下载。",
)
def download_job(
job_id: int,
service: PdfService = Depends(_service),
owner_cookie: str = Depends(_resolve_cookie),
) -> FileResponse:
_job, path, filename = service.get_output_path(job_id, owner_cookie)
return FileResponse(
path=str(path),
media_type="application/pdf",
filename=filename,
)
@router.delete(
"/api/pdf/jobs/{job_id}",
response_model=DeleteResult,
summary="用户软删任务(不再对用户展示)",
description="仅置 user_deleted 标记,磁盘与 DB 行保留;管理页仍可见并标注已删除。",
)
def delete_job(
job_id: int,
service: PdfService = Depends(_service),
owner_cookie: str = Depends(_resolve_cookie),
) -> DeleteResult:
ok = service.user_delete(job_id, owner_cookie)
if not ok:
raise HTTPException(404, "任务不存在")
return DeleteResult(deleted=True)
# ---------------- 管理侧Basic Auth ----------------
@router.get(
"/api/admin/pdf/jobs",
response_model=PdfJobListResponse,
summary="列出全部转换任务(需鉴权)",
description="管理页使用:含 user_deleted 标记,可看到用户是否已软删。",
)
def admin_list_jobs(
service: PdfService = Depends(_service),
_: str = Depends(require_docs_auth),
limit: int = Query(100, ge=1, le=500),
offset: int = Query(0, ge=0),
) -> PdfJobListResponse:
return service.list_for_admin(limit=limit, offset=offset)
@router.delete(
"/api/admin/pdf/jobs/{job_id}",
response_model=DeleteResult,
summary="管理员硬删任务(真正删除)",
description="删原始/产物磁盘文件 + UploadedFile 行 + PdfJob 行,不可恢复。",
)
def admin_delete_job(
job_id: int,
service: PdfService = Depends(_service),
_: str = Depends(require_docs_auth),
) -> DeleteResult:
ok = service.admin_delete(job_id)
if not ok:
raise HTTPException(404, "任务不存在")
return DeleteResult(deleted=True)

View File

@@ -1,10 +1,13 @@
"""白板接口REST访问/管理)+ WebSocket实时同步 """白板接口REST访问/管理)+ WebSocket实时同步
路由: 路由:
GET /whiteboard/{board_id} 公开:访问白板,不存在则新建 GET /api/wb/{board_id} 公开:访问记事本元数据,不存在则新建(前端 init 用)
WS /ws/whiteboard/{board_id} 公开:实时协作 + 心跳 WS /ws/wb/{board_id} 公开:实时协作 + 心跳
GET /api/admin/whiteboards Basic Auth管理页列表 GET /api/admin/wb Basic Auth管理页列表
DELETE /api/admin/whiteboards/{id} Basic Auth删除白板 DELETE /api/admin/wb/{board_id} Basic Auth删除记事本
HTML 页面 /wb/{id} 与管理页 /wb-admin 由 main.py 直接返回静态文件,
不在此 controller 注册,避免与 REST 同路径冲突。
""" """
from __future__ import annotations from __future__ import annotations
@@ -29,17 +32,17 @@ router = APIRouter(tags=["whiteboard"])
def _service(db: Session = Depends(get_db)) -> WhiteboardService: def _service(db: Session = Depends(get_db)) -> WhiteboardService:
"""REST 路径的 service:注入 hub 以便删除时踢出连接""" """REST 路径的 service(纯 DB 操作)"""
return WhiteboardService(WhiteboardDAO(db), hub=get_hub()) return WhiteboardService(WhiteboardDAO(db))
# ---------------- 公开 REST ---------------- # ---------------- 公开 REST ----------------
@router.get( @router.get(
"/whiteboard/{board_id}", "/api/wb/{board_id}",
response_model=WhiteboardOut, response_model=WhiteboardOut,
summary="访问白板(不存在则新建)", summary="访问记事本元数据(不存在则新建)",
description="任何人凭 board_id 即可访问;不存在时自动创建空板并返回", description="前端打开 /wb/{id} 页面后调本接口拉取初始文本;不存在时自动创建空板。",
) )
def get_whiteboard(board_id: str, service: WhiteboardService = Depends(_service)) -> WhiteboardOut: def get_whiteboard(board_id: str, service: WhiteboardService = Depends(_service)) -> WhiteboardOut:
return service.get_or_create(board_id) return service.get_or_create(board_id)
@@ -48,10 +51,10 @@ def get_whiteboard(board_id: str, service: WhiteboardService = Depends(_service)
# ---------------- 管理 RESTBasic Auth ---------------- # ---------------- 管理 RESTBasic Auth ----------------
@router.get( @router.get(
"/api/admin/whiteboards", "/api/admin/wb",
response_model=WhiteboardListResponse, response_model=WhiteboardListResponse,
summary="列出所有白板(需鉴权)", summary="列出所有记事本(需鉴权)",
description="白板管理页使用board_id / 创建时间 / 修改次数 / 上次修改时间。", description="记事本管理页使用board_id / 创建时间 / 编辑次数 / 上次修改时间。",
) )
def list_whiteboards( def list_whiteboards(
limit: int = Query(100, ge=1, le=500), limit: int = Query(100, ge=1, le=500),
@@ -64,11 +67,11 @@ def list_whiteboards(
@router.delete( @router.delete(
"/api/admin/whiteboards/{board_id}", "/api/admin/wb/{board_id}",
summary="删除白板(需鉴权)", summary="删除记事本(需鉴权)",
description="删 DB 行,并关闭该 board 的所有在线 WebSocket 连接。", description="删 DB 行,并关闭该 board 的所有在线 WebSocket 连接。",
) )
def delete_whiteboard( async def delete_whiteboard(
board_id: str, board_id: str,
service: WhiteboardService = Depends(_service), service: WhiteboardService = Depends(_service),
_: str = Depends(require_docs_auth), _: str = Depends(require_docs_auth),
@@ -76,25 +79,27 @@ def delete_whiteboard(
ok = service.delete(board_id) ok = service.delete(board_id)
if not ok: if not ok:
raise HTTPException(404, "白板不存在") raise HTTPException(404, "白板不存在")
# 删除成功后踢出该 board 的所有在线连接close_board 是 async须在事件循环中调用
await get_hub().close_board(board_id)
return {"deleted": True} return {"deleted": True}
# ---------------- WebSocket公开实时同步 + 心跳) ---------------- # ---------------- WebSocket公开实时同步 + 心跳) ----------------
@router.websocket("/ws/whiteboard/{board_id}") @router.websocket("/api/ws/wb/{board_id}")
async def whiteboard_ws(websocket: WebSocket, board_id: str) -> None: async def whiteboard_ws(websocket: WebSocket, board_id: str) -> None:
"""白板实时协作端点。 """白板实时协作端点(文本记事本)
协议JSON 文本帧): 协议JSON 文本帧):
client -> server: client -> server:
{"type":"hello","client_id":"..."} 首帧(可选,未带则服务端生成) {"type":"hello","client_id":"..."} 首帧(可选,未带则服务端生成)
{"type":"ping"} 心跳,服务端回 pong 并刷新计时 {"type":"ping"} 心跳,服务端回 pong 并刷新计时
{"type":"stroke","stroke":{...}} 新增笔画,持久化并广播给他人 {"type":"edit","content":"..."} debounce 后发完整文本,持久化并广播给他人
{"type":"clear"} 清空,持久化并广播给所有人 {"type":"clear"} 清空,持久化并广播给所有人
server -> client: server -> client:
{"type":"init","strokes":[...],"stroke_count":n} {"type":"init","content":"...","version":n,"edit_count":m}
{"type":"pong"} {"type":"pong"}
{"type":"stroke","stroke":{...},"client_id":"..."} {"type":"update","content":"...","version":n,"client_id":"..."} 文本变更
{"type":"cleared","client_id":"..."} {"type":"cleared","client_id":"..."}
{"type":"error","msg":"..."} {"type":"error","msg":"..."}
""" """
@@ -112,9 +117,8 @@ async def whiteboard_ws(websocket: WebSocket, board_id: str) -> None:
client_id = _extract_client_id(first) or uuid.uuid4().hex[:12] client_id = _extract_client_id(first) or uuid.uuid4().hex[:12]
# 校验 board_id 并加载白板(不存在则新建) # 校验 board_id 并加载白板(不存在则新建)
from ..database import get_session_local
try: try:
board = _with_db(lambda db: WhiteboardService(WhiteboardDAO(db), hub=hub).get_or_create(board_id)) board = _with_db(lambda db: WhiteboardService(WhiteboardDAO(db)).get_or_create(board_id))
except HTTPException as exc: except HTTPException as exc:
await _safe_send(websocket, {"type": "error", "msg": exc.detail}) await _safe_send(websocket, {"type": "error", "msg": exc.detail})
await _safe_close(websocket) await _safe_close(websocket)
@@ -122,17 +126,27 @@ async def whiteboard_ws(websocket: WebSocket, board_id: str) -> None:
# 注册连接并下发 init # 注册连接并下发 init
conn = Connection(websocket=websocket, board_id=board_id, client_id=client_id) conn = Connection(websocket=websocket, board_id=board_id, client_id=client_id)
await hub.register(conn) ok = await hub.register(conn)
if not ok:
await _safe_send(websocket, {"type": "error", "msg": "该记事本在线人数已满"})
await _safe_close(websocket)
return
await _safe_send(websocket, { await _safe_send(websocket, {
"type": "init", "type": "init",
"strokes": board.strokes, "content": board.content,
"stroke_count": board.stroke_count, "version": board.version,
"edit_count": board.edit_count,
}) })
# 主循环:收消息 -> 处理 -> 广播 # 主循环:收消息 -> 处理 -> 广播
# 单帧大小上限:与 content 限制对齐256KB 文本 + JSON 开销,留余量到 512KB
MAX_FRAME = 512 * 1024
try: try:
while True: while True:
raw = await websocket.receive_text() raw = await websocket.receive_text()
if len(raw) > MAX_FRAME:
await _safe_send(websocket, {"type": "error", "msg": "消息过大"})
continue
msg = _parse(raw) msg = _parse(raw)
if msg is None: if msg is None:
continue continue
@@ -143,22 +157,27 @@ async def whiteboard_ws(websocket: WebSocket, board_id: str) -> None:
continue continue
# 任何有效业务帧都视为活性证据 # 任何有效业务帧都视为活性证据
conn.touch() conn.touch()
if mtype == "stroke": if mtype == "edit":
stroke = msg.get("stroke") or {} content = msg.get("content")
if not isinstance(content, str):
await _safe_send(websocket, {"type": "error", "msg": "content 必须是字符串"})
continue
try: try:
_with_db(lambda db: WhiteboardService(WhiteboardDAO(db), hub=hub).append_stroke(board_id, stroke)) out = _with_db(
lambda db: WhiteboardService(WhiteboardDAO(db)).update_content(board_id, content)
)
except HTTPException as exc: except HTTPException as exc:
await _safe_send(websocket, {"type": "error", "msg": exc.detail}) await _safe_send(websocket, {"type": "error", "msg": exc.detail})
continue continue
# 广播给他人(发送者本地已,不回推) # 广播给他人(发送者本地已更新,不回推)
await hub.broadcast( await hub.broadcast(
board_id, board_id,
{"type": "stroke", "stroke": stroke, "client_id": client_id}, {"type": "update", "content": out.content, "version": out.version, "client_id": client_id},
exclude=conn, exclude=conn,
) )
elif mtype == "clear": elif mtype == "clear":
try: try:
_with_db(lambda db: WhiteboardService(WhiteboardDAO(db), hub=hub).clear(board_id)) _with_db(lambda db: WhiteboardService(WhiteboardDAO(db)).clear(board_id))
except HTTPException as exc: except HTTPException as exc:
await _safe_send(websocket, {"type": "error", "msg": exc.detail}) await _safe_send(websocket, {"type": "error", "msg": exc.detail})
continue continue
@@ -209,12 +228,12 @@ def _parse(raw: str) -> dict | None:
async def _safe_send(ws: WebSocket, msg: dict) -> None: async def _safe_send(ws: WebSocket, msg: dict) -> None:
try: try:
await ws.send_json(msg) await ws.send_json(msg)
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
pass logger.debug("发送 WS 消息失败: %s", exc)
async def _safe_close(ws: WebSocket) -> None: async def _safe_close(ws: WebSocket) -> None:
try: try:
await ws.close() await ws.close()
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
pass logger.debug("关闭 WS 失败: %s", exc)

71
app/dao/pdf_job_dao.py Normal file
View File

@@ -0,0 +1,71 @@
"""PdfJob 的 DAO。
所有写操作均在该层 commitservice 不直接操作 session。
用户/管理两条查询路径分别按 user_deleted 过滤。
"""
from __future__ import annotations
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from ..models.pdf_job import PdfJob
class PdfJobDAO:
def __init__(self, db: Session) -> None:
self.db = db
def create(self, job: PdfJob) -> PdfJob:
self.db.add(job)
self.db.commit()
self.db.refresh(job)
return job
def get(self, job_id: int) -> PdfJob | None:
return self.db.get(PdfJob, job_id)
def update(self, job: PdfJob) -> PdfJob:
"""提交对 job 的就地修改并刷新。"""
self.db.commit()
self.db.refresh(job)
return job
def delete(self, job: PdfJob) -> None:
"""硬删 PdfJob 行(管理视角真正删除,不可恢复)。"""
self.db.delete(job)
self.db.commit()
def list_for_user(self, owner_cookie: str, limit: int = 100, offset: int = 0) -> list[PdfJob]:
"""用户视角:仅未软删的任务,按创建时间倒序。"""
stmt = (
select(PdfJob)
.where(PdfJob.owner_cookie == owner_cookie)
.where(PdfJob.user_deleted == False) # noqa: E712
.order_by(PdfJob.created_at.desc())
.limit(limit)
.offset(offset)
)
return list(self.db.scalars(stmt).all())
def count_for_user(self, owner_cookie: str) -> int:
stmt = (
select(func.count())
.select_from(PdfJob)
.where(PdfJob.owner_cookie == owner_cookie)
.where(PdfJob.user_deleted == False) # noqa: E712
)
return self.db.scalar(stmt) or 0
def list_all(self, limit: int = 100, offset: int = 0) -> list[PdfJob]:
"""管理视角:全部任务(含已软删),按创建时间倒序。"""
stmt = (
select(PdfJob)
.order_by(PdfJob.created_at.desc())
.limit(limit)
.offset(offset)
)
return list(self.db.scalars(stmt).all())
def count_all(self) -> int:
return self.db.scalar(select(func.count()).select_from(PdfJob)) or 0

View File

@@ -31,15 +31,6 @@ class TunnelSessionDAO:
) )
return self.db.scalars(stmt).first() return self.db.scalars(stmt).first()
def get_active_by_port(self, tunnel_port: int) -> TunnelSession | None:
stmt = (
select(TunnelSession)
.where(TunnelSession.tunnel_port == tunnel_port)
.where(TunnelSession.status == "active")
.limit(1)
)
return self.db.scalars(stmt).first()
def list_active(self) -> list[TunnelSession]: def list_active(self) -> list[TunnelSession]:
stmt = select(TunnelSession).where(TunnelSession.status == "active") stmt = select(TunnelSession).where(TunnelSession.status == "active")
return list(self.db.scalars(stmt).all()) return list(self.db.scalars(stmt).all())

View File

@@ -29,6 +29,11 @@ class UploadedFileDAO:
"""返回数据库中文件总条数。""" """返回数据库中文件总条数。"""
return self.db.scalar(select(func.count()).select_from(UploadedFile)) or 0 return self.db.scalar(select(func.count()).select_from(UploadedFile)) or 0
def list_storage_paths(self) -> set[str]:
"""返回所有已记录的 storage_path 集合(供磁盘扫描比对,识别未入库文件)。"""
stmt = select(UploadedFile.storage_path)
return {row for row in self.db.scalars(stmt).all()}
def list(self, limit: int = 100, offset: int = 0) -> list[UploadedFile]: def list(self, limit: int = 100, offset: int = 0) -> list[UploadedFile]:
stmt = ( stmt = (
select(UploadedFile) select(UploadedFile)

View File

@@ -1,18 +1,21 @@
"""Whiteboard 的 DAO。 """Whiteboard 的 DAO(文本记事本)
所有写操作均在该层 commitservice 不直接操作 session。 所有写操作均在该层 commitservice 不直接操作 session。
get_or_create 用于「访问即新建」语义(路由 GET /whiteboard/{id} 不存在则建)。 get_or_create 用于「访问即新建」语义(路由 GET /api/wb/{id} 不存在则建)。
""" """
from __future__ import annotations from __future__ import annotations
from typing import Any import logging
from sqlalchemy import func, select from sqlalchemy import func, select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from ..models.whiteboard import Whiteboard from ..models.whiteboard import Whiteboard
logger = logging.getLogger("zikai.whiteboard")
class WhiteboardDAO: class WhiteboardDAO:
def __init__(self, db: Session) -> None: def __init__(self, db: Session) -> None:
@@ -29,36 +32,30 @@ class WhiteboardDAO:
return self.db.scalars(stmt).first() return self.db.scalars(stmt).first()
def get_or_create(self, board_id: str) -> Whiteboard: def get_or_create(self, board_id: str) -> Whiteboard:
"""存在则返回,否则新建空板。利用 unique 约束兜底并发首访。""" """存在则返回,否则新建空板。利用 unique 约束兜底并发首访。
仅 IntegrityError并发下另一事务已插入违反唯一约束才回滚重读
其他异常向上抛,避免掩盖 schema/连接等真实故障。
"""
board = self.get(board_id) board = self.get(board_id)
if board is not None: if board is not None:
return board return board
board = Whiteboard(board_id=board_id, strokes=[], stroke_count=0) board = Whiteboard(board_id=board_id, content="", version=0, edit_count=0)
try: try:
return self.create(board) return self.create(board)
except Exception: except IntegrityError:
# 并发下另一事务已插入:回滚后重新读 # 并发下另一事务已插入:回滚后重新读
self.db.rollback() self.db.rollback()
return self.get(board_id) # type: ignore[return-value] return self.get(board_id) # type: ignore[return-value]
def append_strokes(self, board_id: str, new_strokes: list[Any]) -> Whiteboard | None: def update_content(self, board_id: str, content: str) -> Whiteboard | None:
"""把新笔画追加到 strokes 数组尾部stroke_count 自增""" """整体替换文本内容version +1、edit_count +1"""
board = self.get(board_id) board = self.get(board_id)
if board is None: if board is None:
return None return None
board.strokes = [*board.strokes, *new_strokes] board.content = content
board.stroke_count = (board.stroke_count or 0) + len(new_strokes) board.version = (board.version or 0) + 1
self.db.commit() board.edit_count = (board.edit_count or 0) + 1
self.db.refresh(board)
return board
def replace_strokes(self, board_id: str, strokes: list[Any]) -> Whiteboard | None:
"""整体替换 strokes清空时传 []stroke_count 自增 1。"""
board = self.get(board_id)
if board is None:
return None
board.strokes = list(strokes)
board.stroke_count = (board.stroke_count or 0) + 1
self.db.commit() self.db.commit()
self.db.refresh(board) self.db.refresh(board)
return board return board

View File

@@ -64,7 +64,29 @@ def get_db() -> Generator[Session, None, None]:
def init_db_schema() -> None: def init_db_schema() -> None:
"""按需建表(幂等)。先导入 models 以注册映射。""" """按需建表(幂等)并校验既有表列与模型一致fail-fast on schema drift
先导入 models 注册映射create_all 用 IF NOT EXISTS 仅补缺失的表;
随后对每张已存在的表检查模型声明的列是否齐全,缺列即抛 RuntimeError
避免运行期才以晦涩的 OperationalError 暴露 schema 漂移。
"""
from sqlalchemy import inspect
from . import models # noqa: F401 from . import models # noqa: F401
get_engine() engine = get_engine()
Base.metadata.create_all(bind=_engine) Base.metadata.create_all(bind=engine)
inspector = inspect(engine)
missing: list[str] = []
for table, mapper in Base.registry.mappers.items():
if not inspector.has_table(table):
continue
db_cols = {c["name"] for c in inspector.get_columns(table)}
for model_col in mapper.columns.keys():
if model_col not in db_cols:
missing.append(f"{table}.{model_col}")
if missing:
raise RuntimeError(
"数据库 schema 与模型不一致,缺少列: " + ", ".join(missing)
+ "。请执行 sql/schema.sql 或迁移脚本更新表结构。"
)

View File

@@ -6,12 +6,13 @@
GET /redoc -> ReDoc Basic Auth GET /redoc -> ReDoc Basic Auth
GET /openapi.json -> OpenAPI 文档Basic Auth GET /openapi.json -> OpenAPI 文档Basic Auth
GET /health -> 存活探针(公开) GET /health -> 存活探针(公开)
GET /upload -> 上传页面(公开 HTML GET /api/index -> 导航页(公开,收集所有页面入口
GET /files -> 文件浏览页Basic Auth同 docs GET /api/upload -> 上传页面(公开 HTML
GET /whiteboard/{id} -> 白板页面(公开,不存在则新建 GET /api/files-page -> 文件管理页Basic Auth同 docs含 PDF 转换管理
GET /whiteboard-admin -> 白板管理页Basic Auth同 docs GET /api/wb/{id} -> 白板页面(公开,不存在则新建
GET /api/wb-admin -> 白板管理页Basic Auth同 docs
GET /api/... -> 业务接口 GET /api/... -> 业务接口
WS /ws/whiteboard/{id} -> 白板实时同步(公开) WS /ws/wb/{id} -> 白板实时同步(公开)
/static/... -> 前端静态资源JS/CSS /static/... -> 前端静态资源JS/CSS
""" """
@@ -31,6 +32,7 @@ from .controllers import (
chunk_upload_router, chunk_upload_router,
file_admin_router, file_admin_router,
file_router, file_router,
pdf_router,
system_router, system_router,
tunnel_router, tunnel_router,
whiteboard_router, whiteboard_router,
@@ -149,54 +151,91 @@ def create_app() -> FastAPI:
app.include_router(chunk_upload_router) app.include_router(chunk_upload_router)
app.include_router(tunnel_router) app.include_router(tunnel_router)
app.include_router(whiteboard_router) app.include_router(whiteboard_router)
app.include_router(pdf_router)
# 前端静态资源JS/CSSHTML 壳由下面的具名路由返回,便于各自挂 Basic Auth # 前端静态资源JS/CSSHTML 壳由下面的具名路由返回,便于各自挂 Basic Auth
# 统一 /api/ 前缀:所有 zTools2 入口(页面/静态/探针/WS/API都在 /api/ 下,
# 反代与 vite proxy 只需一条 /api/ 规则即可转发,与环境无关
if _STATIC_DIR.is_dir(): if _STATIC_DIR.is_dir():
app.mount("/static", StaticFiles(directory=str(_STATIC_DIR)), name="static") app.mount("/api/static", StaticFiles(directory=str(_STATIC_DIR)), name="static")
# 受 Basic Auth 保护的文档接口 # 受 Basic Auth 保护的文档接口
@app.get("/openapi.json") @app.get("/openapi.json", tags=["docs"], summary="OpenAPI 文档(需鉴权)")
def protected_openapi(_: str = Depends(require_docs_auth)) -> JSONResponse: def protected_openapi(_: str = Depends(require_docs_auth)) -> JSONResponse:
return JSONResponse(app.openapi()) return JSONResponse(app.openapi())
@app.get("/docs") @app.get("/docs", tags=["docs"], summary="Swagger UI需鉴权")
def protected_docs(_: str = Depends(require_docs_auth)): def protected_docs(_: str = Depends(require_docs_auth)):
return get_swagger_ui_html( return get_swagger_ui_html(
openapi_url="/openapi.json", title="zikai docs", swagger_favicon_url="" openapi_url="/openapi.json", title="zikai docs", swagger_favicon_url=""
) )
@app.get("/redoc") @app.get("/redoc", tags=["docs"], summary="ReDoc需鉴权")
def protected_redoc(_: str = Depends(require_docs_auth)): def protected_redoc(_: str = Depends(require_docs_auth)):
return get_redoc_html( return get_redoc_html(
openapi_url="/openapi.json", title="zikai docs", redoc_favicon_url="" openapi_url="/openapi.json", title="zikai docs", redoc_favicon_url=""
) )
@app.get("/", response_class=PlainTextResponse) @app.get("/", tags=["meta"], summary="版本号")
def root() -> PlainTextResponse: def root() -> PlainTextResponse:
return PlainTextResponse(f"zikai {app.version}\n") return PlainTextResponse(f"zikai {app.version}\n")
@app.get("/health") @app.get("/api/health", tags=["meta"], summary="存活探针")
def health() -> dict: def health() -> dict:
return {"status": "ok"} return {"status": "ok"}
@app.get("/upload", response_class=HTMLResponse) @app.get(
"/api/index",
response_class=HTMLResponse,
tags=["pages"],
summary="导航页(公开)",
description="收集所有页面入口的卡片式导航页,各子页脚注可返回此处。",
)
def index_page() -> HTMLResponse:
return _serve_static_html("index.html")
@app.get(
"/api/upload",
response_class=HTMLResponse,
tags=["pages"],
summary="上传页面",
description="拖拽 / 多文件 / 分片4 MiB / 断点续传上传页面(公开)。",
)
def upload_page() -> HTMLResponse: def upload_page() -> HTMLResponse:
"""拖拽 / 多文件 / 分片上传页面(公开,对齐 /api/files/upload"""
return HTMLResponse(render_upload_html()) return HTMLResponse(render_upload_html())
@app.get("/files", response_class=HTMLResponse) @app.get(
"/api/files-page",
response_class=HTMLResponse,
tags=["pages"],
summary="文件管理页(需鉴权)",
description=(
"列出 / 下载 / 删除已上传文件,并合并 PDF 转换管理:以 uploaded_files 为基础,"
"用 pdf_jobs 匹配标注关联文件的转换状态、用户软删标记,可硬删任务。"
"支持多选、批量下载删除与分页。Basic Auth 同 docs。"
),
)
def files_page(_: str = Depends(require_docs_auth)) -> HTMLResponse: def files_page(_: str = Depends(require_docs_auth)) -> HTMLResponse:
"""文件浏览页Basic Auth同 docs列出/下载/删除已上传文件。"""
return _serve_static_html("file_browser.html") return _serve_static_html("file_browser.html")
@app.get("/whiteboard-admin", response_class=HTMLResponse) @app.get(
"/api/wb-admin",
response_class=HTMLResponse,
tags=["pages"],
summary="记事本管理页(需鉴权)",
description="查看所有记事本的创建时间 / 编辑次数 / 上次修改时间并可删除。Basic Auth 同 docs。",
)
def whiteboard_admin_page(_: str = Depends(require_docs_auth)) -> HTMLResponse: def whiteboard_admin_page(_: str = Depends(require_docs_auth)) -> HTMLResponse:
"""白板管理页Basic Auth同 docs查看/删除白板。"""
return _serve_static_html("whiteboard_admin.html") return _serve_static_html("whiteboard_admin.html")
@app.get("/whiteboard/{board_id}", response_class=HTMLResponse) @app.get(
"/api/wb-page/{board_id}",
response_class=HTMLResponse,
tags=["pages"],
summary="记事本页面",
description="公开访问的共享文本记事本,不存在则自动新建;实时协作走 WS /api/ws/wb/{id}",
)
def whiteboard_page(board_id: str) -> HTMLResponse: def whiteboard_page(board_id: str) -> HTMLResponse:
"""白板页面(公开):访问即协作,不存在则前端拉取时自动新建。"""
return _serve_static_html("whiteboard.html") return _serve_static_html("whiteboard.html")
return app return app

View File

@@ -1,8 +1,9 @@
"""ORM 模型包import 本包即把所有实体注册到 Base.metadata。""" """ORM 模型包import 本包即把所有实体注册到 Base.metadata。"""
from .pdf_job import PdfJob
from .tunnel_session import TunnelSession from .tunnel_session import TunnelSession
from .uploaded_file import UploadedFile from .uploaded_file import UploadedFile
from .upload_session import UploadSession from .upload_session import UploadSession
from .whiteboard import Whiteboard from .whiteboard import Whiteboard
__all__ = ["TunnelSession", "UploadedFile", "UploadSession", "Whiteboard"] __all__ = ["PdfJob", "TunnelSession", "UploadedFile", "UploadSession", "Whiteboard"]

56
app/models/pdf_job.py Normal file
View File

@@ -0,0 +1,56 @@
"""PDF 转换任务实体。
一个任务记录一次「上传文件 -> 转为 PDF」的转换。原始上传文件与转换产物 PDF
均复用 UploadedFile 存储(落盘 + 元数据入库),本表只记录两者关系与转换状态,
不重复实现存储逻辑。
删除语义(两级):
用户软删user_deleted=true-- 用户页不再展示,但磁盘与 DB 行保留;
管理页仍可见且标注「已删除」。
管理员硬删 -- 删原始/产物磁盘文件 + UploadedFile 行 + 本表行,真正删除。
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import BigInteger, Boolean, DateTime, Integer, String, func
from sqlalchemy.orm import Mapped, mapped_column
from ..database import Base
class PdfJob(Base):
__tablename__ = "pdf_job"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True)
# 用户标识httpOnly cookie 值uuid4.hex用户凭此查看自己的任务
owner_cookie: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
# 原始上传文件(复用 UploadedFile 存储)
source_file_id: Mapped[int] = mapped_column(BigInteger, nullable=False, index=True)
source_filename: Mapped[str] = mapped_column(String(512), nullable=False)
source_size: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0)
# 转换产物 PDF复用 UploadedFile 存储);转换完成前为 NULL
output_file_id: Mapped[int | None] = mapped_column(BigInteger, nullable=True, default=None)
# pending(排队) / converting(转换中) / done(完成) / failed(失败)
status: Mapped[str] = mapped_column(String(16), nullable=False, default="pending", index=True)
# 转换进度 0-100供前端轮询显示
progress: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
# 失败原因status=failed 时填写)
error_message: Mapped[str] = mapped_column(String(512), nullable=False, default="")
# 用户软删标记true=用户已从其页面删除,不再对用户展示
user_deleted: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, index=True)
# 用户软删时间(管理页展示「是否已删除」时用)
deleted_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True, default=None)
created_at: Mapped[datetime] = mapped_column(
DateTime, server_default=func.now(), nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime, server_default=func.now(), onupdate=func.now(), nullable=False
)
def __repr__(self) -> str: # pragma: no cover
return (
f"<PdfJob id={self.id} status={self.status} "
f"src={self.source_filename!r} progress={self.progress}>"
)

View File

@@ -1,15 +1,16 @@
"""共享白板实体。 """共享白板实体(文本记事本)
一个白板由 board_id 唯一标识(用户可读的 url idstrokes 以 JSON 列保存全部笔画。 一个白板由 board_id 唯一标识(用户可读的 url idcontent 存完整文本,
白板长期留存,进程重启后仍可恢复;实时协作由 WebSocket hub 在内存中维护在线连接, version 是乐观锁版本号(每次编辑 +1白板长期留存,进程重启后仍可恢复;
笔画变更经 service 落库后由 hub 广播给同 board 的其它在线连接。 实时协作由 WebSocket hub 在内存中维护在线连接,文本变更经 service 落库后
由 hub 广播给同 board 的其它在线连接。
""" """
from __future__ import annotations from __future__ import annotations
from datetime import datetime from datetime import datetime
from sqlalchemy import BigInteger, DateTime, Integer, JSON, String, func from sqlalchemy import BigInteger, DateTime, Integer, String, Text, func
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from ..database import Base from ..database import Base
@@ -21,10 +22,12 @@ class Whiteboard(Base):
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True) id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True)
# 用户可读的 url id[a-zA-Z0-9_-]{1,64}),全局唯一 # 用户可读的 url id[a-zA-Z0-9_-]{1,64}),全局唯一
board_id: Mapped[str] = mapped_column(String(64), nullable=False, unique=True, index=True) board_id: Mapped[str] = mapped_column(String(64), nullable=False, unique=True, index=True)
# 笔画数组 [{points:[[x,y],...], color, width}, ...] # 白板文本内容(记事本)
strokes: Mapped[list] = mapped_column(JSON, nullable=False, default=list) content: Mapped[str] = mapped_column(Text, nullable=False, default="")
# 修改次数:每次新增笔画或清空 +1供管理页统计 # 乐观锁版本号:每次编辑 +1
stroke_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0) version: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
# 编辑次数(累计修改次数,含清空),供管理页统计
edit_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
created_at: Mapped[datetime] = mapped_column( created_at: Mapped[datetime] = mapped_column(
DateTime, server_default=func.now(), nullable=False DateTime, server_default=func.now(), nullable=False
) )
@@ -35,5 +38,5 @@ class Whiteboard(Base):
def __repr__(self) -> str: # pragma: no cover def __repr__(self) -> str: # pragma: no cover
return ( return (
f"<Whiteboard board_id={self.board_id!r} " f"<Whiteboard board_id={self.board_id!r} "
f"strokes={len(self.strokes)} mods={self.stroke_count}>" f"len={len(self.content)} v={self.version} mods={self.edit_count}>"
) )

View File

@@ -10,8 +10,6 @@ from .file import FileListResponse, FileUploadResponse, UploadedFileOut
from .system import DiskUsage, MemoryUsage, SystemStatus from .system import DiskUsage, MemoryUsage, SystemStatus
from .tunnel import TunnelStatusResponse from .tunnel import TunnelStatusResponse
from .whiteboard import ( from .whiteboard import (
Stroke,
StrokeOp,
WhiteboardListItem, WhiteboardListItem,
WhiteboardListResponse, WhiteboardListResponse,
WhiteboardOut, WhiteboardOut,
@@ -26,8 +24,6 @@ __all__ = [
"FileUploadResponse", "FileUploadResponse",
"MemoryUsage", "MemoryUsage",
"SessionStatusResponse", "SessionStatusResponse",
"Stroke",
"StrokeOp",
"SystemStatus", "SystemStatus",
"TunnelStatusResponse", "TunnelStatusResponse",
"UploadedFileOut", "UploadedFileOut",

View File

@@ -36,3 +36,31 @@ class FileUploadResponse(BaseModel):
class FileListResponse(BaseModel): class FileListResponse(BaseModel):
total: int total: int
items: list[UploadedFileOut] items: list[UploadedFileOut]
class PdfJobBrief(BaseModel):
"""文件关联到的 PDF 转换任务摘要(供合并管理页展示)。
一个 uploaded_file 可能同时被多个 job 引用(罕见),故为列表;
通常每行 0 或 1 条。
"""
job_id: int
role: str = Field(..., description='"source"=该文件是 epub 源文件;"output"=该文件是产物 PDF')
status: str = Field(..., description="pending / converting / done / failed")
progress: int = Field(0, description="转换进度 0-100")
user_deleted: bool = Field(False, description="用户是否已软删该任务")
deleted_at: datetime | None = Field(None, description="用户软删时间")
class UploadedFileWithPdfOut(UploadedFileOut):
"""带 PDF 任务关联信息的文件视图(合并管理页用)。"""
pdf_jobs: list[PdfJobBrief] = Field(default_factory=list, description="关联到的 PDF 转换任务")
class FileWithPdfListResponse(BaseModel):
"""合并管理页列表响应:以 uploaded_files 为基础,附带 pdf_jobs 关联。"""
total: int
items: list[UploadedFileWithPdfOut]

46
app/schemas/pdf.py Normal file
View File

@@ -0,0 +1,46 @@
"""PDF 转换接口 DTO。"""
from __future__ import annotations
from datetime import datetime
from pydantic import BaseModel, Field
class PdfJobOut(BaseModel):
"""任务对外视图用户与管理页共用user_deleted 仅管理页关注)。"""
id: int
source_file_id: int = Field(..., description="原始 epub 文件的 uploaded_file.id")
source_filename: str = Field(..., description="原始上传文件名")
source_size: int = Field(..., description="原始文件字节数")
output_file_id: int | None = Field(None, description="产物 PDF 的 uploaded_file.id转换完成前为 null")
status: str = Field(..., description="pending / converting / done / failed")
progress: int = Field(0, description="转换进度 0-100")
error_message: str = Field("", description="失败原因")
user_deleted: bool = Field(False, description="用户是否已软删(管理页用)")
deleted_at: datetime | None = Field(None, description="用户软删时间(管理页用)")
created_at: datetime
updated_at: datetime
model_config = {"from_attributes": True}
class PdfJobListResponse(BaseModel):
"""任务列表响应。"""
total: int
items: list[PdfJobOut]
class PdfSubmitResponse(BaseModel):
"""提交转换任务的响应。"""
job: PdfJobOut = Field(..., description="新建的任务")
set_cookie: bool = Field(
False, description="true=本次请求未带 cookie响应已下发新 cookie"
)
class DeleteResult(BaseModel):
deleted: bool

View File

@@ -1,27 +1,19 @@
"""白板接口 DTO。""" """白板接口 DTO(文本记事本)"""
from __future__ import annotations from __future__ import annotations
from datetime import datetime from datetime import datetime
from typing import Any
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
class Stroke(BaseModel):
"""一条笔画:点序列 + 样式。结构宽松Any以兼容前端扩展字段。"""
points: list[list[float]] = Field(default_factory=list, description="[[x,y],...]")
color: str = Field(default="#1565c0", description="笔画颜色")
width: float = Field(default=3, description="笔画宽度")
class WhiteboardOut(BaseModel): class WhiteboardOut(BaseModel):
"""白板完整内容GET /whiteboard/{id} 与 WS init 帧)。""" """白板完整内容GET /api/wb/{id} 与 WS init 帧)。"""
board_id: str board_id: str
strokes: list[Any] = Field(default_factory=list, description="笔画数组") content: str = Field("", description="白板文本内容")
stroke_count: int = Field(0, description="累计修改次数") version: int = Field(0, description="乐观锁版本号,每次编辑 +1")
edit_count: int = Field(0, description="累计编辑次数")
created_at: datetime created_at: datetime
updated_at: datetime updated_at: datetime
@@ -32,7 +24,7 @@ class WhiteboardListItem(BaseModel):
"""管理页列表项。""" """管理页列表项。"""
board_id: str board_id: str
stroke_count: int edit_count: int
created_at: datetime created_at: datetime
updated_at: datetime updated_at: datetime
@@ -44,10 +36,3 @@ class WhiteboardListResponse(BaseModel):
total: int total: int
items: list[WhiteboardListItem] items: list[WhiteboardListItem]
class StrokeOp(BaseModel):
"""WS 笔画操作type=stroke 时携带)。"""
type: str = Field(..., description="add / clear")
stroke: dict[str, Any] | None = Field(None, description="type=add 时携带的笔画对象")

View File

@@ -0,0 +1,104 @@
"""文件 -> PDF 转换器(纯 Python无 calibre/xvfb 系统依赖)。
当前支持 epub必要能力ebooklib 解析 epub 文档项(按 spine 顺序),拼接为
完整 HTML 后交 weasyprint 渲染为 PDF。epub 内的相对资源(图片/CSS经 base_url
指向 epub 解包目录解析。
接口抽象为 ``convert_to_pdf(src_path, dst_path)``:未来新增格式只需在本模块内
按扩展名分支调用方PdfService无需改动。
"""
from __future__ import annotations
import logging
import tempfile
import zipfile
from pathlib import Path
logger = logging.getLogger("zikai.pdf")
# 支持的输入格式(小写扩展名 -> 是否可转)。新增格式在此登记并实现分支即可。
# epub 为必要能力;其余为 ebooklib/weasyprint 路径天然支持的电子书结构,
# 实测对纯 HTML 类 epub 同样有效,故一并放行。
SUPPORTED_EXTENSIONS = {".epub"}
def is_supported(filename: str) -> bool:
"""文件名扩展名是否在支持列表内。"""
return Path(filename).suffix.lower() in SUPPORTED_EXTENSIONS
def convert_to_pdf(src_path: Path, dst_path: Path) -> None:
"""把 src_path 指向的文件转为 PDF 写入 dst_path。
失败抛 RuntimeError调用方捕获后写 error_message。按扩展名分发
当前仅 epub 分支;新增格式在此 elif 扩展。
"""
ext = src_path.suffix.lower()
if ext == ".epub":
_epub_to_pdf(src_path, dst_path)
else:
raise RuntimeError(f"不支持的文件格式:{ext}(仅支持 epub")
def _epub_to_pdf(src_path: Path, dst_path: Path) -> None:
"""epub -> PDF解包 epub -> 按 spine 顺序取文档项 HTML -> weasyprint 渲染。
epub 本质是 zip。先解包到临时目录用 ebooklib 读取(其内部按 zip 解析,
base_url 指向解包后的 OEBPS/ 内容目录使相对图片/CSS 可被 weasyprint 解析)。
"""
# 延迟导入weasyprint 首次 import 较重(加载 pango/cairo且仅在真正转换时需要
import ebooklib # noqa: F401
from ebooklib import epub
from weasyprint import HTML
with tempfile.TemporaryDirectory(prefix="zpdf_") as tmp:
tmp_dir = Path(tmp)
# epub 是 zip解包到临时目录便于 weasyprint 解析相对资源
try:
with zipfile.ZipFile(src_path, "r") as zf:
zf.extractall(tmp_dir)
except zipfile.BadZipFile as exc:
raise RuntimeError(f"epub 文件损坏(非有效 zip{exc}") from exc
try:
book = epub.read_epub(str(src_path), {"ignore_ncx": True})
except Exception as exc:
raise RuntimeError(f"epub 解析失败:{exc}") from exc
# 按 spine 顺序收集文档项XHTML保证章节顺序正确
docs: list[str] = []
for idref, _linear in book.spine:
item = book.get_item_with_id(idref)
if item is not None:
docs.append(item.get_content().decode("utf-8", errors="replace"))
if not docs:
# spine 为空时回退:取所有文档项
docs = [
it.get_content().decode("utf-8", errors="replace")
for it in book.get_items_of_type(ebooklib.ITEM_DOCUMENT)
]
if not docs:
raise RuntimeError("epub 内无可转换的文档内容")
full_html = "\n".join(docs)
# 定位资源根目录(含图片/CSS 的目录):通常是 OEBPS/ 或根目录。
# epub 内资源(图片/CSS相对文档项引用文档项与资源同处 OPF 所在目录。
# 故以 OPF 文件所在目录作为 base_url使相对路径正确解析。
base_url = str(tmp_dir)
opf_files = list(tmp_dir.rglob("*.opf"))
if opf_files:
opf_dir = opf_files[0].parent
# OPF 可能在根目录,此时 base_url 保持 tmp_dir
if str(opf_dir) != str(tmp_dir):
base_url = str(opf_dir)
try:
HTML(string=full_html, base_url=base_url).write_pdf(str(dst_path))
except Exception as exc:
raise RuntimeError(f"PDF 渲染失败:{exc}") from exc
if not dst_path.exists() or dst_path.stat().st_size == 0:
raise RuntimeError("PDF 渲染未产出有效文件")
logger.info("epub->pdf 完成: %s -> %s (%d bytes)", src_path.name, dst_path.name, dst_path.stat().st_size)

296
app/services/pdf_service.py Normal file
View File

@@ -0,0 +1,296 @@
"""PDF 转换服务:上传落盘 + 异步后台转换 + 进度追踪 + 两级删除。
复用 UploadService 的「流式落盘 + 存储路径生成」能力:原始上传文件与产物 PDF
均作为 UploadedFile 存储,本服务只维护 PdfJob 关系与状态。
转换在后台 asyncio task 中以 to_thread 执行(转换器是同步阻塞调用),
过程中经 DAO 更新 progress/status供前端轮询。worker 限制:依赖进程内
asyncio 事件循环,与现有 reaper/hub 一致,保持单 worker。
删除语义:
user_delete -- 置 user_deleted=true + deleted_at磁盘与 DB 行保留(管理页可见)。
admin_delete -- 删原始/产物磁盘文件 + UploadedFile 行 + PdfJob 行(真正删除)。
"""
from __future__ import annotations
import asyncio
import logging
import secrets
from datetime import datetime, timezone
from pathlib import Path
from fastapi import HTTPException, UploadFile
from ..config import get_settings
from ..dao.pdf_job_dao import PdfJobDAO
from ..dao.uploaded_file_dao import UploadedFileDAO
from ..models.pdf_job import PdfJob
from ..schemas.pdf import PdfJobOut, PdfJobListResponse
from . import pdf_converter
from .upload_service import UploadService
logger = logging.getLogger("zikai.pdf")
def new_owner_cookie() -> str:
"""生成新的用户标识 cookie 值uuid4 hex"""
return secrets.token_hex(16)
class PdfService:
def __init__(self, job_dao: PdfJobDAO, file_dao: UploadedFileDAO) -> None:
s = get_settings()
self.job_dao = job_dao
self.file_dao = file_dao
self.upload_root = s.resolved_upload_dir()
self.cfg = s.pdf
# 复用 UploadService 的存储路径生成与落盘能力
self._upload = UploadService(file_dao)
# ---------------- 提交 ----------------
def submit(self, file: UploadFile, owner_cookie: str) -> tuple[PdfJobOut, int]:
"""上传原始文件并创建 pending 任务,返回 (任务视图, source_file_id)。
校验大小 ≤ max_size_bytes 与扩展名白名单;复用 UploadService 流式落盘
入库为 UploadedFile再建 PdfJob 关联。转换不在此处执行(由 controller
调 schedule_convert 异步触发)。
"""
filename = file.filename or "upload.epub"
if not pdf_converter.is_supported(filename):
raise HTTPException(400, "仅支持 epub 文件")
# 大小校验UploadFile 流式无已知长度,先读一遍统计并重置(小文件可行),
# 对大文件更优的做法是流式计数,这里复用 stream_to_disk 后按 size 校验。
resp = self._upload.stream_to_disk(file, source="pdf", uploaded_by=owner_cookie)
if resp.size_bytes > self.cfg.max_size_bytes:
# 超限:清理刚落盘的文件与 DB 行,保持无副作用
try:
self._upload.delete_file(resp.id)
except Exception: # pragma: no cover
logger.warning("清理超限文件失败 id=%s", resp.id)
raise HTTPException(
413,
f"文件过大({resp.size_bytes} > {self.cfg.max_size_bytes},上限 250MB",
)
job = PdfJob(
owner_cookie=owner_cookie,
source_file_id=resp.id,
source_filename=filename,
source_size=resp.size_bytes,
status="pending",
progress=0,
)
job = self.job_dao.create(job)
logger.info("PDF 任务已创建 job=%s file=%s size=%d", job.id, filename, resp.size_bytes)
return PdfJobOut.model_validate(job), resp.id
def schedule_convert(self, job_id: int) -> None:
"""在当前事件循环起一个后台 task 执行转换(不阻塞调用方)。
用 to_thread 跑同步转换器;转换中分段更新 progress。
重要:后台 task 必须用独立 DB Session请求 Session 在请求结束后即关闭),
故 _convert_async 内部经 _fresh_service 重建带新 Session 的 service。
"""
asyncio.create_task(self._convert_async(job_id))
async def _convert_async(self, job_id: int) -> None:
"""后台转换pending -> converting(进度) -> done/failed。
每个阶段用独立 Session_fresh_service避免引用请求 Session已关闭
"""
try:
await asyncio.to_thread(self._run_with_fresh_session, "_mark_converting", job_id)
await asyncio.wait_for(
asyncio.to_thread(self._run_with_fresh_session, "_do_convert", job_id),
timeout=self.cfg.convert_timeout_seconds,
)
except asyncio.TimeoutError:
await asyncio.to_thread(self._run_with_fresh_session, "_mark_failed", job_id, "转换超时")
except Exception as exc:
await asyncio.to_thread(self._run_with_fresh_session, "_mark_failed", job_id, f"转换失败:{exc}")
@staticmethod
def _run_with_fresh_session(method_name: str, *args) -> None:
"""用独立 DB Session 构造新 PdfService 实例执行其方法。
后台线程不能复用请求的 Session请求结束即关闭故每次操作新建 Session。
method_name 是 PdfService 实例方法名,在此用新 service 调用对应方法。
"""
from ..database import get_session_local
db = get_session_local()()
try:
svc = PdfService(PdfJobDAO(db), UploadedFileDAO(db))
getattr(svc, method_name)(*args)
finally:
db.close()
# ---------------- 同步转换实现(在线程中执行,用独立 Session ----------------
def _mark_converting(self, job_id: int) -> None:
job = self.job_dao.get(job_id)
if job is None:
return
job.status = "converting"
job.progress = 5
self.job_dao.update(job)
def _do_convert(self, job_id: int) -> None:
"""执行转换并落产物 PDF 为 UploadedFile。"""
job = self.job_dao.get(job_id)
if job is None:
return
src_row = self.file_dao.get_by_id(job.source_file_id)
if src_row is None:
self._mark_failed(job_id, "原始文件记录丢失")
return
src_path = (self.upload_root / src_row.storage_path).resolve()
if not src_path.exists():
self._mark_failed(job_id, "原始文件实体不存在")
return
# 产物 PDF 存储路径(复用 UploadService 的路径生成,扩展名固定 .pdf
rel_path, abs_path = self._upload.make_storage_path("result.pdf")
part_path = abs_path.with_name(abs_path.name + ".part")
# 更新进度到「渲染中」
job.status = "converting"
job.progress = 30
self.job_dao.update(job)
try:
pdf_converter.convert_to_pdf(src_path, part_path)
except Exception:
part_path.unlink(missing_ok=True)
raise
job.progress = 80
self.job_dao.update(job)
# 落产物 UploadedFile复用 commit_entity 的原子改名 + 入库)
from ..models.uploaded_file import UploadedFile
size = part_path.stat().st_size
sha256 = self._hash_file(part_path)
entity = UploadedFile(
storage_path=str(rel_path),
original_filename=Path(job.source_filename).stem + ".pdf",
content_type="application/pdf",
size_bytes=size,
sha256=sha256,
source="pdf-convert",
uploaded_by=job.owner_cookie,
)
saved = self._upload.commit_entity(entity, part_path, abs_path)
job.output_file_id = saved.id
job.status = "done"
job.progress = 100
self.job_dao.update(job)
logger.info("PDF 转换完成 job=%s output_file_id=%s", job_id, saved.id)
def _mark_failed(self, job_id: int, message: str) -> None:
job = self.job_dao.get(job_id)
if job is None:
return
job.status = "failed"
job.error_message = message[:500]
self.job_dao.update(job)
logger.warning("PDF 转换失败 job=%s: %s", job_id, message)
def _hash_file(self, path: Path) -> str:
import hashlib
h = hashlib.sha256()
with path.open("rb") as f:
while chunk := f.read(self._upload.chunk_bytes):
h.update(chunk)
return h.hexdigest()
# ---------------- 查询 ----------------
def list_for_user(self, owner_cookie: str, limit: int = 100, offset: int = 0) -> PdfJobListResponse:
limit = min(max(limit, 1), self.cfg.list_limit)
offset = max(offset, 0)
total = self.job_dao.count_for_user(owner_cookie)
rows = self.job_dao.list_for_user(owner_cookie, limit=limit, offset=offset)
return PdfJobListResponse(total=total, items=[PdfJobOut.model_validate(r) for r in rows])
def list_for_admin(self, limit: int = 100, offset: int = 0) -> PdfJobListResponse:
limit = min(max(limit, 1), self.cfg.list_limit)
offset = max(offset, 0)
total = self.job_dao.count_all()
rows = self.job_dao.list_all(limit=limit, offset=offset)
return PdfJobListResponse(total=total, items=[PdfJobOut.model_validate(r) for r in rows])
def get_job(self, job_id: int, owner_cookie: str) -> PdfJobOut:
"""用户查询单任务:仅当归属本人且未软删时可见。"""
job = self.job_dao.get(job_id)
if job is None or job.owner_cookie != owner_cookie or job.user_deleted:
raise HTTPException(404, "任务不存在")
return PdfJobOut.model_validate(job)
def get_output_path(self, job_id: int, owner_cookie: str) -> tuple[PdfJobOut, Path, str]:
"""返回 (任务视图, 产物磁盘绝对路径, 下载文件名) 供下载。
用户仅可下载自己未软删且已完成的任务产物。
"""
job = self.job_dao.get(job_id)
if job is None or job.owner_cookie != owner_cookie or job.user_deleted:
raise HTTPException(404, "任务不存在")
if job.status != "done" or job.output_file_id is None:
raise HTTPException(409, "任务尚未完成,无法下载")
out_row = self.file_dao.get_by_id(job.output_file_id)
if out_row is None:
raise HTTPException(410, "产物文件记录丢失")
path = (self.upload_root / out_row.storage_path).resolve()
if not path.exists():
raise HTTPException(410, "产物文件实体不存在")
return PdfJobOut.model_validate(job), path, out_row.original_filename
# ---------------- 删除 ----------------
def user_delete(self, job_id: int, owner_cookie: str) -> bool:
"""用户软删:仅置 user_deleted=true磁盘与 DB 行保留(管理页仍可见)。"""
job = self.job_dao.get(job_id)
if job is None or job.owner_cookie != owner_cookie or job.user_deleted:
return False
job.user_deleted = True
job.deleted_at = datetime.now(timezone.utc)
self.job_dao.update(job)
logger.info("用户软删 PDF 任务 job=%s", job_id)
return True
def admin_delete(self, job_id: int) -> bool:
"""管理员硬删:删原始/产物磁盘文件 + UploadedFile 行 + PdfJob 行。
真正删除,不可恢复。磁盘删除失败仅记日志,仍清 DB 行保证列表不再显示。
"""
job = self.job_dao.get(job_id)
if job is None:
return False
# 删原始文件
self._safe_delete_file(job.source_file_id)
# 删产物文件(若有)
if job.output_file_id is not None:
self._safe_delete_file(job.output_file_id)
# 删 PdfJob 行
self.job_dao.delete(job)
logger.info("管理员硬删 PDF 任务 job=%s", job_id)
return True
def _safe_delete_file(self, file_id: int) -> None:
"""删 UploadedFile 磁盘文件 + DB 行;失败只记日志不阻断。"""
row = self.file_dao.get_by_id(file_id)
if row is None:
return
path = (self.upload_root / row.storage_path).resolve()
try:
Path(path).unlink(missing_ok=True)
except Exception as exc: # pragma: no cover
logger.warning("删除文件失败 file_id=%s path=%s: %s", file_id, path, exc)
try:
self.file_dao.delete(file_id)
except Exception as exc: # pragma: no cover
logger.warning("删除文件 DB 行失败 file_id=%s: %s", file_id, exc)

View File

@@ -27,8 +27,9 @@ class ZikaiSFTPServer(asyncssh.SFTPServer):
super().__init__(chan, chroot=str(upload_root).encode()) super().__init__(chan, chroot=str(upload_root).encode())
try: try:
self._username = chan.get_extra_info("username") or "unknown" self._username = chan.get_extra_info("username") or "unknown"
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
self._username = "unknown" self._username = "unknown"
logger.debug("读取 SFTP 会话用户名失败: %s", exc)
logger.info("SFTP 会话开始 user=%s chroot=%s", self._username, upload_root) logger.info("SFTP 会话开始 user=%s chroot=%s", self._username, upload_root)
def exit(self) -> None: def exit(self) -> None:
@@ -45,8 +46,8 @@ def _tunnel_dao():
def _close_tunnel_dao(dao) -> None: def _close_tunnel_dao(dao) -> None:
try: try:
dao.db.close() dao.db.close()
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
pass logger.debug("关闭隧道 DAO 会话失败: %s", exc)
class ZikaiSSHServer(asyncssh.SSHServer): class ZikaiSSHServer(asyncssh.SSHServer):
@@ -101,7 +102,8 @@ class ZikaiSSHServer(asyncssh.SSHServer):
try: try:
# asyncssh 命中返回 dict可能为空未命中返回 None # asyncssh 命中返回 dict可能为空未命中返回 None
result = self._authorized_keys.validate(key, client_host=addr, client_addr=addr) result = self._authorized_keys.validate(key, client_host=addr, client_addr=addr)
except Exception: except Exception as exc: # pragma: no cover
logger.warning("公钥校验异常 user=%s: %s", username, exc)
result = None result = None
ok = result is not None ok = result is not None
if ok: if ok:

View File

@@ -52,11 +52,6 @@ class TunnelService:
def get_active(self, user_name: str) -> TunnelSession | None: def get_active(self, user_name: str) -> TunnelSession | None:
return self.dao.get_active_by_user(user_name) return self.dao.get_active_by_user(user_name)
def is_port_allowed(self, user_name: str, tunnel_port: int) -> bool:
"""校验该 user 是否被允许绑定该隧道端口(防 user 乱绑端口)。"""
user = self.settings.find_user(user_name)
return user is not None and user.tunnel_port == tunnel_port
def reap_orphans(self) -> int: def reap_orphans(self) -> int:
"""兜底清理:关闭所有 active 会话(进程重启时 DB 里残留的孤儿记录)。 """兜底清理:关闭所有 active 会话(进程重启时 DB 里残留的孤儿记录)。

View File

@@ -8,7 +8,9 @@ upload_root不再有 HTTP 登记接口。
from __future__ import annotations from __future__ import annotations
import hashlib import hashlib
import logging
import os import os
import time
import uuid import uuid
from collections.abc import Iterator from collections.abc import Iterator
from datetime import datetime, timezone from datetime import datetime, timezone
@@ -21,6 +23,12 @@ from ..dao.uploaded_file_dao import UploadedFileDAO
from ..models.uploaded_file import UploadedFile from ..models.uploaded_file import UploadedFile
from ..schemas.file import FileUploadResponse, UploadedFileOut from ..schemas.file import FileUploadResponse, UploadedFileOut
logger = logging.getLogger("zikai.upload")
# 磁盘扫描频率限制:两次扫描至少间隔此秒数,否则跳过(只返回 DB 缓存)。
_SCAN_MIN_INTERVAL = 3.0
_last_scan_time: float = 0.0
def hash_stream(chunks: Iterator[bytes]) -> tuple[int, str]: def hash_stream(chunks: Iterator[bytes]) -> tuple[int, str]:
"""对一段字节块序列流式计算 (size, sha256)。供多条上传路径复用。""" """对一段字节块序列流式计算 (size, sha256)。供多条上传路径复用。"""
@@ -66,6 +74,72 @@ class UploadService:
# ---------------- 查询 ---------------- # ---------------- 查询 ----------------
def scan_sftp_files(self, force: bool = False) -> int:
"""扫描 uploads/ 目录,把磁盘上有但 DB 未记录的文件补录入库。
SFTP 上传的文件直接落盘到 upload_rootchroot 根),不经过 HTTP 路径,
因此没有 DB 记录。本方法遍历磁盘文件,与 DB 已有 storage_path 比对,
为缺失项创建记录source="sftp",计算 size + sha256
频率限制:两次扫描间隔 < _SCAN_MIN_INTERVAL3s则跳过force=True 强制扫。
返回本次新补录的条数(跳过时返回 0
"""
global _last_scan_time
now = time.monotonic()
if not force and (now - _last_scan_time) < _SCAN_MIN_INTERVAL:
return 0
_last_scan_time = now
known = self.dao.list_storage_paths()
new_count = 0
# 排除分片会话暂存目录与 .part 残品
skip_dirs = {".work"}
for abs_path in self.upload_root.rglob("*"):
if not abs_path.is_file():
continue
if abs_path.suffix == ".part":
continue
# 跳过 .work 目录下的任何文件
rel = abs_path.relative_to(self.upload_root)
if rel.parts and rel.parts[0] in skip_dirs:
continue
storage_path = str(rel).replace("\\", "/")
if storage_path in known:
continue
# 磁盘有、DB 无:补录
try:
size = abs_path.stat().st_size
sha256 = self._hash_file(abs_path)
except Exception as exc:
logger.warning("扫描文件失败 path=%s: %s", abs_path, exc)
continue
entity = UploadedFile(
storage_path=storage_path,
original_filename=abs_path.name,
content_type="",
size_bytes=size,
sha256=sha256,
source="sftp",
uploaded_by="sftp",
)
try:
self.dao.create(entity)
new_count += 1
known.add(storage_path)
except Exception as exc:
logger.warning("补录 SFTP 文件失败 path=%s: %s", storage_path, exc)
if new_count:
logger.info("SFTP 文件扫描补录 %d", new_count)
return new_count
def _hash_file(self, path: Path) -> str:
"""流式计算文件 sha256避免大文件一次性读入内存"""
h = hashlib.sha256()
with path.open("rb") as f:
while chunk := f.read(self.chunk_bytes):
h.update(chunk)
return h.hexdigest()
def list_files(self, limit: int = 100, offset: int = 0) -> tuple[int, list[UploadedFileOut]]: def list_files(self, limit: int = 100, offset: int = 0) -> tuple[int, list[UploadedFileOut]]:
total = self.dao.count() total = self.dao.count()
rows = self.dao.list(limit=limit, offset=offset) rows = self.dao.list(limit=limit, offset=offset)
@@ -95,18 +169,17 @@ class UploadService:
def delete_file(self, file_id: int) -> bool: def delete_file(self, file_id: int) -> bool:
"""硬删除:删磁盘文件 + 删 DB 行。磁盘缺失不阻断 DB 清理。返回是否命中。 """硬删除:删磁盘文件 + 删 DB 行。磁盘缺失不阻断 DB 清理。返回是否命中。
供单删/批删复用,保证删除语义一致。 供单删/批删复用,保证删除语义一致。磁盘删除失败仅记日志,仍清 DB 行
(保证列表不再显示),避免磁盘文件泄漏却无任何记录。
""" """
_, path = self.get_out_with_disk_path(file_id) out, path = self.get_out_with_disk_path(file_id)
row = self.dao.get_by_id(file_id) if out is None:
if row is None:
return False return False
if path is not None: if path is not None:
try: try:
Path(path).unlink(missing_ok=True) Path(path).unlink(missing_ok=True)
except Exception: except Exception as exc:
# 即使磁盘删除失败也继续清 DB 行,保证列表不再显示 logger.warning("删除磁盘文件失败 file_id=%s path=%s: %s", file_id, path, exc)
pass
self.dao.delete(file_id) self.dao.delete(file_id)
return True return True

View File

@@ -18,6 +18,7 @@ from __future__ import annotations
import asyncio import asyncio
import logging import logging
import threading
import time import time
from dataclasses import dataclass, field from dataclasses import dataclass, field
@@ -58,19 +59,26 @@ class WhiteboardHub:
self.heartbeat_interval = cfg.heartbeat_interval_seconds self.heartbeat_interval = cfg.heartbeat_interval_seconds
self.heartbeat_miss_threshold = cfg.heartbeat_miss_threshold self.heartbeat_miss_threshold = cfg.heartbeat_miss_threshold
self.timeout_seconds = self.heartbeat_interval * self.heartbeat_miss_threshold self.timeout_seconds = self.heartbeat_interval * self.heartbeat_miss_threshold
self.max_connections_per_board = cfg.max_connections_per_board
# {board_id: set[Connection]} # {board_id: set[Connection]}
self._boards: dict[str, set[Connection]] = {} self._boards: dict[str, set[Connection]] = {}
self._lock = asyncio.Lock() self._lock = asyncio.Lock()
# ---------------- 连接生命周期 ---------------- # ---------------- 连接生命周期 ----------------
async def register(self, conn: Connection) -> None: async def register(self, conn: Connection) -> bool:
"""把已 accept 的连接加入 board 集合WebSocket accept 由 controller 负责)。""" """把已 accept 的连接加入 board 集合
返回 False 表示该 board 连接数已达上限(调用方应关闭连接)。
"""
async with self._lock: async with self._lock:
conns = self._boards.setdefault(conn.board_id, set()) conns = self._boards.setdefault(conn.board_id, set())
if len(conns) >= self.max_connections_per_board:
return False
conns.add(conn) conns.add(conn)
logger.info("连接接入 board=%s client=%s(当前 %d 人)", logger.info("连接接入 board=%s client=%s(当前 %d 人)",
conn.board_id, conn.client_id, self.connection_count(conn.board_id)) conn.board_id, conn.client_id, self.connection_count(conn.board_id))
return True
async def disconnect(self, conn: Connection) -> None: async def disconnect(self, conn: Connection) -> None:
"""幂等移除连接;空 set 从 dict 删除以防内存泄漏。""" """幂等移除连接;空 set 从 dict 删除以防内存泄漏。"""
@@ -84,8 +92,9 @@ class WhiteboardHub:
# 尽力关闭 websocket可能已关闭 # 尽力关闭 websocket可能已关闭
try: try:
await conn.websocket.close() await conn.websocket.close()
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
pass logger.debug("关闭 websocket 时出错 board=%s client=%s: %s",
conn.board_id, conn.client_id, exc)
logger.info("连接移除 board=%s client=%s(剩余 %d 人)", logger.info("连接移除 board=%s client=%s(剩余 %d 人)",
conn.board_id, conn.client_id, self.connection_count(conn.board_id)) conn.board_id, conn.client_id, self.connection_count(conn.board_id))
@@ -153,23 +162,22 @@ class WhiteboardHub:
for conn in conns: for conn in conns:
try: try:
await conn.websocket.close() await conn.websocket.close()
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
pass logger.debug("关闭 websocket 时出错 board=%s client=%s: %s",
conn.board_id, conn.client_id, exc)
logger.info("关闭白板 board=%s,踢出 %d 个连接", board_id, len(conns)) logger.info("关闭白板 board=%s,踢出 %d 个连接", board_id, len(conns))
# 进程内单例(由 main.py lifespan / controller 共享) # 进程内单例(由 main.py lifespan / controller 共享)
_hub: WhiteboardHub | None = None _hub: WhiteboardHub | None = None
_hub_lock = threading.Lock()
def get_hub() -> WhiteboardHub: def get_hub() -> WhiteboardHub:
"""获取/创建进程内单例 hub。线程安全lifespan 预热后通常不再进锁)。"""
global _hub global _hub
if _hub is None: if _hub is None:
_hub = WhiteboardHub() with _hub_lock:
if _hub is None:
_hub = WhiteboardHub()
return _hub return _hub
def reset_hub() -> None:
"""测试用:重置单例。"""
global _hub
_hub = None

View File

@@ -1,14 +1,13 @@
"""白板服务CRUD + 笔画操作 """白板服务(文本记事本)CRUD + 文本更新 + 清空
不持有 WebSocket 连接状态(那是 hub 的职责)删除白板时通过可选的 hub 回调 不持有 WebSocket 连接状态(那是 hub 的职责)删除白板时由 controller 层负责
通知 hub 踢出该 board 的所有在线连接,由 controller 在装配时注入,避免 service -> hub 通知 hub 踢出在线连接(因为 close_board 是 async需在事件循环中调用
的硬依赖(保持低耦合 service 只管 DB 层面的删除,保持低耦合。
""" """
from __future__ import annotations from __future__ import annotations
import re import re
from typing import TYPE_CHECKING, Any
from fastapi import HTTPException from fastapi import HTTPException
@@ -16,20 +15,18 @@ from ..config import get_settings
from ..dao.whiteboard_dao import WhiteboardDAO from ..dao.whiteboard_dao import WhiteboardDAO
from ..schemas.whiteboard import WhiteboardListItem, WhiteboardOut from ..schemas.whiteboard import WhiteboardListItem, WhiteboardOut
if TYPE_CHECKING: # 避免运行时循环导入
from .whiteboard_hub import WhiteboardHub
# board_id 合法字符集:字母数字下划线短横线 # board_id 合法字符集:字母数字下划线短横线
_BOARD_ID_RE = re.compile(r"^[a-zA-Z0-9_-]+$") _BOARD_ID_RE = re.compile(r"^[a-zA-Z0-9_-]+$")
class WhiteboardService: class WhiteboardService:
def __init__(self, dao: WhiteboardDAO, hub: "WhiteboardHub | None" = None) -> None: def __init__(self, dao: WhiteboardDAO) -> None:
self.dao = dao self.dao = dao
self.hub = hub
cfg = get_settings().whiteboard cfg = get_settings().whiteboard
self.max_board_id_length = cfg.max_board_id_length self.max_board_id_length = cfg.max_board_id_length
self.list_limit = cfg.list_limit self.list_limit = cfg.list_limit
# 文本内容长度上限(防滥用)
self.max_content_length = 256 * 1024
# ---------------- 校验 ---------------- # ---------------- 校验 ----------------
@@ -42,6 +39,15 @@ class WhiteboardService:
): ):
raise HTTPException(400, "board_id 非法仅允许字母数字下划线短横线1-64 字符)") raise HTTPException(400, "board_id 非法仅允许字母数字下划线短横线1-64 字符)")
def validate_content(self, content: str) -> None:
if not isinstance(content, str):
raise HTTPException(400, "content 必须是字符串")
if len(content) > self.max_content_length:
raise HTTPException(
413,
f"文本过长({len(content)} > {self.max_content_length}),请缩减内容",
)
# ---------------- 读 ---------------- # ---------------- 读 ----------------
def get_or_create(self, board_id: str) -> WhiteboardOut: def get_or_create(self, board_id: str) -> WhiteboardOut:
@@ -59,34 +65,20 @@ class WhiteboardService:
# ---------------- 写 ---------------- # ---------------- 写 ----------------
def append_stroke(self, board_id: str, stroke: dict[str, Any]) -> WhiteboardOut: def update_content(self, board_id: str, content: str) -> WhiteboardOut:
"""追加一条笔画并返回最新状态""" """整体替换文本内容(客户端 debounce 后发完整文本)"""
self.validate_board_id(board_id) self.validate_board_id(board_id)
board = self.dao.append_strokes(board_id, [stroke]) self.validate_content(content)
board = self.dao.update_content(board_id, content)
if board is None: if board is None:
raise HTTPException(404, "白板不存在") raise HTTPException(404, "白板不存在")
return WhiteboardOut.model_validate(board) return WhiteboardOut.model_validate(board)
def clear(self, board_id: str) -> WhiteboardOut: def clear(self, board_id: str) -> WhiteboardOut:
"""清空白板stroke_count 仍自增以记录这次修改。""" """清空白板内容置空edit_count 仍自增以记录这次修改。"""
self.validate_board_id(board_id) return self.update_content(board_id, "")
board = self.dao.replace_strokes(board_id, [])
if board is None:
raise HTTPException(404, "白板不存在")
return WhiteboardOut.model_validate(board)
def delete(self, board_id: str) -> bool: def delete(self, board_id: str) -> bool:
"""删除白板;同时通知 hub 踢出该 board 的所有在线连接""" """删除白板 DB 行。踢出在线连接由 controller 层负责close_board 是 async"""
self.validate_board_id(board_id) self.validate_board_id(board_id)
ok = self.dao.delete(board_id) return self.dao.delete(board_id)
if ok and self.hub is not None:
# hub.close_board 是 async但删除走 REST 同步路径;安排到事件循环里执行
import asyncio
try:
loop = asyncio.get_running_loop()
loop.create_task(self.hub.close_board(board_id))
except RuntimeError:
# 无运行中事件循环(如脚本调用):同步调用会报错,忽略即可
pass
return ok

View File

@@ -128,7 +128,7 @@ def render(status: SystemStatus) -> str:
</table> </table>
</div> </div>
<p class="foot">zikai file service · 数据来源 psutil</p> <p class="foot"><a class="json" href="/api/index">导航</a> · zikai file service · 数据来源 psutil</p>
</body> </body>
</html> </html>
""" """

View File

@@ -6,8 +6,6 @@
from __future__ import annotations from __future__ import annotations
from html import escape
# 默认分片大小 4 MiB大于 Apache 300s 限制下单片可数秒传完,小到内存恒定。 # 默认分片大小 4 MiB大于 Apache 300s 限制下单片可数秒传完,小到内存恒定。
DEFAULT_CHUNK_SIZE = 4 * 1024 * 1024 DEFAULT_CHUNK_SIZE = 4 * 1024 * 1024
# 同一文件分片并发数 # 同一文件分片并发数
@@ -38,7 +36,7 @@ def render() -> str:
<div id="summary" class="foot"></div> <div id="summary" class="foot"></div>
<p class="foot"><a class="json" href="/api/system/status">系统状态</a> · zikai file service</p> <p class="foot"><a class="json" href="/api/index">导航</a> · <a class="json" href="/api/system/status">系统状态</a> · zikai file service</p>
<script> <script>
const CHUNK_SIZE = {DEFAULT_CHUNK_SIZE}; const CHUNK_SIZE = {DEFAULT_CHUNK_SIZE};

View File

@@ -64,3 +64,13 @@ whiteboard:
heartbeat_miss_threshold: 5 heartbeat_miss_threshold: 5
max_board_id_length: 64 # board_id 合法字符 [a-zA-Z0-9_-],长度上限 max_board_id_length: 64 # board_id 合法字符 [a-zA-Z0-9_-],长度上限
list_limit: 100 # 管理页单次列表上限 list_limit: 100 # 管理页单次列表上限
pdf:
# PDF 转换服务:用户上传 epub -> 后台转换为 PDF -> 显示进度并下载。
# 用户侧(上传/查看/下载/软删)凭 httpOnly cookie 标识;管理侧(列表/硬删)走 docs 同款 Basic Auth。
# 原始文件与产物 PDF 复用 storage.upload_dir 落盘。转换用纯 Pythonebooklib + weasyprint
max_size_bytes: 262144000 # 单文件上限 250 MiB
convert_timeout_seconds: 600 # 转换超时(秒),超大文件兜底
list_limit: 100 # 列表单次上限
cookie_name: zk_pdf # 用户标识 cookie 名
cookie_max_age_seconds: 31536000 # cookie 有效期 1 年

56
deploy/install-systemd.sh Executable file
View File

@@ -0,0 +1,56 @@
#!/usr/bin/env bash
# 安装 systemd 服务,实现 zTools2 持久化部署(开机自启 + 崩溃自动重启)。
#
# 做的事:
# 1. 用实际路径填充 deploy/ztools2.service 模板,写入 /etc/systemd/system/
# 2. 停掉旧方式start.sh 启动的 uvicorn避免端口冲突
# 3. systemctl daemon-reload + enable + start
#
# 用法:./deploy/install-systemd.sh
# 卸载systemctl disable --now ztools2 && rm /etc/systemd/system/ztools2.service && systemctl daemon-reload
set -euo pipefail
cd "$(dirname "$0")/.."
ROOT="$(pwd)"
# 解析 uvicorn 可执行路径:优先 .venvsetup.sh 标准),回退系统 python -m uvicorn
if [[ -x "$ROOT/.venv/bin/uvicorn" ]]; then
UVICORN="$ROOT/.venv/bin/uvicorn"
PYBIN="$ROOT/.venv/bin/python"
elif command -v uvicorn >/dev/null 2>&1; then
UVICORN="$(command -v uvicorn)"
PYBIN="$(command -v python3)"
else
echo "未找到 uvicorn请先运行 ./setup.sh建 .venv或 pip install uvicorn" >&2
exit 1
fi
[[ -f "$ROOT/config.yaml" ]] || { echo "未找到 config.yaml请先运行 ./setup.sh" >&2; exit 1; }
UNIT_SRC="$ROOT/deploy/ztools2.service"
UNIT_DST="/etc/systemd/system/ztools2.service"
echo "==> 生成 systemd unit路径 $ROOTuvicorn=$UVICORN"
# 同时填充 ROOT 与 UVICORNPYBIN 备用ExecStart 用 UVICORN 直接启动)
sudo sed -e "s|__ZTOOLS2_DIR__|$ROOT|g" -e "s|__UVICORN__|$UVICORN|g" "$UNIT_SRC" > /tmp/ztools2.service
sudo mv /tmp/ztools2.service "$UNIT_DST"
sudo chmod 644 "$UNIT_DST"
echo "==> 停止旧方式start.sh 启动的进程,若有)"
./stop.sh >/dev/null 2>&1 || true
echo "==> 启用并启动 ztools2 服务"
sudo systemctl daemon-reload
sudo systemctl enable ztools2
sudo systemctl restart ztools2
sleep 2
if systemctl is-active --quiet ztools2; then
echo "✓ ztools2 已启动并设为开机自启"
echo " 状态systemctl status ztools2"
echo " 日志journalctl -u ztools2 -f"
echo " 停止sudo systemctl stop ztools2"
echo " 重启sudo systemctl restart ztools2"
else
echo "✗ ztools2 启动失败查看日志journalctl -u ztools2 -n 50" >&2
exit 1
fi

23
deploy/ztools2.service Normal file
View File

@@ -0,0 +1,23 @@
[Unit]
Description=zikai file service (zTools2) - FastAPI backend
Documentation=https://git.zikai.wang/zikai/zTools2
After=network.target mysql.service
Wants=mysql.service
[Service]
Type=simple
# 工作目录与 uvicorn 路径:安装时由 install-systemd.sh 用实际路径替换
WorkingDirectory=__ZTOOLS2_DIR__
Environment=PYTHONUNBUFFERED=1
ExecStart=__UVICORN__ app.main:app --host 127.0.0.1 --port 6867 --workers 1
Restart=on-failure
RestartSec=3
# 后台转换可能耗时较长,放宽超时
TimeoutStopSec=30
KillSignal=SIGINT
# 日志走 journaldjournalctl -u ztools2
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target

39
docs/configuration.md Normal file
View File

@@ -0,0 +1,39 @@
# 配置说明
所有运行时配置在 `config.yaml`git-ignored。完整 schema 见 `config.example.yaml`
## 配置项
| 段 | 关键项 | 说明 |
|----|--------|------|
| `server` | `host`/`port` | 绑定地址,保持 `127.0.0.1:6867`Apache 反代) |
| `database` | `host`/`port`/`user`/`password`/`database` | MySQL 连接;密码由 `setup.sh` 自动生成写回 |
| `storage` | `upload_dir` | 文件存储根目录(默认 `uploads` |
| | `chunk_bytes` | 流式上传分片大小(默认 1 MiB |
| | `chunk_session_dir` | 分片会话暂存目录(默认 `uploads/.work` |
| | `chunk_session_ttl_seconds` | 被放弃会话存活秒数(默认 300 |
| `docs` | `username`/`password` | `/docs``/api/files-page``/api/wb-admin``/api/admin/*` 的 Basic Auth明文常量时间比较 |
| `sftp` | `enabled`/`host`/`port` | SFTP 服务,默认 `0.0.0.0:2022` |
| | `users[].username`/`password_hash` | SFTP 用户bcrypt |
| | `host_key_path`/`authorized_keys_path` | 主机密钥与公钥白名单路径 |
| `tunnel` | `enabled`/`users[]` | 反向隧道:`username`/`password_hash`/`tunnel_port`/`local_port` |
| `whiteboard` | `heartbeat_interval_seconds` | 心跳间隔(默认 3s |
| | `heartbeat_miss_threshold` | 失活阈值(默认 5 次 = 15s |
| | `max_board_id_length` | board_id 长度上限(默认 64 |
| | `max_connections_per_board` | 单白板并发连接上限(默认 50 |
| | `list_limit` | 管理页单次列表上限(默认 100 |
## 生成 bcrypt hashSFTP/隧道用户密码)
```bash
.venv/bin/python -c "import bcrypt;print(bcrypt.hashpw(b'yourpass',bcrypt.gensalt()).decode())"
```
把输出填入 `config.yaml` 对应 `password_hash` 字段。
## 重新生成 DB 密码
```bash
.venv/bin/python -m app.scripts.init_db # 重置密码(写回 config.yaml
KEEP_DB_PASSWORD=1 .venv/bin/python -m app.scripts.init_db # 保留现有密码,仅建库建账
```

29
docs/error-handling.md Normal file
View File

@@ -0,0 +1,29 @@
# 错误处理与日志约定
zTools2 采用 Spring 风格分层架构错误处理分三层DAO fail-fast抛异常Service 捕获后转换业务异常并记日志Controller 捕获后转 HTTP 状态码。
## DAO 层app/dao/
- **唯一发 SQL 的层**所有写操作create/update/delete均在该层 commitservice 不直接操作 session。
- `get_or_create`whiteboard_dao.py`IntegrityError`(并发下另一事务已插入违反唯一约束)才回滚重读;其他异常向上抛,避免掩盖 schema/连接等真实故障。
- `delete`pdf_job_dao.py / uploaded_file_dao.py硬删 DB 行并 commit不存在返回 False。
- schema 漂移检测:`init_db_schema`database.py建表后用 inspector 检查既有表的列是否与模型声明齐全,缺列即抛 `RuntimeError`fail-fast避免运行期才以晦涩的 `OperationalError` 暴露。
## Service 层app/services/
- **PdfService**`submit` 校验扩展名/大小,超限清理已落盘文件后抛 `HTTPException`;后台转换 `_convert_async` 捕获 `TimeoutError` / 通用异常,经 `_mark_failed` 落库 + `logger.warning``admin_delete` 磁盘删除失败仅 `logger.warning`,仍清 DB 行保证列表不再显示。
- **UploadService / ChunkUploadService**:流式落盘出错清理临时文件后 `raise`(向上传播);分片会话被放弃由后台 reaper 每 60s 清理。
- **WhiteboardHub**`disconnect` / `close_board` 关闭 websocket 出错 `logger.debug`(尽力关闭,可能已关闭);`broadcast` 单连接发送失败立即 disconnect不影响其他连接reaper 循环异常 `logger.warning` 后继续。
- **TunnelService**`register` / `close` / `reap_orphans` 均记 `logger.info`SSH 连接断开时清理会话失败 `logger.warning`
- **sftp_server**`validate_public_key` 校验异常 `logger.warning`auth 路径,避免静默失败);`_close_tunnel_dao` / 读会话用户名失败 `logger.debug`(尽力清理)。
## Controller 层app/controllers/
- **pdf_controller**`_resolve_cookie` 解析用户 cookie无则生成新值挂 request.state 供响应 set_cookie。
- **whiteboard_controller**`_safe_send` / `_safe_close` 发送/关闭 WS 失败 `logger.debug`best-effortWS 主循环异常 `logger.warning` 后正常关闭连接。
- 所有管理 API`/api/admin/*`)经 `require_docs_auth` Basic Auth 守卫(常量时间比较)。
## 日志位置
- `logs/app.log`HTTP`logs/sftp.log`SFTPpidfile`app.pid``sftp.pid`
- 日志器命名:`zikai.pdf` / `zikai.whiteboard` / `zikai.tunnel` / `sftp`,便于按模块过滤。

51
docs/routes.md Normal file
View File

@@ -0,0 +1,51 @@
# 路由与访问入口
## 路由约定
**所有 zTools2 托管的入口(页面 / 静态资源 / 健康探针 / WebSocket / REST API统一挂在 `/api/` 前缀下**,只有元信息/文档例外(`/``/docs``/redoc``/openapi.json`)。这样反向代理与 vite dev proxy 都只需一条 `/api/` 规则即可把请求转给当前环境的 zTools2前端用**同源相对路径**(如 `/api/pdf/jobs``/api/health`)即可,与运行环境(本地 / 测试 / 生产)无关,无需区分 dev/prod 指向。
> PDF 转换的用户侧 UI 由 zMainPage 的 zPDF_package 组件提供(构建期 import非 iframezTools2 仅提供 `/api/pdf/jobs` 等 REST API。PDF 转换管理(查看全部任务含软删标记、硬删)已并入文件管理页 `/api/files-page`。
页面类入口为避免与同名 REST API 冲突,统一加 `-page` 后缀:
| 类型 | 页面入口 | REST API同名不加后缀 |
|------|---------|------------------------|
| 文件管理 | `GET /api/files-page` | `GET /api/files``/api/files/{id}` 等 |
| 记事本 | `GET /api/wb-page/{id}` | `GET /api/wb/{id}` |
其余入口:`/api/index`(导航页)、`/api/health`(探针)、`/api/static/*`JS/CSS`/api/ws/wb/{id}`WebSocket`/api/upload``/api/wb-admin`
## 功能一览
| 模块 | 页面 / 接口 | 鉴权 |
|------|------------|------|
| 导航页 | `GET /api/index`(卡片式入口索引,各子页可返回) | 公开 |
| 文件上传 | `POST /api/files/upload`(流式)/ `POST /api/files/chunk-uploads/*`(分片+断点续传) | 公开 |
| 上传页 | `GET /api/upload`(拖拽/多文件/分片/去重) | 公开 |
| 文件管理 | `GET /api/files-page`(多选/批量下载删除/分页,合并 PDF 转换管理:状态、软删标记、硬删任务) | Basic Auth |
| 文件管理 API | `GET /api/admin/files``GET /api/admin/files/with-pdf`(合并视图,附带 pdf_jobs 关联)、`GET/DELETE /api/admin/files/{id}``GET /api/admin/files/{id}/download` | Basic Auth |
| 共享记事本 | `GET /api/wb-page/{id}`(公开,不存在则新建) | 公开 |
| 记事本实时同步 | `WS /api/ws/wb/{id}`(心跳 3s5 次失活移除) | 公开 |
| 记事本管理 | `GET /api/wb-admin`(查看/删除) | Basic Auth |
| 记事本管理 API | `GET /api/admin/wb``DELETE /api/admin/wb/{id}` | Basic Auth |
| PDF 转换 API | `POST /api/pdf/jobs``GET /api/pdf/jobs[/{id}]``GET /api/pdf/jobs/{id}/download``DELETE /api/pdf/jobs/{id}` | 公开cookie |
| PDF 转换管理 API | `GET /api/admin/pdf/jobs``DELETE /api/admin/pdf/jobs/{id}`(管理入口已并入 `/api/files-page` | Basic Auth |
| 主机监控 | `GET /api/system/status`CPU/内存/磁盘HTML+JSON 内容协商) | 公开 |
| 反向隧道反代 | `ALL /api/userPort/{userName}`(经 SSH 隧道转发到 user 本地服务) | 公开 |
| SFTP/SSH | 端口 2022密码+公钥chroot 到上传目录,承载隧道转发) | SSH |
| API 文档 | `GET /docs`Swagger/ `GET /redoc` | Basic Auth |
## 访问入口
| 入口 | URL |
|------|-----|
| 导航页 | https://f.zikai.wang/api/index |
| API 文档 | https://f.zikai.wang/docsBasic Auth |
| 上传页 | https://f.zikai.wang/api/upload |
| 文件管理 | https://f.zikai.wang/api/files-pageBasic Auth含 PDF 转换管理) |
| 共享记事本 | https://f.zikai.wang/api/wb-page/{id}(公开,`{id}``[a-zA-Z0-9_-]{1,64}` |
| 记事本管理 | https://f.zikai.wang/api/wb-adminBasic Auth |
| PDF 转换 | 经 zMainPage 的 PDF 页签zPDF_package 组件)调用 `/api/pdf/jobs` 等 REST API公开凭 cookie |
| 系统状态 | https://f.zikai.wang/api/system/statusHTML`?format=json` 切 JSON |
| curl 上传 | `curl -F file=@big.iso https://f.zikai.wang/api/files/upload` |
| SFTP | `sftp -P 2022 uploader@f.zikai.wang` |

View File

@@ -2,10 +2,13 @@ fastapi==0.115.6
uvicorn[standard]==0.34.0 uvicorn[standard]==0.34.0
python-multipart==0.0.20 python-multipart==0.0.20
psutil==6.1.1 psutil==6.1.1
SQLAlchemy==2.0.36 SQLAlchemy==2.0.51
PyMySQL==1.1.1 PyMySQL==1.1.1
pydantic-settings==2.7.0 pydantic-settings==2.7.0
PyYAML==6.0.2 PyYAML==6.0.2
asyncssh==2.18.0 asyncssh==2.18.0
bcrypt==4.2.1 bcrypt==4.2.1
httpx==0.28.1 httpx==0.28.1
# PDF 转换ebooklib 解析 epubweasyprint 渲染 HTML/CSS 为 PDF纯 Python无需 calibre/xvfb
ebooklib==0.20
weasyprint==69.0

View File

@@ -53,16 +53,42 @@ CREATE TABLE IF NOT EXISTS `tunnel_session` (
KEY `idx_status` (`status`) KEY `idx_status` (`status`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- 共享白板表。由 app/services/whiteboard_service.py 使用。 -- 共享白板表(文本记事本)。由 app/services/whiteboard_service.py 使用。
-- 白板长期留存strokes 以 JSON 保存全部笔画;实时同步由 WebSocket hub 在内存维护 -- content 存完整文本version 是乐观锁版本号(每次编辑 +1edit_count 累计编辑次数
-- 白板长期留存,实时同步由 WebSocket hub 在内存维护在线连接。
CREATE TABLE IF NOT EXISTS `whiteboard` ( CREATE TABLE IF NOT EXISTS `whiteboard` (
`id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, `id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
`board_id` VARCHAR(64) NOT NULL COMMENT '用户可读的 url id[a-zA-Z0-9_-]{1,64}', `board_id` VARCHAR(64) NOT NULL COMMENT '用户可读的 url id[a-zA-Z0-9_-]{1,64}',
`strokes` JSON NOT NULL COMMENT '笔画数组 [{points,color,width}, ...]', `content` MEDIUMTEXT NOT NULL COMMENT '白板文本内容',
`stroke_count` INT NOT NULL DEFAULT 0 COMMENT '累计修改次数(新增笔画/清空各 +1', `version` INT NOT NULL DEFAULT 0 COMMENT '乐观锁版本号,每次编辑 +1',
`edit_count` INT NOT NULL DEFAULT 0 COMMENT '累计编辑次数(含清空)',
`created_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, `created_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, `updated_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`), PRIMARY KEY (`id`),
UNIQUE KEY `uq_board_id` (`board_id`), UNIQUE KEY `uq_board_id` (`board_id`),
KEY `idx_updated_at` (`updated_at`) KEY `idx_updated_at` (`updated_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- PDF 转换任务表。由 app/services/pdf_service.py 使用。
-- 原始文件与产物 PDF 复用 uploaded_file 存储,本表只记录关系与转换状态。
-- 删除两级user_deleted用户软删管理页仍可见/ admin 硬删(真正删除)。
CREATE TABLE IF NOT EXISTS `pdf_job` (
`id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
`owner_cookie` VARCHAR(64) NOT NULL COMMENT '用户标识httpOnly cookie 值uuid4 hex',
`source_file_id` BIGINT UNSIGNED NOT NULL COMMENT '原始上传文件 -> uploaded_file.id',
`source_filename` VARCHAR(512) NOT NULL,
`source_size` BIGINT UNSIGNED NOT NULL DEFAULT 0,
`output_file_id` BIGINT UNSIGNED NULL DEFAULT NULL COMMENT '产物 PDF -> uploaded_file.id转换完成前 NULL',
`status` VARCHAR(16) NOT NULL DEFAULT 'pending' COMMENT 'pending/converting/done/failed',
`progress` INT NOT NULL DEFAULT 0 COMMENT '转换进度 0-100',
`error_message` VARCHAR(512) NOT NULL DEFAULT '',
`user_deleted` TINYINT(1) NOT NULL DEFAULT 0 COMMENT '用户软删标记',
`deleted_at` DATETIME NULL DEFAULT NULL COMMENT '用户软删时间',
`created_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `idx_owner_cookie` (`owner_cookie`),
KEY `idx_status` (`status`),
KEY `idx_user_deleted` (`user_deleted`),
KEY `idx_created_at` (`created_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

View File

@@ -3,6 +3,12 @@
(function (global) { (function (global) {
"use strict"; "use strict";
// 布尔属性:用属性赋值而非 setAttributesetAttribute("checked","false") 仍会勾选)
const BOOL_PROPS = new Set([
"checked", "disabled", "readonly", "selected", "hidden", "multiple", "open",
"autofocus", "required", "async", "defer", "controls", "autoplay", "loop", "muted",
]);
function el(tag, attrs, ...children) { function el(tag, attrs, ...children) {
const node = document.createElement(tag); const node = document.createElement(tag);
if (attrs) { if (attrs) {
@@ -11,7 +17,8 @@
else if (k === "dataset") Object.assign(node.dataset, v); else if (k === "dataset") Object.assign(node.dataset, v);
else if (k.startsWith("on") && typeof v === "function") else if (k.startsWith("on") && typeof v === "function")
node.addEventListener(k.slice(2).toLowerCase(), v); node.addEventListener(k.slice(2).toLowerCase(), v);
else if (v !== null && v !== undefined) node.setAttribute(k, v); else if (BOOL_PROPS.has(k)) node[k] = !!v;
else if (v !== null && v !== undefined && v !== false) node.setAttribute(k, v);
} }
} }
for (const c of children) { for (const c of children) {
@@ -54,15 +61,15 @@
} }
function fmtBytes(n) { function fmtBytes(n) {
if (n == null) return "-"; let x = Number(n);
const x = Number(n); if (n == null || !isFinite(x)) return "-";
if (!isFinite(x)) return "-"; const units = ["B", "KB", "MB", "GB", "TB", "PB"];
for (const unit of ["B", "KiB", "MiB", "GiB", "TiB", "PiB"]) { let i = 0;
if (Math.abs(x) < 1024 || unit === "PiB") while (Math.abs(x) >= 1000 && i < units.length - 1) {
return unit === "B" ? `${x} B` : `${x.toFixed(1)} ${unit}`; x /= 1000;
n = x / 1024; i++;
} }
return `${n.toFixed(1)} PiB`; return i === 0 ? `${Math.round(x)} ${units[i]}` : `${x.toFixed(1)} ${units[i]}`;
} }
function fmtTime(s) { function fmtTime(s) {
@@ -76,8 +83,15 @@
async function api(path, opts) { async function api(path, opts) {
const res = await fetch(path, opts); const res = await fetch(path, opts);
if (res.status === 401) { if (res.status === 401) {
// 触发浏览器 Basic Auth 弹窗(同源 reload 即可带上凭据) // fetch 不会触发浏览器 Basic Auth 弹窗(只有导航/form 会)。
// 重载当前页:浏览器对页面导航的 401 会弹凭据框,凭据缓存后重试即可带上。
toast("需要登录"); toast("需要登录");
if (location.href.indexOf("/api/") !== -1) {
// 纯 API 调用页(无页面壳),跳转到来源页触发鉴权
location.reload();
} else {
location.reload();
}
throw new Error("UNAUTHORIZED"); throw new Error("UNAUTHORIZED");
} }
return res; return res;

View File

@@ -5,6 +5,7 @@
.files-table th.col-size { width: 9em; } .files-table th.col-size { width: 9em; }
.files-table th.col-src { width: 6em; } .files-table th.col-src { width: 6em; }
.files-table th.col-time { width: 11em; } .files-table th.col-time { width: 11em; }
.files-table th.col-pdf { width: 13em; }
.files-table th.col-act { width: 9em; text-align: right; } .files-table th.col-act { width: 9em; text-align: right; }
.files-table td.col-act { text-align: right; white-space: nowrap; } .files-table td.col-act { text-align: right; white-space: nowrap; }
.files-table td.col-name .fname { font-weight: 600; word-break: break-all; } .files-table td.col-name .fname { font-weight: 600; word-break: break-all; }
@@ -12,9 +13,24 @@
.files-table td.col-act .btn { padding: 0.3em 0.8em; font-size: 0.85em; margin-left: 0.3em; } .files-table td.col-act .btn { padding: 0.3em 0.8em; font-size: 0.85em; margin-left: 0.3em; }
.files-table tbody tr.sel { background: var(--primary-soft); } .files-table tbody tr.sel { background: var(--primary-soft); }
.files-table tbody tr.sel:hover td { background: var(--primary-soft); } .files-table tbody tr.sel:hover td { background: var(--primary-soft); }
/* 关联到已软删 PDF 任务的行:淡化背景提示 */
.files-table tbody tr.row-pdf-deleted td { background: var(--danger-soft); }
.files-table tbody tr.row-pdf-deleted:hover td { background: var(--danger-soft); }
.files-table input[type="checkbox"] { width: 16px; height: 16px; cursor: pointer; accent-color: var(--primary); } .files-table input[type="checkbox"] { width: 16px; height: 16px; cursor: pointer; accent-color: var(--primary); }
.sha-short { cursor: pointer; } .sha-short { cursor: pointer; }
.sha-short:hover { color: var(--primary); } .sha-short:hover { color: var(--primary); }
/* PDF 任务列:角色徽标 + 状态徽标堆叠 */
.pdf-cell { display: flex; flex-direction: column; gap: 0.2em; font-size: 0.82em; }
.pdf-cell .pdf-roles { display: flex; flex-wrap: wrap; gap: 0.3em; align-items: center; }
.pdf-cell .role-tag {
display: inline-block; padding: 0.05em 0.5em; border-radius: 8px;
font-size: 0.78em; background: var(--surface-2); color: var(--text-dim);
}
.pdf-cell .role-tag.source { background: var(--warn-soft); color: var(--warn); }
.pdf-cell .role-tag.output { background: var(--success-soft); color: var(--success); }
.pdf-cell .del-mark { color: var(--danger); font-size: 0.78em; }
.pdf-cell .pdf-act { display: flex; gap: 0.3em; flex-wrap: wrap; }
.pdf-cell .pdf-act .btn { padding: 0.2em 0.6em; font-size: 0.8em; }
.row-removed { opacity: 0; transition: opacity 0.25s; } .row-removed { opacity: 0; transition: opacity 0.25s; }
.toolbar-spacer { flex: 1; } .toolbar-spacer { flex: 1; }
@@ -47,5 +63,6 @@
.files-table th, .files-table td { padding: 0.5em 0.4em; } .files-table th, .files-table td { padding: 0.5em 0.4em; }
.files-table th.col-src, .files-table td.col-src { display: none; } .files-table th.col-src, .files-table td.col-src { display: none; }
.files-table th.col-time, .files-table td.col-time { font-size: 0.78em; } .files-table th.col-time, .files-table td.col-time { font-size: 0.78em; }
.files-table th.col-pdf, .files-table td.col-pdf { font-size: 0.78em; }
.batchbar { font-size: 0.85em; } .batchbar { font-size: 0.85em; }
} }

View File

@@ -3,14 +3,14 @@
<head> <head>
<meta charset="utf-8"> <meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title>文件浏览 - zikai</title> <title>文件管理 - zikai</title>
<link rel="stylesheet" href="/static/common.css"> <link rel="stylesheet" href="/api/static/common.css">
<link rel="stylesheet" href="/static/file_browser.css"> <link rel="stylesheet" href="/api/static/file_browser.css">
</head> </head>
<body> <body>
<div class="wrap"> <div class="wrap">
<h1>文件浏览</h1> <h1>文件管理</h1>
<p class="sub">查看已上传的文件、下载或删除。支持多选与分页。删除后不再显示</p> <p class="sub">浏览 / 下载 / 删除已上传文件,并合并 PDF 转换管理:关联文件会标注转换状态与用户软删标记,可硬删任务。支持多选与分页</p>
<div class="toolbar"> <div class="toolbar">
<button class="btn primary" id="refresh">刷新</button> <button class="btn primary" id="refresh">刷新</button>
@@ -39,9 +39,9 @@
<div id="pager" class="pager hidden"></div> <div id="pager" class="pager hidden"></div>
<p class="foot"><a class="link" href="/upload">上传文件</a> · zikai file service</p> <p class="foot"><a class="link" href="/api/index">导航</a> · <a class="link" href="/api/upload">上传文件</a> · zikai file service</p>
</div> </div>
<script src="/static/common.js"></script> <script src="/api/static/common.js"></script>
<script src="/static/file_browser.js"></script> <script src="/api/static/file_browser.js"></script>
</body> </body>
</html> </html>

View File

@@ -1,4 +1,6 @@
/* 文件浏览页:分页拉取 /api/admin/files、复选框多选 + 全选、批量下载/删除、复制 sha。 /* 文件管理页:分页拉取 /api/admin/files-with-pdf、复选框多选 + 全选、批量下载/删除、复制 sha。
合并 PDF 转换管理:每个文件附带 pdf_jobsrole=source epub / role=output 产物 PDF
渲染转换状态徽标、用户软删标记并支持硬删任务DELETE /api/admin/pdf/jobs/{id})。
state.items 缓存当前页数据;切换页/页大小重新拉取;删除后若当前页空则回退一页。 */ state.items 缓存当前页数据;切换页/页大小重新拉取;删除后若当前页空则回退一页。 */
(function () { (function () {
"use strict"; "use strict";
@@ -53,7 +55,7 @@
countEl.textContent = ""; countEl.textContent = "";
pagerEl.classList.add("hidden"); pagerEl.classList.add("hidden");
try { try {
const res = await api(`/api/admin/files?limit=${limit}&offset=${offset}`); const res = await api(`/api/admin/files/with-pdf?limit=${limit}&offset=${offset}`);
if (!res.ok) throw new Error("HTTP " + res.status); if (!res.ok) throw new Error("HTTP " + res.status);
const body = await res.json(); const body = await res.json();
state.items = body.items || []; state.items = body.items || [];
@@ -73,7 +75,7 @@
function render() { function render() {
if (!state.items.length) { if (!state.items.length) {
listEl.innerHTML = '<div class="empty">还没有文件。去 <a class="link" href="/upload">上传</a> 一个吧。</div>'; listEl.innerHTML = '<div class="empty">还没有文件。去 <a class="link" href="/api/upload">上传</a> 一个吧。</div>';
renderSelection(); renderSelection();
return; return;
} }
@@ -88,13 +90,15 @@
el("th", { class: "col-size" }, "大小"), el("th", { class: "col-size" }, "大小"),
el("th", { class: "col-src" }, "来源"), el("th", { class: "col-src" }, "来源"),
el("th", { class: "col-time" }, "上传时间"), el("th", { class: "col-time" }, "上传时间"),
el("th", { class: "col-pdf" }, "PDF 任务"),
el("th", { class: "col-act" }, "操作") el("th", { class: "col-act" }, "操作")
) )
); );
const tbody = el("tbody", null); const tbody = el("tbody", null);
for (const f of state.items) { for (const f of state.items) {
const checked = state.selected.has(f.id); const checked = state.selected.has(f.id);
const row = el("tr", { class: checked ? "sel" : "", dataset: { id: f.id } }, const pdfDeleted = (f.pdf_jobs || []).some((j) => j.user_deleted);
const row = el("tr", { class: [checked ? "sel" : "", pdfDeleted ? "row-pdf-deleted" : ""].filter(Boolean).join(" "), dataset: { id: f.id } },
el("td", { class: "col-sel" }, el("td", { class: "col-sel" },
el("input", { type: "checkbox", class: "row-sel", checked, dataset: { id: f.id } }) el("input", { type: "checkbox", class: "row-sel", checked, dataset: { id: f.id } })
), ),
@@ -105,6 +109,7 @@
el("td", { class: "col-size mono" }, fmtBytes(f.size_bytes)), el("td", { class: "col-size mono" }, fmtBytes(f.size_bytes)),
el("td", { class: "col-src" }, el("span", { class: "tag" }, f.source || "-")), el("td", { class: "col-src" }, el("span", { class: "tag" }, f.source || "-")),
el("td", { class: "col-time muted" }, fmtTime(f.uploaded_at)), el("td", { class: "col-time muted" }, fmtTime(f.uploaded_at)),
el("td", { class: "col-pdf" }, renderPdfCell(f)),
el("td", { class: "col-act" }, el("td", { class: "col-act" },
el("a", { class: "btn primary", href: `/api/admin/files/${f.id}/download`, download: "" }, "下载"), el("a", { class: "btn primary", href: `/api/admin/files/${f.id}/download`, download: "" }, "下载"),
el("button", { class: "btn danger", onclick: () => removeOne(f) }, "删除") el("button", { class: "btn danger", onclick: () => removeOne(f) }, "删除")
@@ -190,6 +195,58 @@
return sha.length > 16 ? sha.slice(0, 12) + "…" + sha.slice(-4) : sha; return sha.length > 16 ? sha.slice(0, 12) + "…" + sha.slice(-4) : sha;
} }
// 渲染 PDF 任务列:展示角色(源 epub / 产物 PDF徽标、转换状态、用户软删标记与硬删按钮。
// 一个文件可能同时被多个 job 引用(罕见),全列出;无关联则显示 -。
function renderPdfCell(f) {
const jobs = f.pdf_jobs || [];
if (!jobs.length) return el("span", { class: "muted" }, "-");
const cell = el("div", { class: "pdf-cell" });
const roles = el("div", { class: "pdf-roles" });
for (const j of jobs) {
roles.appendChild(el("span", { class: "role-tag " + j.role, title: j.role === "source" ? "PDF 转换的 epub 源文件" : "PDF 转换的产物 PDF" },
j.role === "source" ? "源" : "产物"));
roles.appendChild(statusTag(j));
if (j.user_deleted) {
roles.appendChild(el("span", { class: "del-mark", title: "用户已软删:" + fmtTime(j.deleted_at) }, "已删"));
}
roles.appendChild(el("span", { class: "muted" }, "#"+j.job_id));
}
cell.appendChild(roles);
// 硬删按钮:对每个关联 job 提供source 与 output 共属同一 job去重后只显示一个
const seenJobIds = new Set();
const actBar = el("div", { class: "pdf-act" });
for (const j of jobs) {
if (seenJobIds.has(j.job_id)) continue;
seenJobIds.add(j.job_id);
actBar.appendChild(el("button", {
class: "btn danger", onclick: () => removePdfJob(j, f)
}, "硬删任务"));
}
cell.appendChild(actBar);
return cell;
}
function statusTag(j) {
if (j.status === "done") return el("span", { class: "tag ok" }, "完成");
if (j.status === "failed") return el("span", { class: "tag err", title: j.error_message || "" }, "失败");
if (j.status === "converting") return el("span", { class: "tag warn" }, "转换中 " + j.progress + "%");
return el("span", { class: "tag" }, "排队中");
}
// 硬删 PDF 任务:删 job + 其源/产物文件。删除后重载当前页。
async function removePdfJob(j, f) {
if (!confirm(`确定硬删 PDF 任务 #${j.job_id}\n将同时删除关联的源文件与产物 PDF不可恢复。`)) return;
try {
const res = await api(`/api/admin/pdf/jobs/${j.job_id}`, { method: "DELETE" });
if (res.status === 404) { toast("任务已不存在"); }
else if (!res.ok) throw new Error("HTTP " + res.status);
toast("已硬删任务");
await load(state.page);
} catch (e) {
toast("硬删失败:" + (e.message || e), "err");
}
}
async function removeOne(f) { async function removeOne(f) {
if (!confirm(`确定删除「${f.original_filename}」?\n此操作不可恢复,将同时删除磁盘文件。`)) return; if (!confirm(`确定删除「${f.original_filename}」?\n此操作不可恢复,将同时删除磁盘文件。`)) return;
try { try {

74
static/index.html Normal file
View File

@@ -0,0 +1,74 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>导航 - zikai</title>
<link rel="stylesheet" href="/api/static/common.css">
<style>
/* 导航页:卡片网格,复用 common.css 变量 */
.nav-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(220px, 1fr));
gap: 1em;
margin-top: 1.5em;
}
.nav-card {
display: flex; flex-direction: column; gap: 0.3em;
padding: 1.2em 1.3em;
border: 1px solid var(--border);
border-radius: var(--radius);
background: var(--surface);
box-shadow: var(--shadow);
text-decoration: none; color: var(--text);
transition: transform 0.05s, border-color 0.15s;
}
.nav-card:hover { transform: translateY(-2px); border-color: var(--primary); }
.nav-card .ico { font-size: 1.6em; }
.nav-card .name { font-size: 1.05em; font-weight: 600; }
.nav-card .desc { color: var(--text-dim); font-size: 0.82em; line-height: 1.4; }
.nav-card .lock { font-size: 0.75em; color: var(--text-dim); }
</style>
</head>
<body>
<div class="wrap">
<h1>zikai 工具箱</h1>
<p class="sub">个人 Web 服务入口导航。带 🔒 标记的页面需 Basic Auth凭据见 config.yaml 的 docs 段)。</p>
<div class="nav-grid">
<a class="nav-card" href="/api/upload">
<span class="ico">📤</span>
<span class="name">上传文件</span>
<span class="desc">拖拽 / 多文件 / 分片4 MiB/ 断点续传</span>
<span class="lock"></span>
</a>
<a class="nav-card" href="/api/files-page">
<span class="ico">📁</span>
<span class="name">文件管理</span>
<span class="desc">浏览 / 下载 / 删除已上传文件,合并 PDF 转换管理(状态、软删标记、硬删任务)</span>
<span class="lock">🔒 需鉴权</span>
</a>
<a class="nav-card" href="/api/wb-admin">
<span class="ico">📝</span>
<span class="name">记事本管理</span>
<span class="desc">查看 / 删除共享文本记事本,可新建并打开</span>
<span class="lock">🔒 需鉴权</span>
</a>
<a class="nav-card" href="/api/system/status">
<span class="ico">📊</span>
<span class="name">系统状态</span>
<span class="desc">CPU / 内存 / 磁盘实时使用率(?format=json 切 JSON</span>
<span class="lock"></span>
</a>
<a class="nav-card" href="/docs">
<span class="ico">📚</span>
<span class="name">API 文档</span>
<span class="desc">Swagger UI全部 REST 接口在线调试</span>
<span class="lock">🔒 需鉴权</span>
</a>
</div>
<p class="foot">zikai file service · <a class="link" href="/api/health">健康探针</a></p>
</div>
</body>
</html>

View File

@@ -1,5 +1,17 @@
/* 白板页专属样式:全屏画布、悬浮工具栏、移动端适配。 */ /* 记事本页专属样式:全屏 textarea、悬浮工具栏、移动端适配。 */
:root { --bar-h: 52px; } /* 强制浅色:本页常被 iframe 嵌入浅色站点,覆盖 common.css 的
color-scheme: light dark 与 prefers-color-scheme: dark避免深色背景。 */
:root {
--bar-h: 52px;
color-scheme: light;
--bg: #f6f7f9;
--surface: #ffffff;
--surface-2: rgba(0, 0, 0, 0.02);
--border: #e0e3e7;
--text: #1a1a1a;
--text-dim: #6b7280;
--shadow: 0 1px 3px rgba(0, 0, 0, 0.04), 0 1px 2px rgba(0, 0, 0, 0.06);
}
body { overflow: hidden; background: var(--bg); } body { overflow: hidden; background: var(--bg); }
.wb-app { display: flex; flex-direction: column; height: 100vh; height: 100dvh; } .wb-app { display: flex; flex-direction: column; height: 100vh; height: 100dvh; }
.wb-bar { .wb-bar {
@@ -14,22 +26,20 @@ body { overflow: hidden; background: var(--bg); }
.wb-id { background: var(--surface-2); padding: 0.15em 0.5em; border-radius: 6px; font-size: 0.82em; color: var(--text-dim); } .wb-id { background: var(--surface-2); padding: 0.15em 0.5em; border-radius: 6px; font-size: 0.82em; color: var(--text-dim); }
.wb-online { color: var(--success); font-size: 0.7em; } .wb-online { color: var(--success); font-size: 0.7em; }
.wb-online.off { color: var(--text-dim); } .wb-online.off { color: var(--text-dim); }
.wb-tool { display: inline-flex; align-items: center; gap: 0.3em; font-size: 0.85em; color: var(--text-dim); }
.wb-tool input[type="color"] { width: 28px; height: 28px; padding: 0; border: 1px solid var(--border); border-radius: 6px; background: transparent; cursor: pointer; }
.wb-tool input[type="range"] { width: 80px; accent-color: var(--primary); }
.wb-width-val { width: 1.4em; text-align: center; }
.wb-bar .btn { padding: 0.4em 0.9em; font-size: 0.86em; } .wb-bar .btn { padding: 0.4em 0.9em; font-size: 0.86em; }
.wb-stage { position: relative; flex: 1; overflow: hidden; } .wb-stage { position: relative; flex: 1; overflow: hidden; }
#canvas { .wb-editor {
position: absolute; inset: 0; width: 100%; height: 100%; position: absolute; inset: 0;
display: block; touch-action: none; cursor: crosshair; width: 100%; height: 100%;
background: display: block; resize: none; border: none; outline: none;
linear-gradient(var(--border) 1px, transparent 1px) 0 0 / 24px 24px, padding: 1em 1.2em;
linear-gradient(90deg, var(--border) 1px, transparent 1px) 0 0 / 24px 24px, font-family: ui-monospace, "SF Mono", Menlo, Consolas, "JetBrains Mono", monospace;
var(--surface); font-size: 14px; line-height: 1.6;
background-blend-mode: normal; background: var(--surface); color: var(--text);
touch-action: manipulation;
} }
.wb-editor::placeholder { color: var(--text-dim); }
.wb-status { .wb-status {
position: absolute; bottom: 0.8em; left: 50%; transform: translateX(-50%); position: absolute; bottom: 0.8em; left: 50%; transform: translateX(-50%);
background: var(--surface); border: 1px solid var(--border); background: var(--surface); border: 1px solid var(--border);
@@ -44,7 +54,7 @@ body { overflow: hidden; background: var(--bg); }
@media (max-width: 640px) { @media (max-width: 640px) {
.wb-bar { padding: 0.4em 0.5em; gap: 0.4em; } .wb-bar { padding: 0.4em 0.5em; gap: 0.4em; }
.wb-id { max-width: 8em; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } .wb-id { max-width: 8em; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.wb-tool input[type="range"] { width: 56px; }
.wb-bar .btn { padding: 0.45em 0.7em; } .wb-bar .btn { padding: 0.45em 0.7em; }
.wb-title { display: none; } .wb-title { display: none; }
.wb-editor { font-size: 15px; padding: 0.8em; }
} }

View File

@@ -4,36 +4,34 @@
<meta charset="utf-8"> <meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no"> <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
<meta name="theme-color" content="#1565c0"> <meta name="theme-color" content="#1565c0">
<title>白板 - zikai</title> <!-- 强制浅色:该页面会被 iframe 嵌入到浅色主题站点mainPage
<link rel="stylesheet" href="/static/common.css"> 禁用 common.css 的 prefers-color-scheme: dark保持背景与嵌入站一致 -->
<link rel="stylesheet" href="/static/whiteboard.css"> <meta name="color-scheme" content="light">
<title>记事本 - zikai</title>
<link rel="stylesheet" href="/api/static/common.css">
<link rel="stylesheet" href="/api/static/whiteboard.css">
</head> </head>
<body> <body>
<div class="wb-app"> <div class="wb-app">
<header class="wb-bar"> <header class="wb-bar">
<div class="wb-bar-left"> <div class="wb-bar-left">
<span class="wb-title">白板</span> <span class="wb-title">记事本</span>
<code class="wb-id mono" id="boardId" title="白板 ID"></code> <code class="wb-id mono" id="boardId" title="记事本 ID"></code>
<span class="wb-online" id="online" title="在线人数"></span> <span class="wb-online" id="online" title="在线人数"></span>
</div> </div>
<div class="wb-bar-right"> <div class="wb-bar-right">
<label class="wb-tool" title="笔画颜色"> <a class="btn" href="/api/index" target="_top" title="返回导航">导航</a>
<input type="color" id="color" value="#1565c0"> <button class="btn" id="copyLinkBtn" title="复制分享链接">复制链接</button>
</label> <button class="btn" id="copyBtn" title="复制全部文本">复制文本</button>
<label class="wb-tool" title="笔画粗细"> <button class="btn danger" id="clearBtn" title="清空全部内容(所有人)">清空</button>
<input type="range" id="width" min="1" max="24" value="3">
<span class="wb-width-val mono" id="widthVal">3</span>
</label>
<button class="btn" id="copyBtn" title="复制分享链接">复制链接</button>
<button class="btn danger" id="clearBtn" title="清空白板(所有人)">清空</button>
</div> </div>
</header> </header>
<main class="wb-stage"> <main class="wb-stage">
<canvas id="canvas"></canvas> <textarea id="editor" class="wb-editor" placeholder="在此输入文本,所有人会实时看到你的编辑…" spellcheck="false" autocomplete="off"></textarea>
<div class="wb-status" id="status">连接中…</div> <div class="wb-status" id="status">连接中…</div>
</main> </main>
</div> </div>
<script src="/static/common.js"></script> <script src="/api/static/common.js"></script>
<script src="/static/whiteboard.js"></script> <script src="/api/static/whiteboard.js"></script>
</body> </body>
</html> </html>

View File

@@ -1,153 +1,149 @@
/* 白板Canvas 绘画 + WebSocket 实时同步 + 心跳。 /* 记事本textarea + WebSocket 实时同步 + 心跳。
- 笔画以 {points:[[x,y],...], color, width} 表示,增量广播 - 文本以整文本 debounce 400ms 后发服务端,服务端存为新版本并广播给其他端
- 心跳 3s 一次 ping服务端 15s 无心跳判失活会主动断连,前端据此重连 - 收到他人 update 时用 diff 应用变更,保留本地光标位置(按相对偏移调整)
- 收到他人 stroke 增量重绘该笔;收到 cleared 清空本地画布 - 收到 cleared 清空本地 textarea
- 兼容鼠标 + 触摸:统一用 pointer eventstouch-action:none 防滚动缩放。 */ - 心跳 3s 一次 ping服务端 15s 无心跳判失活会主动断连,前端据此重连。 */
(function () { (function () {
"use strict"; "use strict";
const { el, toast, copyText } = window.ZK; const { toast, copyText } = window.ZK;
// ---------- 从 URL 解析 board_id ---------- // ---------- 从 URL 解析 board_id ----------
// 路径形如 /whiteboard/{id}id 为 [a-zA-Z0-9_-]{1,64} const m = location.pathname.match(/^\/api\/wb-page\/([^/]+)\/?$/);
const m = location.pathname.match(/^\/whiteboard\/([^/]+)\/?$/);
let boardId = m ? decodeURIComponent(m[1]) : "default"; let boardId = m ? decodeURIComponent(m[1]) : "default";
// 合法性兜底:前端非法字符直接回退到 default真正校验在服务端
if (!/^[a-zA-Z0-9_-]{1,64}$/.test(boardId)) boardId = "default"; if (!/^[a-zA-Z0-9_-]{1,64}$/.test(boardId)) boardId = "default";
document.getElementById("boardId").textContent = boardId; document.getElementById("boardId").textContent = boardId;
// ---------- DOM ---------- // ---------- DOM ----------
const canvas = document.getElementById("canvas"); const editor = document.getElementById("editor");
const ctx = canvas.getContext("2d");
const colorInput = document.getElementById("color");
const widthInput = document.getElementById("width");
const widthVal = document.getElementById("widthVal");
const clearBtn = document.getElementById("clearBtn"); const clearBtn = document.getElementById("clearBtn");
const copyBtn = document.getElementById("copyBtn"); const copyBtn = document.getElementById("copyBtn");
const copyLinkBtn = document.getElementById("copyLinkBtn");
const statusEl = document.getElementById("status"); const statusEl = document.getElementById("status");
const onlineEl = document.getElementById("online"); const onlineEl = document.getElementById("online");
// ---------- 状态 ---------- // ---------- 状态 ----------
let strokes = []; // 已确认的笔画
let current = null; // 正在画的笔画(本地未提交)
let drawing = false;
let ws = null; let ws = null;
let clientId = localStorage.getItem("wb_cid") || ""; // 每个 tab 独立的 client_id用 sessionStorage 而非 localStorage避免同浏览器
// 多 tab 共享 id 导致收到的 update 被误判为「自己的」而跳过不同步)。
// 服务端已用 exclude=conn 排除发送者,前端不再用 client_id 跳过 update。
let clientId = sessionStorage.getItem("wb_cid") || "";
if (!clientId) { if (!clientId) {
clientId = "c_" + Math.random().toString(36).slice(2, 10); clientId = "c_" + Math.random().toString(36).slice(2, 10);
localStorage.setItem("wb_cid", clientId); sessionStorage.setItem("wb_cid", clientId);
} }
let heartbeatTimer = null; let heartbeatTimer = null;
let reconnectTimer = null; let reconnectTimer = null;
let connected = false; let connected = false;
let lastSentText = ""; // 上次发到服务端的文本(避免无变更时重复发)
let suppressInput = false; // 应用远端更新时抑制 input 事件,防回环
let debounceTimer = null;
// ---------- 画布尺寸 ---------- // ---------- 本地编辑 -> debounce -> 发送 ----------
function resize() { editor.addEventListener("input", () => {
const dpr = window.devicePixelRatio || 1; if (suppressInput) return;
const w = canvas.clientWidth; scheduleSend();
const h = canvas.clientHeight;
canvas.width = Math.max(1, Math.floor(w * dpr));
canvas.height = Math.max(1, Math.floor(h * dpr));
ctx.setTransform(dpr, 0, 0, dpr, 0, 0);
redraw();
}
window.addEventListener("resize", resize);
// ---------- 绘制 ----------
function drawStroke(s) {
if (!s || !s.points || s.points.length < 1) return;
ctx.strokeStyle = s.color || "#1565c0";
ctx.lineWidth = Number(s.width) || 3;
ctx.lineCap = "round";
ctx.lineJoin = "round";
const pts = s.points;
ctx.beginPath();
ctx.moveTo(pts[0][0], pts[0][1]);
if (pts.length === 1) {
// 单点:画一个小圆点
ctx.arc(pts[0][0], pts[0][1], (ctx.lineWidth || 3) / 2, 0, Math.PI * 2);
ctx.fillStyle = ctx.strokeStyle;
ctx.fill();
return;
}
for (let i = 1; i < pts.length; i++) ctx.lineTo(pts[i][0], pts[i][1]);
ctx.stroke();
}
function redraw() {
ctx.clearRect(0, 0, canvas.width, canvas.height);
for (const s of strokes) drawStroke(s);
if (current) drawStroke(current);
}
// ---------- 指针事件 ----------
function pos(e) {
const r = canvas.getBoundingClientRect();
return [e.clientX - r.left, e.clientY - r.top];
}
canvas.addEventListener("pointerdown", (e) => {
if (!connected) { flashStatus("未连接,正在重连…"); return; }
e.preventDefault();
canvas.setPointerCapture(e.pointerId);
drawing = true;
current = { points: [pos(e)], color: colorInput.value, width: Number(widthInput.value) };
drawStroke(current);
});
canvas.addEventListener("pointermove", (e) => {
if (!drawing) return;
e.preventDefault();
const p = pos(e);
const last = current.points[current.points.length - 1];
// 跳过过近的点,减少数据量
if (Math.hypot(p[0] - last[0], p[1] - last[1]) < 1.5) return;
current.points.push(p);
// 增量画最后一段
ctx.strokeStyle = current.color;
ctx.lineWidth = current.width;
ctx.lineCap = "round"; ctx.lineJoin = "round";
ctx.beginPath();
ctx.moveTo(last[0], last[1]);
ctx.lineTo(p[0], p[1]);
ctx.stroke();
});
function endStroke(e) {
if (!drawing) return;
drawing = false;
if (e && e.pointerId !== undefined) {
try { canvas.releasePointerCapture(e.pointerId); } catch {}
}
if (current && current.points.length) {
strokes.push(current);
send({ type: "stroke", stroke: current });
}
current = null;
}
canvas.addEventListener("pointerup", endStroke);
canvas.addEventListener("pointercancel", endStroke);
canvas.addEventListener("pointerleave", (e) => {
// 仅在抬起时结束离开但按住不放不结束pointer capture 已处理)
if (!drawing) return;
if (e.buttons === 0) endStroke(e);
}); });
widthInput.addEventListener("input", () => (widthVal.textContent = widthInput.value)); function scheduleSend() {
if (debounceTimer) clearTimeout(debounceTimer);
debounceTimer = setTimeout(() => {
debounceTimer = null;
const text = editor.value;
if (text === lastSentText) return;
lastSentText = text;
send({ type: "edit", content: text });
}, 400);
}
function flushSend() {
if (debounceTimer) {
clearTimeout(debounceTimer);
debounceTimer = null;
const text = editor.value;
if (text !== lastSentText) {
lastSentText = text;
send({ type: "edit", content: text });
}
}
}
clearBtn.addEventListener("click", () => { clearBtn.addEventListener("click", () => {
if (!connected) { flashStatus("未连接"); return; } if (!connected) { flashStatus("未连接"); return; }
if (!confirm("确定清空白板?所有人的内容都会被清除。")) return; if (!confirm("确定清空全部内容?所有人的内容都会被清除。")) return;
send({ type: "clear" }); send({ type: "clear" });
}); });
copyBtn.addEventListener("click", async () => { copyBtn.addEventListener("click", async () => {
const url = `${location.origin}/whiteboard/${boardId}`; const text = editor.value;
if (!text) { toast("内容为空"); return; }
const ok = await copyText(text);
toast(ok ? "已复制全部文本" : "复制失败");
});
copyLinkBtn.addEventListener("click", async () => {
const url = `${location.origin}/api/wb-page/${boardId}`;
const ok = await copyText(url); const ok = await copyText(url);
toast(ok ? "链接已复制" : "复制失败"); toast(ok ? "链接已复制" : "复制失败");
}); });
// ---------- 应用远端更新(保留光标) ----------
// 策略:用最长公共前后缀算出变更区间,仅替换该区间,光标按相对位置调整。
// 若本地有未发送的编辑editor.value !== lastSentText合并后重新 scheduleSend
// 避免本地编辑被远端覆盖后因 lastSentText 短路而丢弃。
function applyRemoteUpdate(newText) {
const oldText = editor.value;
if (newText === oldText) return;
const selStart = editor.selectionStart;
const selEnd = editor.selectionEnd;
// 算公共前缀
let prefix = 0;
const minLen = Math.min(oldText.length, newText.length);
while (prefix < minLen && oldText[prefix] === newText[prefix]) prefix++;
// 算公共后缀(不能与前缀重叠)
let suffixOld = oldText.length;
let suffixNew = newText.length;
while (suffixOld > prefix && suffixNew > prefix && oldText[suffixOld - 1] === newText[suffixNew - 1]) {
suffixOld--;
suffixNew--;
}
const hadPending = editor.value !== lastSentText;
suppressInput = true;
// 用 setRangeText 替换 [prefix, suffixOld) 为 newText[prefix, suffixNew)
editor.setRangeText(newText.slice(prefix, suffixNew), prefix, suffixOld, "end");
suppressInput = false;
lastSentText = editor.value;
// 调整光标:若光标在变更区间之前,不动;在之后,平移差值;在区间内,移到区间末尾
const delta = (suffixNew - prefix) - (suffixOld - prefix);
let newStart = selStart, newEnd = selEnd;
if (selStart <= prefix) {
// 光标在变更前,不变
} else if (selStart >= suffixOld) {
// 光标在变更后,平移
newStart = selStart + delta;
newEnd = selEnd + delta;
} else {
// 光标在变更区间内,移到区间末尾
newStart = newEnd = suffixNew;
}
try {
editor.setSelectionRange(newStart, newEnd);
} catch {}
// 仅当编辑器当前有焦点时恢复焦点,避免抢其它控件焦点
if (document.activeElement === editor) editor.focus();
// 本地有未发送编辑被合并了,重新安排发送,避免被丢弃
if (hadPending) scheduleSend();
}
// ---------- WebSocket ---------- // ---------- WebSocket ----------
function wsUrl() { function wsUrl() {
const proto = location.protocol === "https:" ? "wss:" : "ws:"; const proto = location.protocol === "https:" ? "wss:" : "ws:";
return `${proto}//${location.host}/ws/whiteboard/${encodeURIComponent(boardId)}`; return `${proto}//${location.host}/api/ws/wb/${encodeURIComponent(boardId)}`;
} }
function connect() { function connect() {
@@ -160,7 +156,7 @@
} }
ws.onopen = () => { ws.onopen = () => {
connected = true; connected = true;
setStatus("已连接", true); setStatus("已连接", false);
onlineEl.classList.remove("off"); onlineEl.classList.remove("off");
send({ type: "hello", client_id: clientId }); send({ type: "hello", client_id: clientId });
startHeartbeat(); startHeartbeat();
@@ -175,23 +171,34 @@
try { msg = JSON.parse(raw); } catch { return; } try { msg = JSON.parse(raw); } catch { return; }
switch (msg.type) { switch (msg.type) {
case "init": case "init":
strokes = Array.isArray(msg.strokes) ? msg.strokes : []; // 连接建立时服务端下发当前文本。若本地有未发送编辑(断线期间输入的),
redraw(); // 不直接覆盖而是把本地编辑作为最新版本发上去last-writer-wins
setStatus(`已同步 ${strokes.length}`, true); // 避免断线期间的编辑被静默丢弃。
if (editor.value && editor.value !== lastSentText) {
lastSentText = editor.value;
send({ type: "edit", content: editor.value });
} else {
suppressInput = true;
editor.value = msg.content || "";
lastSentText = editor.value;
suppressInput = false;
editor.focus();
}
setStatus("已同步", false);
break; break;
case "pong": case "pong":
// 心跳回声,保持连接
break; break;
case "stroke": case "update":
if (msg.client_id === clientId) break; // 自己的,已本地画 // 服务端 broadcast 已用 exclude=conn 排除发送者,收到即他人编辑,直接应用。
strokes.push(msg.stroke); applyRemoteUpdate(msg.content || "");
drawStroke(msg.stroke); flashStatus("对方有更新");
break; break;
case "cleared": case "cleared":
strokes = []; suppressInput = true;
current = null; editor.value = "";
redraw(); lastSentText = "";
flashStatus(msg.client_id === clientId ? "已清空" : "对方清空了白板"); suppressInput = false;
flashStatus(msg.client_id === clientId ? "已清空" : "对方清空了内容");
break; break;
case "error": case "error":
flashStatus(msg.msg || "错误", true); flashStatus(msg.msg || "错误", true);
@@ -242,28 +249,33 @@
} }
// ---------- 启动 ---------- // ---------- 启动 ----------
// 先 GET /whiteboard/{id} 确保白板存在(不存在则服务端新建),再连 WS // 先 GET /api/wb/{id} 确保白板存在(不存在则服务端新建),再连 WS
fetch(`/whiteboard/${encodeURIComponent(boardId)}`, { headers: { accept: "application/json" } }) fetch(`/api/wb/${encodeURIComponent(boardId)}`, { headers: { accept: "application/json" } })
.then((r) => r.ok ? r.json() : null) .then((r) => (r.ok ? r.json() : null))
.then((body) => { .then((body) => {
if (body && Array.isArray(body.strokes)) { if (body && typeof body.content === "string") {
strokes = body.strokes; editor.value = body.content;
redraw(); lastSentText = body.content;
} }
resize();
connect(); connect();
}) })
.catch(() => { resize(); connect(); }); .catch(() => { connect(); });
// 页面隐藏时不发心跳可被服务端判失活;重新可见时主动重连 // 页面可见性:重新可见时若已断连则主动重连;隐藏时 flush 未发送的编辑
document.addEventListener("visibilitychange", () => { document.addEventListener("visibilitychange", () => {
if (document.visibilityState === "visible" && (!ws || ws.readyState !== WebSocket.OPEN)) { if (document.visibilityState === "visible") {
if (reconnectTimer) { clearTimeout(reconnectTimer); reconnectTimer = null; } if (!ws || ws.readyState !== WebSocket.OPEN) {
connect(); if (reconnectTimer) { clearTimeout(reconnectTimer); reconnectTimer = null; }
connect();
}
} else {
flushSend();
} }
}); });
// 离开前 flush 未发送编辑
window.addEventListener("beforeunload", () => { window.addEventListener("beforeunload", () => {
flushSend();
stopHeartbeat(); stopHeartbeat();
if (reconnectTimer) clearTimeout(reconnectTimer); if (reconnectTimer) clearTimeout(reconnectTimer);
try { ws && ws.close(); } catch {} try { ws && ws.close(); } catch {}

View File

@@ -4,8 +4,8 @@
<meta charset="utf-8"> <meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title>白板管理 - zikai</title> <title>白板管理 - zikai</title>
<link rel="stylesheet" href="/static/common.css"> <link rel="stylesheet" href="/api/static/common.css">
<link rel="stylesheet" href="/static/whiteboard_admin.css"> <link rel="stylesheet" href="/api/static/whiteboard_admin.css">
</head> </head>
<body> <body>
<div class="wrap"> <div class="wrap">
@@ -22,9 +22,9 @@
<div class="skel">加载中…</div> <div class="skel">加载中…</div>
</div> </div>
<p class="foot"><a class="link" href="/upload">上传文件</a> · <a class="link" href="/files">文件浏览</a> · zikai</p> <p class="foot"><a class="link" href="/api/index">导航</a> · <a class="link" href="/api/upload">上传文件</a> · <a class="link" href="/api/files-page">文件管理</a> · zikai</p>
</div> </div>
<script src="/static/common.js"></script> <script src="/api/static/common.js"></script>
<script src="/static/whiteboard_admin.js"></script> <script src="/api/static/whiteboard_admin.js"></script>
</body> </body>
</html> </html>

View File

@@ -1,4 +1,4 @@
/* 白板管理页:拉取 /api/admin/whiteboards、渲染表格、删除、新建并跳转。 */ /* 白板管理页:拉取 /api/admin/wb、渲染表格、删除、新建并跳转。 */
(function () { (function () {
"use strict"; "use strict";
const { el, toast, fmtTime, api } = window.ZK; const { el, toast, fmtTime, api } = window.ZK;
@@ -11,14 +11,14 @@
newBtn.addEventListener("click", () => { newBtn.addEventListener("click", () => {
// 生成一个随机 board_id 并打开(访问即创建) // 生成一个随机 board_id 并打开(访问即创建)
const id = "b_" + Math.random().toString(36).slice(2, 10); const id = "b_" + Math.random().toString(36).slice(2, 10);
window.open(`/whiteboard/${id}`, "_blank"); window.open(`/api/wb-page/${id}`, "_blank");
}); });
async function load() { async function load() {
listEl.innerHTML = '<div class="skel">加载中…</div>'; listEl.innerHTML = '<div class="skel">加载中…</div>';
countEl.textContent = ""; countEl.textContent = "";
try { try {
const res = await api("/api/admin/whiteboards?limit=500&offset=0"); const res = await api("/api/admin/wb?limit=500&offset=0");
if (!res.ok) throw new Error("HTTP " + res.status); if (!res.ok) throw new Error("HTTP " + res.status);
const body = await res.json(); const body = await res.json();
render(body.items || []); render(body.items || []);
@@ -47,9 +47,9 @@
for (const b of items) { for (const b of items) {
const row = el("tr", null, const row = el("tr", null,
el("td", { class: "col-id" }, el("td", { class: "col-id" },
el("a", { class: "bid link", href: `/whiteboard/${b.board_id}`, target: "_blank" }, b.board_id) el("a", { class: "bid link", href: `/api/wb-page/${b.board_id}`, target: "_blank" }, b.board_id)
), ),
el("td", { class: "col-mods mono" }, String(b.stroke_count ?? 0)), el("td", { class: "col-mods mono" }, String(b.edit_count ?? 0)),
el("td", { class: "col-created muted" }, fmtTime(b.created_at)), el("td", { class: "col-created muted" }, fmtTime(b.created_at)),
el("td", { class: "col-updated muted" }, fmtTime(b.updated_at)), el("td", { class: "col-updated muted" }, fmtTime(b.updated_at)),
el("td", { class: "col-act" }, el("td", { class: "col-act" },
@@ -67,7 +67,7 @@
async function remove(b, row) { async function remove(b, row) {
if (!confirm(`确定删除白板「${b.board_id}」?\n所有在线协作者会被断开,内容不可恢复。`)) return; if (!confirm(`确定删除白板「${b.board_id}」?\n所有在线协作者会被断开,内容不可恢复。`)) return;
try { try {
const res = await api(`/api/admin/whiteboards/${encodeURIComponent(b.board_id)}`, { method: "DELETE" }); const res = await api(`/api/admin/wb/${encodeURIComponent(b.board_id)}`, { method: "DELETE" });
if (res.status === 404) { toast("白板已不存在"); } if (res.status === 404) { toast("白板已不存在"); }
else if (!res.ok) throw new Error("HTTP " + res.status); else if (!res.ok) throw new Error("HTTP " + res.status);
row.classList.add("row-removed"); row.classList.add("row-removed");

View File

@@ -8,7 +8,7 @@ import urllib.request
import websockets import websockets
BASE_WS = "ws://127.0.0.1:6890/ws/whiteboard" BASE_WS = "ws://127.0.0.1:6867/api/ws/wb"
BOARD = "kicktest" BOARD = "kicktest"
AUTH = "Basic YTo2NjUxMTMxNQ==" # a:66511315 AUTH = "Basic YTo2NjUxMTMxNQ==" # a:66511315
@@ -22,7 +22,7 @@ async def main() -> None:
# 通过 REST 删除白板(带 Basic Auth # 通过 REST 删除白板(带 Basic Auth
req = urllib.request.Request( req = urllib.request.Request(
f"http://127.0.0.1:6890/api/admin/whiteboards/{BOARD}", f"http://127.0.0.1:6867/api/admin/wb/{BOARD}",
method="DELETE", method="DELETE",
headers={"Authorization": AUTH}, headers={"Authorization": AUTH},
) )

View File

@@ -1,22 +1,24 @@
"""白板 WebSocket 端到端烟测:两客户端实时同步 + 心跳 + 清空。 """白板 WebSocket 端到端烟测(文本记事本):两客户端实时同步 + 心跳 + 清空。
验证: 验证:
1. 客户端 A 连入 -> 收到 init 1. 客户端 A 连入 -> 收到 init(含 content/version
2. 客户端 B 连入 -> 收到 init 2. 客户端 B 连入 -> 收到 init
3. A 画一笔 -> B 收到 stroke 广播A 不收自己的) 3. A 编辑文本 -> B 收到 updateA 不收自己的)
4. 心跳 ping -> pong 4. 心跳 ping -> pong
5. A 清空 -> A、B 都收到 cleared 5. A 清空 -> A、B 都收到 cleared
6. 停发心跳的连接会被服务端 reaper 移除15s这里只验证 ping/pong 即可reaper 已在 hub 单测覆盖) 6. 持久化:重连后 init 应返回清空后的内容
7. edit_count 累计
""" """
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import json import json
import urllib.request
import websockets import websockets
BASE_WS = "ws://127.0.0.1:6890/ws/whiteboard" BASE_WS = "ws://127.0.0.1:6867/api/ws/wb"
BOARD = "e2etest" BOARD = "e2etest"
@@ -31,7 +33,6 @@ async def recv_msg(ws, timeout=2.0) -> dict | None:
async def main() -> None: async def main() -> None:
async with websockets.connect(f"{BASE_WS}/{BOARD}") as a, \ async with websockets.connect(f"{BASE_WS}/{BOARD}") as a, \
websockets.connect(f"{BASE_WS}/{BOARD}") as b: websockets.connect(f"{BASE_WS}/{BOARD}") as b:
# hello
await a.send(json.dumps({"type": "hello", "client_id": "A"})) await a.send(json.dumps({"type": "hello", "client_id": "A"}))
await b.send(json.dumps({"type": "hello", "client_id": "B"})) await b.send(json.dumps({"type": "hello", "client_id": "B"}))
@@ -42,18 +43,18 @@ async def main() -> None:
assert init_a and init_a["type"] == "init" assert init_a and init_a["type"] == "init"
assert init_b and init_b["type"] == "init" assert init_b and init_b["type"] == "init"
# A 画一笔 # A 编辑文本
stroke = {"points": [[10, 10], [20, 20]], "color": "#1565c0", "width": 3} text = "Hello, this is a shared note.\nSecond line."
await a.send(json.dumps({"type": "stroke", "stroke": stroke})) await a.send(json.dumps({"type": "edit", "content": text}))
# A 不应收到自己的(排除发送者) # A 不应收到自己的 update
echo = await recv_msg(a, timeout=1.0) echo = await recv_msg(a, timeout=1.0)
print("A self-echo (expect None):", echo) print("A self-echo (expect None):", echo)
assert echo is None, "发送者不应收到自己的 stroke" assert echo is None, "发送者不应收到自己的 update"
# B 应收到 # B 应收到 update
got = await recv_msg(b) got = await recv_msg(b)
print("B recv:", got.get("type") if got else None, "client_id=", got.get("client_id") if got else None) print("B recv:", got.get("type") if got else None, "content=", repr(got.get("content")) if got else None)
assert got and got["type"] == "stroke" and got["client_id"] == "A" assert got and got["type"] == "update" and got["client_id"] == "A"
assert got["stroke"] == stroke assert got["content"] == text
# 心跳 # 心跳
await a.send(json.dumps({"type": "ping"})) await a.send(json.dumps({"type": "ping"}))
@@ -70,20 +71,19 @@ async def main() -> None:
assert cleared_b and cleared_b["type"] == "cleared" and cleared_b["client_id"] == "B" assert cleared_b and cleared_b["type"] == "cleared" and cleared_b["client_id"] == "B"
assert cleared_a and cleared_a["type"] == "cleared" and cleared_a["client_id"] == "B" assert cleared_a and cleared_a["type"] == "cleared" and cleared_a["client_id"] == "B"
# 验证持久化:重连后 init 应 strokes 为空(已清空) # 验证持久化:重连后 init 应 content 为空(已清空)
async with websockets.connect(f"{BASE_WS}/{BOARD}") as c: async with websockets.connect(f"{BASE_WS}/{BOARD}") as c:
await c.send(json.dumps({"type": "hello", "client_id": "C"})) await c.send(json.dumps({"type": "hello", "client_id": "C"}))
init_c = await recv_msg(c) init_c = await recv_msg(c)
print("C init after clear, strokes=", init_c.get("strokes") if init_c else None) print("C init after clear, content=", repr(init_c.get("content")) if init_c else None)
assert init_c and init_c["type"] == "init" assert init_c and init_c["type"] == "init"
assert init_c["strokes"] == [], "清空后重连应得到空 strokes" assert init_c["content"] == "", "清空后重连应得到空 content"
# 验证 stroke_count 累计(之前 1 笔 + 1 次清空 = 2 # 验证 edit_count 累计(1 次编辑 + 1 次清空 = 2
import urllib.request with urllib.request.urlopen(f"http://127.0.0.1:6867/api/wb/{BOARD}") as r:
with urllib.request.urlopen(f"http://127.0.0.1:6890/whiteboard/{BOARD}") as r:
meta = json.load(r) meta = json.load(r)
print("stroke_count after ops:", meta["stroke_count"]) print("edit_count after ops:", meta["edit_count"])
assert meta["stroke_count"] == 2, "1 + 1 清空 = 2 次修改" assert meta["edit_count"] == 2, "1 编辑 + 1 清空 = 2 次"
print("\nWS 端到端全部通过 ✅") print("\nWS 端到端全部通过 ✅")

284
tests/test_pdf_service.py Normal file
View File

@@ -0,0 +1,284 @@
"""PDF 转换服务测试pytest + TestClient
用 SQLite 内存库覆盖 get_db免依赖 MySQL用临时目录覆盖上传根目录。
覆盖完整链路:上传 epub -> 轮询至 done -> 下载有效 PDF -> 用户软删 ->
管理页仍可见 -> 管理员硬删 -> 真正删除。另测大小超限与格式校验。
"""
from __future__ import annotations
import io
import time
import zipfile
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import create_engine, text
from sqlalchemy.orm import sessionmaker
from app import config as config_module
from app import database as db_module
from app.database import Base, get_db
from app.models.uploaded_file import UploadedFile # noqa: F401 注册映射
# ---------------- fixtures ----------------
def _make_epub_bytes() -> bytes:
"""构造一个最小的合法 epub含 1 章节 + 1 PNG 图片 + CSS"""
from PIL import Image # 已是 weasyprint 依赖间接项,环境可用
png = io.BytesIO()
Image.new("RGBA", (8, 8), (26, 95, 180, 255)).save(png, format="PNG")
png_bytes = png.getvalue()
ch = (
'<?xml version="1.0" encoding="UTF-8"?>'
'<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Ch1</title>'
'<link rel="stylesheet" type="text/css" href="style.css"/></head>'
'<body><h1>Hello PDF</h1><p>Test paragraph for conversion.</p>'
'<p><img src="img.png" alt="pic"/></p></body></html>'
)
opf = (
'<?xml version="1.0" encoding="UTF-8"?>'
'<package xmlns="http://www.idpf.org/2007/opf" version="2.0" unique-identifier="Bid">'
'<metadata xmlns:dc="http://purl.org/dc/elements/1.1/">'
'<dc:title>T</dc:title><dc:identifier id="Bid">urn:uuid:t</dc:identifier>'
'<dc:language>en</dc:language></metadata>'
'<manifest>'
'<item id="ncx" href="toc.ncx" media-type="application/x-dtbncx+xml"/>'
'<item id="css" href="style.css" media-type="text/css"/>'
'<item id="img" href="img.png" media-type="image/png"/>'
'<item id="ch1" href="ch1.xhtml" media-type="application/xhtml+xml"/>'
'</manifest><spine toc="ncx"><itemref idref="ch1"/></spine></package>'
)
ncx = (
'<?xml version="1.0" encoding="UTF-8"?>'
'<ncx xmlns="http://www.daisy.org/z3986/2005/ncx/" version="2005-1">'
'<head><meta name="dtb:uid" content="urn:uuid:t"/></head>'
'<docTitle><text>T</text></docTitle>'
'<navMap><navPoint id="n1" playOrder="1"><navLabel><text>Ch1</text></navLabel>'
'<content src="ch1.xhtml"/></navPoint></navMap></ncx>'
)
css = "h1{color:#1a5fb4} p{line-height:1.6}"
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w") as z:
z.writestr("mimetype", "application/epub+zip", compress_type=zipfile.ZIP_STORED)
z.writestr("OEBPS/content.opf", opf, compress_type=zipfile.ZIP_DEFLATED)
z.writestr("OEBPS/toc.ncx", ncx, compress_type=zipfile.ZIP_DEFLATED)
z.writestr("OEBPS/style.css", css, compress_type=zipfile.ZIP_DEFLATED)
z.writestr("OEBPS/img.png", png_bytes, compress_type=zipfile.ZIP_DEFLATED)
z.writestr("OEBPS/ch1.xhtml", ch, compress_type=zipfile.ZIP_DEFLATED)
z.writestr("META-INF/container.xml",
'<?xml version="1.0"?><container version="1.0" '
'xmlns="urn:oasis:names:tc:opendocument:xmlns:container">'
'<rootfiles><rootfile full-path="OEBPS/content.opf" '
'media-type="application/oebps-package+xml"/></rootfiles></container>',
compress_type=zipfile.ZIP_DEFLATED)
return buf.getvalue()
@pytest.fixture()
def client(tmp_path, monkeypatch):
"""构造用隔离 MySQL 测试库 + 临时上传目录的 TestClient。
用独立库 zikai_filesvc_test与生产库隔离每个 fixture 重建表,准确测试生产
行为MySQL BigInteger 自增、线程安全连接)。后台转换经 asyncio.to_thread 在
独立线程执行MySQL 连接池天然支持跨线程。
"""
real_settings = config_module.get_settings()
# 用独立测试库 zikai_filesvc_test与生产库隔离准确测试生产行为
test_settings = real_settings.model_copy(deep=True)
test_settings.database.database = "zikai_filesvc_test"
test_settings.pdf.convert_timeout_seconds = 60
# 全局 get_settings 被 lru_cacheconfig 与 database 两个模块各自 import 了它,
# 都需 patch使后台线程经 database.get_session_local也读到测试库
monkeypatch.setattr(config_module, "get_settings", lambda: test_settings)
monkeypatch.setattr(db_module, "get_settings", lambda: test_settings)
# 清掉已建的全局引擎/SessionLocal下次 get_session_local() 用测试库重建
db_module.dispose_engine()
upload_dir = tmp_path / "uploads"
upload_dir.mkdir()
# 复用全局 SessionLocal指向测试库保证请求路径与后台线程用同一库
SessionLocal = db_module.get_session_local()
# 覆盖 get_db仍用全局 SessionLocal但确保请求结束关闭
def override_get_db():
db = SessionLocal()
try:
yield db
finally:
db.close()
# 上传目录指向临时目录
monkeypatch.setattr(test_settings, "storage", test_settings.storage)
test_settings.storage.upload_dir = str(upload_dir)
# upload_service / pdf_service 内部 import 了 get_settings需 patch 其引用
import app.services.upload_service as us
import app.services.pdf_service as ps
import app.services.pdf_converter as pc # noqa: F401
monkeypatch.setattr(us, "get_settings", lambda: test_settings)
monkeypatch.setattr(ps, "get_settings", lambda: test_settings)
# 每次测试前清空重建表,保证隔离
Base.metadata.drop_all(bind=db_module.get_engine())
Base.metadata.create_all(bind=db_module.get_engine())
from app.main import app
app.dependency_overrides[get_db] = override_get_db
with TestClient(app) as c:
yield c
app.dependency_overrides.clear()
Base.metadata.drop_all(bind=db_module.get_engine())
db_module.dispose_engine()
def _wait_done(client, job_id, cookie, timeout=60):
"""轮询任务状态直到 done/failed 或超时。"""
deadline = time.time() + timeout
last = None
while time.time() < deadline:
r = client.get(f"/api/pdf/jobs/{job_id}", cookies=cookie)
assert r.status_code == 200, r.text
last = r.json()
if last["status"] in ("done", "failed"):
return last
time.sleep(0.3)
raise AssertionError(f"任务未在 {timeout}s 内完成,最后状态: {last}")
# 管理页 Basic Auth 凭据(与 config.yaml 的 docs 段保持一致)
ADMIN_AUTH = ("a", "66511315")
# ---------------- 测试 ----------------
class TestSubmitAndConvert:
def test_upload_epub_converts_and_downloads(self, client):
epub = _make_epub_bytes()
# 首次上传:无 cookie应下发新 cookie
r = client.post(
"/api/pdf/jobs",
files={"file": ("test.epub", epub, "application/epub+zip")},
)
assert r.status_code == 200, r.text
body = r.json()
assert body["set_cookie"] is True
assert "zk_pdf" in r.headers.get("set-cookie", "")
job = body["job"]
assert job["status"] == "pending"
job_id = job["id"]
cookie = {"zk_pdf": r.cookies.get("zk_pdf")}
# 轮询至完成
final = _wait_done(client, job_id, cookie)
assert final["status"] == "done", final
assert final["progress"] == 100
# 下载产物:应为有效 PDF
d = client.get(f"/api/pdf/jobs/{job_id}/download", cookies=cookie)
assert d.status_code == 200, d.text
assert d.headers["content-type"] == "application/pdf"
assert d.content[:5] == b"%PDF-"
assert len(d.content) > 100
def test_user_list_shows_own_jobs(self, client):
epub = _make_epub_bytes()
r = client.post("/api/pdf/jobs", files={"file": ("a.epub", epub, "application/epub+zip")})
cookie = {"zk_pdf": r.cookies.get("zk_pdf")}
lst = client.get("/api/pdf/jobs", cookies=cookie)
assert lst.status_code == 200
assert lst.json()["total"] == 1
# 另一用户(清空 client cookie jar 模拟全新浏览器,应下发新 cookie
client.cookies.clear()
r2 = client.post("/api/pdf/jobs", files={"file": ("b.epub", epub, "application/epub+zip")})
cookie2 = {"zk_pdf": r2.cookies.get("zk_pdf")}
assert cookie2["zk_pdf"] != cookie["zk_pdf"] # 确是不同用户
lst2 = client.get("/api/pdf/jobs", cookies=cookie2)
assert lst2.json()["total"] == 1 # 只有自己的 1 个
def test_rejects_non_epub(self, client):
r = client.post(
"/api/pdf/jobs",
files={"file": ("note.txt", b"hello world", "text/plain")},
)
assert r.status_code == 400
def test_rejects_oversize(self, client, monkeypatch):
# 把上限调到极小,避免真的构造 250MB 文件
import app.services.pdf_service as ps
real = ps.get_settings()
small = real.model_copy(deep=True)
small.pdf.max_size_bytes = 100
monkeypatch.setattr(ps, "get_settings", lambda: small)
monkeypatch.setattr(ps, "get_settings", lambda: small)
# controller 的 _resolve_cookie 也读 get_settings但 submit 内 service 用 ps.get_settings
epub = _make_epub_bytes()
r = client.post("/api/pdf/jobs", files={"file": ("big.epub", epub, "application/epub+zip")})
assert r.status_code == 413
class TestDeleteSemantics:
def test_user_soft_delete_admin_still_visible_then_admin_hard_delete(self, client):
epub = _make_epub_bytes()
r = client.post("/api/pdf/jobs", files={"file": ("del.epub", epub, "application/epub+zip")})
cookie = {"zk_pdf": r.cookies.get("zk_pdf")}
job_id = r.json()["job"]["id"]
_wait_done(client, job_id, cookie)
# 用户软删
d = client.delete(f"/api/pdf/jobs/{job_id}", cookies=cookie)
assert d.status_code == 200
assert d.json()["deleted"] is True
# 用户列表不再可见
lst = client.get("/api/pdf/jobs", cookies=cookie)
assert lst.json()["total"] == 0
# 管理页仍可见且标注已删除
al = client.get("/api/admin/pdf/jobs", auth=ADMIN_AUTH)
assert al.status_code == 200
items = al.json()["items"]
assert any(it["id"] == job_id and it["user_deleted"] is True for it in items)
assert any(it["id"] == job_id and it["deleted_at"] is not None for it in items)
# 用户已无法下载(已软删)
dl = client.get(f"/api/pdf/jobs/{job_id}/download", cookies=cookie)
assert dl.status_code == 404
# 管理员硬删
ad = client.delete(f"/api/admin/pdf/jobs/{job_id}", auth=ADMIN_AUTH)
assert ad.status_code == 200
assert ad.json()["deleted"] is True
# 管理页也不再可见
al2 = client.get("/api/admin/pdf/jobs", auth=ADMIN_AUTH)
assert not any(it["id"] == job_id for it in al2.json()["items"])
def test_admin_requires_auth(self, client):
r = client.get("/api/admin/pdf/jobs")
assert r.status_code == 401
r = client.get("/api/admin/pdf/jobs", auth=("a", "wrong"))
assert r.status_code == 401
r = client.get("/api/admin/pdf/jobs", auth=ADMIN_AUTH)
assert r.status_code == 200
def test_user_cannot_access_others_job(self, client):
epub = _make_epub_bytes()
r1 = client.post("/api/pdf/jobs", files={"file": ("x.epub", epub, "application/epub+zip")})
job_id = r1.json()["job"]["id"]
# 另一用户访问(清空 cookie jar 模拟全新浏览器)
client.cookies.clear()
r2 = client.post("/api/pdf/jobs", files={"file": ("y.epub", epub, "application/epub+zip")})
cookie2 = {"zk_pdf": r2.cookies.get("zk_pdf")}
assert client.get(f"/api/pdf/jobs/{job_id}", cookies=cookie2).status_code == 404
assert client.get(f"/api/pdf/jobs/{job_id}/download", cookies=cookie2).status_code == 404
assert client.delete(f"/api/pdf/jobs/{job_id}", cookies=cookie2).status_code == 404