Compare commits

..

9 Commits

Author SHA1 Message Date
7c193ca46e 前端整理: 合并文件管理页 + 新增导航页
1. 合并 files-page 与 pdf-admin 为统一文件管理页 /api/files-page:
   - 新增 GET /api/admin/files/with-pdf 接口,以 uploaded_files 为基础,
     用 pdf_jobs.source_file_id / output_file_id 内存匹配,为关联文件标注
     转换状态、用户软删标记与角色(源epub/产物PDF)
   - PdfJobOut 补 source_file_id / output_file_id 字段
   - file_browser 新增 PDF 任务列(状态徽标/软删标记/硬删任务按钮)
   - 删除 /api/pdf-admin 路由及 static/pdf_admin.* 三个文件

2. 新增导航页 /api/index,卡片式收集所有页面入口;
   各子页(upload/whiteboard/system_status/files-page)脚注加返回导航链接

3. 更新 docs/routes.md、docs/configuration.md 同步说明

测试: pytest tests/test_pdf_service.py 7 passed; 手动校验各页面路由与合并接口响应
2026-07-28 14:04:12 +08:00
9af28f41b4 refactor: 清理死代码/提前失败/日志/高内聚低耦合
死代码移除:
- whiteboard_hub.py: 移除未引用的 reset_hub 单例重置函数
- tunnel_service.py: 移除未引用的 is_port_allowed (逻辑已在 sftp_server 内联)
- tunnel_session_dao.py: 移除未引用的 get_active_by_port
- pdf_job_dao.py: 移除未用 datetime 导入
- pdf_converter.py: 移除未用 shutil 导入
- pdf_service.py: 移除未用 PdfSubmitResponse 导入 + _do_convert 内未用 hashlib 导入
- upload_html.py: 移除未用 escape 导入 (JS 侧自有 escapeHtml)
- pdf_controller.py: 移除 _resolve_cookie 内未用 cfg 局部变量

提前失败/分层修复:
- database.py init_db_schema: 建表后用 inspector 校验既有表列与模型一致,
  缺列即抛 RuntimeError (fail-fast on schema drift), 避免运行期才暴露
- whiteboard_dao.get_or_create: 仅 IntegrityError 才回滚重读, 其他异常向上抛
  (原 except Exception 会掩盖 schema/连接等真实故障)
- pdf_service.admin_delete/_safe_delete_file: 改用 PdfJobDAO.delete /
  UploadedFileDAO.delete, 不再直接操作 job_dao.db / file_dao.db (修复分层契约:
  DAO 头注释声明 service 不直接操作 session)
- PdfJobDAO 新增 delete(job) 方法

日志补全 (8 处 silent catch):
- whiteboard_hub.py disconnect/close_board 关闭 ws: logger.debug
- whiteboard_controller _safe_send/_safe_close: logger.debug
- sftp_server _close_tunnel_dao/读用户名: logger.debug
- sftp_server validate_public_key: logger.warning (auth 路径, 避免静默失败)

文档:
- 新增 docs/routes.md, docs/configuration.md, docs/error-handling.md
- README.md 精简为简介/结构/外部依赖/apache2 配置/Ubuntu 安装/docs 链接
2026-07-28 11:34:35 +08:00
1c0f776571 feat(pdf-admin): 支持全选/多选批量硬删并去掉删除二次确认
- 工具栏新增「批量硬删」按钮,实时显示选中数,无选中时禁用
- 表头复选框全选/反选,行复选框多选,行级选中状态同步全选框
- 批量删除并发执行 DELETE,逐项淡出移除并汇总结果 toast
- 单条与批量删除均直接执行,移除 confirm() 二次确认
- 新增 .col-check 复选框列样式(窄宽居中)
2026-07-28 09:39:39 +08:00
e7b4a3d5e3 feat: 移除冗余 PDF 用户服务端页,前后端分离由 zMainPage 组件接管
PDF 用户侧 UI 改由 zMainPage 的 zPDF_package 组件提供(构建期 import,非 iframe),
zTools2 仅保留 /api/pdf/* REST API 与 /api/pdf-admin 管理页。cookie 认证不受影响:
zk_pdf 仍在首次 POST /api/pdf/jobs 时种下,与已删的 /api/pdf 页面路由无关。

- app/main.py:删除 GET /api/pdf 页面路由(pdf_page)
- static/pdf.html、static/pdf.js、static/pdf.css:删除用户页 HTML/JS/CSS
  (功能已被 zPDF_package 的 App.vue/Uploader.vue/JobList.vue 取代)
- static/pdf_admin.css:合并原 pdf.css 中管理页依赖的样式,成为自包含样式表
- static/pdf_admin.html:移除指向已删 /api/pdf 的页脚死链与 pdf.css 引用
- README.md:功能一览/访问入口表去 /api/pdf 用户页条目,路由约定去 iframe 表述
2026-07-27 17:14:33 +08:00
ff2ad3fcb3 feat: 所有入口统一到 /api/ 前缀
将 zTools2 托管的页面/静态/探针/WebSocket 路由全部从顶级路径迁移到 /api/ 下:
- 页面:/pdf -> /api/pdf、/pdf-admin -> /api/pdf-admin、/upload -> /api/upload、
  /files -> /api/files-page、/wb/{id} -> /api/wb-page/{id}、/wb-admin -> /api/wb-admin
  (页面类加 -page 后缀以规避同名 REST API /api/files、/api/wb/{id})
- 静态资源:/static -> /api/static
- 探针:/health -> /api/health
- WebSocket:/ws/wb/{id} -> /api/ws/wb/{id}
- 前端 HTML 壳与 JS 中的资源/页间链接/WS URL 同步更新
- 手动测试脚本 BASE_WS 同步

这样反代与 vite proxy 只需一条 /api/ 规则即可转发全部入口;
前端 iframe 用同源相对路径 /api/pdf,与环境无关,不再误打到其它环境域名。
README 新增「路由约定」说明。
2026-07-27 15:42:47 +08:00
729c77e98b feat: 隐藏 PDF 用户页的管理页入口,凭据改为 账号a / 66511315
- pdf.html 移除页脚"管理页"链接:管理页入口对普通用户隐藏,
  仅管理员知晓 /pdf-admin 直达路径(权限同 /files 文件浏览页,
  复用 require_docs_auth Basic Auth)
- test_pdf_service.py 同步测试凭据为 a/66511315

注:实际凭据存放于 config.yaml 的 docs 段(git-ignored,仅本机 root 持有),
本次仅隐藏入口;管理页鉴权机制(require_docs_auth)此前已与
文件浏览页 /files 完全一致,无需改动。
2026-07-27 14:17:16 +08:00
29c5462734 feat: 新增 systemd 持久化部署(deploy/ztools2.service + install-systemd.sh)
systemd 管理 zTools2 后端实现开机自启 + 崩溃自动重启:
- deploy/ztools2.service 模板(Restart=on-failure,after mysql)
- deploy/install-systemd.sh 自动检测 .venv 或系统 uvicorn,填充路径安装
- README 补持久化部署章节
2026-07-27 13:52:23 +08:00
aa08a1cf60 feat: 新增 PDF 转换服务(epub->pdf,cookie 用户隔离,软删/硬删两级删除)
纯 Python 转换(ebooklib 解析 epub + weasyprint 渲染 HTML/CSS 为 PDF),
无需 calibre/xvfb 系统依赖。复用 UploadedFile 存储(原始文件与产物 PDF)。

- model/dao/schema:PdfJob 记录转换任务(owner_cookie/source/output/进度/状态)
- pdf_converter:epub->PDF 转换器(按 spine 顺序拼 HTML,OPF 目录解析相对资源)
- pdf_service:上传落盘 + asyncio 后台转换(独立 Session)+ 进度追踪 + 两级删除
  · 用户软删(user_deleted,管理页仍可见)· 管理员硬删(真正删磁盘+记录)
- pdf_controller:/api/pdf/*(用户,cookie)+ /api/admin/pdf/*(Basic Auth)
- /pdf、/pdf-admin 页面(原生 JS,复用 common.css/js)
- 配置 pdf 段(max_size 250MB、转换超时、cookie)
- pytest 7 项(SQLite→MySQL 隔离测试库):上传/转换/下载/软删/硬删/超限/越权
2026-07-27 11:33:43 +08:00
fa980e74e0 fix: 白板页强制浅色,避免被 iframe 嵌入到浅色站点时出现黑色背景
- whiteboard.html 加 <meta name="color-scheme" content="light">
- whiteboard.css 在 :root 钉死 color-scheme: light 与浅色变量,
  覆盖 common.css 的 prefers-color-scheme: dark
2026-07-22 02:44:03 +00:00
42 changed files with 1725 additions and 208 deletions

185
README.md
View File

@@ -1,104 +1,47 @@
# zikai file service # zTools2 - 个人 Web 服务后端
基于 FastAPI 的个人 Web 服务,提供**文件上传/浏览/下载、共享记事本(实时协作)、 基于 FastAPI 的个人 Web 服务后端,提供文件上传/浏览/下载、共享记事本(实时协作)、主机监控、SFTP 暂存、反向隧道、PDF 转换。采用 Spring 风格分层架构controller -> service -> dao -> ORM model -> MySQL自带 API 文档。
主机监控、SFTP 暂存、反向隧道**。采用 Spring 风格分层架构,自带 API 文档。
## 功能一览 所有入口统一挂在 `/api/` 前缀下,前端用同源相对路径调用,无需区分 dev/prod。前端子项目[timeTableFix](https://git.zikai.wang/zikai/timeTableFix)、[zPDF_package](https://git.zikai.wang/zikai/zPDF_package)、[zWhiteBoard](https://git.zikai.wang/zikai/zWhiteBoard))经 [zMainPage](https://git.zikai.wang/zikai/zMainPage) 构建期组件 import 集成,生产由 apache2 静态托管 + `/api` 反代到本服务。
| 模块 | 页面 / 接口 | 鉴权 |
|------|------------|------|
| 文件上传 | `POST /api/files/upload`(流式)/ `POST /api/files/chunk-uploads/*`(分片+断点续传) | 公开 |
| 上传页 | `GET /upload`(拖拽/多文件/分片/去重) | 公开 |
| 文件浏览 | `GET /files`(多选/批量下载删除/分页) | Basic Auth |
| 文件管理 API | `GET /api/admin/files``GET/DELETE /api/admin/files/{id}``GET /api/admin/files/{id}/download` | Basic Auth |
| 共享记事本 | `GET /wb/{id}`(公开,不存在则新建) | 公开 |
| 记事本实时同步 | `WS /ws/wb/{id}`(心跳 3s5 次失活移除) | 公开 |
| 记事本管理 | `GET /wb-admin`(查看/删除) | Basic Auth |
| 记事本管理 API | `GET /api/admin/wb``DELETE /api/admin/wb/{id}` | Basic Auth |
| 主机监控 | `GET /api/system/status`CPU/内存/磁盘HTML+JSON 内容协商) | 公开 |
| 反向隧道反代 | `ALL /api/userPort/{userName}`(经 SSH 隧道转发到 user 本地服务) | 公开 |
| SFTP/SSH | 端口 2022密码+公钥chroot 到上传目录,承载隧道转发) | SSH |
| API 文档 | `GET /docs`Swagger/ `GET /redoc` | Basic Auth |
## 项目结构 ## 项目结构
``` ```
server/ zTools2/
├── app/ ├── app/
│ ├── main.py # FastAPI 应用工厂、路由注册、生命周期reaper │ ├── main.py # FastAPI 应用工厂、路由注册、生命周期reaper
│ ├── config.py # 从 config.yaml 加载的类型化 Settingspydantic-settings │ ├── config.py # 从 config.yaml 加载的类型化 Settingspydantic-settings
│ ├── database.py # SQLAlchemy 引擎/Session/Base/get_db 依赖 │ ├── database.py # SQLAlchemy 引擎/Session/Base/get_db 依赖 + schema 校验
│ ├── security.py # Basic Authrequire_docs_auth常量时间比较 │ ├── security.py # Basic Authrequire_docs_auth常量时间比较
│ ├── controllers/ # 路由层@RestControllerfile/system/chunk/tunnel/whiteboard/admin │ ├── controllers/ # 路由层file/system/chunk/tunnel/whiteboard/pdf/admin
│ ├── services/ # 业务层UploadService/ChunkUploadService/SystemService/ │ ├── services/ # 业务层Upload/ChunkUpload/System/Whiteboard/Tunnel/Pdf/sftp
│ │ # WhiteboardService/WhiteboardHub/TunnelService/sftp_server
│ ├── dao/ # 数据访问层:唯一发 SQL 的层SQLAlchemy ORM 参数化) │ ├── dao/ # 数据访问层:唯一发 SQL 的层SQLAlchemy ORM 参数化)
│ ├── models/ # ORM 实体UploadedFile/UploadSession/Whiteboard/TunnelSession │ ├── models/ # ORM 实体UploadedFile/UploadSession/Whiteboard/TunnelSession/PdfJob
│ ├── schemas/ # pydantic 请求/响应 DTO │ ├── schemas/ # pydantic 请求/响应 DTO
│ ├── views/ # 服务端渲染 HTML系统状态页、上传页 │ ├── views/ # 服务端渲染 HTML系统状态页、上传页
│ ├── static/ # 前端静态资源common + file_browser + whiteboard + whiteboard_admin │ ├── static/ # 前端静态资源
│ └── scripts/init_db.py # 数据库初始化(建库建账、随机密码写回 config.yaml │ └── scripts/init_db.py # 数据库初始化(建库建账、随机密码写回 config.yaml
├── sql/schema.sql # 建表 DDL参考实际由 ORM 自动建表) ├── sql/schema.sql # 建表 DDL参考实际由 ORM 自动建表)
├── config.example.yaml # 配置模板(含注释) ├── config.example.yaml # 配置模板(含注释)
├── config.yaml # 实际配置git-ignored含密码 ├── config.yaml # 实际配置git-ignored含密码
├── requirements.txt ├── requirements.txt
├── setup.sh # 一次性初始化venv + 依赖 + 建库 + SFTP 密钥 ├── setup.sh # 一次性初始化venv + 依赖 + 建库 + SFTP 密钥
├── start.sh / stop.sh # 启停 HTTP6867+ SFTP2022 ├── start.sh / stop.sh # 启停 HTTP127.0.0.1:6867+ SFTP2022
└── logs/ # app.log / sftp.log └── deploy/ # systemd 持久化部署
``` ```
**请求流程**`controller service dao ORM model MySQL` **请求流程**`controller -> service -> dao -> ORM model -> MySQL`DB Session 由 `get_db` 依赖注入。
DB Session 由 `get_db` 依赖注入。前端页面走「StaticFiles 挂载 + 具名 HTML 路由」前后端分离JS 调同源 `/api/...`
## 从零安装Ubuntu 22.04+ ## 外部依赖
### 1. 安装系统依赖 | 项 | 说明 |
|----|------|
| Python | ≥ 3.11(用 `.venv` |
| MySQL | 8.x独立库 `zikai_filesvc`,由 `setup.sh` 建账) |
| Apache2 | 反向代理对外提供 HTTPS服务本身只绑 `127.0.0.1:6867` |
| 系统库 | `libpango/cairo`weasyprint PDF 转换)、`build-essential`bcrypt/asyncssh 编译) |
```bash ### Apache2 反向代理配置
apt update
apt install -y python3-venv python3-pip mysql-server apache2 \
libssl-dev build-essential # build-essential 给 bcrypt/asyncssh 编译
```
### 2. 获取代码
```bash
git clone <repo> /root/zikai
cd /root/zikai/server
```
### 3. 初始化venv + 依赖 + 建库 + SFTP 密钥)
```bash
./setup.sh
```
`setup.sh` 会:
- 创建 `.venv` 并安装 `requirements.txt`
- 复制 `config.example.yaml → config.yaml`(若不存在)
- 通过本机 root socket 建独立 MySQL 库 `zikai_filesvc` + 应用账户,随机密码写回 `config.yaml`
- 生成 SFTP 主机密钥(`keys/ssh_host_*`
### 4. 配置凭据
编辑 `config.yaml`(见下方[配置说明](#配置说明)
- `docs.username` / `docs.password`:管理页与 API 文档的 Basic Auth 凭据
- `sftp.users[].password_hash`SFTP 用户bcrypt生成方式见下
- `tunnel.users[]`:反向隧道用户(可选)
```bash
# 生成 bcrypt hash
.venv/bin/python -c "import bcrypt;print(bcrypt.hashpw(b'yourpass',bcrypt.gensalt()).decode())"
```
### 5. 启动
```bash
./start.sh # 启动 HTTP(127.0.0.1:6867) + SFTP(0.0.0.0:2022)
./stop.sh # 停止
```
### 6. 配置 Apache 反向代理
服务只绑 `127.0.0.1:6867`,通过 Apache 对外提供 HTTPS。安装模块并配置 vhost 服务只绑 `127.0.0.1:6867`,通过 Apache 对外提供 HTTPS。安装模块并配置 vhost
@@ -117,11 +60,10 @@ a2enmod ssl proxy proxy_http proxy_wstunnel rewrite headers
ProxyPreserveHost On ProxyPreserveHost On
ProxyPass /fdata ! ProxyPass /fdata !
# WebSocket 反代:/ws/ 必须在通用 / 规则之前,用 proxy_wstunnel 透传 # 所有 zTools2 入口(页面/静态/探针/WS/API统一在 /api/ 下,一条规则即可;
ProxyPass /ws/ ws://127.0.0.1:6867/ws/ # WebSocket 走 /api/ws/wb/{id},靠 proxy_wstunnel 透传 Upgrade 头
ProxyPassReverse /ws/ ws://127.0.0.1:6867/ws/ ProxyPass /api/ http://127.0.0.1:6867/api/
ProxyPass / http://127.0.0.1:6867/ ProxyPassReverse /api/ http://127.0.0.1:6867/api/
ProxyPassReverse / http://127.0.0.1:6867/
ProxyTimeout 300 ProxyTimeout 300
</VirtualHost> </VirtualHost>
``` ```
@@ -132,48 +74,51 @@ systemctl reload apache2
``` ```
> **防火墙**:放开 443HTTPS与 2022SFTP。6867 不对外(仅 loopback > **防火墙**:放开 443HTTPS与 2022SFTP。6867 不对外(仅 loopback
> **大文件上传**Apache 全局 `Timeout 300`,慢链路建议走分片上传(`/upload`)或 SFTP。 > **大文件上传**Apache 全局 `Timeout 300`,慢链路建议走分片上传(`/api/upload`)或 SFTP。
## 配置说明 ## 从零安装Ubuntu 22.04+
所有运行时配置在 `config.yaml`git-ignored。完整 schema 见 `config.example.yaml` ### 1. 安装系统依赖
| 段 | 关键项 | 说明 | ```bash
|----|--------|------| apt update
| `server` | `host`/`port` | 绑定地址,保持 `127.0.0.1:6867`Apache 反代) | apt install -y python3-venv python3-pip mysql-server apache2 \
| `database` | `host`/`port`/`user`/`password`/`database` | MySQL 连接;密码由 `setup.sh` 自动生成写回 | libssl-dev build-essential # build-essential 给 bcrypt/asyncssh 编译
| `storage` | `upload_dir` | 文件存储根目录(默认 `uploads` | # PDF 转换依赖 weasyprint需 pango/cairo 系统库:
| | `chunk_bytes` | 流式上传分片大小(默认 1 MiB | apt install -y libpango-1.0-0 libpangoft2-1.0-0 libcairo2 libgdk-pixbuf-2.0-0
| | `chunk_session_dir` | 分片会话暂存目录(默认 `uploads/.work` | ```
| | `chunk_session_ttl_seconds` | 被放弃会话存活秒数(默认 300 |
| `docs` | `username`/`password` | `/docs``/files``/wb-admin``/api/admin/*` 的 Basic Auth明文常量时间比较 |
| `sftp` | `enabled`/`host`/`port` | SFTP 服务,默认 `0.0.0.0:2022` |
| | `users[].username`/`password_hash` | SFTP 用户bcrypt |
| | `host_key_path`/`authorized_keys_path` | 主机密钥与公钥白名单路径 |
| `tunnel` | `enabled`/`users[]` | 反向隧道:`username`/`password_hash`/`tunnel_port`/`local_port` |
| `whiteboard` | `heartbeat_interval_seconds` | 心跳间隔(默认 3s |
| | `heartbeat_miss_threshold` | 失活阈值(默认 5 次 = 15s |
| | `max_board_id_length` | board_id 长度上限(默认 64 |
| | `max_connections_per_board` | 单白板并发连接上限(默认 50 |
| | `list_limit` | 管理页单次列表上限(默认 100 |
## 访问入口 ### 2. 获取代码并初始化
| 入口 | URL | ```bash
|------|-----| git clone <repo> /root/zikai
| API 文档 | https://f.zikai.wang/docsBasic Auth | cd /root/zikai/zTools2
| 上传页 | https://f.zikai.wang/upload | ./setup.sh # 创建 .venv + 装依赖 + 复制 config.yaml + 建 MySQL 库账 + 生成 SFTP 密钥
| 文件浏览 | https://f.zikai.wang/filesBasic Auth | ```
| 共享记事本 | https://f.zikai.wang/wb/{id}(公开,`{id}``[a-zA-Z0-9_-]{1,64}` |
| 记事本管理 | https://f.zikai.wang/wb-adminBasic Auth |
| 系统状态 | https://f.zikai.wang/api/system/statusHTML`?format=json` 切 JSON |
| curl 上传 | `curl -F file=@big.iso https://f.zikai.wang/api/files/upload` |
| SFTP | `sftp -P 2022 uploader@f.zikai.wang` |
## 运维 ### 3. 配置凭据
- **日志**`logs/app.log`HTTP`logs/sftp.log`SFTPpidfile`app.pid``sftp.pid` 编辑 `config.yaml`(详见 [`docs/configuration.md`](./docs/configuration.md)`docs.username/password`(管理页 Basic Auth`sftp.users[].password_hash`bcrypt、可选 `tunnel.users[]`
- **临时文件清理**:分片上传完成后立即删 `.work/<id>/`;被放弃会话(`pending` 超 5 分钟)由后台 reaper 每 60s 清理;`start.sh` 启动时兜底清残留。
- **重新生成 DB 密码**`.venv/bin/python -m app.scripts.init_db`(保留现有:`KEEP_DB_PASSWORD=1`)。 ### 4. 启动
- **多 worker 限制**:记事本 hub 是进程内存,多 uvicorn worker 下不互通,保持 `workers: 1`
- **数据库表**ORM 启动时自动建表(`init_db_schema``sql/schema.sql` 供参考/手动初始化。 ```bash
./start.sh # 启动 HTTP(127.0.0.1:6867) + SFTP(0.0.0.0:2022)
./stop.sh # 停止
```
### 5. 持久化部署systemd
```bash
./deploy/install-systemd.sh # 安装并启动 ztools2 服务(开机自启)
systemctl status ztools2
journalctl -u ztools2 -f
```
> 与 zMainPage 整体部署配合systemd 管后端,`zMainPage/deploy.sh --no-restart` 部署前端。
## 了解更多
- [路由与访问入口](./docs/routes.md)
- [配置说明](./docs/configuration.md)
- [错误处理与日志约定](./docs/error-handling.md)

View File

@@ -108,6 +108,24 @@ class WhiteboardConfig(BaseModel):
list_limit: int = 100 list_limit: int = 100
class PdfConfig(BaseModel):
"""PDF 转换服务配置。
用户侧(上传/查看/下载/软删)凭 httpOnly cookie 标识;管理侧(列表/硬删)
走 docs 同款 Basic Auth。原始文件与产物 PDF 复用 storage.upload_dir 落盘。
"""
# 单文件大小上限字节。250 MiB。
max_size_bytes: int = 250 * 1024 * 1024
# 转换超时(秒):超大/复杂文件兜底,避免长期占用 worker。
convert_timeout_seconds: int = 600
# 列表分页默认值
list_limit: int = 100
# 用户 cookie 名与有效期
cookie_name: str = "zk_pdf"
cookie_max_age_seconds: int = 365 * 24 * 3600
class Settings(BaseModel): class Settings(BaseModel):
server: ServerConfig = ServerConfig() server: ServerConfig = ServerConfig()
database: DatabaseConfig = DatabaseConfig() database: DatabaseConfig = DatabaseConfig()
@@ -116,6 +134,7 @@ class Settings(BaseModel):
docs: DocsConfig = DocsConfig() docs: DocsConfig = DocsConfig()
tunnel: TunnelConfig = TunnelConfig() tunnel: TunnelConfig = TunnelConfig()
whiteboard: WhiteboardConfig = WhiteboardConfig() whiteboard: WhiteboardConfig = WhiteboardConfig()
pdf: PdfConfig = PdfConfig()
def db_url(self) -> str: def db_url(self) -> str:
c = self.database c = self.database

View File

@@ -3,6 +3,7 @@
from .chunk_upload_controller import router as chunk_upload_router from .chunk_upload_controller import router as chunk_upload_router
from .file_admin_controller import router as file_admin_router from .file_admin_controller import router as file_admin_router
from .file_controller import router as file_router from .file_controller import router as file_router
from .pdf_controller import router as pdf_router
from .system_controller import router as system_router from .system_controller import router as system_router
from .tunnel_controller import router as tunnel_router from .tunnel_controller import router as tunnel_router
from .whiteboard_controller import router as whiteboard_router from .whiteboard_controller import router as whiteboard_router
@@ -11,6 +12,7 @@ __all__ = [
"chunk_upload_router", "chunk_upload_router",
"file_admin_router", "file_admin_router",
"file_router", "file_router",
"pdf_router",
"system_router", "system_router",
"tunnel_router", "tunnel_router",
"whiteboard_router", "whiteboard_router",

View File

@@ -14,8 +14,15 @@ from pydantic import BaseModel, Field
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from ..database import get_db from ..database import get_db
from ..dao.pdf_job_dao import PdfJobDAO
from ..dao.uploaded_file_dao import UploadedFileDAO from ..dao.uploaded_file_dao import UploadedFileDAO
from ..schemas.file import FileListResponse, UploadedFileOut from ..schemas.file import (
FileListResponse,
FileWithPdfListResponse,
PdfJobBrief,
UploadedFileOut,
UploadedFileWithPdfOut,
)
from ..security import require_docs_auth from ..security import require_docs_auth
from ..services.upload_service import UploadService from ..services.upload_service import UploadService
@@ -57,6 +64,53 @@ def list_files(
return FileListResponse(total=total, items=items) return FileListResponse(total=total, items=items)
@router.get(
"/with-pdf",
response_model=FileWithPdfListResponse,
summary="列出已上传文件并附带 PDF 转换任务关联(需鉴权)",
description=(
"合并管理页使用:以 uploaded_files 为基础分页拉取,再内存匹配 pdf_jobs"
"为每个文件附带它作为 PDF 任务「源文件(epub)」或「产物(PDF)」时的状态、"
"进度与用户软删标记。匹配键为 pdf_job.source_file_id / output_file_id -> uploaded_file.id。"
"返回前会触发磁盘扫描(频率限制 3s把 SFTP 上传但未入库的文件补录进来。"
),
)
def list_files_with_pdf(
limit: int = Query(100, ge=1, le=10000, description="每页条数1-10000"),
offset: int = Query(0, ge=0, description="偏移量"),
service: UploadService = Depends(_service),
db: Session = Depends(get_db),
_: str = Depends(require_docs_auth),
) -> FileWithPdfListResponse:
service.scan_sftp_files() # 频率限制内置3s 内重复只返回 DB 缓存
total, items = service.list_files(limit=limit, offset=offset)
# 拉全部 pdf_jobs数据量小不分页建反查 map
jobs = PdfJobDAO(db).list_all(limit=10000, offset=0)
by_source: dict[int, list] = {}
by_output: dict[int, list] = {}
for j in jobs:
by_source.setdefault(j.source_file_id, []).append(j)
if j.output_file_id is not None:
by_output.setdefault(j.output_file_id, []).append(j)
out_items: list[UploadedFileWithPdfOut] = []
for f in items:
briefs: list[PdfJobBrief] = []
for j in by_source.get(f.id, []):
briefs.append(PdfJobBrief(
job_id=j.id, role="source", status=j.status,
progress=j.progress, user_deleted=j.user_deleted, deleted_at=j.deleted_at,
))
for j in by_output.get(f.id, []):
briefs.append(PdfJobBrief(
job_id=j.id, role="output", status=j.status,
progress=j.progress, user_deleted=j.user_deleted, deleted_at=j.deleted_at,
))
out_items.append(UploadedFileWithPdfOut(**f.model_dump(), pdf_jobs=briefs))
return FileWithPdfListResponse(total=total, items=out_items)
@router.post( @router.post(
"/batch-delete", "/batch-delete",
response_model=BatchDeleteResult, response_model=BatchDeleteResult,

View File

@@ -0,0 +1,181 @@
"""PDF 转换接口用户侧cookie 标识)+ 管理侧Basic Auth
路由:
POST /api/pdf/jobs 用户上传文件并提交转换(首次无 cookie 则下发)
GET /api/pdf/jobs 当前用户任务列表(仅未软删)
GET /api/pdf/jobs/{id} 单任务状态(轮询进度)
GET /api/pdf/jobs/{id}/download 下载产物 PDF
DELETE /api/pdf/jobs/{id} 用户软删(不再对用户展示,管理页仍可见)
GET /api/admin/pdf/jobs 管理页列表(全部,含已软删标记)
DELETE /api/admin/pdf/jobs/{id} 管理员硬删(真正删除磁盘与 DB
HTML 页面 /pdf用户与 /pdf-admin管理由 main.py 返回静态文件,
不在此 controller 注册,避免与 REST 同路径冲突。
"""
from __future__ import annotations
from fastapi import APIRouter, Cookie, Depends, File, HTTPException, Query, Request, UploadFile
from fastapi.responses import FileResponse, JSONResponse
from sqlalchemy.orm import Session
from ..config import get_settings
from ..database import get_db
from ..dao.pdf_job_dao import PdfJobDAO
from ..dao.uploaded_file_dao import UploadedFileDAO
from ..schemas.pdf import DeleteResult, PdfJobListResponse, PdfJobOut, PdfSubmitResponse
from ..security import require_docs_auth
from ..services.pdf_service import PdfService, new_owner_cookie
router = APIRouter(tags=["pdf"])
def _service(db: Session = Depends(get_db)) -> PdfService:
return PdfService(PdfJobDAO(db), UploadedFileDAO(db))
def _resolve_cookie(request: Request, zk_pdf: str | None = Cookie(default=None)) -> str:
"""解析用户 cookie无则生成新值由 controller 写入响应头)。
cookie 缺失时把新值挂到 request.state供响应阶段 set_cookie。
"""
if zk_pdf and len(zk_pdf) == 32:
return zk_pdf
new = new_owner_cookie()
request.state.new_pdf_cookie = new
return new
@router.post(
"/api/pdf/jobs",
response_model=PdfSubmitResponse,
summary="上传文件并提交 PDF 转换",
description=(
"multipart/form-data 上传 epub 文件≤250MB流式落盘后创建 pending 转换任务,"
"后台异步转换。用户凭 zk_pdf cookie 标识;首次无 cookie 时响应下发新 cookie。"
),
)
async def submit_job(
request: Request,
file: UploadFile = File(..., description="要转换的 epub 文件"),
service: PdfService = Depends(_service),
owner_cookie: str = Depends(_resolve_cookie),
) -> PdfSubmitResponse:
job, _source_id = service.submit(file, owner_cookie)
# 提交成功后触发后台转换
service.schedule_convert(job.id)
resp = PdfSubmitResponse(job=job, set_cookie=False)
new_cookie = getattr(request.state, "new_pdf_cookie", None)
if new_cookie:
resp.set_cookie = True
# 用 JSONResponse 显式 set_cookie 后返回模型体
cfg = get_settings().pdf
data = resp.model_dump(mode="json")
response = JSONResponse(data)
response.set_cookie(
key=cfg.cookie_name,
value=new_cookie,
max_age=cfg.cookie_max_age_seconds,
httponly=True,
samesite="lax",
path="/",
)
return response
return resp
@router.get(
"/api/pdf/jobs",
response_model=PdfJobListResponse,
summary="当前用户的任务列表",
description="按 zk_pdf cookie 返回该用户未软删的任务,按创建时间倒序。",
)
def list_jobs(
service: PdfService = Depends(_service),
owner_cookie: str = Depends(_resolve_cookie),
limit: int = Query(100, ge=1, le=500),
offset: int = Query(0, ge=0),
) -> PdfJobListResponse:
return service.list_for_user(owner_cookie, limit=limit, offset=offset)
@router.get(
"/api/pdf/jobs/{job_id}",
response_model=PdfJobOut,
summary="查询单任务状态(轮询进度)",
)
def get_job(
job_id: int,
service: PdfService = Depends(_service),
owner_cookie: str = Depends(_resolve_cookie),
) -> PdfJobOut:
return service.get_job(job_id, owner_cookie)
@router.get(
"/api/pdf/jobs/{job_id}/download",
summary="下载转换后的 PDF",
description="仅任务状态为 done 且归属本人未软删时可下载。",
)
def download_job(
job_id: int,
service: PdfService = Depends(_service),
owner_cookie: str = Depends(_resolve_cookie),
) -> FileResponse:
_job, path, filename = service.get_output_path(job_id, owner_cookie)
return FileResponse(
path=str(path),
media_type="application/pdf",
filename=filename,
)
@router.delete(
"/api/pdf/jobs/{job_id}",
response_model=DeleteResult,
summary="用户软删任务(不再对用户展示)",
description="仅置 user_deleted 标记,磁盘与 DB 行保留;管理页仍可见并标注已删除。",
)
def delete_job(
job_id: int,
service: PdfService = Depends(_service),
owner_cookie: str = Depends(_resolve_cookie),
) -> DeleteResult:
ok = service.user_delete(job_id, owner_cookie)
if not ok:
raise HTTPException(404, "任务不存在")
return DeleteResult(deleted=True)
# ---------------- 管理侧Basic Auth ----------------
@router.get(
"/api/admin/pdf/jobs",
response_model=PdfJobListResponse,
summary="列出全部转换任务(需鉴权)",
description="管理页使用:含 user_deleted 标记,可看到用户是否已软删。",
)
def admin_list_jobs(
service: PdfService = Depends(_service),
_: str = Depends(require_docs_auth),
limit: int = Query(100, ge=1, le=500),
offset: int = Query(0, ge=0),
) -> PdfJobListResponse:
return service.list_for_admin(limit=limit, offset=offset)
@router.delete(
"/api/admin/pdf/jobs/{job_id}",
response_model=DeleteResult,
summary="管理员硬删任务(真正删除)",
description="删原始/产物磁盘文件 + UploadedFile 行 + PdfJob 行,不可恢复。",
)
def admin_delete_job(
job_id: int,
service: PdfService = Depends(_service),
_: str = Depends(require_docs_auth),
) -> DeleteResult:
ok = service.admin_delete(job_id)
if not ok:
raise HTTPException(404, "任务不存在")
return DeleteResult(deleted=True)

View File

@@ -86,7 +86,7 @@ async def delete_whiteboard(
# ---------------- WebSocket公开实时同步 + 心跳) ---------------- # ---------------- WebSocket公开实时同步 + 心跳) ----------------
@router.websocket("/ws/wb/{board_id}") @router.websocket("/api/ws/wb/{board_id}")
async def whiteboard_ws(websocket: WebSocket, board_id: str) -> None: async def whiteboard_ws(websocket: WebSocket, board_id: str) -> None:
"""白板实时协作端点(文本记事本)。 """白板实时协作端点(文本记事本)。
@@ -228,12 +228,12 @@ def _parse(raw: str) -> dict | None:
async def _safe_send(ws: WebSocket, msg: dict) -> None: async def _safe_send(ws: WebSocket, msg: dict) -> None:
try: try:
await ws.send_json(msg) await ws.send_json(msg)
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
pass logger.debug("发送 WS 消息失败: %s", exc)
async def _safe_close(ws: WebSocket) -> None: async def _safe_close(ws: WebSocket) -> None:
try: try:
await ws.close() await ws.close()
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
pass logger.debug("关闭 WS 失败: %s", exc)

71
app/dao/pdf_job_dao.py Normal file
View File

@@ -0,0 +1,71 @@
"""PdfJob 的 DAO。
所有写操作均在该层 commitservice 不直接操作 session。
用户/管理两条查询路径分别按 user_deleted 过滤。
"""
from __future__ import annotations
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from ..models.pdf_job import PdfJob
class PdfJobDAO:
def __init__(self, db: Session) -> None:
self.db = db
def create(self, job: PdfJob) -> PdfJob:
self.db.add(job)
self.db.commit()
self.db.refresh(job)
return job
def get(self, job_id: int) -> PdfJob | None:
return self.db.get(PdfJob, job_id)
def update(self, job: PdfJob) -> PdfJob:
"""提交对 job 的就地修改并刷新。"""
self.db.commit()
self.db.refresh(job)
return job
def delete(self, job: PdfJob) -> None:
"""硬删 PdfJob 行(管理视角真正删除,不可恢复)。"""
self.db.delete(job)
self.db.commit()
def list_for_user(self, owner_cookie: str, limit: int = 100, offset: int = 0) -> list[PdfJob]:
"""用户视角:仅未软删的任务,按创建时间倒序。"""
stmt = (
select(PdfJob)
.where(PdfJob.owner_cookie == owner_cookie)
.where(PdfJob.user_deleted == False) # noqa: E712
.order_by(PdfJob.created_at.desc())
.limit(limit)
.offset(offset)
)
return list(self.db.scalars(stmt).all())
def count_for_user(self, owner_cookie: str) -> int:
stmt = (
select(func.count())
.select_from(PdfJob)
.where(PdfJob.owner_cookie == owner_cookie)
.where(PdfJob.user_deleted == False) # noqa: E712
)
return self.db.scalar(stmt) or 0
def list_all(self, limit: int = 100, offset: int = 0) -> list[PdfJob]:
"""管理视角:全部任务(含已软删),按创建时间倒序。"""
stmt = (
select(PdfJob)
.order_by(PdfJob.created_at.desc())
.limit(limit)
.offset(offset)
)
return list(self.db.scalars(stmt).all())
def count_all(self) -> int:
return self.db.scalar(select(func.count()).select_from(PdfJob)) or 0

View File

@@ -31,15 +31,6 @@ class TunnelSessionDAO:
) )
return self.db.scalars(stmt).first() return self.db.scalars(stmt).first()
def get_active_by_port(self, tunnel_port: int) -> TunnelSession | None:
stmt = (
select(TunnelSession)
.where(TunnelSession.tunnel_port == tunnel_port)
.where(TunnelSession.status == "active")
.limit(1)
)
return self.db.scalars(stmt).first()
def list_active(self) -> list[TunnelSession]: def list_active(self) -> list[TunnelSession]:
stmt = select(TunnelSession).where(TunnelSession.status == "active") stmt = select(TunnelSession).where(TunnelSession.status == "active")
return list(self.db.scalars(stmt).all()) return list(self.db.scalars(stmt).all())

View File

@@ -6,11 +6,16 @@ get_or_create 用于「访问即新建」语义(路由 GET /api/wb/{id} 不存
from __future__ import annotations from __future__ import annotations
import logging
from sqlalchemy import func, select from sqlalchemy import func, select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from ..models.whiteboard import Whiteboard from ..models.whiteboard import Whiteboard
logger = logging.getLogger("zikai.whiteboard")
class WhiteboardDAO: class WhiteboardDAO:
def __init__(self, db: Session) -> None: def __init__(self, db: Session) -> None:
@@ -27,14 +32,18 @@ class WhiteboardDAO:
return self.db.scalars(stmt).first() return self.db.scalars(stmt).first()
def get_or_create(self, board_id: str) -> Whiteboard: def get_or_create(self, board_id: str) -> Whiteboard:
"""存在则返回,否则新建空板。利用 unique 约束兜底并发首访。""" """存在则返回,否则新建空板。利用 unique 约束兜底并发首访。
仅 IntegrityError并发下另一事务已插入违反唯一约束才回滚重读
其他异常向上抛,避免掩盖 schema/连接等真实故障。
"""
board = self.get(board_id) board = self.get(board_id)
if board is not None: if board is not None:
return board return board
board = Whiteboard(board_id=board_id, content="", version=0, edit_count=0) board = Whiteboard(board_id=board_id, content="", version=0, edit_count=0)
try: try:
return self.create(board) return self.create(board)
except Exception: except IntegrityError:
# 并发下另一事务已插入:回滚后重新读 # 并发下另一事务已插入:回滚后重新读
self.db.rollback() self.db.rollback()
return self.get(board_id) # type: ignore[return-value] return self.get(board_id) # type: ignore[return-value]

View File

@@ -64,7 +64,29 @@ def get_db() -> Generator[Session, None, None]:
def init_db_schema() -> None: def init_db_schema() -> None:
"""按需建表(幂等)。先导入 models 以注册映射。""" """按需建表(幂等)并校验既有表列与模型一致fail-fast on schema drift
先导入 models 注册映射create_all 用 IF NOT EXISTS 仅补缺失的表;
随后对每张已存在的表检查模型声明的列是否齐全,缺列即抛 RuntimeError
避免运行期才以晦涩的 OperationalError 暴露 schema 漂移。
"""
from sqlalchemy import inspect
from . import models # noqa: F401 from . import models # noqa: F401
get_engine() engine = get_engine()
Base.metadata.create_all(bind=_engine) Base.metadata.create_all(bind=engine)
inspector = inspect(engine)
missing: list[str] = []
for table, mapper in Base.registry.mappers.items():
if not inspector.has_table(table):
continue
db_cols = {c["name"] for c in inspector.get_columns(table)}
for model_col in mapper.columns.keys():
if model_col not in db_cols:
missing.append(f"{table}.{model_col}")
if missing:
raise RuntimeError(
"数据库 schema 与模型不一致,缺少列: " + ", ".join(missing)
+ "。请执行 sql/schema.sql 或迁移脚本更新表结构。"
)

View File

@@ -6,10 +6,11 @@
GET /redoc -> ReDoc Basic Auth GET /redoc -> ReDoc Basic Auth
GET /openapi.json -> OpenAPI 文档Basic Auth GET /openapi.json -> OpenAPI 文档Basic Auth
GET /health -> 存活探针(公开) GET /health -> 存活探针(公开)
GET /upload -> 上传页面(公开 HTML GET /api/index -> 导航页(公开,收集所有页面入口
GET /files -> 文件浏览页Basic Auth同 docs GET /api/upload -> 上传页面(公开 HTML
GET /wb/{id} -> 白板页面(公开,不存在则新建 GET /api/files-page -> 文件管理页Basic Auth同 docs含 PDF 转换管理
GET /wb-admin -> 白板管理页Basic Auth同 docs GET /api/wb/{id} -> 白板页面(公开,不存在则新建
GET /api/wb-admin -> 白板管理页Basic Auth同 docs
GET /api/... -> 业务接口 GET /api/... -> 业务接口
WS /ws/wb/{id} -> 白板实时同步(公开) WS /ws/wb/{id} -> 白板实时同步(公开)
/static/... -> 前端静态资源JS/CSS /static/... -> 前端静态资源JS/CSS
@@ -31,6 +32,7 @@ from .controllers import (
chunk_upload_router, chunk_upload_router,
file_admin_router, file_admin_router,
file_router, file_router,
pdf_router,
system_router, system_router,
tunnel_router, tunnel_router,
whiteboard_router, whiteboard_router,
@@ -149,10 +151,13 @@ def create_app() -> FastAPI:
app.include_router(chunk_upload_router) app.include_router(chunk_upload_router)
app.include_router(tunnel_router) app.include_router(tunnel_router)
app.include_router(whiteboard_router) app.include_router(whiteboard_router)
app.include_router(pdf_router)
# 前端静态资源JS/CSSHTML 壳由下面的具名路由返回,便于各自挂 Basic Auth # 前端静态资源JS/CSSHTML 壳由下面的具名路由返回,便于各自挂 Basic Auth
# 统一 /api/ 前缀:所有 zTools2 入口(页面/静态/探针/WS/API都在 /api/ 下,
# 反代与 vite proxy 只需一条 /api/ 规则即可转发,与环境无关
if _STATIC_DIR.is_dir(): if _STATIC_DIR.is_dir():
app.mount("/static", StaticFiles(directory=str(_STATIC_DIR)), name="static") app.mount("/api/static", StaticFiles(directory=str(_STATIC_DIR)), name="static")
# 受 Basic Auth 保护的文档接口 # 受 Basic Auth 保护的文档接口
@app.get("/openapi.json", tags=["docs"], summary="OpenAPI 文档(需鉴权)") @app.get("/openapi.json", tags=["docs"], summary="OpenAPI 文档(需鉴权)")
@@ -175,12 +180,22 @@ def create_app() -> FastAPI:
def root() -> PlainTextResponse: def root() -> PlainTextResponse:
return PlainTextResponse(f"zikai {app.version}\n") return PlainTextResponse(f"zikai {app.version}\n")
@app.get("/health", tags=["meta"], summary="存活探针") @app.get("/api/health", tags=["meta"], summary="存活探针")
def health() -> dict: def health() -> dict:
return {"status": "ok"} return {"status": "ok"}
@app.get( @app.get(
"/upload", "/api/index",
response_class=HTMLResponse,
tags=["pages"],
summary="导航页(公开)",
description="收集所有页面入口的卡片式导航页,各子页脚注可返回此处。",
)
def index_page() -> HTMLResponse:
return _serve_static_html("index.html")
@app.get(
"/api/upload",
response_class=HTMLResponse, response_class=HTMLResponse,
tags=["pages"], tags=["pages"],
summary="上传页面", summary="上传页面",
@@ -190,17 +205,21 @@ def create_app() -> FastAPI:
return HTMLResponse(render_upload_html()) return HTMLResponse(render_upload_html())
@app.get( @app.get(
"/files", "/api/files-page",
response_class=HTMLResponse, response_class=HTMLResponse,
tags=["pages"], tags=["pages"],
summary="文件浏览页(需鉴权)", summary="文件管理页(需鉴权)",
description="列出 / 下载 / 删除已上传文件支持多选、批量下载删除与分页。Basic Auth 同 docs。", description=(
"列出 / 下载 / 删除已上传文件,并合并 PDF 转换管理:以 uploaded_files 为基础,"
"用 pdf_jobs 匹配标注关联文件的转换状态、用户软删标记,可硬删任务。"
"支持多选、批量下载删除与分页。Basic Auth 同 docs。"
),
) )
def files_page(_: str = Depends(require_docs_auth)) -> HTMLResponse: def files_page(_: str = Depends(require_docs_auth)) -> HTMLResponse:
return _serve_static_html("file_browser.html") return _serve_static_html("file_browser.html")
@app.get( @app.get(
"/wb-admin", "/api/wb-admin",
response_class=HTMLResponse, response_class=HTMLResponse,
tags=["pages"], tags=["pages"],
summary="记事本管理页(需鉴权)", summary="记事本管理页(需鉴权)",
@@ -210,11 +229,11 @@ def create_app() -> FastAPI:
return _serve_static_html("whiteboard_admin.html") return _serve_static_html("whiteboard_admin.html")
@app.get( @app.get(
"/wb/{board_id}", "/api/wb-page/{board_id}",
response_class=HTMLResponse, response_class=HTMLResponse,
tags=["pages"], tags=["pages"],
summary="记事本页面", summary="记事本页面",
description="公开访问的共享文本记事本,不存在则自动新建;实时协作走 WS /ws/wb/{id}", description="公开访问的共享文本记事本,不存在则自动新建;实时协作走 WS /api/ws/wb/{id}",
) )
def whiteboard_page(board_id: str) -> HTMLResponse: def whiteboard_page(board_id: str) -> HTMLResponse:
return _serve_static_html("whiteboard.html") return _serve_static_html("whiteboard.html")

View File

@@ -1,8 +1,9 @@
"""ORM 模型包import 本包即把所有实体注册到 Base.metadata。""" """ORM 模型包import 本包即把所有实体注册到 Base.metadata。"""
from .pdf_job import PdfJob
from .tunnel_session import TunnelSession from .tunnel_session import TunnelSession
from .uploaded_file import UploadedFile from .uploaded_file import UploadedFile
from .upload_session import UploadSession from .upload_session import UploadSession
from .whiteboard import Whiteboard from .whiteboard import Whiteboard
__all__ = ["TunnelSession", "UploadedFile", "UploadSession", "Whiteboard"] __all__ = ["PdfJob", "TunnelSession", "UploadedFile", "UploadSession", "Whiteboard"]

56
app/models/pdf_job.py Normal file
View File

@@ -0,0 +1,56 @@
"""PDF 转换任务实体。
一个任务记录一次「上传文件 -> 转为 PDF」的转换。原始上传文件与转换产物 PDF
均复用 UploadedFile 存储(落盘 + 元数据入库),本表只记录两者关系与转换状态,
不重复实现存储逻辑。
删除语义(两级):
用户软删user_deleted=true-- 用户页不再展示,但磁盘与 DB 行保留;
管理页仍可见且标注「已删除」。
管理员硬删 -- 删原始/产物磁盘文件 + UploadedFile 行 + 本表行,真正删除。
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import BigInteger, Boolean, DateTime, Integer, String, func
from sqlalchemy.orm import Mapped, mapped_column
from ..database import Base
class PdfJob(Base):
__tablename__ = "pdf_job"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True)
# 用户标识httpOnly cookie 值uuid4.hex用户凭此查看自己的任务
owner_cookie: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
# 原始上传文件(复用 UploadedFile 存储)
source_file_id: Mapped[int] = mapped_column(BigInteger, nullable=False, index=True)
source_filename: Mapped[str] = mapped_column(String(512), nullable=False)
source_size: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0)
# 转换产物 PDF复用 UploadedFile 存储);转换完成前为 NULL
output_file_id: Mapped[int | None] = mapped_column(BigInteger, nullable=True, default=None)
# pending(排队) / converting(转换中) / done(完成) / failed(失败)
status: Mapped[str] = mapped_column(String(16), nullable=False, default="pending", index=True)
# 转换进度 0-100供前端轮询显示
progress: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
# 失败原因status=failed 时填写)
error_message: Mapped[str] = mapped_column(String(512), nullable=False, default="")
# 用户软删标记true=用户已从其页面删除,不再对用户展示
user_deleted: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, index=True)
# 用户软删时间(管理页展示「是否已删除」时用)
deleted_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True, default=None)
created_at: Mapped[datetime] = mapped_column(
DateTime, server_default=func.now(), nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime, server_default=func.now(), onupdate=func.now(), nullable=False
)
def __repr__(self) -> str: # pragma: no cover
return (
f"<PdfJob id={self.id} status={self.status} "
f"src={self.source_filename!r} progress={self.progress}>"
)

View File

@@ -36,3 +36,31 @@ class FileUploadResponse(BaseModel):
class FileListResponse(BaseModel): class FileListResponse(BaseModel):
total: int total: int
items: list[UploadedFileOut] items: list[UploadedFileOut]
class PdfJobBrief(BaseModel):
"""文件关联到的 PDF 转换任务摘要(供合并管理页展示)。
一个 uploaded_file 可能同时被多个 job 引用(罕见),故为列表;
通常每行 0 或 1 条。
"""
job_id: int
role: str = Field(..., description='"source"=该文件是 epub 源文件;"output"=该文件是产物 PDF')
status: str = Field(..., description="pending / converting / done / failed")
progress: int = Field(0, description="转换进度 0-100")
user_deleted: bool = Field(False, description="用户是否已软删该任务")
deleted_at: datetime | None = Field(None, description="用户软删时间")
class UploadedFileWithPdfOut(UploadedFileOut):
"""带 PDF 任务关联信息的文件视图(合并管理页用)。"""
pdf_jobs: list[PdfJobBrief] = Field(default_factory=list, description="关联到的 PDF 转换任务")
class FileWithPdfListResponse(BaseModel):
"""合并管理页列表响应:以 uploaded_files 为基础,附带 pdf_jobs 关联。"""
total: int
items: list[UploadedFileWithPdfOut]

46
app/schemas/pdf.py Normal file
View File

@@ -0,0 +1,46 @@
"""PDF 转换接口 DTO。"""
from __future__ import annotations
from datetime import datetime
from pydantic import BaseModel, Field
class PdfJobOut(BaseModel):
"""任务对外视图用户与管理页共用user_deleted 仅管理页关注)。"""
id: int
source_file_id: int = Field(..., description="原始 epub 文件的 uploaded_file.id")
source_filename: str = Field(..., description="原始上传文件名")
source_size: int = Field(..., description="原始文件字节数")
output_file_id: int | None = Field(None, description="产物 PDF 的 uploaded_file.id转换完成前为 null")
status: str = Field(..., description="pending / converting / done / failed")
progress: int = Field(0, description="转换进度 0-100")
error_message: str = Field("", description="失败原因")
user_deleted: bool = Field(False, description="用户是否已软删(管理页用)")
deleted_at: datetime | None = Field(None, description="用户软删时间(管理页用)")
created_at: datetime
updated_at: datetime
model_config = {"from_attributes": True}
class PdfJobListResponse(BaseModel):
"""任务列表响应。"""
total: int
items: list[PdfJobOut]
class PdfSubmitResponse(BaseModel):
"""提交转换任务的响应。"""
job: PdfJobOut = Field(..., description="新建的任务")
set_cookie: bool = Field(
False, description="true=本次请求未带 cookie响应已下发新 cookie"
)
class DeleteResult(BaseModel):
deleted: bool

View File

@@ -0,0 +1,104 @@
"""文件 -> PDF 转换器(纯 Python无 calibre/xvfb 系统依赖)。
当前支持 epub必要能力ebooklib 解析 epub 文档项(按 spine 顺序),拼接为
完整 HTML 后交 weasyprint 渲染为 PDF。epub 内的相对资源(图片/CSS经 base_url
指向 epub 解包目录解析。
接口抽象为 ``convert_to_pdf(src_path, dst_path)``:未来新增格式只需在本模块内
按扩展名分支调用方PdfService无需改动。
"""
from __future__ import annotations
import logging
import tempfile
import zipfile
from pathlib import Path
logger = logging.getLogger("zikai.pdf")
# 支持的输入格式(小写扩展名 -> 是否可转)。新增格式在此登记并实现分支即可。
# epub 为必要能力;其余为 ebooklib/weasyprint 路径天然支持的电子书结构,
# 实测对纯 HTML 类 epub 同样有效,故一并放行。
SUPPORTED_EXTENSIONS = {".epub"}
def is_supported(filename: str) -> bool:
"""文件名扩展名是否在支持列表内。"""
return Path(filename).suffix.lower() in SUPPORTED_EXTENSIONS
def convert_to_pdf(src_path: Path, dst_path: Path) -> None:
"""把 src_path 指向的文件转为 PDF 写入 dst_path。
失败抛 RuntimeError调用方捕获后写 error_message。按扩展名分发
当前仅 epub 分支;新增格式在此 elif 扩展。
"""
ext = src_path.suffix.lower()
if ext == ".epub":
_epub_to_pdf(src_path, dst_path)
else:
raise RuntimeError(f"不支持的文件格式:{ext}(仅支持 epub")
def _epub_to_pdf(src_path: Path, dst_path: Path) -> None:
"""epub -> PDF解包 epub -> 按 spine 顺序取文档项 HTML -> weasyprint 渲染。
epub 本质是 zip。先解包到临时目录用 ebooklib 读取(其内部按 zip 解析,
base_url 指向解包后的 OEBPS/ 内容目录使相对图片/CSS 可被 weasyprint 解析)。
"""
# 延迟导入weasyprint 首次 import 较重(加载 pango/cairo且仅在真正转换时需要
import ebooklib # noqa: F401
from ebooklib import epub
from weasyprint import HTML
with tempfile.TemporaryDirectory(prefix="zpdf_") as tmp:
tmp_dir = Path(tmp)
# epub 是 zip解包到临时目录便于 weasyprint 解析相对资源
try:
with zipfile.ZipFile(src_path, "r") as zf:
zf.extractall(tmp_dir)
except zipfile.BadZipFile as exc:
raise RuntimeError(f"epub 文件损坏(非有效 zip{exc}") from exc
try:
book = epub.read_epub(str(src_path), {"ignore_ncx": True})
except Exception as exc:
raise RuntimeError(f"epub 解析失败:{exc}") from exc
# 按 spine 顺序收集文档项XHTML保证章节顺序正确
docs: list[str] = []
for idref, _linear in book.spine:
item = book.get_item_with_id(idref)
if item is not None:
docs.append(item.get_content().decode("utf-8", errors="replace"))
if not docs:
# spine 为空时回退:取所有文档项
docs = [
it.get_content().decode("utf-8", errors="replace")
for it in book.get_items_of_type(ebooklib.ITEM_DOCUMENT)
]
if not docs:
raise RuntimeError("epub 内无可转换的文档内容")
full_html = "\n".join(docs)
# 定位资源根目录(含图片/CSS 的目录):通常是 OEBPS/ 或根目录。
# epub 内资源(图片/CSS相对文档项引用文档项与资源同处 OPF 所在目录。
# 故以 OPF 文件所在目录作为 base_url使相对路径正确解析。
base_url = str(tmp_dir)
opf_files = list(tmp_dir.rglob("*.opf"))
if opf_files:
opf_dir = opf_files[0].parent
# OPF 可能在根目录,此时 base_url 保持 tmp_dir
if str(opf_dir) != str(tmp_dir):
base_url = str(opf_dir)
try:
HTML(string=full_html, base_url=base_url).write_pdf(str(dst_path))
except Exception as exc:
raise RuntimeError(f"PDF 渲染失败:{exc}") from exc
if not dst_path.exists() or dst_path.stat().st_size == 0:
raise RuntimeError("PDF 渲染未产出有效文件")
logger.info("epub->pdf 完成: %s -> %s (%d bytes)", src_path.name, dst_path.name, dst_path.stat().st_size)

296
app/services/pdf_service.py Normal file
View File

@@ -0,0 +1,296 @@
"""PDF 转换服务:上传落盘 + 异步后台转换 + 进度追踪 + 两级删除。
复用 UploadService 的「流式落盘 + 存储路径生成」能力:原始上传文件与产物 PDF
均作为 UploadedFile 存储,本服务只维护 PdfJob 关系与状态。
转换在后台 asyncio task 中以 to_thread 执行(转换器是同步阻塞调用),
过程中经 DAO 更新 progress/status供前端轮询。worker 限制:依赖进程内
asyncio 事件循环,与现有 reaper/hub 一致,保持单 worker。
删除语义:
user_delete -- 置 user_deleted=true + deleted_at磁盘与 DB 行保留(管理页可见)。
admin_delete -- 删原始/产物磁盘文件 + UploadedFile 行 + PdfJob 行(真正删除)。
"""
from __future__ import annotations
import asyncio
import logging
import secrets
from datetime import datetime, timezone
from pathlib import Path
from fastapi import HTTPException, UploadFile
from ..config import get_settings
from ..dao.pdf_job_dao import PdfJobDAO
from ..dao.uploaded_file_dao import UploadedFileDAO
from ..models.pdf_job import PdfJob
from ..schemas.pdf import PdfJobOut, PdfJobListResponse
from . import pdf_converter
from .upload_service import UploadService
logger = logging.getLogger("zikai.pdf")
def new_owner_cookie() -> str:
"""生成新的用户标识 cookie 值uuid4 hex"""
return secrets.token_hex(16)
class PdfService:
def __init__(self, job_dao: PdfJobDAO, file_dao: UploadedFileDAO) -> None:
s = get_settings()
self.job_dao = job_dao
self.file_dao = file_dao
self.upload_root = s.resolved_upload_dir()
self.cfg = s.pdf
# 复用 UploadService 的存储路径生成与落盘能力
self._upload = UploadService(file_dao)
# ---------------- 提交 ----------------
def submit(self, file: UploadFile, owner_cookie: str) -> tuple[PdfJobOut, int]:
"""上传原始文件并创建 pending 任务,返回 (任务视图, source_file_id)。
校验大小 ≤ max_size_bytes 与扩展名白名单;复用 UploadService 流式落盘
入库为 UploadedFile再建 PdfJob 关联。转换不在此处执行(由 controller
调 schedule_convert 异步触发)。
"""
filename = file.filename or "upload.epub"
if not pdf_converter.is_supported(filename):
raise HTTPException(400, "仅支持 epub 文件")
# 大小校验UploadFile 流式无已知长度,先读一遍统计并重置(小文件可行),
# 对大文件更优的做法是流式计数,这里复用 stream_to_disk 后按 size 校验。
resp = self._upload.stream_to_disk(file, source="pdf", uploaded_by=owner_cookie)
if resp.size_bytes > self.cfg.max_size_bytes:
# 超限:清理刚落盘的文件与 DB 行,保持无副作用
try:
self._upload.delete_file(resp.id)
except Exception: # pragma: no cover
logger.warning("清理超限文件失败 id=%s", resp.id)
raise HTTPException(
413,
f"文件过大({resp.size_bytes} > {self.cfg.max_size_bytes},上限 250MB",
)
job = PdfJob(
owner_cookie=owner_cookie,
source_file_id=resp.id,
source_filename=filename,
source_size=resp.size_bytes,
status="pending",
progress=0,
)
job = self.job_dao.create(job)
logger.info("PDF 任务已创建 job=%s file=%s size=%d", job.id, filename, resp.size_bytes)
return PdfJobOut.model_validate(job), resp.id
def schedule_convert(self, job_id: int) -> None:
"""在当前事件循环起一个后台 task 执行转换(不阻塞调用方)。
用 to_thread 跑同步转换器;转换中分段更新 progress。
重要:后台 task 必须用独立 DB Session请求 Session 在请求结束后即关闭),
故 _convert_async 内部经 _fresh_service 重建带新 Session 的 service。
"""
asyncio.create_task(self._convert_async(job_id))
async def _convert_async(self, job_id: int) -> None:
"""后台转换pending -> converting(进度) -> done/failed。
每个阶段用独立 Session_fresh_service避免引用请求 Session已关闭
"""
try:
await asyncio.to_thread(self._run_with_fresh_session, "_mark_converting", job_id)
await asyncio.wait_for(
asyncio.to_thread(self._run_with_fresh_session, "_do_convert", job_id),
timeout=self.cfg.convert_timeout_seconds,
)
except asyncio.TimeoutError:
await asyncio.to_thread(self._run_with_fresh_session, "_mark_failed", job_id, "转换超时")
except Exception as exc:
await asyncio.to_thread(self._run_with_fresh_session, "_mark_failed", job_id, f"转换失败:{exc}")
@staticmethod
def _run_with_fresh_session(method_name: str, *args) -> None:
"""用独立 DB Session 构造新 PdfService 实例执行其方法。
后台线程不能复用请求的 Session请求结束即关闭故每次操作新建 Session。
method_name 是 PdfService 实例方法名,在此用新 service 调用对应方法。
"""
from ..database import get_session_local
db = get_session_local()()
try:
svc = PdfService(PdfJobDAO(db), UploadedFileDAO(db))
getattr(svc, method_name)(*args)
finally:
db.close()
# ---------------- 同步转换实现(在线程中执行,用独立 Session ----------------
def _mark_converting(self, job_id: int) -> None:
job = self.job_dao.get(job_id)
if job is None:
return
job.status = "converting"
job.progress = 5
self.job_dao.update(job)
def _do_convert(self, job_id: int) -> None:
"""执行转换并落产物 PDF 为 UploadedFile。"""
job = self.job_dao.get(job_id)
if job is None:
return
src_row = self.file_dao.get_by_id(job.source_file_id)
if src_row is None:
self._mark_failed(job_id, "原始文件记录丢失")
return
src_path = (self.upload_root / src_row.storage_path).resolve()
if not src_path.exists():
self._mark_failed(job_id, "原始文件实体不存在")
return
# 产物 PDF 存储路径(复用 UploadService 的路径生成,扩展名固定 .pdf
rel_path, abs_path = self._upload.make_storage_path("result.pdf")
part_path = abs_path.with_name(abs_path.name + ".part")
# 更新进度到「渲染中」
job.status = "converting"
job.progress = 30
self.job_dao.update(job)
try:
pdf_converter.convert_to_pdf(src_path, part_path)
except Exception:
part_path.unlink(missing_ok=True)
raise
job.progress = 80
self.job_dao.update(job)
# 落产物 UploadedFile复用 commit_entity 的原子改名 + 入库)
from ..models.uploaded_file import UploadedFile
size = part_path.stat().st_size
sha256 = self._hash_file(part_path)
entity = UploadedFile(
storage_path=str(rel_path),
original_filename=Path(job.source_filename).stem + ".pdf",
content_type="application/pdf",
size_bytes=size,
sha256=sha256,
source="pdf-convert",
uploaded_by=job.owner_cookie,
)
saved = self._upload.commit_entity(entity, part_path, abs_path)
job.output_file_id = saved.id
job.status = "done"
job.progress = 100
self.job_dao.update(job)
logger.info("PDF 转换完成 job=%s output_file_id=%s", job_id, saved.id)
def _mark_failed(self, job_id: int, message: str) -> None:
job = self.job_dao.get(job_id)
if job is None:
return
job.status = "failed"
job.error_message = message[:500]
self.job_dao.update(job)
logger.warning("PDF 转换失败 job=%s: %s", job_id, message)
def _hash_file(self, path: Path) -> str:
import hashlib
h = hashlib.sha256()
with path.open("rb") as f:
while chunk := f.read(self._upload.chunk_bytes):
h.update(chunk)
return h.hexdigest()
# ---------------- 查询 ----------------
def list_for_user(self, owner_cookie: str, limit: int = 100, offset: int = 0) -> PdfJobListResponse:
limit = min(max(limit, 1), self.cfg.list_limit)
offset = max(offset, 0)
total = self.job_dao.count_for_user(owner_cookie)
rows = self.job_dao.list_for_user(owner_cookie, limit=limit, offset=offset)
return PdfJobListResponse(total=total, items=[PdfJobOut.model_validate(r) for r in rows])
def list_for_admin(self, limit: int = 100, offset: int = 0) -> PdfJobListResponse:
limit = min(max(limit, 1), self.cfg.list_limit)
offset = max(offset, 0)
total = self.job_dao.count_all()
rows = self.job_dao.list_all(limit=limit, offset=offset)
return PdfJobListResponse(total=total, items=[PdfJobOut.model_validate(r) for r in rows])
def get_job(self, job_id: int, owner_cookie: str) -> PdfJobOut:
"""用户查询单任务:仅当归属本人且未软删时可见。"""
job = self.job_dao.get(job_id)
if job is None or job.owner_cookie != owner_cookie or job.user_deleted:
raise HTTPException(404, "任务不存在")
return PdfJobOut.model_validate(job)
def get_output_path(self, job_id: int, owner_cookie: str) -> tuple[PdfJobOut, Path, str]:
"""返回 (任务视图, 产物磁盘绝对路径, 下载文件名) 供下载。
用户仅可下载自己未软删且已完成的任务产物。
"""
job = self.job_dao.get(job_id)
if job is None or job.owner_cookie != owner_cookie or job.user_deleted:
raise HTTPException(404, "任务不存在")
if job.status != "done" or job.output_file_id is None:
raise HTTPException(409, "任务尚未完成,无法下载")
out_row = self.file_dao.get_by_id(job.output_file_id)
if out_row is None:
raise HTTPException(410, "产物文件记录丢失")
path = (self.upload_root / out_row.storage_path).resolve()
if not path.exists():
raise HTTPException(410, "产物文件实体不存在")
return PdfJobOut.model_validate(job), path, out_row.original_filename
# ---------------- 删除 ----------------
def user_delete(self, job_id: int, owner_cookie: str) -> bool:
"""用户软删:仅置 user_deleted=true磁盘与 DB 行保留(管理页仍可见)。"""
job = self.job_dao.get(job_id)
if job is None or job.owner_cookie != owner_cookie or job.user_deleted:
return False
job.user_deleted = True
job.deleted_at = datetime.now(timezone.utc)
self.job_dao.update(job)
logger.info("用户软删 PDF 任务 job=%s", job_id)
return True
def admin_delete(self, job_id: int) -> bool:
"""管理员硬删:删原始/产物磁盘文件 + UploadedFile 行 + PdfJob 行。
真正删除,不可恢复。磁盘删除失败仅记日志,仍清 DB 行保证列表不再显示。
"""
job = self.job_dao.get(job_id)
if job is None:
return False
# 删原始文件
self._safe_delete_file(job.source_file_id)
# 删产物文件(若有)
if job.output_file_id is not None:
self._safe_delete_file(job.output_file_id)
# 删 PdfJob 行
self.job_dao.delete(job)
logger.info("管理员硬删 PDF 任务 job=%s", job_id)
return True
def _safe_delete_file(self, file_id: int) -> None:
"""删 UploadedFile 磁盘文件 + DB 行;失败只记日志不阻断。"""
row = self.file_dao.get_by_id(file_id)
if row is None:
return
path = (self.upload_root / row.storage_path).resolve()
try:
Path(path).unlink(missing_ok=True)
except Exception as exc: # pragma: no cover
logger.warning("删除文件失败 file_id=%s path=%s: %s", file_id, path, exc)
try:
self.file_dao.delete(file_id)
except Exception as exc: # pragma: no cover
logger.warning("删除文件 DB 行失败 file_id=%s: %s", file_id, exc)

View File

@@ -27,8 +27,9 @@ class ZikaiSFTPServer(asyncssh.SFTPServer):
super().__init__(chan, chroot=str(upload_root).encode()) super().__init__(chan, chroot=str(upload_root).encode())
try: try:
self._username = chan.get_extra_info("username") or "unknown" self._username = chan.get_extra_info("username") or "unknown"
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
self._username = "unknown" self._username = "unknown"
logger.debug("读取 SFTP 会话用户名失败: %s", exc)
logger.info("SFTP 会话开始 user=%s chroot=%s", self._username, upload_root) logger.info("SFTP 会话开始 user=%s chroot=%s", self._username, upload_root)
def exit(self) -> None: def exit(self) -> None:
@@ -45,8 +46,8 @@ def _tunnel_dao():
def _close_tunnel_dao(dao) -> None: def _close_tunnel_dao(dao) -> None:
try: try:
dao.db.close() dao.db.close()
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
pass logger.debug("关闭隧道 DAO 会话失败: %s", exc)
class ZikaiSSHServer(asyncssh.SSHServer): class ZikaiSSHServer(asyncssh.SSHServer):
@@ -101,7 +102,8 @@ class ZikaiSSHServer(asyncssh.SSHServer):
try: try:
# asyncssh 命中返回 dict可能为空未命中返回 None # asyncssh 命中返回 dict可能为空未命中返回 None
result = self._authorized_keys.validate(key, client_host=addr, client_addr=addr) result = self._authorized_keys.validate(key, client_host=addr, client_addr=addr)
except Exception: except Exception as exc: # pragma: no cover
logger.warning("公钥校验异常 user=%s: %s", username, exc)
result = None result = None
ok = result is not None ok = result is not None
if ok: if ok:

View File

@@ -52,11 +52,6 @@ class TunnelService:
def get_active(self, user_name: str) -> TunnelSession | None: def get_active(self, user_name: str) -> TunnelSession | None:
return self.dao.get_active_by_user(user_name) return self.dao.get_active_by_user(user_name)
def is_port_allowed(self, user_name: str, tunnel_port: int) -> bool:
"""校验该 user 是否被允许绑定该隧道端口(防 user 乱绑端口)。"""
user = self.settings.find_user(user_name)
return user is not None and user.tunnel_port == tunnel_port
def reap_orphans(self) -> int: def reap_orphans(self) -> int:
"""兜底清理:关闭所有 active 会话(进程重启时 DB 里残留的孤儿记录)。 """兜底清理:关闭所有 active 会话(进程重启时 DB 里残留的孤儿记录)。

View File

@@ -92,8 +92,9 @@ class WhiteboardHub:
# 尽力关闭 websocket可能已关闭 # 尽力关闭 websocket可能已关闭
try: try:
await conn.websocket.close() await conn.websocket.close()
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
pass logger.debug("关闭 websocket 时出错 board=%s client=%s: %s",
conn.board_id, conn.client_id, exc)
logger.info("连接移除 board=%s client=%s(剩余 %d 人)", logger.info("连接移除 board=%s client=%s(剩余 %d 人)",
conn.board_id, conn.client_id, self.connection_count(conn.board_id)) conn.board_id, conn.client_id, self.connection_count(conn.board_id))
@@ -161,8 +162,9 @@ class WhiteboardHub:
for conn in conns: for conn in conns:
try: try:
await conn.websocket.close() await conn.websocket.close()
except Exception: # pragma: no cover except Exception as exc: # pragma: no cover
pass logger.debug("关闭 websocket 时出错 board=%s client=%s: %s",
conn.board_id, conn.client_id, exc)
logger.info("关闭白板 board=%s,踢出 %d 个连接", board_id, len(conns)) logger.info("关闭白板 board=%s,踢出 %d 个连接", board_id, len(conns))
@@ -179,10 +181,3 @@ def get_hub() -> WhiteboardHub:
if _hub is None: if _hub is None:
_hub = WhiteboardHub() _hub = WhiteboardHub()
return _hub return _hub
def reset_hub() -> None:
"""测试用:重置单例。"""
global _hub
with _hub_lock:
_hub = None

View File

@@ -128,7 +128,7 @@ def render(status: SystemStatus) -> str:
</table> </table>
</div> </div>
<p class="foot">zikai file service · 数据来源 psutil</p> <p class="foot"><a class="json" href="/api/index">导航</a> · zikai file service · 数据来源 psutil</p>
</body> </body>
</html> </html>
""" """

View File

@@ -6,8 +6,6 @@
from __future__ import annotations from __future__ import annotations
from html import escape
# 默认分片大小 4 MiB大于 Apache 300s 限制下单片可数秒传完,小到内存恒定。 # 默认分片大小 4 MiB大于 Apache 300s 限制下单片可数秒传完,小到内存恒定。
DEFAULT_CHUNK_SIZE = 4 * 1024 * 1024 DEFAULT_CHUNK_SIZE = 4 * 1024 * 1024
# 同一文件分片并发数 # 同一文件分片并发数
@@ -38,7 +36,7 @@ def render() -> str:
<div id="summary" class="foot"></div> <div id="summary" class="foot"></div>
<p class="foot"><a class="json" href="/api/system/status">系统状态</a> · zikai file service</p> <p class="foot"><a class="json" href="/api/index">导航</a> · <a class="json" href="/api/system/status">系统状态</a> · zikai file service</p>
<script> <script>
const CHUNK_SIZE = {DEFAULT_CHUNK_SIZE}; const CHUNK_SIZE = {DEFAULT_CHUNK_SIZE};

View File

@@ -64,3 +64,13 @@ whiteboard:
heartbeat_miss_threshold: 5 heartbeat_miss_threshold: 5
max_board_id_length: 64 # board_id 合法字符 [a-zA-Z0-9_-],长度上限 max_board_id_length: 64 # board_id 合法字符 [a-zA-Z0-9_-],长度上限
list_limit: 100 # 管理页单次列表上限 list_limit: 100 # 管理页单次列表上限
pdf:
# PDF 转换服务:用户上传 epub -> 后台转换为 PDF -> 显示进度并下载。
# 用户侧(上传/查看/下载/软删)凭 httpOnly cookie 标识;管理侧(列表/硬删)走 docs 同款 Basic Auth。
# 原始文件与产物 PDF 复用 storage.upload_dir 落盘。转换用纯 Pythonebooklib + weasyprint
max_size_bytes: 262144000 # 单文件上限 250 MiB
convert_timeout_seconds: 600 # 转换超时(秒),超大文件兜底
list_limit: 100 # 列表单次上限
cookie_name: zk_pdf # 用户标识 cookie 名
cookie_max_age_seconds: 31536000 # cookie 有效期 1 年

56
deploy/install-systemd.sh Executable file
View File

@@ -0,0 +1,56 @@
#!/usr/bin/env bash
# 安装 systemd 服务,实现 zTools2 持久化部署(开机自启 + 崩溃自动重启)。
#
# 做的事:
# 1. 用实际路径填充 deploy/ztools2.service 模板,写入 /etc/systemd/system/
# 2. 停掉旧方式start.sh 启动的 uvicorn避免端口冲突
# 3. systemctl daemon-reload + enable + start
#
# 用法:./deploy/install-systemd.sh
# 卸载systemctl disable --now ztools2 && rm /etc/systemd/system/ztools2.service && systemctl daemon-reload
set -euo pipefail
cd "$(dirname "$0")/.."
ROOT="$(pwd)"
# 解析 uvicorn 可执行路径:优先 .venvsetup.sh 标准),回退系统 python -m uvicorn
if [[ -x "$ROOT/.venv/bin/uvicorn" ]]; then
UVICORN="$ROOT/.venv/bin/uvicorn"
PYBIN="$ROOT/.venv/bin/python"
elif command -v uvicorn >/dev/null 2>&1; then
UVICORN="$(command -v uvicorn)"
PYBIN="$(command -v python3)"
else
echo "未找到 uvicorn请先运行 ./setup.sh建 .venv或 pip install uvicorn" >&2
exit 1
fi
[[ -f "$ROOT/config.yaml" ]] || { echo "未找到 config.yaml请先运行 ./setup.sh" >&2; exit 1; }
UNIT_SRC="$ROOT/deploy/ztools2.service"
UNIT_DST="/etc/systemd/system/ztools2.service"
echo "==> 生成 systemd unit路径 $ROOTuvicorn=$UVICORN"
# 同时填充 ROOT 与 UVICORNPYBIN 备用ExecStart 用 UVICORN 直接启动)
sudo sed -e "s|__ZTOOLS2_DIR__|$ROOT|g" -e "s|__UVICORN__|$UVICORN|g" "$UNIT_SRC" > /tmp/ztools2.service
sudo mv /tmp/ztools2.service "$UNIT_DST"
sudo chmod 644 "$UNIT_DST"
echo "==> 停止旧方式start.sh 启动的进程,若有)"
./stop.sh >/dev/null 2>&1 || true
echo "==> 启用并启动 ztools2 服务"
sudo systemctl daemon-reload
sudo systemctl enable ztools2
sudo systemctl restart ztools2
sleep 2
if systemctl is-active --quiet ztools2; then
echo "✓ ztools2 已启动并设为开机自启"
echo " 状态systemctl status ztools2"
echo " 日志journalctl -u ztools2 -f"
echo " 停止sudo systemctl stop ztools2"
echo " 重启sudo systemctl restart ztools2"
else
echo "✗ ztools2 启动失败查看日志journalctl -u ztools2 -n 50" >&2
exit 1
fi

23
deploy/ztools2.service Normal file
View File

@@ -0,0 +1,23 @@
[Unit]
Description=zikai file service (zTools2) - FastAPI backend
Documentation=https://git.zikai.wang/zikai/zTools2
After=network.target mysql.service
Wants=mysql.service
[Service]
Type=simple
# 工作目录与 uvicorn 路径:安装时由 install-systemd.sh 用实际路径替换
WorkingDirectory=__ZTOOLS2_DIR__
Environment=PYTHONUNBUFFERED=1
ExecStart=__UVICORN__ app.main:app --host 127.0.0.1 --port 6867 --workers 1
Restart=on-failure
RestartSec=3
# 后台转换可能耗时较长,放宽超时
TimeoutStopSec=30
KillSignal=SIGINT
# 日志走 journaldjournalctl -u ztools2
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target

39
docs/configuration.md Normal file
View File

@@ -0,0 +1,39 @@
# 配置说明
所有运行时配置在 `config.yaml`git-ignored。完整 schema 见 `config.example.yaml`
## 配置项
| 段 | 关键项 | 说明 |
|----|--------|------|
| `server` | `host`/`port` | 绑定地址,保持 `127.0.0.1:6867`Apache 反代) |
| `database` | `host`/`port`/`user`/`password`/`database` | MySQL 连接;密码由 `setup.sh` 自动生成写回 |
| `storage` | `upload_dir` | 文件存储根目录(默认 `uploads` |
| | `chunk_bytes` | 流式上传分片大小(默认 1 MiB |
| | `chunk_session_dir` | 分片会话暂存目录(默认 `uploads/.work` |
| | `chunk_session_ttl_seconds` | 被放弃会话存活秒数(默认 300 |
| `docs` | `username`/`password` | `/docs``/api/files-page``/api/wb-admin``/api/admin/*` 的 Basic Auth明文常量时间比较 |
| `sftp` | `enabled`/`host`/`port` | SFTP 服务,默认 `0.0.0.0:2022` |
| | `users[].username`/`password_hash` | SFTP 用户bcrypt |
| | `host_key_path`/`authorized_keys_path` | 主机密钥与公钥白名单路径 |
| `tunnel` | `enabled`/`users[]` | 反向隧道:`username`/`password_hash`/`tunnel_port`/`local_port` |
| `whiteboard` | `heartbeat_interval_seconds` | 心跳间隔(默认 3s |
| | `heartbeat_miss_threshold` | 失活阈值(默认 5 次 = 15s |
| | `max_board_id_length` | board_id 长度上限(默认 64 |
| | `max_connections_per_board` | 单白板并发连接上限(默认 50 |
| | `list_limit` | 管理页单次列表上限(默认 100 |
## 生成 bcrypt hashSFTP/隧道用户密码)
```bash
.venv/bin/python -c "import bcrypt;print(bcrypt.hashpw(b'yourpass',bcrypt.gensalt()).decode())"
```
把输出填入 `config.yaml` 对应 `password_hash` 字段。
## 重新生成 DB 密码
```bash
.venv/bin/python -m app.scripts.init_db # 重置密码(写回 config.yaml
KEEP_DB_PASSWORD=1 .venv/bin/python -m app.scripts.init_db # 保留现有密码,仅建库建账
```

29
docs/error-handling.md Normal file
View File

@@ -0,0 +1,29 @@
# 错误处理与日志约定
zTools2 采用 Spring 风格分层架构错误处理分三层DAO fail-fast抛异常Service 捕获后转换业务异常并记日志Controller 捕获后转 HTTP 状态码。
## DAO 层app/dao/
- **唯一发 SQL 的层**所有写操作create/update/delete均在该层 commitservice 不直接操作 session。
- `get_or_create`whiteboard_dao.py`IntegrityError`(并发下另一事务已插入违反唯一约束)才回滚重读;其他异常向上抛,避免掩盖 schema/连接等真实故障。
- `delete`pdf_job_dao.py / uploaded_file_dao.py硬删 DB 行并 commit不存在返回 False。
- schema 漂移检测:`init_db_schema`database.py建表后用 inspector 检查既有表的列是否与模型声明齐全,缺列即抛 `RuntimeError`fail-fast避免运行期才以晦涩的 `OperationalError` 暴露。
## Service 层app/services/
- **PdfService**`submit` 校验扩展名/大小,超限清理已落盘文件后抛 `HTTPException`;后台转换 `_convert_async` 捕获 `TimeoutError` / 通用异常,经 `_mark_failed` 落库 + `logger.warning``admin_delete` 磁盘删除失败仅 `logger.warning`,仍清 DB 行保证列表不再显示。
- **UploadService / ChunkUploadService**:流式落盘出错清理临时文件后 `raise`(向上传播);分片会话被放弃由后台 reaper 每 60s 清理。
- **WhiteboardHub**`disconnect` / `close_board` 关闭 websocket 出错 `logger.debug`(尽力关闭,可能已关闭);`broadcast` 单连接发送失败立即 disconnect不影响其他连接reaper 循环异常 `logger.warning` 后继续。
- **TunnelService**`register` / `close` / `reap_orphans` 均记 `logger.info`SSH 连接断开时清理会话失败 `logger.warning`
- **sftp_server**`validate_public_key` 校验异常 `logger.warning`auth 路径,避免静默失败);`_close_tunnel_dao` / 读会话用户名失败 `logger.debug`(尽力清理)。
## Controller 层app/controllers/
- **pdf_controller**`_resolve_cookie` 解析用户 cookie无则生成新值挂 request.state 供响应 set_cookie。
- **whiteboard_controller**`_safe_send` / `_safe_close` 发送/关闭 WS 失败 `logger.debug`best-effortWS 主循环异常 `logger.warning` 后正常关闭连接。
- 所有管理 API`/api/admin/*`)经 `require_docs_auth` Basic Auth 守卫(常量时间比较)。
## 日志位置
- `logs/app.log`HTTP`logs/sftp.log`SFTPpidfile`app.pid``sftp.pid`
- 日志器命名:`zikai.pdf` / `zikai.whiteboard` / `zikai.tunnel` / `sftp`,便于按模块过滤。

51
docs/routes.md Normal file
View File

@@ -0,0 +1,51 @@
# 路由与访问入口
## 路由约定
**所有 zTools2 托管的入口(页面 / 静态资源 / 健康探针 / WebSocket / REST API统一挂在 `/api/` 前缀下**,只有元信息/文档例外(`/``/docs``/redoc``/openapi.json`)。这样反向代理与 vite dev proxy 都只需一条 `/api/` 规则即可把请求转给当前环境的 zTools2前端用**同源相对路径**(如 `/api/pdf/jobs``/api/health`)即可,与运行环境(本地 / 测试 / 生产)无关,无需区分 dev/prod 指向。
> PDF 转换的用户侧 UI 由 zMainPage 的 zPDF_package 组件提供(构建期 import非 iframezTools2 仅提供 `/api/pdf/jobs` 等 REST API。PDF 转换管理(查看全部任务含软删标记、硬删)已并入文件管理页 `/api/files-page`。
页面类入口为避免与同名 REST API 冲突,统一加 `-page` 后缀:
| 类型 | 页面入口 | REST API同名不加后缀 |
|------|---------|------------------------|
| 文件管理 | `GET /api/files-page` | `GET /api/files``/api/files/{id}` 等 |
| 记事本 | `GET /api/wb-page/{id}` | `GET /api/wb/{id}` |
其余入口:`/api/index`(导航页)、`/api/health`(探针)、`/api/static/*`JS/CSS`/api/ws/wb/{id}`WebSocket`/api/upload``/api/wb-admin`
## 功能一览
| 模块 | 页面 / 接口 | 鉴权 |
|------|------------|------|
| 导航页 | `GET /api/index`(卡片式入口索引,各子页可返回) | 公开 |
| 文件上传 | `POST /api/files/upload`(流式)/ `POST /api/files/chunk-uploads/*`(分片+断点续传) | 公开 |
| 上传页 | `GET /api/upload`(拖拽/多文件/分片/去重) | 公开 |
| 文件管理 | `GET /api/files-page`(多选/批量下载删除/分页,合并 PDF 转换管理:状态、软删标记、硬删任务) | Basic Auth |
| 文件管理 API | `GET /api/admin/files``GET /api/admin/files/with-pdf`(合并视图,附带 pdf_jobs 关联)、`GET/DELETE /api/admin/files/{id}``GET /api/admin/files/{id}/download` | Basic Auth |
| 共享记事本 | `GET /api/wb-page/{id}`(公开,不存在则新建) | 公开 |
| 记事本实时同步 | `WS /api/ws/wb/{id}`(心跳 3s5 次失活移除) | 公开 |
| 记事本管理 | `GET /api/wb-admin`(查看/删除) | Basic Auth |
| 记事本管理 API | `GET /api/admin/wb``DELETE /api/admin/wb/{id}` | Basic Auth |
| PDF 转换 API | `POST /api/pdf/jobs``GET /api/pdf/jobs[/{id}]``GET /api/pdf/jobs/{id}/download``DELETE /api/pdf/jobs/{id}` | 公开cookie |
| PDF 转换管理 API | `GET /api/admin/pdf/jobs``DELETE /api/admin/pdf/jobs/{id}`(管理入口已并入 `/api/files-page` | Basic Auth |
| 主机监控 | `GET /api/system/status`CPU/内存/磁盘HTML+JSON 内容协商) | 公开 |
| 反向隧道反代 | `ALL /api/userPort/{userName}`(经 SSH 隧道转发到 user 本地服务) | 公开 |
| SFTP/SSH | 端口 2022密码+公钥chroot 到上传目录,承载隧道转发) | SSH |
| API 文档 | `GET /docs`Swagger/ `GET /redoc` | Basic Auth |
## 访问入口
| 入口 | URL |
|------|-----|
| 导航页 | https://f.zikai.wang/api/index |
| API 文档 | https://f.zikai.wang/docsBasic Auth |
| 上传页 | https://f.zikai.wang/api/upload |
| 文件管理 | https://f.zikai.wang/api/files-pageBasic Auth含 PDF 转换管理) |
| 共享记事本 | https://f.zikai.wang/api/wb-page/{id}(公开,`{id}``[a-zA-Z0-9_-]{1,64}` |
| 记事本管理 | https://f.zikai.wang/api/wb-adminBasic Auth |
| PDF 转换 | 经 zMainPage 的 PDF 页签zPDF_package 组件)调用 `/api/pdf/jobs` 等 REST API公开凭 cookie |
| 系统状态 | https://f.zikai.wang/api/system/statusHTML`?format=json` 切 JSON |
| curl 上传 | `curl -F file=@big.iso https://f.zikai.wang/api/files/upload` |
| SFTP | `sftp -P 2022 uploader@f.zikai.wang` |

View File

@@ -2,10 +2,13 @@ fastapi==0.115.6
uvicorn[standard]==0.34.0 uvicorn[standard]==0.34.0
python-multipart==0.0.20 python-multipart==0.0.20
psutil==6.1.1 psutil==6.1.1
SQLAlchemy==2.0.36 SQLAlchemy==2.0.51
PyMySQL==1.1.1 PyMySQL==1.1.1
pydantic-settings==2.7.0 pydantic-settings==2.7.0
PyYAML==6.0.2 PyYAML==6.0.2
asyncssh==2.18.0 asyncssh==2.18.0
bcrypt==4.2.1 bcrypt==4.2.1
httpx==0.28.1 httpx==0.28.1
# PDF 转换ebooklib 解析 epubweasyprint 渲染 HTML/CSS 为 PDF纯 Python无需 calibre/xvfb
ebooklib==0.20
weasyprint==69.0

View File

@@ -68,3 +68,27 @@ CREATE TABLE IF NOT EXISTS `whiteboard` (
UNIQUE KEY `uq_board_id` (`board_id`), UNIQUE KEY `uq_board_id` (`board_id`),
KEY `idx_updated_at` (`updated_at`) KEY `idx_updated_at` (`updated_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- PDF 转换任务表。由 app/services/pdf_service.py 使用。
-- 原始文件与产物 PDF 复用 uploaded_file 存储,本表只记录关系与转换状态。
-- 删除两级user_deleted用户软删管理页仍可见/ admin 硬删(真正删除)。
CREATE TABLE IF NOT EXISTS `pdf_job` (
`id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
`owner_cookie` VARCHAR(64) NOT NULL COMMENT '用户标识httpOnly cookie 值uuid4 hex',
`source_file_id` BIGINT UNSIGNED NOT NULL COMMENT '原始上传文件 -> uploaded_file.id',
`source_filename` VARCHAR(512) NOT NULL,
`source_size` BIGINT UNSIGNED NOT NULL DEFAULT 0,
`output_file_id` BIGINT UNSIGNED NULL DEFAULT NULL COMMENT '产物 PDF -> uploaded_file.id转换完成前 NULL',
`status` VARCHAR(16) NOT NULL DEFAULT 'pending' COMMENT 'pending/converting/done/failed',
`progress` INT NOT NULL DEFAULT 0 COMMENT '转换进度 0-100',
`error_message` VARCHAR(512) NOT NULL DEFAULT '',
`user_deleted` TINYINT(1) NOT NULL DEFAULT 0 COMMENT '用户软删标记',
`deleted_at` DATETIME NULL DEFAULT NULL COMMENT '用户软删时间',
`created_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `idx_owner_cookie` (`owner_cookie`),
KEY `idx_status` (`status`),
KEY `idx_user_deleted` (`user_deleted`),
KEY `idx_created_at` (`created_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

View File

@@ -5,6 +5,7 @@
.files-table th.col-size { width: 9em; } .files-table th.col-size { width: 9em; }
.files-table th.col-src { width: 6em; } .files-table th.col-src { width: 6em; }
.files-table th.col-time { width: 11em; } .files-table th.col-time { width: 11em; }
.files-table th.col-pdf { width: 13em; }
.files-table th.col-act { width: 9em; text-align: right; } .files-table th.col-act { width: 9em; text-align: right; }
.files-table td.col-act { text-align: right; white-space: nowrap; } .files-table td.col-act { text-align: right; white-space: nowrap; }
.files-table td.col-name .fname { font-weight: 600; word-break: break-all; } .files-table td.col-name .fname { font-weight: 600; word-break: break-all; }
@@ -12,9 +13,24 @@
.files-table td.col-act .btn { padding: 0.3em 0.8em; font-size: 0.85em; margin-left: 0.3em; } .files-table td.col-act .btn { padding: 0.3em 0.8em; font-size: 0.85em; margin-left: 0.3em; }
.files-table tbody tr.sel { background: var(--primary-soft); } .files-table tbody tr.sel { background: var(--primary-soft); }
.files-table tbody tr.sel:hover td { background: var(--primary-soft); } .files-table tbody tr.sel:hover td { background: var(--primary-soft); }
/* 关联到已软删 PDF 任务的行:淡化背景提示 */
.files-table tbody tr.row-pdf-deleted td { background: var(--danger-soft); }
.files-table tbody tr.row-pdf-deleted:hover td { background: var(--danger-soft); }
.files-table input[type="checkbox"] { width: 16px; height: 16px; cursor: pointer; accent-color: var(--primary); } .files-table input[type="checkbox"] { width: 16px; height: 16px; cursor: pointer; accent-color: var(--primary); }
.sha-short { cursor: pointer; } .sha-short { cursor: pointer; }
.sha-short:hover { color: var(--primary); } .sha-short:hover { color: var(--primary); }
/* PDF 任务列:角色徽标 + 状态徽标堆叠 */
.pdf-cell { display: flex; flex-direction: column; gap: 0.2em; font-size: 0.82em; }
.pdf-cell .pdf-roles { display: flex; flex-wrap: wrap; gap: 0.3em; align-items: center; }
.pdf-cell .role-tag {
display: inline-block; padding: 0.05em 0.5em; border-radius: 8px;
font-size: 0.78em; background: var(--surface-2); color: var(--text-dim);
}
.pdf-cell .role-tag.source { background: var(--warn-soft); color: var(--warn); }
.pdf-cell .role-tag.output { background: var(--success-soft); color: var(--success); }
.pdf-cell .del-mark { color: var(--danger); font-size: 0.78em; }
.pdf-cell .pdf-act { display: flex; gap: 0.3em; flex-wrap: wrap; }
.pdf-cell .pdf-act .btn { padding: 0.2em 0.6em; font-size: 0.8em; }
.row-removed { opacity: 0; transition: opacity 0.25s; } .row-removed { opacity: 0; transition: opacity 0.25s; }
.toolbar-spacer { flex: 1; } .toolbar-spacer { flex: 1; }
@@ -47,5 +63,6 @@
.files-table th, .files-table td { padding: 0.5em 0.4em; } .files-table th, .files-table td { padding: 0.5em 0.4em; }
.files-table th.col-src, .files-table td.col-src { display: none; } .files-table th.col-src, .files-table td.col-src { display: none; }
.files-table th.col-time, .files-table td.col-time { font-size: 0.78em; } .files-table th.col-time, .files-table td.col-time { font-size: 0.78em; }
.files-table th.col-pdf, .files-table td.col-pdf { font-size: 0.78em; }
.batchbar { font-size: 0.85em; } .batchbar { font-size: 0.85em; }
} }

View File

@@ -3,14 +3,14 @@
<head> <head>
<meta charset="utf-8"> <meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title>文件浏览 - zikai</title> <title>文件管理 - zikai</title>
<link rel="stylesheet" href="/static/common.css"> <link rel="stylesheet" href="/api/static/common.css">
<link rel="stylesheet" href="/static/file_browser.css"> <link rel="stylesheet" href="/api/static/file_browser.css">
</head> </head>
<body> <body>
<div class="wrap"> <div class="wrap">
<h1>文件浏览</h1> <h1>文件管理</h1>
<p class="sub">查看已上传的文件、下载或删除。支持多选与分页。删除后不再显示</p> <p class="sub">浏览 / 下载 / 删除已上传文件,并合并 PDF 转换管理:关联文件会标注转换状态与用户软删标记,可硬删任务。支持多选与分页</p>
<div class="toolbar"> <div class="toolbar">
<button class="btn primary" id="refresh">刷新</button> <button class="btn primary" id="refresh">刷新</button>
@@ -39,9 +39,9 @@
<div id="pager" class="pager hidden"></div> <div id="pager" class="pager hidden"></div>
<p class="foot"><a class="link" href="/upload">上传文件</a> · zikai file service</p> <p class="foot"><a class="link" href="/api/index">导航</a> · <a class="link" href="/api/upload">上传文件</a> · zikai file service</p>
</div> </div>
<script src="/static/common.js"></script> <script src="/api/static/common.js"></script>
<script src="/static/file_browser.js"></script> <script src="/api/static/file_browser.js"></script>
</body> </body>
</html> </html>

View File

@@ -1,4 +1,6 @@
/* 文件浏览页:分页拉取 /api/admin/files、复选框多选 + 全选、批量下载/删除、复制 sha。 /* 文件管理页:分页拉取 /api/admin/files-with-pdf、复选框多选 + 全选、批量下载/删除、复制 sha。
合并 PDF 转换管理:每个文件附带 pdf_jobsrole=source epub / role=output 产物 PDF
渲染转换状态徽标、用户软删标记并支持硬删任务DELETE /api/admin/pdf/jobs/{id})。
state.items 缓存当前页数据;切换页/页大小重新拉取;删除后若当前页空则回退一页。 */ state.items 缓存当前页数据;切换页/页大小重新拉取;删除后若当前页空则回退一页。 */
(function () { (function () {
"use strict"; "use strict";
@@ -53,7 +55,7 @@
countEl.textContent = ""; countEl.textContent = "";
pagerEl.classList.add("hidden"); pagerEl.classList.add("hidden");
try { try {
const res = await api(`/api/admin/files?limit=${limit}&offset=${offset}`); const res = await api(`/api/admin/files/with-pdf?limit=${limit}&offset=${offset}`);
if (!res.ok) throw new Error("HTTP " + res.status); if (!res.ok) throw new Error("HTTP " + res.status);
const body = await res.json(); const body = await res.json();
state.items = body.items || []; state.items = body.items || [];
@@ -73,7 +75,7 @@
function render() { function render() {
if (!state.items.length) { if (!state.items.length) {
listEl.innerHTML = '<div class="empty">还没有文件。去 <a class="link" href="/upload">上传</a> 一个吧。</div>'; listEl.innerHTML = '<div class="empty">还没有文件。去 <a class="link" href="/api/upload">上传</a> 一个吧。</div>';
renderSelection(); renderSelection();
return; return;
} }
@@ -88,13 +90,15 @@
el("th", { class: "col-size" }, "大小"), el("th", { class: "col-size" }, "大小"),
el("th", { class: "col-src" }, "来源"), el("th", { class: "col-src" }, "来源"),
el("th", { class: "col-time" }, "上传时间"), el("th", { class: "col-time" }, "上传时间"),
el("th", { class: "col-pdf" }, "PDF 任务"),
el("th", { class: "col-act" }, "操作") el("th", { class: "col-act" }, "操作")
) )
); );
const tbody = el("tbody", null); const tbody = el("tbody", null);
for (const f of state.items) { for (const f of state.items) {
const checked = state.selected.has(f.id); const checked = state.selected.has(f.id);
const row = el("tr", { class: checked ? "sel" : "", dataset: { id: f.id } }, const pdfDeleted = (f.pdf_jobs || []).some((j) => j.user_deleted);
const row = el("tr", { class: [checked ? "sel" : "", pdfDeleted ? "row-pdf-deleted" : ""].filter(Boolean).join(" "), dataset: { id: f.id } },
el("td", { class: "col-sel" }, el("td", { class: "col-sel" },
el("input", { type: "checkbox", class: "row-sel", checked, dataset: { id: f.id } }) el("input", { type: "checkbox", class: "row-sel", checked, dataset: { id: f.id } })
), ),
@@ -105,6 +109,7 @@
el("td", { class: "col-size mono" }, fmtBytes(f.size_bytes)), el("td", { class: "col-size mono" }, fmtBytes(f.size_bytes)),
el("td", { class: "col-src" }, el("span", { class: "tag" }, f.source || "-")), el("td", { class: "col-src" }, el("span", { class: "tag" }, f.source || "-")),
el("td", { class: "col-time muted" }, fmtTime(f.uploaded_at)), el("td", { class: "col-time muted" }, fmtTime(f.uploaded_at)),
el("td", { class: "col-pdf" }, renderPdfCell(f)),
el("td", { class: "col-act" }, el("td", { class: "col-act" },
el("a", { class: "btn primary", href: `/api/admin/files/${f.id}/download`, download: "" }, "下载"), el("a", { class: "btn primary", href: `/api/admin/files/${f.id}/download`, download: "" }, "下载"),
el("button", { class: "btn danger", onclick: () => removeOne(f) }, "删除") el("button", { class: "btn danger", onclick: () => removeOne(f) }, "删除")
@@ -190,6 +195,58 @@
return sha.length > 16 ? sha.slice(0, 12) + "…" + sha.slice(-4) : sha; return sha.length > 16 ? sha.slice(0, 12) + "…" + sha.slice(-4) : sha;
} }
// 渲染 PDF 任务列:展示角色(源 epub / 产物 PDF徽标、转换状态、用户软删标记与硬删按钮。
// 一个文件可能同时被多个 job 引用(罕见),全列出;无关联则显示 -。
function renderPdfCell(f) {
const jobs = f.pdf_jobs || [];
if (!jobs.length) return el("span", { class: "muted" }, "-");
const cell = el("div", { class: "pdf-cell" });
const roles = el("div", { class: "pdf-roles" });
for (const j of jobs) {
roles.appendChild(el("span", { class: "role-tag " + j.role, title: j.role === "source" ? "PDF 转换的 epub 源文件" : "PDF 转换的产物 PDF" },
j.role === "source" ? "源" : "产物"));
roles.appendChild(statusTag(j));
if (j.user_deleted) {
roles.appendChild(el("span", { class: "del-mark", title: "用户已软删:" + fmtTime(j.deleted_at) }, "已删"));
}
roles.appendChild(el("span", { class: "muted" }, "#"+j.job_id));
}
cell.appendChild(roles);
// 硬删按钮:对每个关联 job 提供source 与 output 共属同一 job去重后只显示一个
const seenJobIds = new Set();
const actBar = el("div", { class: "pdf-act" });
for (const j of jobs) {
if (seenJobIds.has(j.job_id)) continue;
seenJobIds.add(j.job_id);
actBar.appendChild(el("button", {
class: "btn danger", onclick: () => removePdfJob(j, f)
}, "硬删任务"));
}
cell.appendChild(actBar);
return cell;
}
function statusTag(j) {
if (j.status === "done") return el("span", { class: "tag ok" }, "完成");
if (j.status === "failed") return el("span", { class: "tag err", title: j.error_message || "" }, "失败");
if (j.status === "converting") return el("span", { class: "tag warn" }, "转换中 " + j.progress + "%");
return el("span", { class: "tag" }, "排队中");
}
// 硬删 PDF 任务:删 job + 其源/产物文件。删除后重载当前页。
async function removePdfJob(j, f) {
if (!confirm(`确定硬删 PDF 任务 #${j.job_id}\n将同时删除关联的源文件与产物 PDF不可恢复。`)) return;
try {
const res = await api(`/api/admin/pdf/jobs/${j.job_id}`, { method: "DELETE" });
if (res.status === 404) { toast("任务已不存在"); }
else if (!res.ok) throw new Error("HTTP " + res.status);
toast("已硬删任务");
await load(state.page);
} catch (e) {
toast("硬删失败:" + (e.message || e), "err");
}
}
async function removeOne(f) { async function removeOne(f) {
if (!confirm(`确定删除「${f.original_filename}」?\n此操作不可恢复,将同时删除磁盘文件。`)) return; if (!confirm(`确定删除「${f.original_filename}」?\n此操作不可恢复,将同时删除磁盘文件。`)) return;
try { try {

74
static/index.html Normal file
View File

@@ -0,0 +1,74 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>导航 - zikai</title>
<link rel="stylesheet" href="/api/static/common.css">
<style>
/* 导航页:卡片网格,复用 common.css 变量 */
.nav-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(220px, 1fr));
gap: 1em;
margin-top: 1.5em;
}
.nav-card {
display: flex; flex-direction: column; gap: 0.3em;
padding: 1.2em 1.3em;
border: 1px solid var(--border);
border-radius: var(--radius);
background: var(--surface);
box-shadow: var(--shadow);
text-decoration: none; color: var(--text);
transition: transform 0.05s, border-color 0.15s;
}
.nav-card:hover { transform: translateY(-2px); border-color: var(--primary); }
.nav-card .ico { font-size: 1.6em; }
.nav-card .name { font-size: 1.05em; font-weight: 600; }
.nav-card .desc { color: var(--text-dim); font-size: 0.82em; line-height: 1.4; }
.nav-card .lock { font-size: 0.75em; color: var(--text-dim); }
</style>
</head>
<body>
<div class="wrap">
<h1>zikai 工具箱</h1>
<p class="sub">个人 Web 服务入口导航。带 🔒 标记的页面需 Basic Auth凭据见 config.yaml 的 docs 段)。</p>
<div class="nav-grid">
<a class="nav-card" href="/api/upload">
<span class="ico">📤</span>
<span class="name">上传文件</span>
<span class="desc">拖拽 / 多文件 / 分片4 MiB/ 断点续传</span>
<span class="lock"></span>
</a>
<a class="nav-card" href="/api/files-page">
<span class="ico">📁</span>
<span class="name">文件管理</span>
<span class="desc">浏览 / 下载 / 删除已上传文件,合并 PDF 转换管理(状态、软删标记、硬删任务)</span>
<span class="lock">🔒 需鉴权</span>
</a>
<a class="nav-card" href="/api/wb-admin">
<span class="ico">📝</span>
<span class="name">记事本管理</span>
<span class="desc">查看 / 删除共享文本记事本,可新建并打开</span>
<span class="lock">🔒 需鉴权</span>
</a>
<a class="nav-card" href="/api/system/status">
<span class="ico">📊</span>
<span class="name">系统状态</span>
<span class="desc">CPU / 内存 / 磁盘实时使用率(?format=json 切 JSON</span>
<span class="lock"></span>
</a>
<a class="nav-card" href="/docs">
<span class="ico">📚</span>
<span class="name">API 文档</span>
<span class="desc">Swagger UI全部 REST 接口在线调试</span>
<span class="lock">🔒 需鉴权</span>
</a>
</div>
<p class="foot">zikai file service · <a class="link" href="/api/health">健康探针</a></p>
</div>
</body>
</html>

View File

@@ -1,5 +1,17 @@
/* 记事本页专属样式:全屏 textarea、悬浮工具栏、移动端适配。 */ /* 记事本页专属样式:全屏 textarea、悬浮工具栏、移动端适配。 */
:root { --bar-h: 52px; } /* 强制浅色:本页常被 iframe 嵌入浅色站点,覆盖 common.css 的
color-scheme: light dark 与 prefers-color-scheme: dark避免深色背景。 */
:root {
--bar-h: 52px;
color-scheme: light;
--bg: #f6f7f9;
--surface: #ffffff;
--surface-2: rgba(0, 0, 0, 0.02);
--border: #e0e3e7;
--text: #1a1a1a;
--text-dim: #6b7280;
--shadow: 0 1px 3px rgba(0, 0, 0, 0.04), 0 1px 2px rgba(0, 0, 0, 0.06);
}
body { overflow: hidden; background: var(--bg); } body { overflow: hidden; background: var(--bg); }
.wb-app { display: flex; flex-direction: column; height: 100vh; height: 100dvh; } .wb-app { display: flex; flex-direction: column; height: 100vh; height: 100dvh; }
.wb-bar { .wb-bar {

View File

@@ -4,9 +4,12 @@
<meta charset="utf-8"> <meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no"> <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
<meta name="theme-color" content="#1565c0"> <meta name="theme-color" content="#1565c0">
<!-- 强制浅色:该页面会被 iframe 嵌入到浅色主题站点mainPage
禁用 common.css 的 prefers-color-scheme: dark保持背景与嵌入站一致 -->
<meta name="color-scheme" content="light">
<title>记事本 - zikai</title> <title>记事本 - zikai</title>
<link rel="stylesheet" href="/static/common.css"> <link rel="stylesheet" href="/api/static/common.css">
<link rel="stylesheet" href="/static/whiteboard.css"> <link rel="stylesheet" href="/api/static/whiteboard.css">
</head> </head>
<body> <body>
<div class="wb-app"> <div class="wb-app">
@@ -17,6 +20,7 @@
<span class="wb-online" id="online" title="在线人数"></span> <span class="wb-online" id="online" title="在线人数"></span>
</div> </div>
<div class="wb-bar-right"> <div class="wb-bar-right">
<a class="btn" href="/api/index" target="_top" title="返回导航">导航</a>
<button class="btn" id="copyLinkBtn" title="复制分享链接">复制链接</button> <button class="btn" id="copyLinkBtn" title="复制分享链接">复制链接</button>
<button class="btn" id="copyBtn" title="复制全部文本">复制文本</button> <button class="btn" id="copyBtn" title="复制全部文本">复制文本</button>
<button class="btn danger" id="clearBtn" title="清空全部内容(所有人)">清空</button> <button class="btn danger" id="clearBtn" title="清空全部内容(所有人)">清空</button>
@@ -27,7 +31,7 @@
<div class="wb-status" id="status">连接中…</div> <div class="wb-status" id="status">连接中…</div>
</main> </main>
</div> </div>
<script src="/static/common.js"></script> <script src="/api/static/common.js"></script>
<script src="/static/whiteboard.js"></script> <script src="/api/static/whiteboard.js"></script>
</body> </body>
</html> </html>

View File

@@ -8,7 +8,7 @@
const { toast, copyText } = window.ZK; const { toast, copyText } = window.ZK;
// ---------- 从 URL 解析 board_id ---------- // ---------- 从 URL 解析 board_id ----------
const m = location.pathname.match(/^\/wb\/([^/]+)\/?$/); const m = location.pathname.match(/^\/api\/wb-page\/([^/]+)\/?$/);
let boardId = m ? decodeURIComponent(m[1]) : "default"; let boardId = m ? decodeURIComponent(m[1]) : "default";
if (!/^[a-zA-Z0-9_-]{1,64}$/.test(boardId)) boardId = "default"; if (!/^[a-zA-Z0-9_-]{1,64}$/.test(boardId)) boardId = "default";
document.getElementById("boardId").textContent = boardId; document.getElementById("boardId").textContent = boardId;
@@ -81,7 +81,7 @@
}); });
copyLinkBtn.addEventListener("click", async () => { copyLinkBtn.addEventListener("click", async () => {
const url = `${location.origin}/wb/${boardId}`; const url = `${location.origin}/api/wb-page/${boardId}`;
const ok = await copyText(url); const ok = await copyText(url);
toast(ok ? "链接已复制" : "复制失败"); toast(ok ? "链接已复制" : "复制失败");
}); });
@@ -143,7 +143,7 @@
// ---------- WebSocket ---------- // ---------- WebSocket ----------
function wsUrl() { function wsUrl() {
const proto = location.protocol === "https:" ? "wss:" : "ws:"; const proto = location.protocol === "https:" ? "wss:" : "ws:";
return `${proto}//${location.host}/ws/wb/${encodeURIComponent(boardId)}`; return `${proto}//${location.host}/api/ws/wb/${encodeURIComponent(boardId)}`;
} }
function connect() { function connect() {

View File

@@ -4,8 +4,8 @@
<meta charset="utf-8"> <meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<title>白板管理 - zikai</title> <title>白板管理 - zikai</title>
<link rel="stylesheet" href="/static/common.css"> <link rel="stylesheet" href="/api/static/common.css">
<link rel="stylesheet" href="/static/whiteboard_admin.css"> <link rel="stylesheet" href="/api/static/whiteboard_admin.css">
</head> </head>
<body> <body>
<div class="wrap"> <div class="wrap">
@@ -22,9 +22,9 @@
<div class="skel">加载中…</div> <div class="skel">加载中…</div>
</div> </div>
<p class="foot"><a class="link" href="/upload">上传文件</a> · <a class="link" href="/files">文件浏览</a> · zikai</p> <p class="foot"><a class="link" href="/api/index">导航</a> · <a class="link" href="/api/upload">上传文件</a> · <a class="link" href="/api/files-page">文件管理</a> · zikai</p>
</div> </div>
<script src="/static/common.js"></script> <script src="/api/static/common.js"></script>
<script src="/static/whiteboard_admin.js"></script> <script src="/api/static/whiteboard_admin.js"></script>
</body> </body>
</html> </html>

View File

@@ -11,7 +11,7 @@
newBtn.addEventListener("click", () => { newBtn.addEventListener("click", () => {
// 生成一个随机 board_id 并打开(访问即创建) // 生成一个随机 board_id 并打开(访问即创建)
const id = "b_" + Math.random().toString(36).slice(2, 10); const id = "b_" + Math.random().toString(36).slice(2, 10);
window.open(`/wb/${id}`, "_blank"); window.open(`/api/wb-page/${id}`, "_blank");
}); });
async function load() { async function load() {
@@ -47,7 +47,7 @@
for (const b of items) { for (const b of items) {
const row = el("tr", null, const row = el("tr", null,
el("td", { class: "col-id" }, el("td", { class: "col-id" },
el("a", { class: "bid link", href: `/wb/${b.board_id}`, target: "_blank" }, b.board_id) el("a", { class: "bid link", href: `/api/wb-page/${b.board_id}`, target: "_blank" }, b.board_id)
), ),
el("td", { class: "col-mods mono" }, String(b.edit_count ?? 0)), el("td", { class: "col-mods mono" }, String(b.edit_count ?? 0)),
el("td", { class: "col-created muted" }, fmtTime(b.created_at)), el("td", { class: "col-created muted" }, fmtTime(b.created_at)),

View File

@@ -8,7 +8,7 @@ import urllib.request
import websockets import websockets
BASE_WS = "ws://127.0.0.1:6867/ws/wb" BASE_WS = "ws://127.0.0.1:6867/api/ws/wb"
BOARD = "kicktest" BOARD = "kicktest"
AUTH = "Basic YTo2NjUxMTMxNQ==" # a:66511315 AUTH = "Basic YTo2NjUxMTMxNQ==" # a:66511315

View File

@@ -18,7 +18,7 @@ import urllib.request
import websockets import websockets
BASE_WS = "ws://127.0.0.1:6867/ws/wb" BASE_WS = "ws://127.0.0.1:6867/api/ws/wb"
BOARD = "e2etest" BOARD = "e2etest"

284
tests/test_pdf_service.py Normal file
View File

@@ -0,0 +1,284 @@
"""PDF 转换服务测试pytest + TestClient
用 SQLite 内存库覆盖 get_db免依赖 MySQL用临时目录覆盖上传根目录。
覆盖完整链路:上传 epub -> 轮询至 done -> 下载有效 PDF -> 用户软删 ->
管理页仍可见 -> 管理员硬删 -> 真正删除。另测大小超限与格式校验。
"""
from __future__ import annotations
import io
import time
import zipfile
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import create_engine, text
from sqlalchemy.orm import sessionmaker
from app import config as config_module
from app import database as db_module
from app.database import Base, get_db
from app.models.uploaded_file import UploadedFile # noqa: F401 注册映射
# ---------------- fixtures ----------------
def _make_epub_bytes() -> bytes:
"""构造一个最小的合法 epub含 1 章节 + 1 PNG 图片 + CSS"""
from PIL import Image # 已是 weasyprint 依赖间接项,环境可用
png = io.BytesIO()
Image.new("RGBA", (8, 8), (26, 95, 180, 255)).save(png, format="PNG")
png_bytes = png.getvalue()
ch = (
'<?xml version="1.0" encoding="UTF-8"?>'
'<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Ch1</title>'
'<link rel="stylesheet" type="text/css" href="style.css"/></head>'
'<body><h1>Hello PDF</h1><p>Test paragraph for conversion.</p>'
'<p><img src="img.png" alt="pic"/></p></body></html>'
)
opf = (
'<?xml version="1.0" encoding="UTF-8"?>'
'<package xmlns="http://www.idpf.org/2007/opf" version="2.0" unique-identifier="Bid">'
'<metadata xmlns:dc="http://purl.org/dc/elements/1.1/">'
'<dc:title>T</dc:title><dc:identifier id="Bid">urn:uuid:t</dc:identifier>'
'<dc:language>en</dc:language></metadata>'
'<manifest>'
'<item id="ncx" href="toc.ncx" media-type="application/x-dtbncx+xml"/>'
'<item id="css" href="style.css" media-type="text/css"/>'
'<item id="img" href="img.png" media-type="image/png"/>'
'<item id="ch1" href="ch1.xhtml" media-type="application/xhtml+xml"/>'
'</manifest><spine toc="ncx"><itemref idref="ch1"/></spine></package>'
)
ncx = (
'<?xml version="1.0" encoding="UTF-8"?>'
'<ncx xmlns="http://www.daisy.org/z3986/2005/ncx/" version="2005-1">'
'<head><meta name="dtb:uid" content="urn:uuid:t"/></head>'
'<docTitle><text>T</text></docTitle>'
'<navMap><navPoint id="n1" playOrder="1"><navLabel><text>Ch1</text></navLabel>'
'<content src="ch1.xhtml"/></navPoint></navMap></ncx>'
)
css = "h1{color:#1a5fb4} p{line-height:1.6}"
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w") as z:
z.writestr("mimetype", "application/epub+zip", compress_type=zipfile.ZIP_STORED)
z.writestr("OEBPS/content.opf", opf, compress_type=zipfile.ZIP_DEFLATED)
z.writestr("OEBPS/toc.ncx", ncx, compress_type=zipfile.ZIP_DEFLATED)
z.writestr("OEBPS/style.css", css, compress_type=zipfile.ZIP_DEFLATED)
z.writestr("OEBPS/img.png", png_bytes, compress_type=zipfile.ZIP_DEFLATED)
z.writestr("OEBPS/ch1.xhtml", ch, compress_type=zipfile.ZIP_DEFLATED)
z.writestr("META-INF/container.xml",
'<?xml version="1.0"?><container version="1.0" '
'xmlns="urn:oasis:names:tc:opendocument:xmlns:container">'
'<rootfiles><rootfile full-path="OEBPS/content.opf" '
'media-type="application/oebps-package+xml"/></rootfiles></container>',
compress_type=zipfile.ZIP_DEFLATED)
return buf.getvalue()
@pytest.fixture()
def client(tmp_path, monkeypatch):
"""构造用隔离 MySQL 测试库 + 临时上传目录的 TestClient。
用独立库 zikai_filesvc_test与生产库隔离每个 fixture 重建表,准确测试生产
行为MySQL BigInteger 自增、线程安全连接)。后台转换经 asyncio.to_thread 在
独立线程执行MySQL 连接池天然支持跨线程。
"""
real_settings = config_module.get_settings()
# 用独立测试库 zikai_filesvc_test与生产库隔离准确测试生产行为
test_settings = real_settings.model_copy(deep=True)
test_settings.database.database = "zikai_filesvc_test"
test_settings.pdf.convert_timeout_seconds = 60
# 全局 get_settings 被 lru_cacheconfig 与 database 两个模块各自 import 了它,
# 都需 patch使后台线程经 database.get_session_local也读到测试库
monkeypatch.setattr(config_module, "get_settings", lambda: test_settings)
monkeypatch.setattr(db_module, "get_settings", lambda: test_settings)
# 清掉已建的全局引擎/SessionLocal下次 get_session_local() 用测试库重建
db_module.dispose_engine()
upload_dir = tmp_path / "uploads"
upload_dir.mkdir()
# 复用全局 SessionLocal指向测试库保证请求路径与后台线程用同一库
SessionLocal = db_module.get_session_local()
# 覆盖 get_db仍用全局 SessionLocal但确保请求结束关闭
def override_get_db():
db = SessionLocal()
try:
yield db
finally:
db.close()
# 上传目录指向临时目录
monkeypatch.setattr(test_settings, "storage", test_settings.storage)
test_settings.storage.upload_dir = str(upload_dir)
# upload_service / pdf_service 内部 import 了 get_settings需 patch 其引用
import app.services.upload_service as us
import app.services.pdf_service as ps
import app.services.pdf_converter as pc # noqa: F401
monkeypatch.setattr(us, "get_settings", lambda: test_settings)
monkeypatch.setattr(ps, "get_settings", lambda: test_settings)
# 每次测试前清空重建表,保证隔离
Base.metadata.drop_all(bind=db_module.get_engine())
Base.metadata.create_all(bind=db_module.get_engine())
from app.main import app
app.dependency_overrides[get_db] = override_get_db
with TestClient(app) as c:
yield c
app.dependency_overrides.clear()
Base.metadata.drop_all(bind=db_module.get_engine())
db_module.dispose_engine()
def _wait_done(client, job_id, cookie, timeout=60):
"""轮询任务状态直到 done/failed 或超时。"""
deadline = time.time() + timeout
last = None
while time.time() < deadline:
r = client.get(f"/api/pdf/jobs/{job_id}", cookies=cookie)
assert r.status_code == 200, r.text
last = r.json()
if last["status"] in ("done", "failed"):
return last
time.sleep(0.3)
raise AssertionError(f"任务未在 {timeout}s 内完成,最后状态: {last}")
# 管理页 Basic Auth 凭据(与 config.yaml 的 docs 段保持一致)
ADMIN_AUTH = ("a", "66511315")
# ---------------- 测试 ----------------
class TestSubmitAndConvert:
def test_upload_epub_converts_and_downloads(self, client):
epub = _make_epub_bytes()
# 首次上传:无 cookie应下发新 cookie
r = client.post(
"/api/pdf/jobs",
files={"file": ("test.epub", epub, "application/epub+zip")},
)
assert r.status_code == 200, r.text
body = r.json()
assert body["set_cookie"] is True
assert "zk_pdf" in r.headers.get("set-cookie", "")
job = body["job"]
assert job["status"] == "pending"
job_id = job["id"]
cookie = {"zk_pdf": r.cookies.get("zk_pdf")}
# 轮询至完成
final = _wait_done(client, job_id, cookie)
assert final["status"] == "done", final
assert final["progress"] == 100
# 下载产物:应为有效 PDF
d = client.get(f"/api/pdf/jobs/{job_id}/download", cookies=cookie)
assert d.status_code == 200, d.text
assert d.headers["content-type"] == "application/pdf"
assert d.content[:5] == b"%PDF-"
assert len(d.content) > 100
def test_user_list_shows_own_jobs(self, client):
epub = _make_epub_bytes()
r = client.post("/api/pdf/jobs", files={"file": ("a.epub", epub, "application/epub+zip")})
cookie = {"zk_pdf": r.cookies.get("zk_pdf")}
lst = client.get("/api/pdf/jobs", cookies=cookie)
assert lst.status_code == 200
assert lst.json()["total"] == 1
# 另一用户(清空 client cookie jar 模拟全新浏览器,应下发新 cookie
client.cookies.clear()
r2 = client.post("/api/pdf/jobs", files={"file": ("b.epub", epub, "application/epub+zip")})
cookie2 = {"zk_pdf": r2.cookies.get("zk_pdf")}
assert cookie2["zk_pdf"] != cookie["zk_pdf"] # 确是不同用户
lst2 = client.get("/api/pdf/jobs", cookies=cookie2)
assert lst2.json()["total"] == 1 # 只有自己的 1 个
def test_rejects_non_epub(self, client):
r = client.post(
"/api/pdf/jobs",
files={"file": ("note.txt", b"hello world", "text/plain")},
)
assert r.status_code == 400
def test_rejects_oversize(self, client, monkeypatch):
# 把上限调到极小,避免真的构造 250MB 文件
import app.services.pdf_service as ps
real = ps.get_settings()
small = real.model_copy(deep=True)
small.pdf.max_size_bytes = 100
monkeypatch.setattr(ps, "get_settings", lambda: small)
monkeypatch.setattr(ps, "get_settings", lambda: small)
# controller 的 _resolve_cookie 也读 get_settings但 submit 内 service 用 ps.get_settings
epub = _make_epub_bytes()
r = client.post("/api/pdf/jobs", files={"file": ("big.epub", epub, "application/epub+zip")})
assert r.status_code == 413
class TestDeleteSemantics:
def test_user_soft_delete_admin_still_visible_then_admin_hard_delete(self, client):
epub = _make_epub_bytes()
r = client.post("/api/pdf/jobs", files={"file": ("del.epub", epub, "application/epub+zip")})
cookie = {"zk_pdf": r.cookies.get("zk_pdf")}
job_id = r.json()["job"]["id"]
_wait_done(client, job_id, cookie)
# 用户软删
d = client.delete(f"/api/pdf/jobs/{job_id}", cookies=cookie)
assert d.status_code == 200
assert d.json()["deleted"] is True
# 用户列表不再可见
lst = client.get("/api/pdf/jobs", cookies=cookie)
assert lst.json()["total"] == 0
# 管理页仍可见且标注已删除
al = client.get("/api/admin/pdf/jobs", auth=ADMIN_AUTH)
assert al.status_code == 200
items = al.json()["items"]
assert any(it["id"] == job_id and it["user_deleted"] is True for it in items)
assert any(it["id"] == job_id and it["deleted_at"] is not None for it in items)
# 用户已无法下载(已软删)
dl = client.get(f"/api/pdf/jobs/{job_id}/download", cookies=cookie)
assert dl.status_code == 404
# 管理员硬删
ad = client.delete(f"/api/admin/pdf/jobs/{job_id}", auth=ADMIN_AUTH)
assert ad.status_code == 200
assert ad.json()["deleted"] is True
# 管理页也不再可见
al2 = client.get("/api/admin/pdf/jobs", auth=ADMIN_AUTH)
assert not any(it["id"] == job_id for it in al2.json()["items"])
def test_admin_requires_auth(self, client):
r = client.get("/api/admin/pdf/jobs")
assert r.status_code == 401
r = client.get("/api/admin/pdf/jobs", auth=("a", "wrong"))
assert r.status_code == 401
r = client.get("/api/admin/pdf/jobs", auth=ADMIN_AUTH)
assert r.status_code == 200
def test_user_cannot_access_others_job(self, client):
epub = _make_epub_bytes()
r1 = client.post("/api/pdf/jobs", files={"file": ("x.epub", epub, "application/epub+zip")})
job_id = r1.json()["job"]["id"]
# 另一用户访问(清空 cookie jar 模拟全新浏览器)
client.cookies.clear()
r2 = client.post("/api/pdf/jobs", files={"file": ("y.epub", epub, "application/epub+zip")})
cookie2 = {"zk_pdf": r2.cookies.get("zk_pdf")}
assert client.get(f"/api/pdf/jobs/{job_id}", cookies=cookie2).status_code == 404
assert client.get(f"/api/pdf/jobs/{job_id}/download", cookies=cookie2).status_code == 404
assert client.delete(f"/api/pdf/jobs/{job_id}", cookies=cookie2).status_code == 404