Compare commits
19 Commits
41580a9025
...
refactor/c
| Author | SHA1 | Date | |
|---|---|---|---|
| 7c193ca46e | |||
| 9af28f41b4 | |||
| 1c0f776571 | |||
| e7b4a3d5e3 | |||
| ff2ad3fcb3 | |||
| 729c77e98b | |||
| 29c5462734 | |||
| aa08a1cf60 | |||
| fa980e74e0 | |||
|
|
09705a8843 | ||
|
|
30a263ed50 | ||
|
|
4de2648178 | ||
|
|
1030801712 | ||
|
|
da7d2fb4b4 | ||
|
|
374c3d150f | ||
|
|
655e039aad | ||
|
|
7bacc33679 | ||
|
|
7188c62d3a | ||
|
|
3284269399 |
269
README.md
269
README.md
@@ -1,213 +1,124 @@
|
||||
# zikai file service
|
||||
# zTools2 - 个人 Web 服务后端
|
||||
|
||||
`f.zikai.wang` 的 Python Web 服务(FastAPI),提供主机监控、大文件上传、共享白板与
|
||||
文件浏览:**HTTP(整文件 + 分片/断点续传)**,并内置 **SFTP 服务器** 用于原始文件暂存。
|
||||
采用 Spring 风格分层架构(`controllers` -> `services` -> `dao`,外加 `models` 与
|
||||
`schemas`),自带自动生成的 API 文档,全部运行在自包含的 `.venv` 中。
|
||||
基于 FastAPI 的个人 Web 服务后端,提供文件上传/浏览/下载、共享记事本(实时协作)、主机监控、SFTP 暂存、反向隧道、PDF 转换。采用 Spring 风格分层架构(controller -> service -> dao -> ORM model -> MySQL),自带 API 文档。
|
||||
|
||||
## 功能
|
||||
所有入口统一挂在 `/api/` 前缀下,前端用同源相对路径调用,无需区分 dev/prod。前端子项目([timeTableFix](https://git.zikai.wang/zikai/timeTableFix)、[zPDF_package](https://git.zikai.wang/zikai/zPDF_package)、[zWhiteBoard](https://git.zikai.wang/zikai/zWhiteBoard))经 [zMainPage](https://git.zikai.wang/zikai/zMainPage) 构建期组件 import 集成,生产由 apache2 静态托管 + `/api` 反代到本服务。
|
||||
|
||||
- `GET /api/system/status` - CPU、内存、各磁盘使用率(via `psutil`)。
|
||||
- `POST /api/files/upload` - **流式** multipart 上传(内存恒定,支持多 GB),落盘时算 SHA-256。
|
||||
- `GET /upload` - 拖拽上传页面:多文件、**分片(4 MiB)**、**断点续传**、sha256 去重。
|
||||
- `POST /api/files/chunk-uploads/*` - 支撑 `/upload` 的分片上传 API(建会话 / 查状态 / 传分片 / 完成)。
|
||||
- `GET /api/files`、`GET /api/files/{id}`、`GET /api/files/{id}/download`。
|
||||
- **文件浏览页** `GET /files`(Basic Auth,同 docs):列出/下载/**硬删除**已上传文件;删除后不再显示。
|
||||
管理 API:`GET /api/admin/files`、`GET /api/admin/files/{id}`、`GET /api/admin/files/{id}/download`、
|
||||
`DELETE /api/admin/files/{id}`(均 Basic Auth)。
|
||||
- **共享白板** `GET /whiteboard/{id}`(公开,不存在则新建):Canvas 实时协作 + **清空 / 复制链接**,兼容移动端。
|
||||
实时同步走 `WS /ws/whiteboard/{id}`(**心跳 3s,连续 5 次丢失判失活并移除**)。
|
||||
- **白板管理页** `GET /whiteboard-admin`(Basic Auth,同 docs):查看创建时间/修改次数/上次修改时间/删除。
|
||||
管理 API:`GET /api/admin/whiteboards`、`DELETE /api/admin/whiteboards/{id}`(均 Basic Auth)。
|
||||
- **反向隧道反代**:`ALL /api/userPort/{userName}` -- 把请求经 SSH 反向隧道转发到该 user 的本机服务。
|
||||
- **内置 SFTP/SSH 服务器**(asyncssh),支持 **密码 + 公钥** 鉴权,同时承载 SFTP 文件暂存与反向隧道。
|
||||
- `/docs`(Swagger UI)与 `/redoc` - 交互式文档,自动列出所有 API。
|
||||
- 元数据持久化在 **独立的 MySQL 数据库**(`zikai_filesvc`)。
|
||||
- `start.sh` / `stop.sh` 生命周期管理;`setup.sh` 一次性初始化。
|
||||
|
||||
## 架构(Spring 风格分层)
|
||||
## 项目结构
|
||||
|
||||
```
|
||||
app/
|
||||
├── controllers/ # FastAPI 路由 -- HTTP 边界(类似 @RestController)
|
||||
├── services/ # 业务逻辑(SystemService, UploadService, ChunkUploadService,
|
||||
│ # WhiteboardService, WhiteboardHub, SFTP 服务)
|
||||
├── dao/ # 数据访问对象 -- 唯一发出 SQL/ORM 的层
|
||||
├── models/ # SQLAlchemy ORM 实体(UploadedFile, UploadSession, Whiteboard, ...)
|
||||
├── schemas/ # pydantic DTO(请求/响应校验)
|
||||
├── views/ # 服务端渲染的 HTML 页面(系统状态、上传页)
|
||||
├── static/ # 前端静态资源(文件浏览/白板/白板管理的 HTML+JS+CSS,经 StaticFiles 挂载)
|
||||
├── database.py # 引擎、Session、Base、get_db() 依赖
|
||||
├── config.py # 从 config.yaml 加载的类型化 Settings
|
||||
└── scripts/ # init_db.py -- 数据库初始化
|
||||
zTools2/
|
||||
├── app/
|
||||
│ ├── main.py # FastAPI 应用工厂、路由注册、生命周期(reaper)
|
||||
│ ├── config.py # 从 config.yaml 加载的类型化 Settings(pydantic-settings)
|
||||
│ ├── database.py # SQLAlchemy 引擎/Session/Base/get_db 依赖 + schema 校验
|
||||
│ ├── security.py # Basic Auth(require_docs_auth,常量时间比较)
|
||||
│ ├── controllers/ # 路由层:file/system/chunk/tunnel/whiteboard/pdf/admin
|
||||
│ ├── services/ # 业务层:Upload/ChunkUpload/System/Whiteboard/Tunnel/Pdf/sftp
|
||||
│ ├── dao/ # 数据访问层:唯一发 SQL 的层(SQLAlchemy ORM 参数化)
|
||||
│ ├── models/ # ORM 实体:UploadedFile/UploadSession/Whiteboard/TunnelSession/PdfJob
|
||||
│ ├── schemas/ # pydantic 请求/响应 DTO
|
||||
│ ├── views/ # 服务端渲染 HTML(系统状态页、上传页)
|
||||
│ ├── static/ # 前端静态资源
|
||||
│ └── scripts/init_db.py # 数据库初始化(建库建账、随机密码写回 config.yaml)
|
||||
├── sql/schema.sql # 建表 DDL(参考;实际由 ORM 自动建表)
|
||||
├── config.example.yaml # 配置模板(含注释)
|
||||
├── config.yaml # 实际配置(git-ignored,含密码)
|
||||
├── requirements.txt
|
||||
├── setup.sh # 一次性初始化:venv + 依赖 + 建库 + SFTP 密钥
|
||||
├── start.sh / stop.sh # 启停 HTTP(127.0.0.1:6867)+ SFTP(2022)
|
||||
└── deploy/ # systemd 持久化部署
|
||||
```
|
||||
|
||||
请求流程:**controller** -> **service** -> **dao** -> **ORM model** -> MySQL。
|
||||
DB Session 由 FastAPI 的 `get_db` 依赖注入并向下传递。前端三套页面走「独立静态文件 +
|
||||
StaticFiles 挂载」的前后端分离模式,HTML 壳由具名路由返回(便于各自挂 Basic Auth),
|
||||
JS 调用同源 `/api/...`。
|
||||
**请求流程**:`controller -> service -> dao -> ORM model -> MySQL`。DB Session 由 `get_db` 依赖注入。
|
||||
|
||||
## 快速开始
|
||||
## 外部依赖
|
||||
|
||||
| 项 | 说明 |
|
||||
|----|------|
|
||||
| Python | ≥ 3.11(用 `.venv`) |
|
||||
| MySQL | 8.x(独立库 `zikai_filesvc`,由 `setup.sh` 建账) |
|
||||
| Apache2 | 反向代理对外提供 HTTPS;服务本身只绑 `127.0.0.1:6867` |
|
||||
| 系统库 | `libpango/cairo`(weasyprint PDF 转换)、`build-essential`(bcrypt/asyncssh 编译) |
|
||||
|
||||
### Apache2 反向代理配置
|
||||
|
||||
服务只绑 `127.0.0.1:6867`,通过 Apache 对外提供 HTTPS。安装模块并配置 vhost:
|
||||
|
||||
```bash
|
||||
cd /root/zikai
|
||||
./setup.sh # 一次性:venv、依赖、建库建账、SFTP 主机密钥
|
||||
./start.sh # 启动 HTTP(127.0.0.1:6867)+ SFTP(0.0.0.0:2022)
|
||||
./stop.sh # 停止两者
|
||||
a2enmod ssl proxy proxy_http proxy_wstunnel rewrite headers
|
||||
```
|
||||
|
||||
`setup.sh` 可重复执行。它会创建 `.venv`、安装 `requirements.txt`、复制
|
||||
`config.example.yaml` → `config.yaml`(若不存在)、通过本机 root socket 建一个
|
||||
**全新的独立 MySQL 数据库与应用账户**,并生成 SFTP 主机密钥。
|
||||
创建 `/etc/apache2/sites-available/f.zikai.wang.conf`(关键部分):
|
||||
|
||||
## 访问方式
|
||||
```apache
|
||||
<VirtualHost *:443>
|
||||
ServerName f.zikai.wang
|
||||
SSLEngine on
|
||||
SSLCertificateFile /etc/letsencrypt/live/f.zikai.wang/fullchain.pem
|
||||
SSLCertificateKeyFile /etc/letsencrypt/live/f.zikai.wang/privkey.pem
|
||||
|
||||
| 入口 | URL |
|
||||
|------|-----|
|
||||
| 状态页(HTML) | https://f.zikai.wang/api/system/status |
|
||||
| 状态页(JSON) | https://f.zikai.wang/api/system/status?format=json(或 `Accept: application/json`) |
|
||||
| API 文档(Swagger) | https://f.zikai.wang/docs **(HTTP Basic Auth -- 见 config.yaml 的 `docs:` 段)** |
|
||||
| API 文档(ReDoc) | https://f.zikai.wang/redoc(同样鉴权) |
|
||||
| 上传页(拖拽、分片、断点续传) | https://f.zikai.wang/upload |
|
||||
| 文件浏览页(列出/下载/删除) | https://f.zikai.wang/files **(Basic Auth,同 docs)** |
|
||||
| 共享白板(实时协作) | https://f.zikai.wang/whiteboard/{id}(公开,`{id}` 为 `[a-zA-Z0-9_-]{1,64}`,不存在则新建) |
|
||||
| 白板管理页 | https://f.zikai.wang/whiteboard-admin **(Basic Auth,同 docs)** |
|
||||
| 上传(curl) | `curl -F file=@big.iso https://f.zikai.wang/api/files/upload` |
|
||||
| SFTP | `sftp -P 2022 uploader@f.zikai.wang` |
|
||||
|
||||
`/api/system/status` 做内容协商:浏览器(`Accept: text/html`)拿到带进度条的可读页面;
|
||||
API 客户端拿到 JSON。可用 `?format=html` 或 `?format=json` 强制指定。
|
||||
|
||||
`/docs`、`/redoc`、`/openapi.json` 需要 HTTP Basic Auth —— 浏览器会弹出登录框。用户名与
|
||||
明文密码写在 `config.yaml` 的 `docs:` 段。`/health` 与 `/` 保持公开。
|
||||
|
||||
Apache(`/etc/apache2/sites-available/f.zikai.wang-le-ssl.conf`)把 `f.zikai.wang` 反代到
|
||||
`127.0.0.1:6867`(`ProxyPreserveHost On`),因此服务只绑 loopback。
|
||||
|
||||
> **大文件/慢速 HTTP 上传:** Apache 代理段继承全局 `Timeout 300`。多 GB 慢链路传输建议走
|
||||
> **分片上传**(`/upload` 页面或 `/api/files/chunk-uploads`,单片 4 MiB 在超时内可传完)或
|
||||
> **SFTP**(完全绕过 HTTP 代理)。要提高 HTTP 上限可在 Apache vhost 加 `ProxyTimeout`/`Timeout`。
|
||||
|
||||
## 配置
|
||||
|
||||
所有运行时配置都在 **`config.yaml`**(git-ignored)。完整 schema 见 `config.example.yaml`。
|
||||
关键配置项:
|
||||
|
||||
- `server` — 绑定 host/port(保持 `127.0.0.1:6867` 以对齐 Apache)。
|
||||
- `database` — host/port/user/password/database。密码由 `setup.sh`/`init_db.py` 自动生成并写回。
|
||||
- `storage.upload_dir`、`storage.chunk_bytes`(默认 1 MiB 流式分片)、`storage.chunk_session_dir`
|
||||
(分片会话暂存目录)、`storage.chunk_session_ttl_seconds`(被放弃会话的存活秒数,默认 300)。
|
||||
- `sftp` — enabled、host/port、host key + authorized_keys 路径、`users`。
|
||||
- `whiteboard` - `heartbeat_interval_seconds`(默认 3)、`heartbeat_miss_threshold`(默认 5)、`max_board_id_length`(默认 64)、`list_limit`(默认 100)。
|
||||
|
||||
### 设置 /docs 管理密码
|
||||
|
||||
直接编辑 `config.yaml`,无需哈希:
|
||||
|
||||
```yaml
|
||||
docs:
|
||||
enabled: true
|
||||
username: admin
|
||||
password: "your-plaintext-password"
|
||||
realm: "zikai docs"
|
||||
ProxyPreserveHost On
|
||||
ProxyPass /fdata !
|
||||
# 所有 zTools2 入口(页面/静态/探针/WS/API)统一在 /api/ 下,一条规则即可;
|
||||
# WebSocket 走 /api/ws/wb/{id},靠 proxy_wstunnel 透传 Upgrade 头
|
||||
ProxyPass /api/ http://127.0.0.1:6867/api/
|
||||
ProxyPassReverse /api/ http://127.0.0.1:6867/api/
|
||||
ProxyTimeout 300
|
||||
</VirtualHost>
|
||||
```
|
||||
|
||||
然后 `./stop.sh && ./start.sh`。该文件 root 持有且仅在本机;比较使用常量时间
|
||||
(`secrets.compare_digest`)。
|
||||
|
||||
### 设置 SFTP 凭据
|
||||
|
||||
**密码鉴权** —— 生成 bcrypt hash 写入 `config.yaml`:
|
||||
|
||||
```bash
|
||||
.venv/bin/python -c "import bcrypt;print(bcrypt.hashpw(b'yourpass',bcrypt.gensalt()).decode())"
|
||||
# 输出粘贴到 sftp.users[].password_hash,然后 ./stop.sh && ./start.sh
|
||||
a2ensite f.zikai.wang
|
||||
systemctl reload apache2
|
||||
```
|
||||
|
||||
**公钥鉴权** —— 把每个客户端的公钥(OpenSSH 格式)追加到 `keys/authorized_keys`(每行一个)。
|
||||
`sftp.users[]` 中的用户随后可用任一方式登录。
|
||||
> **防火墙**:放开 443(HTTPS)与 2022(SFTP)。6867 不对外(仅 loopback)。
|
||||
> **大文件上传**:Apache 全局 `Timeout 300`,慢链路建议走分片上传(`/api/upload`)或 SFTP。
|
||||
|
||||
### 重新生成数据库密码
|
||||
## 从零安装(Ubuntu 22.04+)
|
||||
|
||||
### 1. 安装系统依赖
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m app.scripts.init_db # 生成新随机密码
|
||||
KEEP_DB_PASSWORD=1 .venv/bin/python -m app.scripts.init_db # 保留现有密码
|
||||
apt update
|
||||
apt install -y python3-venv python3-pip mysql-server apache2 \
|
||||
libssl-dev build-essential # build-essential 给 bcrypt/asyncssh 编译
|
||||
# PDF 转换依赖 weasyprint,需 pango/cairo 系统库:
|
||||
apt install -y libpango-1.0-0 libpangoft2-1.0-0 libcairo2 libgdk-pixbuf-2.0-0
|
||||
```
|
||||
|
||||
## SFTP 说明
|
||||
### 2. 获取代码并初始化
|
||||
|
||||
- SFTP 服务无法穿透 Apache 的 HTTP 代理,因此直接绑 `0.0.0.0:2022`。**请在防火墙放开
|
||||
2022 端口** 供外部客户端(FileZilla/WinSCP/scp)连接。
|
||||
- 会话 chroot 到上传根目录(`uploads/`),与 HTTP 共用存储。
|
||||
- 仅 `sftp.users` 中列出的用户可连接;只允许 SFTP(无 shell/exec)。
|
||||
- SFTP 服务器作为文件暂存通道;不再提供 HTTP 登记接口。
|
||||
```bash
|
||||
git clone <repo> /root/zikai
|
||||
cd /root/zikai/zTools2
|
||||
./setup.sh # 创建 .venv + 装依赖 + 复制 config.yaml + 建 MySQL 库账 + 生成 SFTP 密钥
|
||||
```
|
||||
|
||||
## 反向隧道
|
||||
### 3. 配置凭据
|
||||
|
||||
SSH 服务器(2022)同时承载 SFTP 文件暂存与反向隧道。隧道 user 在 `config.yaml` 的
|
||||
`tunnel.users[]` 独立配置(与 `sftp.users[]` 分开):
|
||||
编辑 `config.yaml`(详见 [`docs/configuration.md`](./docs/configuration.md)):`docs.username/password`(管理页 Basic Auth)、`sftp.users[].password_hash`(bcrypt)、可选 `tunnel.users[]`。
|
||||
|
||||
- user 端跑 `user/tunnel.py`,连 2022 请求 remote port forward 绑 `tunnel_port`。
|
||||
- `ZikaiSSHServer.server_requested` 校验该 user 是否允许绑该端口,记一条 `tunnel_session`
|
||||
到 DB(user IP、local_port、tunnel_port、起止时间)。
|
||||
- `GET /api/userPort/{userName}` 查 DB 该 user 活跃隧道的端口,反代到 `127.0.0.1:tunnel_port`
|
||||
(经隧道回指 user 本地服务)。无活跃隧道返回 502。
|
||||
- user 断开时 `connection_lost` 回调关闭 DB 会话(记 `ended_at`);另有启动 reaper 兜底
|
||||
清理进程异常重启后的孤儿记录。
|
||||
### 4. 启动
|
||||
|
||||
配置示例见 `config.example.yaml` 的 `tunnel:` 段。生成 bcrypt hash 的方式同 SFTP。
|
||||
```bash
|
||||
./start.sh # 启动 HTTP(127.0.0.1:6867) + SFTP(0.0.0.0:2022)
|
||||
./stop.sh # 停止
|
||||
```
|
||||
|
||||
## 文件浏览页
|
||||
### 5. 持久化部署(systemd)
|
||||
|
||||
- `GET /files`(Basic Auth,同 docs)渲染 `static/file_browser.html`,JS 调同源管理 API。
|
||||
- 管理 API(均 Basic Auth):
|
||||
- `GET /api/admin/files?limit=&offset=` -> `{total, items:[UploadedFileOut]}`
|
||||
- `GET /api/admin/files/{id}` -> `UploadedFileOut`
|
||||
- `GET /api/admin/files/{id}/download` -> 文件流(磁盘缺失返回 410)
|
||||
- `DELETE /api/admin/files/{id}` -> 硬删除:删 DB 行 + 删磁盘文件(`unlink missing_ok`)。
|
||||
- **删除后不再显示**:列表每次进入或删除后重新 fetch,前端不缓存;DB 行已删,列表自然不含。
|
||||
- 公开 `/api/files` 系列(user.py 依赖的查重/查询/下载)保留不变。
|
||||
```bash
|
||||
./deploy/install-systemd.sh # 安装并启动 ztools2 服务(开机自启)
|
||||
systemctl status ztools2
|
||||
journalctl -u ztools2 -f
|
||||
```
|
||||
|
||||
## 共享白板
|
||||
> 与 zMainPage 整体部署配合:systemd 管后端,`zMainPage/deploy.sh --no-restart` 部署前端。
|
||||
|
||||
白板无鉴权,任何人凭 `/whiteboard/{id}` 即可访问并实时协作;`{id}` 须匹配
|
||||
`[a-zA-Z0-9_-]{1,64}`,非法返回 400。访问不存在的 id 自动新建空板。白板长期留存
|
||||
(存 MySQL `whiteboard` 表),进程重启后内容仍在。
|
||||
## 了解更多
|
||||
|
||||
### 实时同步与心跳
|
||||
|
||||
- 连接:`WS /ws/whiteboard/{id}`(公开)。JSON 文本帧协议:
|
||||
- client -> server:`{"type":"hello","client_id":"..."}`(首帧,可选)、
|
||||
`{"type":"ping"}`(心跳)、`{"type":"stroke","stroke":{points,color,width}}`、`{"type":"clear"}`
|
||||
- server -> client:`{"type":"init","strokes":[...],"stroke_count":n}`、`{"type":"pong"}`、
|
||||
`{"type":"stroke","stroke":{...},"client_id":"..."}`(广播给他人,不含发送者)、
|
||||
`{"type":"cleared","client_id":"..."}`(广播给所有人)、`{"type":"error","msg":"..."}`
|
||||
- **心跳**:客户端每 `whiteboard.heartbeat_interval_seconds`(默认 3s)发一次 `ping`,服务端回 `pong`
|
||||
并刷新计时。后台 reaper 每秒扫描,连续 `heartbeat_miss_threshold`(默认 5)次未收到心跳
|
||||
(即 15s)判失活,**关闭该连接并从 hub 移除**。
|
||||
- **内存安全**:`WhiteboardHub` 维护 `{board_id: set[Connection]}`:
|
||||
- `disconnect` 幂等,空 set 从 dict 删除(防 board 键无限增长);
|
||||
- WS 主循环 `try/finally` 必调 `disconnect`,异常/断连均清理;
|
||||
- `broadcast` 对单连接发送失败立即 `disconnect`,不影响其他连接;
|
||||
- 删除白板时 `close_board` 关闭并清理该 board 的全部连接。
|
||||
- **多 worker 限制**:hub 是进程内存,多 uvicorn worker 下不同进程的连接不互通。生产部署需
|
||||
保持 `server.workers: 1`,或后续接 Redis pub/sub 跨进程广播。
|
||||
|
||||
### 白板管理
|
||||
|
||||
- `GET /whiteboard-admin`(Basic Auth,同 docs)渲染 `static/whiteboard_admin.html`。
|
||||
- 管理 API(均 Basic Auth):
|
||||
- `GET /api/admin/whiteboards?limit=&offset=` -> `{total, items:[{board_id, stroke_count, created_at, updated_at}]}`
|
||||
- `DELETE /api/admin/whiteboards/{id}` -> 删 DB 行 + 关闭该 board 所有在线 WS 连接。
|
||||
|
||||
## 临时文件清理
|
||||
|
||||
- `complete` 成功(含去重命中)后,会话目录 `uploads/.work/<upload_id>/` 立即删除。
|
||||
- 被放弃的上传(`pending` 状态且超过 `chunk_session_ttl_seconds` 无活动,默认 5 分钟)由
|
||||
**后台 reaper** 清理:每 60 秒扫一次,删 `.work/<upload_id>/` 目录 + DB 会话行。
|
||||
- `start.sh` 启动时仍会兜底清掉残留的 `.work/` 与 `*.part`(进程异常退出时的半成品)。
|
||||
|
||||
## 日志与 pidfile
|
||||
|
||||
- HTTP 日志 → `logs/app.log`;SFTP 日志 → `logs/sftp.log`。
|
||||
- pidfile:`app.pid`、`sftp.pid`(`stop.sh` 使用)。
|
||||
- [路由与访问入口](./docs/routes.md)
|
||||
- [配置说明](./docs/configuration.md)
|
||||
- [错误处理与日志约定](./docs/error-handling.md)
|
||||
|
||||
@@ -102,10 +102,30 @@ class WhiteboardConfig(BaseModel):
|
||||
heartbeat_miss_threshold: int = 5
|
||||
# board_id 合法字符集与长度上限,防路径/注入
|
||||
max_board_id_length: int = 64
|
||||
# 单 board 并发连接上限,防资源耗尽(同 board 同时在线人数)
|
||||
max_connections_per_board: int = 50
|
||||
# 列表/管理页分页默认值
|
||||
list_limit: int = 100
|
||||
|
||||
|
||||
class PdfConfig(BaseModel):
|
||||
"""PDF 转换服务配置。
|
||||
|
||||
用户侧(上传/查看/下载/软删)凭 httpOnly cookie 标识;管理侧(列表/硬删)
|
||||
走 docs 同款 Basic Auth。原始文件与产物 PDF 复用 storage.upload_dir 落盘。
|
||||
"""
|
||||
|
||||
# 单文件大小上限(字节)。250 MiB。
|
||||
max_size_bytes: int = 250 * 1024 * 1024
|
||||
# 转换超时(秒):超大/复杂文件兜底,避免长期占用 worker。
|
||||
convert_timeout_seconds: int = 600
|
||||
# 列表分页默认值
|
||||
list_limit: int = 100
|
||||
# 用户 cookie 名与有效期
|
||||
cookie_name: str = "zk_pdf"
|
||||
cookie_max_age_seconds: int = 365 * 24 * 3600
|
||||
|
||||
|
||||
class Settings(BaseModel):
|
||||
server: ServerConfig = ServerConfig()
|
||||
database: DatabaseConfig = DatabaseConfig()
|
||||
@@ -114,6 +134,7 @@ class Settings(BaseModel):
|
||||
docs: DocsConfig = DocsConfig()
|
||||
tunnel: TunnelConfig = TunnelConfig()
|
||||
whiteboard: WhiteboardConfig = WhiteboardConfig()
|
||||
pdf: PdfConfig = PdfConfig()
|
||||
|
||||
def db_url(self) -> str:
|
||||
c = self.database
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from .chunk_upload_controller import router as chunk_upload_router
|
||||
from .file_admin_controller import router as file_admin_router
|
||||
from .file_controller import router as file_router
|
||||
from .pdf_controller import router as pdf_router
|
||||
from .system_controller import router as system_router
|
||||
from .tunnel_controller import router as tunnel_router
|
||||
from .whiteboard_controller import router as whiteboard_router
|
||||
@@ -11,6 +12,7 @@ __all__ = [
|
||||
"chunk_upload_router",
|
||||
"file_admin_router",
|
||||
"file_router",
|
||||
"pdf_router",
|
||||
"system_router",
|
||||
"tunnel_router",
|
||||
"whiteboard_router",
|
||||
|
||||
@@ -14,8 +14,15 @@ from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ..database import get_db
|
||||
from ..dao.pdf_job_dao import PdfJobDAO
|
||||
from ..dao.uploaded_file_dao import UploadedFileDAO
|
||||
from ..schemas.file import FileListResponse, UploadedFileOut
|
||||
from ..schemas.file import (
|
||||
FileListResponse,
|
||||
FileWithPdfListResponse,
|
||||
PdfJobBrief,
|
||||
UploadedFileOut,
|
||||
UploadedFileWithPdfOut,
|
||||
)
|
||||
from ..security import require_docs_auth
|
||||
from ..services.upload_service import UploadService
|
||||
|
||||
@@ -43,7 +50,8 @@ class BatchDeleteResult(BaseModel):
|
||||
"",
|
||||
response_model=FileListResponse,
|
||||
summary="列出已上传文件(需鉴权)",
|
||||
description="供文件浏览页使用;字段同公开 /api/files,但要求 docs Basic Auth。支持分页。",
|
||||
description="供文件浏览页使用;字段同公开 /api/files,但要求 docs Basic Auth。支持分页。"
|
||||
"返回前会触发磁盘扫描(频率限制 3s),把 SFTP 上传但未入库的文件补录进来。",
|
||||
)
|
||||
def list_files(
|
||||
limit: int = Query(100, ge=1, le=10000, description="每页条数,1-10000"),
|
||||
@@ -51,10 +59,58 @@ def list_files(
|
||||
service: UploadService = Depends(_service),
|
||||
_: str = Depends(require_docs_auth),
|
||||
) -> FileListResponse:
|
||||
service.scan_sftp_files() # 频率限制内置:3s 内重复只返回 DB 缓存
|
||||
total, items = service.list_files(limit=limit, offset=offset)
|
||||
return FileListResponse(total=total, items=items)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/with-pdf",
|
||||
response_model=FileWithPdfListResponse,
|
||||
summary="列出已上传文件并附带 PDF 转换任务关联(需鉴权)",
|
||||
description=(
|
||||
"合并管理页使用:以 uploaded_files 为基础分页拉取,再内存匹配 pdf_jobs,"
|
||||
"为每个文件附带它作为 PDF 任务「源文件(epub)」或「产物(PDF)」时的状态、"
|
||||
"进度与用户软删标记。匹配键为 pdf_job.source_file_id / output_file_id -> uploaded_file.id。"
|
||||
"返回前会触发磁盘扫描(频率限制 3s),把 SFTP 上传但未入库的文件补录进来。"
|
||||
),
|
||||
)
|
||||
def list_files_with_pdf(
|
||||
limit: int = Query(100, ge=1, le=10000, description="每页条数,1-10000"),
|
||||
offset: int = Query(0, ge=0, description="偏移量"),
|
||||
service: UploadService = Depends(_service),
|
||||
db: Session = Depends(get_db),
|
||||
_: str = Depends(require_docs_auth),
|
||||
) -> FileWithPdfListResponse:
|
||||
service.scan_sftp_files() # 频率限制内置:3s 内重复只返回 DB 缓存
|
||||
total, items = service.list_files(limit=limit, offset=offset)
|
||||
|
||||
# 拉全部 pdf_jobs(数据量小,不分页),建反查 map
|
||||
jobs = PdfJobDAO(db).list_all(limit=10000, offset=0)
|
||||
by_source: dict[int, list] = {}
|
||||
by_output: dict[int, list] = {}
|
||||
for j in jobs:
|
||||
by_source.setdefault(j.source_file_id, []).append(j)
|
||||
if j.output_file_id is not None:
|
||||
by_output.setdefault(j.output_file_id, []).append(j)
|
||||
|
||||
out_items: list[UploadedFileWithPdfOut] = []
|
||||
for f in items:
|
||||
briefs: list[PdfJobBrief] = []
|
||||
for j in by_source.get(f.id, []):
|
||||
briefs.append(PdfJobBrief(
|
||||
job_id=j.id, role="source", status=j.status,
|
||||
progress=j.progress, user_deleted=j.user_deleted, deleted_at=j.deleted_at,
|
||||
))
|
||||
for j in by_output.get(f.id, []):
|
||||
briefs.append(PdfJobBrief(
|
||||
job_id=j.id, role="output", status=j.status,
|
||||
progress=j.progress, user_deleted=j.user_deleted, deleted_at=j.deleted_at,
|
||||
))
|
||||
out_items.append(UploadedFileWithPdfOut(**f.model_dump(), pdf_jobs=briefs))
|
||||
return FileWithPdfListResponse(total=total, items=out_items)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/batch-delete",
|
||||
response_model=BatchDeleteResult,
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from fastapi import APIRouter, Depends, File, HTTPException, UploadFile
|
||||
from fastapi import APIRouter, Depends, File, HTTPException, Query, UploadFile
|
||||
from fastapi.responses import FileResponse
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
@@ -43,8 +43,8 @@ async def upload_file(
|
||||
|
||||
@router.get("", response_model=FileListResponse, summary="列出已上传的文件")
|
||||
def list_files(
|
||||
limit: int = 100,
|
||||
offset: int = 0,
|
||||
limit: int = Query(100, ge=1, le=10000),
|
||||
offset: int = Query(0, ge=0),
|
||||
service: UploadService = Depends(_service),
|
||||
) -> FileListResponse:
|
||||
total, items = service.list_files(limit=limit, offset=offset)
|
||||
|
||||
181
app/controllers/pdf_controller.py
Normal file
181
app/controllers/pdf_controller.py
Normal file
@@ -0,0 +1,181 @@
|
||||
"""PDF 转换接口:用户侧(cookie 标识)+ 管理侧(Basic Auth)。
|
||||
|
||||
路由:
|
||||
POST /api/pdf/jobs 用户上传文件并提交转换(首次无 cookie 则下发)
|
||||
GET /api/pdf/jobs 当前用户任务列表(仅未软删)
|
||||
GET /api/pdf/jobs/{id} 单任务状态(轮询进度)
|
||||
GET /api/pdf/jobs/{id}/download 下载产物 PDF
|
||||
DELETE /api/pdf/jobs/{id} 用户软删(不再对用户展示,管理页仍可见)
|
||||
GET /api/admin/pdf/jobs 管理页列表(全部,含已软删标记)
|
||||
DELETE /api/admin/pdf/jobs/{id} 管理员硬删(真正删除磁盘与 DB)
|
||||
|
||||
HTML 页面 /pdf(用户)与 /pdf-admin(管理)由 main.py 返回静态文件,
|
||||
不在此 controller 注册,避免与 REST 同路径冲突。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from fastapi import APIRouter, Cookie, Depends, File, HTTPException, Query, Request, UploadFile
|
||||
from fastapi.responses import FileResponse, JSONResponse
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ..config import get_settings
|
||||
from ..database import get_db
|
||||
from ..dao.pdf_job_dao import PdfJobDAO
|
||||
from ..dao.uploaded_file_dao import UploadedFileDAO
|
||||
from ..schemas.pdf import DeleteResult, PdfJobListResponse, PdfJobOut, PdfSubmitResponse
|
||||
from ..security import require_docs_auth
|
||||
from ..services.pdf_service import PdfService, new_owner_cookie
|
||||
|
||||
router = APIRouter(tags=["pdf"])
|
||||
|
||||
|
||||
def _service(db: Session = Depends(get_db)) -> PdfService:
|
||||
return PdfService(PdfJobDAO(db), UploadedFileDAO(db))
|
||||
|
||||
|
||||
def _resolve_cookie(request: Request, zk_pdf: str | None = Cookie(default=None)) -> str:
|
||||
"""解析用户 cookie;无则生成新值(由 controller 写入响应头)。
|
||||
|
||||
cookie 缺失时把新值挂到 request.state,供响应阶段 set_cookie。
|
||||
"""
|
||||
if zk_pdf and len(zk_pdf) == 32:
|
||||
return zk_pdf
|
||||
new = new_owner_cookie()
|
||||
request.state.new_pdf_cookie = new
|
||||
return new
|
||||
|
||||
|
||||
@router.post(
|
||||
"/api/pdf/jobs",
|
||||
response_model=PdfSubmitResponse,
|
||||
summary="上传文件并提交 PDF 转换",
|
||||
description=(
|
||||
"multipart/form-data 上传 epub 文件(≤250MB),流式落盘后创建 pending 转换任务,"
|
||||
"后台异步转换。用户凭 zk_pdf cookie 标识;首次无 cookie 时响应下发新 cookie。"
|
||||
),
|
||||
)
|
||||
async def submit_job(
|
||||
request: Request,
|
||||
file: UploadFile = File(..., description="要转换的 epub 文件"),
|
||||
service: PdfService = Depends(_service),
|
||||
owner_cookie: str = Depends(_resolve_cookie),
|
||||
) -> PdfSubmitResponse:
|
||||
job, _source_id = service.submit(file, owner_cookie)
|
||||
# 提交成功后触发后台转换
|
||||
service.schedule_convert(job.id)
|
||||
resp = PdfSubmitResponse(job=job, set_cookie=False)
|
||||
new_cookie = getattr(request.state, "new_pdf_cookie", None)
|
||||
if new_cookie:
|
||||
resp.set_cookie = True
|
||||
# 用 JSONResponse 显式 set_cookie 后返回模型体
|
||||
cfg = get_settings().pdf
|
||||
data = resp.model_dump(mode="json")
|
||||
response = JSONResponse(data)
|
||||
response.set_cookie(
|
||||
key=cfg.cookie_name,
|
||||
value=new_cookie,
|
||||
max_age=cfg.cookie_max_age_seconds,
|
||||
httponly=True,
|
||||
samesite="lax",
|
||||
path="/",
|
||||
)
|
||||
return response
|
||||
return resp
|
||||
|
||||
|
||||
@router.get(
|
||||
"/api/pdf/jobs",
|
||||
response_model=PdfJobListResponse,
|
||||
summary="当前用户的任务列表",
|
||||
description="按 zk_pdf cookie 返回该用户未软删的任务,按创建时间倒序。",
|
||||
)
|
||||
def list_jobs(
|
||||
service: PdfService = Depends(_service),
|
||||
owner_cookie: str = Depends(_resolve_cookie),
|
||||
limit: int = Query(100, ge=1, le=500),
|
||||
offset: int = Query(0, ge=0),
|
||||
) -> PdfJobListResponse:
|
||||
return service.list_for_user(owner_cookie, limit=limit, offset=offset)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/api/pdf/jobs/{job_id}",
|
||||
response_model=PdfJobOut,
|
||||
summary="查询单任务状态(轮询进度)",
|
||||
)
|
||||
def get_job(
|
||||
job_id: int,
|
||||
service: PdfService = Depends(_service),
|
||||
owner_cookie: str = Depends(_resolve_cookie),
|
||||
) -> PdfJobOut:
|
||||
return service.get_job(job_id, owner_cookie)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/api/pdf/jobs/{job_id}/download",
|
||||
summary="下载转换后的 PDF",
|
||||
description="仅任务状态为 done 且归属本人未软删时可下载。",
|
||||
)
|
||||
def download_job(
|
||||
job_id: int,
|
||||
service: PdfService = Depends(_service),
|
||||
owner_cookie: str = Depends(_resolve_cookie),
|
||||
) -> FileResponse:
|
||||
_job, path, filename = service.get_output_path(job_id, owner_cookie)
|
||||
return FileResponse(
|
||||
path=str(path),
|
||||
media_type="application/pdf",
|
||||
filename=filename,
|
||||
)
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/api/pdf/jobs/{job_id}",
|
||||
response_model=DeleteResult,
|
||||
summary="用户软删任务(不再对用户展示)",
|
||||
description="仅置 user_deleted 标记,磁盘与 DB 行保留;管理页仍可见并标注已删除。",
|
||||
)
|
||||
def delete_job(
|
||||
job_id: int,
|
||||
service: PdfService = Depends(_service),
|
||||
owner_cookie: str = Depends(_resolve_cookie),
|
||||
) -> DeleteResult:
|
||||
ok = service.user_delete(job_id, owner_cookie)
|
||||
if not ok:
|
||||
raise HTTPException(404, "任务不存在")
|
||||
return DeleteResult(deleted=True)
|
||||
|
||||
|
||||
# ---------------- 管理侧(Basic Auth) ----------------
|
||||
|
||||
@router.get(
|
||||
"/api/admin/pdf/jobs",
|
||||
response_model=PdfJobListResponse,
|
||||
summary="列出全部转换任务(需鉴权)",
|
||||
description="管理页使用:含 user_deleted 标记,可看到用户是否已软删。",
|
||||
)
|
||||
def admin_list_jobs(
|
||||
service: PdfService = Depends(_service),
|
||||
_: str = Depends(require_docs_auth),
|
||||
limit: int = Query(100, ge=1, le=500),
|
||||
offset: int = Query(0, ge=0),
|
||||
) -> PdfJobListResponse:
|
||||
return service.list_for_admin(limit=limit, offset=offset)
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/api/admin/pdf/jobs/{job_id}",
|
||||
response_model=DeleteResult,
|
||||
summary="管理员硬删任务(真正删除)",
|
||||
description="删原始/产物磁盘文件 + UploadedFile 行 + PdfJob 行,不可恢复。",
|
||||
)
|
||||
def admin_delete_job(
|
||||
job_id: int,
|
||||
service: PdfService = Depends(_service),
|
||||
_: str = Depends(require_docs_auth),
|
||||
) -> DeleteResult:
|
||||
ok = service.admin_delete(job_id)
|
||||
if not ok:
|
||||
raise HTTPException(404, "任务不存在")
|
||||
return DeleteResult(deleted=True)
|
||||
@@ -1,10 +1,13 @@
|
||||
"""白板接口:REST(访问/管理)+ WebSocket(实时同步)。
|
||||
|
||||
路由:
|
||||
GET /whiteboard/{board_id} 公开:访问白板,不存在则新建
|
||||
WS /ws/whiteboard/{board_id} 公开:实时协作 + 心跳
|
||||
GET /api/admin/whiteboards Basic Auth:管理页列表
|
||||
DELETE /api/admin/whiteboards/{id} Basic Auth:删除白板
|
||||
GET /api/wb/{board_id} 公开:访问记事本元数据,不存在则新建(前端 init 用)
|
||||
WS /ws/wb/{board_id} 公开:实时协作 + 心跳
|
||||
GET /api/admin/wb Basic Auth:管理页列表
|
||||
DELETE /api/admin/wb/{board_id} Basic Auth:删除记事本
|
||||
|
||||
HTML 页面 /wb/{id} 与管理页 /wb-admin 由 main.py 直接返回静态文件,
|
||||
不在此 controller 注册,避免与 REST 同路径冲突。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -29,17 +32,17 @@ router = APIRouter(tags=["whiteboard"])
|
||||
|
||||
|
||||
def _service(db: Session = Depends(get_db)) -> WhiteboardService:
|
||||
"""REST 路径的 service:注入 hub 以便删除时踢出连接。"""
|
||||
return WhiteboardService(WhiteboardDAO(db), hub=get_hub())
|
||||
"""REST 路径的 service(纯 DB 操作)。"""
|
||||
return WhiteboardService(WhiteboardDAO(db))
|
||||
|
||||
|
||||
# ---------------- 公开 REST ----------------
|
||||
|
||||
@router.get(
|
||||
"/whiteboard/{board_id}",
|
||||
"/api/wb/{board_id}",
|
||||
response_model=WhiteboardOut,
|
||||
summary="访问白板(不存在则新建)",
|
||||
description="任何人凭 board_id 即可访问;不存在时自动创建空板并返回。",
|
||||
summary="访问记事本元数据(不存在则新建)",
|
||||
description="前端打开 /wb/{id} 页面后调本接口拉取初始文本;不存在时自动创建空板。",
|
||||
)
|
||||
def get_whiteboard(board_id: str, service: WhiteboardService = Depends(_service)) -> WhiteboardOut:
|
||||
return service.get_or_create(board_id)
|
||||
@@ -48,10 +51,10 @@ def get_whiteboard(board_id: str, service: WhiteboardService = Depends(_service)
|
||||
# ---------------- 管理 REST(Basic Auth) ----------------
|
||||
|
||||
@router.get(
|
||||
"/api/admin/whiteboards",
|
||||
"/api/admin/wb",
|
||||
response_model=WhiteboardListResponse,
|
||||
summary="列出所有白板(需鉴权)",
|
||||
description="供白板管理页使用:board_id / 创建时间 / 修改次数 / 上次修改时间。",
|
||||
summary="列出所有记事本(需鉴权)",
|
||||
description="供记事本管理页使用:board_id / 创建时间 / 编辑次数 / 上次修改时间。",
|
||||
)
|
||||
def list_whiteboards(
|
||||
limit: int = Query(100, ge=1, le=500),
|
||||
@@ -64,11 +67,11 @@ def list_whiteboards(
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/api/admin/whiteboards/{board_id}",
|
||||
summary="删除白板(需鉴权)",
|
||||
"/api/admin/wb/{board_id}",
|
||||
summary="删除记事本(需鉴权)",
|
||||
description="删 DB 行,并关闭该 board 的所有在线 WebSocket 连接。",
|
||||
)
|
||||
def delete_whiteboard(
|
||||
async def delete_whiteboard(
|
||||
board_id: str,
|
||||
service: WhiteboardService = Depends(_service),
|
||||
_: str = Depends(require_docs_auth),
|
||||
@@ -76,25 +79,27 @@ def delete_whiteboard(
|
||||
ok = service.delete(board_id)
|
||||
if not ok:
|
||||
raise HTTPException(404, "白板不存在")
|
||||
# 删除成功后踢出该 board 的所有在线连接(close_board 是 async,须在事件循环中调用)
|
||||
await get_hub().close_board(board_id)
|
||||
return {"deleted": True}
|
||||
|
||||
|
||||
# ---------------- WebSocket(公开,实时同步 + 心跳) ----------------
|
||||
|
||||
@router.websocket("/ws/whiteboard/{board_id}")
|
||||
@router.websocket("/api/ws/wb/{board_id}")
|
||||
async def whiteboard_ws(websocket: WebSocket, board_id: str) -> None:
|
||||
"""白板实时协作端点。
|
||||
"""白板实时协作端点(文本记事本)。
|
||||
|
||||
协议(JSON 文本帧):
|
||||
client -> server:
|
||||
{"type":"hello","client_id":"..."} 首帧(可选,未带则服务端生成)
|
||||
{"type":"ping"} 心跳,服务端回 pong 并刷新计时
|
||||
{"type":"stroke","stroke":{...}} 新增笔画,持久化并广播给他人
|
||||
{"type":"edit","content":"..."} debounce 后发完整文本,持久化并广播给他人
|
||||
{"type":"clear"} 清空,持久化并广播给所有人
|
||||
server -> client:
|
||||
{"type":"init","strokes":[...],"stroke_count":n}
|
||||
{"type":"init","content":"...","version":n,"edit_count":m}
|
||||
{"type":"pong"}
|
||||
{"type":"stroke","stroke":{...},"client_id":"..."}
|
||||
{"type":"update","content":"...","version":n,"client_id":"..."} 文本变更
|
||||
{"type":"cleared","client_id":"..."}
|
||||
{"type":"error","msg":"..."}
|
||||
"""
|
||||
@@ -112,9 +117,8 @@ async def whiteboard_ws(websocket: WebSocket, board_id: str) -> None:
|
||||
client_id = _extract_client_id(first) or uuid.uuid4().hex[:12]
|
||||
|
||||
# 校验 board_id 并加载白板(不存在则新建)
|
||||
from ..database import get_session_local
|
||||
try:
|
||||
board = _with_db(lambda db: WhiteboardService(WhiteboardDAO(db), hub=hub).get_or_create(board_id))
|
||||
board = _with_db(lambda db: WhiteboardService(WhiteboardDAO(db)).get_or_create(board_id))
|
||||
except HTTPException as exc:
|
||||
await _safe_send(websocket, {"type": "error", "msg": exc.detail})
|
||||
await _safe_close(websocket)
|
||||
@@ -122,17 +126,27 @@ async def whiteboard_ws(websocket: WebSocket, board_id: str) -> None:
|
||||
|
||||
# 注册连接并下发 init
|
||||
conn = Connection(websocket=websocket, board_id=board_id, client_id=client_id)
|
||||
await hub.register(conn)
|
||||
ok = await hub.register(conn)
|
||||
if not ok:
|
||||
await _safe_send(websocket, {"type": "error", "msg": "该记事本在线人数已满"})
|
||||
await _safe_close(websocket)
|
||||
return
|
||||
await _safe_send(websocket, {
|
||||
"type": "init",
|
||||
"strokes": board.strokes,
|
||||
"stroke_count": board.stroke_count,
|
||||
"content": board.content,
|
||||
"version": board.version,
|
||||
"edit_count": board.edit_count,
|
||||
})
|
||||
|
||||
# 主循环:收消息 -> 处理 -> 广播
|
||||
# 单帧大小上限:与 content 限制对齐(256KB 文本 + JSON 开销,留余量到 512KB)
|
||||
MAX_FRAME = 512 * 1024
|
||||
try:
|
||||
while True:
|
||||
raw = await websocket.receive_text()
|
||||
if len(raw) > MAX_FRAME:
|
||||
await _safe_send(websocket, {"type": "error", "msg": "消息过大"})
|
||||
continue
|
||||
msg = _parse(raw)
|
||||
if msg is None:
|
||||
continue
|
||||
@@ -143,22 +157,27 @@ async def whiteboard_ws(websocket: WebSocket, board_id: str) -> None:
|
||||
continue
|
||||
# 任何有效业务帧都视为活性证据
|
||||
conn.touch()
|
||||
if mtype == "stroke":
|
||||
stroke = msg.get("stroke") or {}
|
||||
if mtype == "edit":
|
||||
content = msg.get("content")
|
||||
if not isinstance(content, str):
|
||||
await _safe_send(websocket, {"type": "error", "msg": "content 必须是字符串"})
|
||||
continue
|
||||
try:
|
||||
_with_db(lambda db: WhiteboardService(WhiteboardDAO(db), hub=hub).append_stroke(board_id, stroke))
|
||||
out = _with_db(
|
||||
lambda db: WhiteboardService(WhiteboardDAO(db)).update_content(board_id, content)
|
||||
)
|
||||
except HTTPException as exc:
|
||||
await _safe_send(websocket, {"type": "error", "msg": exc.detail})
|
||||
continue
|
||||
# 广播给他人(发送者本地已画,不回推)
|
||||
# 广播给他人(发送者本地已更新,不回推)
|
||||
await hub.broadcast(
|
||||
board_id,
|
||||
{"type": "stroke", "stroke": stroke, "client_id": client_id},
|
||||
{"type": "update", "content": out.content, "version": out.version, "client_id": client_id},
|
||||
exclude=conn,
|
||||
)
|
||||
elif mtype == "clear":
|
||||
try:
|
||||
_with_db(lambda db: WhiteboardService(WhiteboardDAO(db), hub=hub).clear(board_id))
|
||||
_with_db(lambda db: WhiteboardService(WhiteboardDAO(db)).clear(board_id))
|
||||
except HTTPException as exc:
|
||||
await _safe_send(websocket, {"type": "error", "msg": exc.detail})
|
||||
continue
|
||||
@@ -209,12 +228,12 @@ def _parse(raw: str) -> dict | None:
|
||||
async def _safe_send(ws: WebSocket, msg: dict) -> None:
|
||||
try:
|
||||
await ws.send_json(msg)
|
||||
except Exception: # pragma: no cover
|
||||
pass
|
||||
except Exception as exc: # pragma: no cover
|
||||
logger.debug("发送 WS 消息失败: %s", exc)
|
||||
|
||||
|
||||
async def _safe_close(ws: WebSocket) -> None:
|
||||
try:
|
||||
await ws.close()
|
||||
except Exception: # pragma: no cover
|
||||
pass
|
||||
except Exception as exc: # pragma: no cover
|
||||
logger.debug("关闭 WS 失败: %s", exc)
|
||||
|
||||
71
app/dao/pdf_job_dao.py
Normal file
71
app/dao/pdf_job_dao.py
Normal file
@@ -0,0 +1,71 @@
|
||||
"""PdfJob 的 DAO。
|
||||
|
||||
所有写操作均在该层 commit,service 不直接操作 session。
|
||||
用户/管理两条查询路径分别按 user_deleted 过滤。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ..models.pdf_job import PdfJob
|
||||
|
||||
|
||||
class PdfJobDAO:
|
||||
def __init__(self, db: Session) -> None:
|
||||
self.db = db
|
||||
|
||||
def create(self, job: PdfJob) -> PdfJob:
|
||||
self.db.add(job)
|
||||
self.db.commit()
|
||||
self.db.refresh(job)
|
||||
return job
|
||||
|
||||
def get(self, job_id: int) -> PdfJob | None:
|
||||
return self.db.get(PdfJob, job_id)
|
||||
|
||||
def update(self, job: PdfJob) -> PdfJob:
|
||||
"""提交对 job 的就地修改并刷新。"""
|
||||
self.db.commit()
|
||||
self.db.refresh(job)
|
||||
return job
|
||||
|
||||
def delete(self, job: PdfJob) -> None:
|
||||
"""硬删 PdfJob 行(管理视角真正删除,不可恢复)。"""
|
||||
self.db.delete(job)
|
||||
self.db.commit()
|
||||
|
||||
def list_for_user(self, owner_cookie: str, limit: int = 100, offset: int = 0) -> list[PdfJob]:
|
||||
"""用户视角:仅未软删的任务,按创建时间倒序。"""
|
||||
stmt = (
|
||||
select(PdfJob)
|
||||
.where(PdfJob.owner_cookie == owner_cookie)
|
||||
.where(PdfJob.user_deleted == False) # noqa: E712
|
||||
.order_by(PdfJob.created_at.desc())
|
||||
.limit(limit)
|
||||
.offset(offset)
|
||||
)
|
||||
return list(self.db.scalars(stmt).all())
|
||||
|
||||
def count_for_user(self, owner_cookie: str) -> int:
|
||||
stmt = (
|
||||
select(func.count())
|
||||
.select_from(PdfJob)
|
||||
.where(PdfJob.owner_cookie == owner_cookie)
|
||||
.where(PdfJob.user_deleted == False) # noqa: E712
|
||||
)
|
||||
return self.db.scalar(stmt) or 0
|
||||
|
||||
def list_all(self, limit: int = 100, offset: int = 0) -> list[PdfJob]:
|
||||
"""管理视角:全部任务(含已软删),按创建时间倒序。"""
|
||||
stmt = (
|
||||
select(PdfJob)
|
||||
.order_by(PdfJob.created_at.desc())
|
||||
.limit(limit)
|
||||
.offset(offset)
|
||||
)
|
||||
return list(self.db.scalars(stmt).all())
|
||||
|
||||
def count_all(self) -> int:
|
||||
return self.db.scalar(select(func.count()).select_from(PdfJob)) or 0
|
||||
@@ -31,15 +31,6 @@ class TunnelSessionDAO:
|
||||
)
|
||||
return self.db.scalars(stmt).first()
|
||||
|
||||
def get_active_by_port(self, tunnel_port: int) -> TunnelSession | None:
|
||||
stmt = (
|
||||
select(TunnelSession)
|
||||
.where(TunnelSession.tunnel_port == tunnel_port)
|
||||
.where(TunnelSession.status == "active")
|
||||
.limit(1)
|
||||
)
|
||||
return self.db.scalars(stmt).first()
|
||||
|
||||
def list_active(self) -> list[TunnelSession]:
|
||||
stmt = select(TunnelSession).where(TunnelSession.status == "active")
|
||||
return list(self.db.scalars(stmt).all())
|
||||
|
||||
@@ -29,6 +29,11 @@ class UploadedFileDAO:
|
||||
"""返回数据库中文件总条数。"""
|
||||
return self.db.scalar(select(func.count()).select_from(UploadedFile)) or 0
|
||||
|
||||
def list_storage_paths(self) -> set[str]:
|
||||
"""返回所有已记录的 storage_path 集合(供磁盘扫描比对,识别未入库文件)。"""
|
||||
stmt = select(UploadedFile.storage_path)
|
||||
return {row for row in self.db.scalars(stmt).all()}
|
||||
|
||||
def list(self, limit: int = 100, offset: int = 0) -> list[UploadedFile]:
|
||||
stmt = (
|
||||
select(UploadedFile)
|
||||
|
||||
@@ -1,18 +1,21 @@
|
||||
"""Whiteboard 的 DAO。
|
||||
"""Whiteboard 的 DAO(文本记事本)。
|
||||
|
||||
所有写操作均在该层 commit,service 不直接操作 session。
|
||||
get_or_create 用于「访问即新建」语义(路由 GET /whiteboard/{id} 不存在则建)。
|
||||
get_or_create 用于「访问即新建」语义(路由 GET /api/wb/{id} 不存在则建)。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
import logging
|
||||
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ..models.whiteboard import Whiteboard
|
||||
|
||||
logger = logging.getLogger("zikai.whiteboard")
|
||||
|
||||
|
||||
class WhiteboardDAO:
|
||||
def __init__(self, db: Session) -> None:
|
||||
@@ -29,36 +32,30 @@ class WhiteboardDAO:
|
||||
return self.db.scalars(stmt).first()
|
||||
|
||||
def get_or_create(self, board_id: str) -> Whiteboard:
|
||||
"""存在则返回,否则新建空板。利用 unique 约束兜底并发首访。"""
|
||||
"""存在则返回,否则新建空板。利用 unique 约束兜底并发首访。
|
||||
|
||||
仅 IntegrityError(并发下另一事务已插入违反唯一约束)才回滚重读;
|
||||
其他异常向上抛,避免掩盖 schema/连接等真实故障。
|
||||
"""
|
||||
board = self.get(board_id)
|
||||
if board is not None:
|
||||
return board
|
||||
board = Whiteboard(board_id=board_id, strokes=[], stroke_count=0)
|
||||
board = Whiteboard(board_id=board_id, content="", version=0, edit_count=0)
|
||||
try:
|
||||
return self.create(board)
|
||||
except Exception:
|
||||
except IntegrityError:
|
||||
# 并发下另一事务已插入:回滚后重新读
|
||||
self.db.rollback()
|
||||
return self.get(board_id) # type: ignore[return-value]
|
||||
|
||||
def append_strokes(self, board_id: str, new_strokes: list[Any]) -> Whiteboard | None:
|
||||
"""把新笔画追加到 strokes 数组尾部,stroke_count 自增。"""
|
||||
def update_content(self, board_id: str, content: str) -> Whiteboard | None:
|
||||
"""整体替换文本内容,version +1、edit_count +1。"""
|
||||
board = self.get(board_id)
|
||||
if board is None:
|
||||
return None
|
||||
board.strokes = [*board.strokes, *new_strokes]
|
||||
board.stroke_count = (board.stroke_count or 0) + len(new_strokes)
|
||||
self.db.commit()
|
||||
self.db.refresh(board)
|
||||
return board
|
||||
|
||||
def replace_strokes(self, board_id: str, strokes: list[Any]) -> Whiteboard | None:
|
||||
"""整体替换 strokes(清空时传 []),stroke_count 自增 1。"""
|
||||
board = self.get(board_id)
|
||||
if board is None:
|
||||
return None
|
||||
board.strokes = list(strokes)
|
||||
board.stroke_count = (board.stroke_count or 0) + 1
|
||||
board.content = content
|
||||
board.version = (board.version or 0) + 1
|
||||
board.edit_count = (board.edit_count or 0) + 1
|
||||
self.db.commit()
|
||||
self.db.refresh(board)
|
||||
return board
|
||||
|
||||
@@ -64,7 +64,29 @@ def get_db() -> Generator[Session, None, None]:
|
||||
|
||||
|
||||
def init_db_schema() -> None:
|
||||
"""按需建表(幂等)。先导入 models 以注册映射。"""
|
||||
"""按需建表(幂等)并校验既有表列与模型一致(fail-fast on schema drift)。
|
||||
|
||||
先导入 models 注册映射;create_all 用 IF NOT EXISTS 仅补缺失的表;
|
||||
随后对每张已存在的表检查模型声明的列是否齐全,缺列即抛 RuntimeError,
|
||||
避免运行期才以晦涩的 OperationalError 暴露 schema 漂移。
|
||||
"""
|
||||
from sqlalchemy import inspect
|
||||
|
||||
from . import models # noqa: F401
|
||||
get_engine()
|
||||
Base.metadata.create_all(bind=_engine)
|
||||
engine = get_engine()
|
||||
Base.metadata.create_all(bind=engine)
|
||||
|
||||
inspector = inspect(engine)
|
||||
missing: list[str] = []
|
||||
for table, mapper in Base.registry.mappers.items():
|
||||
if not inspector.has_table(table):
|
||||
continue
|
||||
db_cols = {c["name"] for c in inspector.get_columns(table)}
|
||||
for model_col in mapper.columns.keys():
|
||||
if model_col not in db_cols:
|
||||
missing.append(f"{table}.{model_col}")
|
||||
if missing:
|
||||
raise RuntimeError(
|
||||
"数据库 schema 与模型不一致,缺少列: " + ", ".join(missing)
|
||||
+ "。请执行 sql/schema.sql 或迁移脚本更新表结构。"
|
||||
)
|
||||
|
||||
77
app/main.py
77
app/main.py
@@ -6,12 +6,13 @@
|
||||
GET /redoc -> ReDoc (Basic Auth)
|
||||
GET /openapi.json -> OpenAPI 文档(Basic Auth)
|
||||
GET /health -> 存活探针(公开)
|
||||
GET /upload -> 上传页面(公开 HTML)
|
||||
GET /files -> 文件浏览页(Basic Auth,同 docs)
|
||||
GET /whiteboard/{id} -> 白板页面(公开,不存在则新建)
|
||||
GET /whiteboard-admin -> 白板管理页(Basic Auth,同 docs)
|
||||
GET /api/index -> 导航页(公开,收集所有页面入口)
|
||||
GET /api/upload -> 上传页面(公开 HTML)
|
||||
GET /api/files-page -> 文件管理页(Basic Auth,同 docs;含 PDF 转换管理)
|
||||
GET /api/wb/{id} -> 白板页面(公开,不存在则新建)
|
||||
GET /api/wb-admin -> 白板管理页(Basic Auth,同 docs)
|
||||
GET /api/... -> 业务接口
|
||||
WS /ws/whiteboard/{id} -> 白板实时同步(公开)
|
||||
WS /ws/wb/{id} -> 白板实时同步(公开)
|
||||
/static/... -> 前端静态资源(JS/CSS)
|
||||
"""
|
||||
|
||||
@@ -31,6 +32,7 @@ from .controllers import (
|
||||
chunk_upload_router,
|
||||
file_admin_router,
|
||||
file_router,
|
||||
pdf_router,
|
||||
system_router,
|
||||
tunnel_router,
|
||||
whiteboard_router,
|
||||
@@ -149,54 +151,91 @@ def create_app() -> FastAPI:
|
||||
app.include_router(chunk_upload_router)
|
||||
app.include_router(tunnel_router)
|
||||
app.include_router(whiteboard_router)
|
||||
app.include_router(pdf_router)
|
||||
|
||||
# 前端静态资源(JS/CSS);HTML 壳由下面的具名路由返回,便于各自挂 Basic Auth
|
||||
# 统一 /api/ 前缀:所有 zTools2 入口(页面/静态/探针/WS/API)都在 /api/ 下,
|
||||
# 反代与 vite proxy 只需一条 /api/ 规则即可转发,与环境无关
|
||||
if _STATIC_DIR.is_dir():
|
||||
app.mount("/static", StaticFiles(directory=str(_STATIC_DIR)), name="static")
|
||||
app.mount("/api/static", StaticFiles(directory=str(_STATIC_DIR)), name="static")
|
||||
|
||||
# 受 Basic Auth 保护的文档接口
|
||||
@app.get("/openapi.json")
|
||||
@app.get("/openapi.json", tags=["docs"], summary="OpenAPI 文档(需鉴权)")
|
||||
def protected_openapi(_: str = Depends(require_docs_auth)) -> JSONResponse:
|
||||
return JSONResponse(app.openapi())
|
||||
|
||||
@app.get("/docs")
|
||||
@app.get("/docs", tags=["docs"], summary="Swagger UI(需鉴权)")
|
||||
def protected_docs(_: str = Depends(require_docs_auth)):
|
||||
return get_swagger_ui_html(
|
||||
openapi_url="/openapi.json", title="zikai docs", swagger_favicon_url=""
|
||||
)
|
||||
|
||||
@app.get("/redoc")
|
||||
@app.get("/redoc", tags=["docs"], summary="ReDoc(需鉴权)")
|
||||
def protected_redoc(_: str = Depends(require_docs_auth)):
|
||||
return get_redoc_html(
|
||||
openapi_url="/openapi.json", title="zikai docs", redoc_favicon_url=""
|
||||
)
|
||||
|
||||
@app.get("/", response_class=PlainTextResponse)
|
||||
@app.get("/", tags=["meta"], summary="版本号")
|
||||
def root() -> PlainTextResponse:
|
||||
return PlainTextResponse(f"zikai {app.version}\n")
|
||||
|
||||
@app.get("/health")
|
||||
@app.get("/api/health", tags=["meta"], summary="存活探针")
|
||||
def health() -> dict:
|
||||
return {"status": "ok"}
|
||||
|
||||
@app.get("/upload", response_class=HTMLResponse)
|
||||
@app.get(
|
||||
"/api/index",
|
||||
response_class=HTMLResponse,
|
||||
tags=["pages"],
|
||||
summary="导航页(公开)",
|
||||
description="收集所有页面入口的卡片式导航页,各子页脚注可返回此处。",
|
||||
)
|
||||
def index_page() -> HTMLResponse:
|
||||
return _serve_static_html("index.html")
|
||||
|
||||
@app.get(
|
||||
"/api/upload",
|
||||
response_class=HTMLResponse,
|
||||
tags=["pages"],
|
||||
summary="上传页面",
|
||||
description="拖拽 / 多文件 / 分片(4 MiB) / 断点续传上传页面(公开)。",
|
||||
)
|
||||
def upload_page() -> HTMLResponse:
|
||||
"""拖拽 / 多文件 / 分片上传页面(公开,对齐 /api/files/upload)。"""
|
||||
return HTMLResponse(render_upload_html())
|
||||
|
||||
@app.get("/files", response_class=HTMLResponse)
|
||||
@app.get(
|
||||
"/api/files-page",
|
||||
response_class=HTMLResponse,
|
||||
tags=["pages"],
|
||||
summary="文件管理页(需鉴权)",
|
||||
description=(
|
||||
"列出 / 下载 / 删除已上传文件,并合并 PDF 转换管理:以 uploaded_files 为基础,"
|
||||
"用 pdf_jobs 匹配标注关联文件的转换状态、用户软删标记,可硬删任务。"
|
||||
"支持多选、批量下载删除与分页。Basic Auth 同 docs。"
|
||||
),
|
||||
)
|
||||
def files_page(_: str = Depends(require_docs_auth)) -> HTMLResponse:
|
||||
"""文件浏览页(Basic Auth,同 docs):列出/下载/删除已上传文件。"""
|
||||
return _serve_static_html("file_browser.html")
|
||||
|
||||
@app.get("/whiteboard-admin", response_class=HTMLResponse)
|
||||
@app.get(
|
||||
"/api/wb-admin",
|
||||
response_class=HTMLResponse,
|
||||
tags=["pages"],
|
||||
summary="记事本管理页(需鉴权)",
|
||||
description="查看所有记事本的创建时间 / 编辑次数 / 上次修改时间,并可删除。Basic Auth 同 docs。",
|
||||
)
|
||||
def whiteboard_admin_page(_: str = Depends(require_docs_auth)) -> HTMLResponse:
|
||||
"""白板管理页(Basic Auth,同 docs):查看/删除白板。"""
|
||||
return _serve_static_html("whiteboard_admin.html")
|
||||
|
||||
@app.get("/whiteboard/{board_id}", response_class=HTMLResponse)
|
||||
@app.get(
|
||||
"/api/wb-page/{board_id}",
|
||||
response_class=HTMLResponse,
|
||||
tags=["pages"],
|
||||
summary="记事本页面",
|
||||
description="公开访问的共享文本记事本,不存在则自动新建;实时协作走 WS /api/ws/wb/{id}。",
|
||||
)
|
||||
def whiteboard_page(board_id: str) -> HTMLResponse:
|
||||
"""白板页面(公开):访问即协作,不存在则前端拉取时自动新建。"""
|
||||
return _serve_static_html("whiteboard.html")
|
||||
|
||||
return app
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
"""ORM 模型包;import 本包即把所有实体注册到 Base.metadata。"""
|
||||
|
||||
from .pdf_job import PdfJob
|
||||
from .tunnel_session import TunnelSession
|
||||
from .uploaded_file import UploadedFile
|
||||
from .upload_session import UploadSession
|
||||
from .whiteboard import Whiteboard
|
||||
|
||||
__all__ = ["TunnelSession", "UploadedFile", "UploadSession", "Whiteboard"]
|
||||
__all__ = ["PdfJob", "TunnelSession", "UploadedFile", "UploadSession", "Whiteboard"]
|
||||
|
||||
56
app/models/pdf_job.py
Normal file
56
app/models/pdf_job.py
Normal file
@@ -0,0 +1,56 @@
|
||||
"""PDF 转换任务实体。
|
||||
|
||||
一个任务记录一次「上传文件 -> 转为 PDF」的转换。原始上传文件与转换产物 PDF
|
||||
均复用 UploadedFile 存储(落盘 + 元数据入库),本表只记录两者关系与转换状态,
|
||||
不重复实现存储逻辑。
|
||||
|
||||
删除语义(两级):
|
||||
用户软删(user_deleted=true)-- 用户页不再展示,但磁盘与 DB 行保留;
|
||||
管理页仍可见且标注「已删除」。
|
||||
管理员硬删 -- 删原始/产物磁盘文件 + UploadedFile 行 + 本表行,真正删除。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import BigInteger, Boolean, DateTime, Integer, String, func
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from ..database import Base
|
||||
|
||||
|
||||
class PdfJob(Base):
|
||||
__tablename__ = "pdf_job"
|
||||
|
||||
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True)
|
||||
# 用户标识:httpOnly cookie 值(uuid4.hex),用户凭此查看自己的任务
|
||||
owner_cookie: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
|
||||
# 原始上传文件(复用 UploadedFile 存储)
|
||||
source_file_id: Mapped[int] = mapped_column(BigInteger, nullable=False, index=True)
|
||||
source_filename: Mapped[str] = mapped_column(String(512), nullable=False)
|
||||
source_size: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0)
|
||||
# 转换产物 PDF(复用 UploadedFile 存储);转换完成前为 NULL
|
||||
output_file_id: Mapped[int | None] = mapped_column(BigInteger, nullable=True, default=None)
|
||||
# pending(排队) / converting(转换中) / done(完成) / failed(失败)
|
||||
status: Mapped[str] = mapped_column(String(16), nullable=False, default="pending", index=True)
|
||||
# 转换进度 0-100,供前端轮询显示
|
||||
progress: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
# 失败原因(status=failed 时填写)
|
||||
error_message: Mapped[str] = mapped_column(String(512), nullable=False, default="")
|
||||
# 用户软删标记:true=用户已从其页面删除,不再对用户展示
|
||||
user_deleted: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, index=True)
|
||||
# 用户软删时间(管理页展示「是否已删除」时用)
|
||||
deleted_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True, default=None)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime, server_default=func.now(), nullable=False
|
||||
)
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime, server_default=func.now(), onupdate=func.now(), nullable=False
|
||||
)
|
||||
|
||||
def __repr__(self) -> str: # pragma: no cover
|
||||
return (
|
||||
f"<PdfJob id={self.id} status={self.status} "
|
||||
f"src={self.source_filename!r} progress={self.progress}>"
|
||||
)
|
||||
@@ -1,15 +1,16 @@
|
||||
"""共享白板实体。
|
||||
"""共享白板实体(文本记事本)。
|
||||
|
||||
一个白板由 board_id 唯一标识(用户可读的 url id),strokes 以 JSON 列保存全部笔画。
|
||||
白板长期留存,进程重启后仍可恢复;实时协作由 WebSocket hub 在内存中维护在线连接,
|
||||
笔画变更经 service 落库后由 hub 广播给同 board 的其它在线连接。
|
||||
一个白板由 board_id 唯一标识(用户可读的 url id),content 存完整文本,
|
||||
version 是乐观锁版本号(每次编辑 +1)。白板长期留存,进程重启后仍可恢复;
|
||||
实时协作由 WebSocket hub 在内存中维护在线连接,文本变更经 service 落库后
|
||||
由 hub 广播给同 board 的其它在线连接。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import BigInteger, DateTime, Integer, JSON, String, func
|
||||
from sqlalchemy import BigInteger, DateTime, Integer, String, Text, func
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from ..database import Base
|
||||
@@ -21,10 +22,12 @@ class Whiteboard(Base):
|
||||
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True)
|
||||
# 用户可读的 url id([a-zA-Z0-9_-]{1,64}),全局唯一
|
||||
board_id: Mapped[str] = mapped_column(String(64), nullable=False, unique=True, index=True)
|
||||
# 笔画数组 [{points:[[x,y],...], color, width}, ...]
|
||||
strokes: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
|
||||
# 修改次数:每次新增笔画或清空 +1,供管理页统计
|
||||
stroke_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
# 白板文本内容(记事本)
|
||||
content: Mapped[str] = mapped_column(Text, nullable=False, default="")
|
||||
# 乐观锁版本号:每次编辑 +1
|
||||
version: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
# 编辑次数(累计修改次数,含清空),供管理页统计
|
||||
edit_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime, server_default=func.now(), nullable=False
|
||||
)
|
||||
@@ -35,5 +38,5 @@ class Whiteboard(Base):
|
||||
def __repr__(self) -> str: # pragma: no cover
|
||||
return (
|
||||
f"<Whiteboard board_id={self.board_id!r} "
|
||||
f"strokes={len(self.strokes)} mods={self.stroke_count}>"
|
||||
f"len={len(self.content)} v={self.version} mods={self.edit_count}>"
|
||||
)
|
||||
|
||||
@@ -10,8 +10,6 @@ from .file import FileListResponse, FileUploadResponse, UploadedFileOut
|
||||
from .system import DiskUsage, MemoryUsage, SystemStatus
|
||||
from .tunnel import TunnelStatusResponse
|
||||
from .whiteboard import (
|
||||
Stroke,
|
||||
StrokeOp,
|
||||
WhiteboardListItem,
|
||||
WhiteboardListResponse,
|
||||
WhiteboardOut,
|
||||
@@ -26,8 +24,6 @@ __all__ = [
|
||||
"FileUploadResponse",
|
||||
"MemoryUsage",
|
||||
"SessionStatusResponse",
|
||||
"Stroke",
|
||||
"StrokeOp",
|
||||
"SystemStatus",
|
||||
"TunnelStatusResponse",
|
||||
"UploadedFileOut",
|
||||
|
||||
@@ -36,3 +36,31 @@ class FileUploadResponse(BaseModel):
|
||||
class FileListResponse(BaseModel):
|
||||
total: int
|
||||
items: list[UploadedFileOut]
|
||||
|
||||
|
||||
class PdfJobBrief(BaseModel):
|
||||
"""文件关联到的 PDF 转换任务摘要(供合并管理页展示)。
|
||||
|
||||
一个 uploaded_file 可能同时被多个 job 引用(罕见),故为列表;
|
||||
通常每行 0 或 1 条。
|
||||
"""
|
||||
|
||||
job_id: int
|
||||
role: str = Field(..., description='"source"=该文件是 epub 源文件;"output"=该文件是产物 PDF')
|
||||
status: str = Field(..., description="pending / converting / done / failed")
|
||||
progress: int = Field(0, description="转换进度 0-100")
|
||||
user_deleted: bool = Field(False, description="用户是否已软删该任务")
|
||||
deleted_at: datetime | None = Field(None, description="用户软删时间")
|
||||
|
||||
|
||||
class UploadedFileWithPdfOut(UploadedFileOut):
|
||||
"""带 PDF 任务关联信息的文件视图(合并管理页用)。"""
|
||||
|
||||
pdf_jobs: list[PdfJobBrief] = Field(default_factory=list, description="关联到的 PDF 转换任务")
|
||||
|
||||
|
||||
class FileWithPdfListResponse(BaseModel):
|
||||
"""合并管理页列表响应:以 uploaded_files 为基础,附带 pdf_jobs 关联。"""
|
||||
|
||||
total: int
|
||||
items: list[UploadedFileWithPdfOut]
|
||||
|
||||
46
app/schemas/pdf.py
Normal file
46
app/schemas/pdf.py
Normal file
@@ -0,0 +1,46 @@
|
||||
"""PDF 转换接口 DTO。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class PdfJobOut(BaseModel):
|
||||
"""任务对外视图(用户与管理页共用,user_deleted 仅管理页关注)。"""
|
||||
|
||||
id: int
|
||||
source_file_id: int = Field(..., description="原始 epub 文件的 uploaded_file.id")
|
||||
source_filename: str = Field(..., description="原始上传文件名")
|
||||
source_size: int = Field(..., description="原始文件字节数")
|
||||
output_file_id: int | None = Field(None, description="产物 PDF 的 uploaded_file.id,转换完成前为 null")
|
||||
status: str = Field(..., description="pending / converting / done / failed")
|
||||
progress: int = Field(0, description="转换进度 0-100")
|
||||
error_message: str = Field("", description="失败原因")
|
||||
user_deleted: bool = Field(False, description="用户是否已软删(管理页用)")
|
||||
deleted_at: datetime | None = Field(None, description="用户软删时间(管理页用)")
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class PdfJobListResponse(BaseModel):
|
||||
"""任务列表响应。"""
|
||||
|
||||
total: int
|
||||
items: list[PdfJobOut]
|
||||
|
||||
|
||||
class PdfSubmitResponse(BaseModel):
|
||||
"""提交转换任务的响应。"""
|
||||
|
||||
job: PdfJobOut = Field(..., description="新建的任务")
|
||||
set_cookie: bool = Field(
|
||||
False, description="true=本次请求未带 cookie,响应已下发新 cookie"
|
||||
)
|
||||
|
||||
|
||||
class DeleteResult(BaseModel):
|
||||
deleted: bool
|
||||
@@ -1,27 +1,19 @@
|
||||
"""白板接口 DTO。"""
|
||||
"""白板接口 DTO(文本记事本)。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class Stroke(BaseModel):
|
||||
"""一条笔画:点序列 + 样式。结构宽松(Any)以兼容前端扩展字段。"""
|
||||
|
||||
points: list[list[float]] = Field(default_factory=list, description="[[x,y],...]")
|
||||
color: str = Field(default="#1565c0", description="笔画颜色")
|
||||
width: float = Field(default=3, description="笔画宽度")
|
||||
|
||||
|
||||
class WhiteboardOut(BaseModel):
|
||||
"""白板完整内容(GET /whiteboard/{id} 与 WS init 帧)。"""
|
||||
"""白板完整内容(GET /api/wb/{id} 与 WS init 帧)。"""
|
||||
|
||||
board_id: str
|
||||
strokes: list[Any] = Field(default_factory=list, description="笔画数组")
|
||||
stroke_count: int = Field(0, description="累计修改次数")
|
||||
content: str = Field("", description="白板文本内容")
|
||||
version: int = Field(0, description="乐观锁版本号,每次编辑 +1")
|
||||
edit_count: int = Field(0, description="累计编辑次数")
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
@@ -32,7 +24,7 @@ class WhiteboardListItem(BaseModel):
|
||||
"""管理页列表项。"""
|
||||
|
||||
board_id: str
|
||||
stroke_count: int
|
||||
edit_count: int
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
@@ -44,10 +36,3 @@ class WhiteboardListResponse(BaseModel):
|
||||
|
||||
total: int
|
||||
items: list[WhiteboardListItem]
|
||||
|
||||
|
||||
class StrokeOp(BaseModel):
|
||||
"""WS 笔画操作(type=stroke 时携带)。"""
|
||||
|
||||
type: str = Field(..., description="add / clear")
|
||||
stroke: dict[str, Any] | None = Field(None, description="type=add 时携带的笔画对象")
|
||||
|
||||
104
app/services/pdf_converter.py
Normal file
104
app/services/pdf_converter.py
Normal file
@@ -0,0 +1,104 @@
|
||||
"""文件 -> PDF 转换器(纯 Python,无 calibre/xvfb 系统依赖)。
|
||||
|
||||
当前支持 epub(必要能力):ebooklib 解析 epub 文档项(按 spine 顺序),拼接为
|
||||
完整 HTML 后交 weasyprint 渲染为 PDF。epub 内的相对资源(图片/CSS)经 base_url
|
||||
指向 epub 解包目录解析。
|
||||
|
||||
接口抽象为 ``convert_to_pdf(src_path, dst_path)``:未来新增格式只需在本模块内
|
||||
按扩展名分支,调用方(PdfService)无需改动。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import tempfile
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger("zikai.pdf")
|
||||
|
||||
# 支持的输入格式(小写扩展名 -> 是否可转)。新增格式在此登记并实现分支即可。
|
||||
# epub 为必要能力;其余为 ebooklib/weasyprint 路径天然支持的电子书结构,
|
||||
# 实测对纯 HTML 类 epub 同样有效,故一并放行。
|
||||
SUPPORTED_EXTENSIONS = {".epub"}
|
||||
|
||||
|
||||
def is_supported(filename: str) -> bool:
|
||||
"""文件名扩展名是否在支持列表内。"""
|
||||
return Path(filename).suffix.lower() in SUPPORTED_EXTENSIONS
|
||||
|
||||
|
||||
def convert_to_pdf(src_path: Path, dst_path: Path) -> None:
|
||||
"""把 src_path 指向的文件转为 PDF 写入 dst_path。
|
||||
|
||||
失败抛 RuntimeError(调用方捕获后写 error_message)。按扩展名分发,
|
||||
当前仅 epub 分支;新增格式在此 elif 扩展。
|
||||
"""
|
||||
ext = src_path.suffix.lower()
|
||||
if ext == ".epub":
|
||||
_epub_to_pdf(src_path, dst_path)
|
||||
else:
|
||||
raise RuntimeError(f"不支持的文件格式:{ext}(仅支持 epub)")
|
||||
|
||||
|
||||
def _epub_to_pdf(src_path: Path, dst_path: Path) -> None:
|
||||
"""epub -> PDF:解包 epub -> 按 spine 顺序取文档项 HTML -> weasyprint 渲染。
|
||||
|
||||
epub 本质是 zip。先解包到临时目录,用 ebooklib 读取(其内部按 zip 解析,
|
||||
base_url 指向解包后的 OEBPS/ 内容目录使相对图片/CSS 可被 weasyprint 解析)。
|
||||
"""
|
||||
# 延迟导入:weasyprint 首次 import 较重(加载 pango/cairo),且仅在真正转换时需要
|
||||
import ebooklib # noqa: F401
|
||||
from ebooklib import epub
|
||||
from weasyprint import HTML
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="zpdf_") as tmp:
|
||||
tmp_dir = Path(tmp)
|
||||
# epub 是 zip,解包到临时目录便于 weasyprint 解析相对资源
|
||||
try:
|
||||
with zipfile.ZipFile(src_path, "r") as zf:
|
||||
zf.extractall(tmp_dir)
|
||||
except zipfile.BadZipFile as exc:
|
||||
raise RuntimeError(f"epub 文件损坏(非有效 zip):{exc}") from exc
|
||||
|
||||
try:
|
||||
book = epub.read_epub(str(src_path), {"ignore_ncx": True})
|
||||
except Exception as exc:
|
||||
raise RuntimeError(f"epub 解析失败:{exc}") from exc
|
||||
|
||||
# 按 spine 顺序收集文档项(XHTML),保证章节顺序正确
|
||||
docs: list[str] = []
|
||||
for idref, _linear in book.spine:
|
||||
item = book.get_item_with_id(idref)
|
||||
if item is not None:
|
||||
docs.append(item.get_content().decode("utf-8", errors="replace"))
|
||||
if not docs:
|
||||
# spine 为空时回退:取所有文档项
|
||||
docs = [
|
||||
it.get_content().decode("utf-8", errors="replace")
|
||||
for it in book.get_items_of_type(ebooklib.ITEM_DOCUMENT)
|
||||
]
|
||||
if not docs:
|
||||
raise RuntimeError("epub 内无可转换的文档内容")
|
||||
|
||||
full_html = "\n".join(docs)
|
||||
|
||||
# 定位资源根目录(含图片/CSS 的目录):通常是 OEBPS/ 或根目录。
|
||||
# epub 内资源(图片/CSS)相对文档项引用,文档项与资源同处 OPF 所在目录。
|
||||
# 故以 OPF 文件所在目录作为 base_url,使相对路径正确解析。
|
||||
base_url = str(tmp_dir)
|
||||
opf_files = list(tmp_dir.rglob("*.opf"))
|
||||
if opf_files:
|
||||
opf_dir = opf_files[0].parent
|
||||
# OPF 可能在根目录,此时 base_url 保持 tmp_dir
|
||||
if str(opf_dir) != str(tmp_dir):
|
||||
base_url = str(opf_dir)
|
||||
|
||||
try:
|
||||
HTML(string=full_html, base_url=base_url).write_pdf(str(dst_path))
|
||||
except Exception as exc:
|
||||
raise RuntimeError(f"PDF 渲染失败:{exc}") from exc
|
||||
|
||||
if not dst_path.exists() or dst_path.stat().st_size == 0:
|
||||
raise RuntimeError("PDF 渲染未产出有效文件")
|
||||
logger.info("epub->pdf 完成: %s -> %s (%d bytes)", src_path.name, dst_path.name, dst_path.stat().st_size)
|
||||
296
app/services/pdf_service.py
Normal file
296
app/services/pdf_service.py
Normal file
@@ -0,0 +1,296 @@
|
||||
"""PDF 转换服务:上传落盘 + 异步后台转换 + 进度追踪 + 两级删除。
|
||||
|
||||
复用 UploadService 的「流式落盘 + 存储路径生成」能力:原始上传文件与产物 PDF
|
||||
均作为 UploadedFile 存储,本服务只维护 PdfJob 关系与状态。
|
||||
|
||||
转换在后台 asyncio task 中以 to_thread 执行(转换器是同步阻塞调用),
|
||||
过程中经 DAO 更新 progress/status,供前端轮询。worker 限制:依赖进程内
|
||||
asyncio 事件循环,与现有 reaper/hub 一致,保持单 worker。
|
||||
|
||||
删除语义:
|
||||
user_delete -- 置 user_deleted=true + deleted_at,磁盘与 DB 行保留(管理页可见)。
|
||||
admin_delete -- 删原始/产物磁盘文件 + UploadedFile 行 + PdfJob 行(真正删除)。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import secrets
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import HTTPException, UploadFile
|
||||
|
||||
from ..config import get_settings
|
||||
from ..dao.pdf_job_dao import PdfJobDAO
|
||||
from ..dao.uploaded_file_dao import UploadedFileDAO
|
||||
from ..models.pdf_job import PdfJob
|
||||
from ..schemas.pdf import PdfJobOut, PdfJobListResponse
|
||||
from . import pdf_converter
|
||||
from .upload_service import UploadService
|
||||
|
||||
logger = logging.getLogger("zikai.pdf")
|
||||
|
||||
|
||||
def new_owner_cookie() -> str:
|
||||
"""生成新的用户标识 cookie 值(uuid4 hex)。"""
|
||||
return secrets.token_hex(16)
|
||||
|
||||
|
||||
class PdfService:
|
||||
def __init__(self, job_dao: PdfJobDAO, file_dao: UploadedFileDAO) -> None:
|
||||
s = get_settings()
|
||||
self.job_dao = job_dao
|
||||
self.file_dao = file_dao
|
||||
self.upload_root = s.resolved_upload_dir()
|
||||
self.cfg = s.pdf
|
||||
# 复用 UploadService 的存储路径生成与落盘能力
|
||||
self._upload = UploadService(file_dao)
|
||||
|
||||
# ---------------- 提交 ----------------
|
||||
|
||||
def submit(self, file: UploadFile, owner_cookie: str) -> tuple[PdfJobOut, int]:
|
||||
"""上传原始文件并创建 pending 任务,返回 (任务视图, source_file_id)。
|
||||
|
||||
校验大小 ≤ max_size_bytes 与扩展名白名单;复用 UploadService 流式落盘
|
||||
入库为 UploadedFile,再建 PdfJob 关联。转换不在此处执行(由 controller
|
||||
调 schedule_convert 异步触发)。
|
||||
"""
|
||||
filename = file.filename or "upload.epub"
|
||||
if not pdf_converter.is_supported(filename):
|
||||
raise HTTPException(400, "仅支持 epub 文件")
|
||||
|
||||
# 大小校验:UploadFile 流式无已知长度,先读一遍统计并重置(小文件可行),
|
||||
# 对大文件更优的做法是流式计数,这里复用 stream_to_disk 后按 size 校验。
|
||||
resp = self._upload.stream_to_disk(file, source="pdf", uploaded_by=owner_cookie)
|
||||
if resp.size_bytes > self.cfg.max_size_bytes:
|
||||
# 超限:清理刚落盘的文件与 DB 行,保持无副作用
|
||||
try:
|
||||
self._upload.delete_file(resp.id)
|
||||
except Exception: # pragma: no cover
|
||||
logger.warning("清理超限文件失败 id=%s", resp.id)
|
||||
raise HTTPException(
|
||||
413,
|
||||
f"文件过大({resp.size_bytes} > {self.cfg.max_size_bytes},上限 250MB)",
|
||||
)
|
||||
|
||||
job = PdfJob(
|
||||
owner_cookie=owner_cookie,
|
||||
source_file_id=resp.id,
|
||||
source_filename=filename,
|
||||
source_size=resp.size_bytes,
|
||||
status="pending",
|
||||
progress=0,
|
||||
)
|
||||
job = self.job_dao.create(job)
|
||||
logger.info("PDF 任务已创建 job=%s file=%s size=%d", job.id, filename, resp.size_bytes)
|
||||
return PdfJobOut.model_validate(job), resp.id
|
||||
|
||||
def schedule_convert(self, job_id: int) -> None:
|
||||
"""在当前事件循环起一个后台 task 执行转换(不阻塞调用方)。
|
||||
|
||||
用 to_thread 跑同步转换器;转换中分段更新 progress。
|
||||
重要:后台 task 必须用独立 DB Session(请求 Session 在请求结束后即关闭),
|
||||
故 _convert_async 内部经 _fresh_service 重建带新 Session 的 service。
|
||||
"""
|
||||
asyncio.create_task(self._convert_async(job_id))
|
||||
|
||||
async def _convert_async(self, job_id: int) -> None:
|
||||
"""后台转换:pending -> converting(进度) -> done/failed。
|
||||
|
||||
每个阶段用独立 Session(_fresh_service),避免引用请求 Session(已关闭)。
|
||||
"""
|
||||
try:
|
||||
await asyncio.to_thread(self._run_with_fresh_session, "_mark_converting", job_id)
|
||||
await asyncio.wait_for(
|
||||
asyncio.to_thread(self._run_with_fresh_session, "_do_convert", job_id),
|
||||
timeout=self.cfg.convert_timeout_seconds,
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
await asyncio.to_thread(self._run_with_fresh_session, "_mark_failed", job_id, "转换超时")
|
||||
except Exception as exc:
|
||||
await asyncio.to_thread(self._run_with_fresh_session, "_mark_failed", job_id, f"转换失败:{exc}")
|
||||
|
||||
@staticmethod
|
||||
def _run_with_fresh_session(method_name: str, *args) -> None:
|
||||
"""用独立 DB Session 构造新 PdfService 实例执行其方法。
|
||||
|
||||
后台线程不能复用请求的 Session(请求结束即关闭),故每次操作新建 Session。
|
||||
method_name 是 PdfService 实例方法名,在此用新 service 调用对应方法。
|
||||
"""
|
||||
from ..database import get_session_local
|
||||
db = get_session_local()()
|
||||
try:
|
||||
svc = PdfService(PdfJobDAO(db), UploadedFileDAO(db))
|
||||
getattr(svc, method_name)(*args)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
# ---------------- 同步转换实现(在线程中执行,用独立 Session) ----------------
|
||||
|
||||
def _mark_converting(self, job_id: int) -> None:
|
||||
job = self.job_dao.get(job_id)
|
||||
if job is None:
|
||||
return
|
||||
job.status = "converting"
|
||||
job.progress = 5
|
||||
self.job_dao.update(job)
|
||||
|
||||
def _do_convert(self, job_id: int) -> None:
|
||||
"""执行转换并落产物 PDF 为 UploadedFile。"""
|
||||
job = self.job_dao.get(job_id)
|
||||
if job is None:
|
||||
return
|
||||
|
||||
src_row = self.file_dao.get_by_id(job.source_file_id)
|
||||
if src_row is None:
|
||||
self._mark_failed(job_id, "原始文件记录丢失")
|
||||
return
|
||||
src_path = (self.upload_root / src_row.storage_path).resolve()
|
||||
if not src_path.exists():
|
||||
self._mark_failed(job_id, "原始文件实体不存在")
|
||||
return
|
||||
|
||||
# 产物 PDF 存储路径(复用 UploadService 的路径生成,扩展名固定 .pdf)
|
||||
rel_path, abs_path = self._upload.make_storage_path("result.pdf")
|
||||
part_path = abs_path.with_name(abs_path.name + ".part")
|
||||
|
||||
# 更新进度到「渲染中」
|
||||
job.status = "converting"
|
||||
job.progress = 30
|
||||
self.job_dao.update(job)
|
||||
|
||||
try:
|
||||
pdf_converter.convert_to_pdf(src_path, part_path)
|
||||
except Exception:
|
||||
part_path.unlink(missing_ok=True)
|
||||
raise
|
||||
|
||||
job.progress = 80
|
||||
self.job_dao.update(job)
|
||||
|
||||
# 落产物 UploadedFile(复用 commit_entity 的原子改名 + 入库)
|
||||
from ..models.uploaded_file import UploadedFile
|
||||
size = part_path.stat().st_size
|
||||
sha256 = self._hash_file(part_path)
|
||||
entity = UploadedFile(
|
||||
storage_path=str(rel_path),
|
||||
original_filename=Path(job.source_filename).stem + ".pdf",
|
||||
content_type="application/pdf",
|
||||
size_bytes=size,
|
||||
sha256=sha256,
|
||||
source="pdf-convert",
|
||||
uploaded_by=job.owner_cookie,
|
||||
)
|
||||
saved = self._upload.commit_entity(entity, part_path, abs_path)
|
||||
|
||||
job.output_file_id = saved.id
|
||||
job.status = "done"
|
||||
job.progress = 100
|
||||
self.job_dao.update(job)
|
||||
logger.info("PDF 转换完成 job=%s output_file_id=%s", job_id, saved.id)
|
||||
|
||||
def _mark_failed(self, job_id: int, message: str) -> None:
|
||||
job = self.job_dao.get(job_id)
|
||||
if job is None:
|
||||
return
|
||||
job.status = "failed"
|
||||
job.error_message = message[:500]
|
||||
self.job_dao.update(job)
|
||||
logger.warning("PDF 转换失败 job=%s: %s", job_id, message)
|
||||
|
||||
def _hash_file(self, path: Path) -> str:
|
||||
import hashlib
|
||||
h = hashlib.sha256()
|
||||
with path.open("rb") as f:
|
||||
while chunk := f.read(self._upload.chunk_bytes):
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
|
||||
# ---------------- 查询 ----------------
|
||||
|
||||
def list_for_user(self, owner_cookie: str, limit: int = 100, offset: int = 0) -> PdfJobListResponse:
|
||||
limit = min(max(limit, 1), self.cfg.list_limit)
|
||||
offset = max(offset, 0)
|
||||
total = self.job_dao.count_for_user(owner_cookie)
|
||||
rows = self.job_dao.list_for_user(owner_cookie, limit=limit, offset=offset)
|
||||
return PdfJobListResponse(total=total, items=[PdfJobOut.model_validate(r) for r in rows])
|
||||
|
||||
def list_for_admin(self, limit: int = 100, offset: int = 0) -> PdfJobListResponse:
|
||||
limit = min(max(limit, 1), self.cfg.list_limit)
|
||||
offset = max(offset, 0)
|
||||
total = self.job_dao.count_all()
|
||||
rows = self.job_dao.list_all(limit=limit, offset=offset)
|
||||
return PdfJobListResponse(total=total, items=[PdfJobOut.model_validate(r) for r in rows])
|
||||
|
||||
def get_job(self, job_id: int, owner_cookie: str) -> PdfJobOut:
|
||||
"""用户查询单任务:仅当归属本人且未软删时可见。"""
|
||||
job = self.job_dao.get(job_id)
|
||||
if job is None or job.owner_cookie != owner_cookie or job.user_deleted:
|
||||
raise HTTPException(404, "任务不存在")
|
||||
return PdfJobOut.model_validate(job)
|
||||
|
||||
def get_output_path(self, job_id: int, owner_cookie: str) -> tuple[PdfJobOut, Path, str]:
|
||||
"""返回 (任务视图, 产物磁盘绝对路径, 下载文件名) 供下载。
|
||||
|
||||
用户仅可下载自己未软删且已完成的任务产物。
|
||||
"""
|
||||
job = self.job_dao.get(job_id)
|
||||
if job is None or job.owner_cookie != owner_cookie or job.user_deleted:
|
||||
raise HTTPException(404, "任务不存在")
|
||||
if job.status != "done" or job.output_file_id is None:
|
||||
raise HTTPException(409, "任务尚未完成,无法下载")
|
||||
out_row = self.file_dao.get_by_id(job.output_file_id)
|
||||
if out_row is None:
|
||||
raise HTTPException(410, "产物文件记录丢失")
|
||||
path = (self.upload_root / out_row.storage_path).resolve()
|
||||
if not path.exists():
|
||||
raise HTTPException(410, "产物文件实体不存在")
|
||||
return PdfJobOut.model_validate(job), path, out_row.original_filename
|
||||
|
||||
# ---------------- 删除 ----------------
|
||||
|
||||
def user_delete(self, job_id: int, owner_cookie: str) -> bool:
|
||||
"""用户软删:仅置 user_deleted=true,磁盘与 DB 行保留(管理页仍可见)。"""
|
||||
job = self.job_dao.get(job_id)
|
||||
if job is None or job.owner_cookie != owner_cookie or job.user_deleted:
|
||||
return False
|
||||
job.user_deleted = True
|
||||
job.deleted_at = datetime.now(timezone.utc)
|
||||
self.job_dao.update(job)
|
||||
logger.info("用户软删 PDF 任务 job=%s", job_id)
|
||||
return True
|
||||
|
||||
def admin_delete(self, job_id: int) -> bool:
|
||||
"""管理员硬删:删原始/产物磁盘文件 + UploadedFile 行 + PdfJob 行。
|
||||
|
||||
真正删除,不可恢复。磁盘删除失败仅记日志,仍清 DB 行保证列表不再显示。
|
||||
"""
|
||||
job = self.job_dao.get(job_id)
|
||||
if job is None:
|
||||
return False
|
||||
# 删原始文件
|
||||
self._safe_delete_file(job.source_file_id)
|
||||
# 删产物文件(若有)
|
||||
if job.output_file_id is not None:
|
||||
self._safe_delete_file(job.output_file_id)
|
||||
# 删 PdfJob 行
|
||||
self.job_dao.delete(job)
|
||||
logger.info("管理员硬删 PDF 任务 job=%s", job_id)
|
||||
return True
|
||||
|
||||
def _safe_delete_file(self, file_id: int) -> None:
|
||||
"""删 UploadedFile 磁盘文件 + DB 行;失败只记日志不阻断。"""
|
||||
row = self.file_dao.get_by_id(file_id)
|
||||
if row is None:
|
||||
return
|
||||
path = (self.upload_root / row.storage_path).resolve()
|
||||
try:
|
||||
Path(path).unlink(missing_ok=True)
|
||||
except Exception as exc: # pragma: no cover
|
||||
logger.warning("删除文件失败 file_id=%s path=%s: %s", file_id, path, exc)
|
||||
try:
|
||||
self.file_dao.delete(file_id)
|
||||
except Exception as exc: # pragma: no cover
|
||||
logger.warning("删除文件 DB 行失败 file_id=%s: %s", file_id, exc)
|
||||
@@ -27,8 +27,9 @@ class ZikaiSFTPServer(asyncssh.SFTPServer):
|
||||
super().__init__(chan, chroot=str(upload_root).encode())
|
||||
try:
|
||||
self._username = chan.get_extra_info("username") or "unknown"
|
||||
except Exception: # pragma: no cover
|
||||
except Exception as exc: # pragma: no cover
|
||||
self._username = "unknown"
|
||||
logger.debug("读取 SFTP 会话用户名失败: %s", exc)
|
||||
logger.info("SFTP 会话开始 user=%s chroot=%s", self._username, upload_root)
|
||||
|
||||
def exit(self) -> None:
|
||||
@@ -45,8 +46,8 @@ def _tunnel_dao():
|
||||
def _close_tunnel_dao(dao) -> None:
|
||||
try:
|
||||
dao.db.close()
|
||||
except Exception: # pragma: no cover
|
||||
pass
|
||||
except Exception as exc: # pragma: no cover
|
||||
logger.debug("关闭隧道 DAO 会话失败: %s", exc)
|
||||
|
||||
|
||||
class ZikaiSSHServer(asyncssh.SSHServer):
|
||||
@@ -101,7 +102,8 @@ class ZikaiSSHServer(asyncssh.SSHServer):
|
||||
try:
|
||||
# asyncssh 命中返回 dict(可能为空),未命中返回 None
|
||||
result = self._authorized_keys.validate(key, client_host=addr, client_addr=addr)
|
||||
except Exception:
|
||||
except Exception as exc: # pragma: no cover
|
||||
logger.warning("公钥校验异常 user=%s: %s", username, exc)
|
||||
result = None
|
||||
ok = result is not None
|
||||
if ok:
|
||||
|
||||
@@ -52,11 +52,6 @@ class TunnelService:
|
||||
def get_active(self, user_name: str) -> TunnelSession | None:
|
||||
return self.dao.get_active_by_user(user_name)
|
||||
|
||||
def is_port_allowed(self, user_name: str, tunnel_port: int) -> bool:
|
||||
"""校验该 user 是否被允许绑定该隧道端口(防 user 乱绑端口)。"""
|
||||
user = self.settings.find_user(user_name)
|
||||
return user is not None and user.tunnel_port == tunnel_port
|
||||
|
||||
def reap_orphans(self) -> int:
|
||||
"""兜底清理:关闭所有 active 会话(进程重启时 DB 里残留的孤儿记录)。
|
||||
|
||||
|
||||
@@ -8,7 +8,9 @@ upload_root),不再有 HTTP 登记接口。
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
import uuid
|
||||
from collections.abc import Iterator
|
||||
from datetime import datetime, timezone
|
||||
@@ -21,6 +23,12 @@ from ..dao.uploaded_file_dao import UploadedFileDAO
|
||||
from ..models.uploaded_file import UploadedFile
|
||||
from ..schemas.file import FileUploadResponse, UploadedFileOut
|
||||
|
||||
logger = logging.getLogger("zikai.upload")
|
||||
|
||||
# 磁盘扫描频率限制:两次扫描至少间隔此秒数,否则跳过(只返回 DB 缓存)。
|
||||
_SCAN_MIN_INTERVAL = 3.0
|
||||
_last_scan_time: float = 0.0
|
||||
|
||||
|
||||
def hash_stream(chunks: Iterator[bytes]) -> tuple[int, str]:
|
||||
"""对一段字节块序列流式计算 (size, sha256)。供多条上传路径复用。"""
|
||||
@@ -66,6 +74,72 @@ class UploadService:
|
||||
|
||||
# ---------------- 查询 ----------------
|
||||
|
||||
def scan_sftp_files(self, force: bool = False) -> int:
|
||||
"""扫描 uploads/ 目录,把磁盘上有但 DB 未记录的文件补录入库。
|
||||
|
||||
SFTP 上传的文件直接落盘到 upload_root(chroot 根),不经过 HTTP 路径,
|
||||
因此没有 DB 记录。本方法遍历磁盘文件,与 DB 已有 storage_path 比对,
|
||||
为缺失项创建记录(source="sftp",计算 size + sha256)。
|
||||
|
||||
频率限制:两次扫描间隔 < _SCAN_MIN_INTERVAL(3s)则跳过,force=True 强制扫。
|
||||
返回本次新补录的条数(跳过时返回 0)。
|
||||
"""
|
||||
global _last_scan_time
|
||||
now = time.monotonic()
|
||||
if not force and (now - _last_scan_time) < _SCAN_MIN_INTERVAL:
|
||||
return 0
|
||||
_last_scan_time = now
|
||||
|
||||
known = self.dao.list_storage_paths()
|
||||
new_count = 0
|
||||
# 排除分片会话暂存目录与 .part 残品
|
||||
skip_dirs = {".work"}
|
||||
for abs_path in self.upload_root.rglob("*"):
|
||||
if not abs_path.is_file():
|
||||
continue
|
||||
if abs_path.suffix == ".part":
|
||||
continue
|
||||
# 跳过 .work 目录下的任何文件
|
||||
rel = abs_path.relative_to(self.upload_root)
|
||||
if rel.parts and rel.parts[0] in skip_dirs:
|
||||
continue
|
||||
storage_path = str(rel).replace("\\", "/")
|
||||
if storage_path in known:
|
||||
continue
|
||||
# 磁盘有、DB 无:补录
|
||||
try:
|
||||
size = abs_path.stat().st_size
|
||||
sha256 = self._hash_file(abs_path)
|
||||
except Exception as exc:
|
||||
logger.warning("扫描文件失败 path=%s: %s", abs_path, exc)
|
||||
continue
|
||||
entity = UploadedFile(
|
||||
storage_path=storage_path,
|
||||
original_filename=abs_path.name,
|
||||
content_type="",
|
||||
size_bytes=size,
|
||||
sha256=sha256,
|
||||
source="sftp",
|
||||
uploaded_by="sftp",
|
||||
)
|
||||
try:
|
||||
self.dao.create(entity)
|
||||
new_count += 1
|
||||
known.add(storage_path)
|
||||
except Exception as exc:
|
||||
logger.warning("补录 SFTP 文件失败 path=%s: %s", storage_path, exc)
|
||||
if new_count:
|
||||
logger.info("SFTP 文件扫描补录 %d 个", new_count)
|
||||
return new_count
|
||||
|
||||
def _hash_file(self, path: Path) -> str:
|
||||
"""流式计算文件 sha256(避免大文件一次性读入内存)。"""
|
||||
h = hashlib.sha256()
|
||||
with path.open("rb") as f:
|
||||
while chunk := f.read(self.chunk_bytes):
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
|
||||
def list_files(self, limit: int = 100, offset: int = 0) -> tuple[int, list[UploadedFileOut]]:
|
||||
total = self.dao.count()
|
||||
rows = self.dao.list(limit=limit, offset=offset)
|
||||
@@ -95,18 +169,17 @@ class UploadService:
|
||||
def delete_file(self, file_id: int) -> bool:
|
||||
"""硬删除:删磁盘文件 + 删 DB 行。磁盘缺失不阻断 DB 清理。返回是否命中。
|
||||
|
||||
供单删/批删复用,保证删除语义一致。
|
||||
供单删/批删复用,保证删除语义一致。磁盘删除失败仅记日志,仍清 DB 行
|
||||
(保证列表不再显示),避免磁盘文件泄漏却无任何记录。
|
||||
"""
|
||||
_, path = self.get_out_with_disk_path(file_id)
|
||||
row = self.dao.get_by_id(file_id)
|
||||
if row is None:
|
||||
out, path = self.get_out_with_disk_path(file_id)
|
||||
if out is None:
|
||||
return False
|
||||
if path is not None:
|
||||
try:
|
||||
Path(path).unlink(missing_ok=True)
|
||||
except Exception:
|
||||
# 即使磁盘删除失败也继续清 DB 行,保证列表不再显示
|
||||
pass
|
||||
except Exception as exc:
|
||||
logger.warning("删除磁盘文件失败 file_id=%s path=%s: %s", file_id, path, exc)
|
||||
self.dao.delete(file_id)
|
||||
return True
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
@@ -58,19 +59,26 @@ class WhiteboardHub:
|
||||
self.heartbeat_interval = cfg.heartbeat_interval_seconds
|
||||
self.heartbeat_miss_threshold = cfg.heartbeat_miss_threshold
|
||||
self.timeout_seconds = self.heartbeat_interval * self.heartbeat_miss_threshold
|
||||
self.max_connections_per_board = cfg.max_connections_per_board
|
||||
# {board_id: set[Connection]}
|
||||
self._boards: dict[str, set[Connection]] = {}
|
||||
self._lock = asyncio.Lock()
|
||||
|
||||
# ---------------- 连接生命周期 ----------------
|
||||
|
||||
async def register(self, conn: Connection) -> None:
|
||||
"""把已 accept 的连接加入 board 集合(WebSocket accept 由 controller 负责)。"""
|
||||
async def register(self, conn: Connection) -> bool:
|
||||
"""把已 accept 的连接加入 board 集合。
|
||||
|
||||
返回 False 表示该 board 连接数已达上限(调用方应关闭连接)。
|
||||
"""
|
||||
async with self._lock:
|
||||
conns = self._boards.setdefault(conn.board_id, set())
|
||||
if len(conns) >= self.max_connections_per_board:
|
||||
return False
|
||||
conns.add(conn)
|
||||
logger.info("连接接入 board=%s client=%s(当前 %d 人)",
|
||||
conn.board_id, conn.client_id, self.connection_count(conn.board_id))
|
||||
return True
|
||||
|
||||
async def disconnect(self, conn: Connection) -> None:
|
||||
"""幂等移除连接;空 set 从 dict 删除以防内存泄漏。"""
|
||||
@@ -84,8 +92,9 @@ class WhiteboardHub:
|
||||
# 尽力关闭 websocket(可能已关闭)
|
||||
try:
|
||||
await conn.websocket.close()
|
||||
except Exception: # pragma: no cover
|
||||
pass
|
||||
except Exception as exc: # pragma: no cover
|
||||
logger.debug("关闭 websocket 时出错 board=%s client=%s: %s",
|
||||
conn.board_id, conn.client_id, exc)
|
||||
logger.info("连接移除 board=%s client=%s(剩余 %d 人)",
|
||||
conn.board_id, conn.client_id, self.connection_count(conn.board_id))
|
||||
|
||||
@@ -153,23 +162,22 @@ class WhiteboardHub:
|
||||
for conn in conns:
|
||||
try:
|
||||
await conn.websocket.close()
|
||||
except Exception: # pragma: no cover
|
||||
pass
|
||||
except Exception as exc: # pragma: no cover
|
||||
logger.debug("关闭 websocket 时出错 board=%s client=%s: %s",
|
||||
conn.board_id, conn.client_id, exc)
|
||||
logger.info("关闭白板 board=%s,踢出 %d 个连接", board_id, len(conns))
|
||||
|
||||
|
||||
# 进程内单例(由 main.py lifespan / controller 共享)
|
||||
_hub: WhiteboardHub | None = None
|
||||
_hub_lock = threading.Lock()
|
||||
|
||||
|
||||
def get_hub() -> WhiteboardHub:
|
||||
"""获取/创建进程内单例 hub。线程安全(lifespan 预热后通常不再进锁)。"""
|
||||
global _hub
|
||||
if _hub is None:
|
||||
_hub = WhiteboardHub()
|
||||
with _hub_lock:
|
||||
if _hub is None:
|
||||
_hub = WhiteboardHub()
|
||||
return _hub
|
||||
|
||||
|
||||
def reset_hub() -> None:
|
||||
"""测试用:重置单例。"""
|
||||
global _hub
|
||||
_hub = None
|
||||
|
||||
@@ -1,14 +1,13 @@
|
||||
"""白板服务:CRUD + 笔画操作。
|
||||
"""白板服务(文本记事本):CRUD + 文本更新 + 清空。
|
||||
|
||||
不持有 WebSocket 连接状态(那是 hub 的职责);删除白板时通过可选的 hub 回调
|
||||
通知 hub 踢出该 board 的所有在线连接,由 controller 在装配时注入,避免 service -> hub
|
||||
的硬依赖(保持低耦合)。
|
||||
不持有 WebSocket 连接状态(那是 hub 的职责)。删除白板时由 controller 层负责
|
||||
通知 hub 踢出在线连接(因为 close_board 是 async,需在事件循环中调用),
|
||||
service 只管 DB 层面的删除,保持低耦合。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from fastapi import HTTPException
|
||||
|
||||
@@ -16,20 +15,18 @@ from ..config import get_settings
|
||||
from ..dao.whiteboard_dao import WhiteboardDAO
|
||||
from ..schemas.whiteboard import WhiteboardListItem, WhiteboardOut
|
||||
|
||||
if TYPE_CHECKING: # 避免运行时循环导入
|
||||
from .whiteboard_hub import WhiteboardHub
|
||||
|
||||
# board_id 合法字符集:字母数字下划线短横线
|
||||
_BOARD_ID_RE = re.compile(r"^[a-zA-Z0-9_-]+$")
|
||||
|
||||
|
||||
class WhiteboardService:
|
||||
def __init__(self, dao: WhiteboardDAO, hub: "WhiteboardHub | None" = None) -> None:
|
||||
def __init__(self, dao: WhiteboardDAO) -> None:
|
||||
self.dao = dao
|
||||
self.hub = hub
|
||||
cfg = get_settings().whiteboard
|
||||
self.max_board_id_length = cfg.max_board_id_length
|
||||
self.list_limit = cfg.list_limit
|
||||
# 文本内容长度上限(防滥用)
|
||||
self.max_content_length = 256 * 1024
|
||||
|
||||
# ---------------- 校验 ----------------
|
||||
|
||||
@@ -42,6 +39,15 @@ class WhiteboardService:
|
||||
):
|
||||
raise HTTPException(400, "board_id 非法(仅允许字母数字下划线短横线,1-64 字符)")
|
||||
|
||||
def validate_content(self, content: str) -> None:
|
||||
if not isinstance(content, str):
|
||||
raise HTTPException(400, "content 必须是字符串")
|
||||
if len(content) > self.max_content_length:
|
||||
raise HTTPException(
|
||||
413,
|
||||
f"文本过长({len(content)} > {self.max_content_length}),请缩减内容",
|
||||
)
|
||||
|
||||
# ---------------- 读 ----------------
|
||||
|
||||
def get_or_create(self, board_id: str) -> WhiteboardOut:
|
||||
@@ -59,34 +65,20 @@ class WhiteboardService:
|
||||
|
||||
# ---------------- 写 ----------------
|
||||
|
||||
def append_stroke(self, board_id: str, stroke: dict[str, Any]) -> WhiteboardOut:
|
||||
"""追加一条笔画并返回最新状态。"""
|
||||
def update_content(self, board_id: str, content: str) -> WhiteboardOut:
|
||||
"""整体替换文本内容(客户端 debounce 后发完整文本)。"""
|
||||
self.validate_board_id(board_id)
|
||||
board = self.dao.append_strokes(board_id, [stroke])
|
||||
self.validate_content(content)
|
||||
board = self.dao.update_content(board_id, content)
|
||||
if board is None:
|
||||
raise HTTPException(404, "白板不存在")
|
||||
return WhiteboardOut.model_validate(board)
|
||||
|
||||
def clear(self, board_id: str) -> WhiteboardOut:
|
||||
"""清空白板;stroke_count 仍自增以记录这次修改。"""
|
||||
self.validate_board_id(board_id)
|
||||
board = self.dao.replace_strokes(board_id, [])
|
||||
if board is None:
|
||||
raise HTTPException(404, "白板不存在")
|
||||
return WhiteboardOut.model_validate(board)
|
||||
"""清空白板(内容置空),edit_count 仍自增以记录这次修改。"""
|
||||
return self.update_content(board_id, "")
|
||||
|
||||
def delete(self, board_id: str) -> bool:
|
||||
"""删除白板;同时通知 hub 踢出该 board 的所有在线连接。"""
|
||||
"""删除白板 DB 行。踢出在线连接由 controller 层负责(close_board 是 async)。"""
|
||||
self.validate_board_id(board_id)
|
||||
ok = self.dao.delete(board_id)
|
||||
if ok and self.hub is not None:
|
||||
# hub.close_board 是 async,但删除走 REST 同步路径;安排到事件循环里执行
|
||||
import asyncio
|
||||
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
loop.create_task(self.hub.close_board(board_id))
|
||||
except RuntimeError:
|
||||
# 无运行中事件循环(如脚本调用):同步调用会报错,忽略即可
|
||||
pass
|
||||
return ok
|
||||
return self.dao.delete(board_id)
|
||||
|
||||
@@ -128,7 +128,7 @@ def render(status: SystemStatus) -> str:
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<p class="foot">zikai file service · 数据来源 psutil</p>
|
||||
<p class="foot"><a class="json" href="/api/index">导航</a> · zikai file service · 数据来源 psutil</p>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
@@ -6,8 +6,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from html import escape
|
||||
|
||||
# 默认分片大小 4 MiB:大于 Apache 300s 限制下单片可数秒传完,小到内存恒定。
|
||||
DEFAULT_CHUNK_SIZE = 4 * 1024 * 1024
|
||||
# 同一文件分片并发数
|
||||
@@ -38,7 +36,7 @@ def render() -> str:
|
||||
|
||||
<div id="summary" class="foot"></div>
|
||||
|
||||
<p class="foot"><a class="json" href="/api/system/status">系统状态</a> · zikai file service</p>
|
||||
<p class="foot"><a class="json" href="/api/index">导航</a> · <a class="json" href="/api/system/status">系统状态</a> · zikai file service</p>
|
||||
|
||||
<script>
|
||||
const CHUNK_SIZE = {DEFAULT_CHUNK_SIZE};
|
||||
|
||||
@@ -64,3 +64,13 @@ whiteboard:
|
||||
heartbeat_miss_threshold: 5
|
||||
max_board_id_length: 64 # board_id 合法字符 [a-zA-Z0-9_-],长度上限
|
||||
list_limit: 100 # 管理页单次列表上限
|
||||
|
||||
pdf:
|
||||
# PDF 转换服务:用户上传 epub -> 后台转换为 PDF -> 显示进度并下载。
|
||||
# 用户侧(上传/查看/下载/软删)凭 httpOnly cookie 标识;管理侧(列表/硬删)走 docs 同款 Basic Auth。
|
||||
# 原始文件与产物 PDF 复用 storage.upload_dir 落盘。转换用纯 Python(ebooklib + weasyprint)。
|
||||
max_size_bytes: 262144000 # 单文件上限 250 MiB
|
||||
convert_timeout_seconds: 600 # 转换超时(秒),超大文件兜底
|
||||
list_limit: 100 # 列表单次上限
|
||||
cookie_name: zk_pdf # 用户标识 cookie 名
|
||||
cookie_max_age_seconds: 31536000 # cookie 有效期 1 年
|
||||
|
||||
56
deploy/install-systemd.sh
Executable file
56
deploy/install-systemd.sh
Executable file
@@ -0,0 +1,56 @@
|
||||
#!/usr/bin/env bash
|
||||
# 安装 systemd 服务,实现 zTools2 持久化部署(开机自启 + 崩溃自动重启)。
|
||||
#
|
||||
# 做的事:
|
||||
# 1. 用实际路径填充 deploy/ztools2.service 模板,写入 /etc/systemd/system/
|
||||
# 2. 停掉旧方式(start.sh 启动的 uvicorn),避免端口冲突
|
||||
# 3. systemctl daemon-reload + enable + start
|
||||
#
|
||||
# 用法:./deploy/install-systemd.sh
|
||||
# 卸载:systemctl disable --now ztools2 && rm /etc/systemd/system/ztools2.service && systemctl daemon-reload
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
ROOT="$(pwd)"
|
||||
|
||||
# 解析 uvicorn 可执行路径:优先 .venv(setup.sh 标准),回退系统 python -m uvicorn
|
||||
if [[ -x "$ROOT/.venv/bin/uvicorn" ]]; then
|
||||
UVICORN="$ROOT/.venv/bin/uvicorn"
|
||||
PYBIN="$ROOT/.venv/bin/python"
|
||||
elif command -v uvicorn >/dev/null 2>&1; then
|
||||
UVICORN="$(command -v uvicorn)"
|
||||
PYBIN="$(command -v python3)"
|
||||
else
|
||||
echo "未找到 uvicorn:请先运行 ./setup.sh(建 .venv)或 pip install uvicorn" >&2
|
||||
exit 1
|
||||
fi
|
||||
[[ -f "$ROOT/config.yaml" ]] || { echo "未找到 config.yaml,请先运行 ./setup.sh" >&2; exit 1; }
|
||||
|
||||
UNIT_SRC="$ROOT/deploy/ztools2.service"
|
||||
UNIT_DST="/etc/systemd/system/ztools2.service"
|
||||
|
||||
echo "==> 生成 systemd unit(路径 $ROOT,uvicorn=$UVICORN)"
|
||||
# 同时填充 ROOT 与 UVICORN;PYBIN 备用(ExecStart 用 UVICORN 直接启动)
|
||||
sudo sed -e "s|__ZTOOLS2_DIR__|$ROOT|g" -e "s|__UVICORN__|$UVICORN|g" "$UNIT_SRC" > /tmp/ztools2.service
|
||||
sudo mv /tmp/ztools2.service "$UNIT_DST"
|
||||
sudo chmod 644 "$UNIT_DST"
|
||||
|
||||
echo "==> 停止旧方式(start.sh 启动的进程,若有)"
|
||||
./stop.sh >/dev/null 2>&1 || true
|
||||
|
||||
echo "==> 启用并启动 ztools2 服务"
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable ztools2
|
||||
sudo systemctl restart ztools2
|
||||
|
||||
sleep 2
|
||||
if systemctl is-active --quiet ztools2; then
|
||||
echo "✓ ztools2 已启动并设为开机自启"
|
||||
echo " 状态:systemctl status ztools2"
|
||||
echo " 日志:journalctl -u ztools2 -f"
|
||||
echo " 停止:sudo systemctl stop ztools2"
|
||||
echo " 重启:sudo systemctl restart ztools2"
|
||||
else
|
||||
echo "✗ ztools2 启动失败,查看日志:journalctl -u ztools2 -n 50" >&2
|
||||
exit 1
|
||||
fi
|
||||
23
deploy/ztools2.service
Normal file
23
deploy/ztools2.service
Normal file
@@ -0,0 +1,23 @@
|
||||
[Unit]
|
||||
Description=zikai file service (zTools2) - FastAPI backend
|
||||
Documentation=https://git.zikai.wang/zikai/zTools2
|
||||
After=network.target mysql.service
|
||||
Wants=mysql.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
# 工作目录与 uvicorn 路径:安装时由 install-systemd.sh 用实际路径替换
|
||||
WorkingDirectory=__ZTOOLS2_DIR__
|
||||
Environment=PYTHONUNBUFFERED=1
|
||||
ExecStart=__UVICORN__ app.main:app --host 127.0.0.1 --port 6867 --workers 1
|
||||
Restart=on-failure
|
||||
RestartSec=3
|
||||
# 后台转换可能耗时较长,放宽超时
|
||||
TimeoutStopSec=30
|
||||
KillSignal=SIGINT
|
||||
# 日志走 journald(journalctl -u ztools2)
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
39
docs/configuration.md
Normal file
39
docs/configuration.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# 配置说明
|
||||
|
||||
所有运行时配置在 `config.yaml`(git-ignored)。完整 schema 见 `config.example.yaml`。
|
||||
|
||||
## 配置项
|
||||
|
||||
| 段 | 关键项 | 说明 |
|
||||
|----|--------|------|
|
||||
| `server` | `host`/`port` | 绑定地址,保持 `127.0.0.1:6867`(Apache 反代) |
|
||||
| `database` | `host`/`port`/`user`/`password`/`database` | MySQL 连接;密码由 `setup.sh` 自动生成写回 |
|
||||
| `storage` | `upload_dir` | 文件存储根目录(默认 `uploads`) |
|
||||
| | `chunk_bytes` | 流式上传分片大小(默认 1 MiB) |
|
||||
| | `chunk_session_dir` | 分片会话暂存目录(默认 `uploads/.work`) |
|
||||
| | `chunk_session_ttl_seconds` | 被放弃会话存活秒数(默认 300) |
|
||||
| `docs` | `username`/`password` | `/docs`、`/api/files-page`、`/api/wb-admin` 及 `/api/admin/*` 的 Basic Auth(明文,常量时间比较) |
|
||||
| `sftp` | `enabled`/`host`/`port` | SFTP 服务,默认 `0.0.0.0:2022` |
|
||||
| | `users[].username`/`password_hash` | SFTP 用户(bcrypt) |
|
||||
| | `host_key_path`/`authorized_keys_path` | 主机密钥与公钥白名单路径 |
|
||||
| `tunnel` | `enabled`/`users[]` | 反向隧道:`username`/`password_hash`/`tunnel_port`/`local_port` |
|
||||
| `whiteboard` | `heartbeat_interval_seconds` | 心跳间隔(默认 3s) |
|
||||
| | `heartbeat_miss_threshold` | 失活阈值(默认 5 次 = 15s) |
|
||||
| | `max_board_id_length` | board_id 长度上限(默认 64) |
|
||||
| | `max_connections_per_board` | 单白板并发连接上限(默认 50) |
|
||||
| | `list_limit` | 管理页单次列表上限(默认 100) |
|
||||
|
||||
## 生成 bcrypt hash(SFTP/隧道用户密码)
|
||||
|
||||
```bash
|
||||
.venv/bin/python -c "import bcrypt;print(bcrypt.hashpw(b'yourpass',bcrypt.gensalt()).decode())"
|
||||
```
|
||||
|
||||
把输出填入 `config.yaml` 对应 `password_hash` 字段。
|
||||
|
||||
## 重新生成 DB 密码
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m app.scripts.init_db # 重置密码(写回 config.yaml)
|
||||
KEEP_DB_PASSWORD=1 .venv/bin/python -m app.scripts.init_db # 保留现有密码,仅建库建账
|
||||
```
|
||||
29
docs/error-handling.md
Normal file
29
docs/error-handling.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# 错误处理与日志约定
|
||||
|
||||
zTools2 采用 Spring 风格分层架构,错误处理分三层:DAO fail-fast(抛异常),Service 捕获后转换业务异常并记日志,Controller 捕获后转 HTTP 状态码。
|
||||
|
||||
## DAO 层(app/dao/)
|
||||
|
||||
- **唯一发 SQL 的层**:所有写操作(create/update/delete)均在该层 commit,service 不直接操作 session。
|
||||
- `get_or_create`(whiteboard_dao.py):仅 `IntegrityError`(并发下另一事务已插入违反唯一约束)才回滚重读;其他异常向上抛,避免掩盖 schema/连接等真实故障。
|
||||
- `delete`(pdf_job_dao.py / uploaded_file_dao.py):硬删 DB 行并 commit;不存在返回 False。
|
||||
- schema 漂移检测:`init_db_schema`(database.py)建表后用 inspector 检查既有表的列是否与模型声明齐全,缺列即抛 `RuntimeError`(fail-fast),避免运行期才以晦涩的 `OperationalError` 暴露。
|
||||
|
||||
## Service 层(app/services/)
|
||||
|
||||
- **PdfService**:`submit` 校验扩展名/大小,超限清理已落盘文件后抛 `HTTPException`;后台转换 `_convert_async` 捕获 `TimeoutError` / 通用异常,经 `_mark_failed` 落库 + `logger.warning`。`admin_delete` 磁盘删除失败仅 `logger.warning`,仍清 DB 行保证列表不再显示。
|
||||
- **UploadService / ChunkUploadService**:流式落盘出错清理临时文件后 `raise`(向上传播);分片会话被放弃由后台 reaper 每 60s 清理。
|
||||
- **WhiteboardHub**:`disconnect` / `close_board` 关闭 websocket 出错 `logger.debug`(尽力关闭,可能已关闭);`broadcast` 单连接发送失败立即 disconnect,不影响其他连接;reaper 循环异常 `logger.warning` 后继续。
|
||||
- **TunnelService**:`register` / `close` / `reap_orphans` 均记 `logger.info`;SSH 连接断开时清理会话失败 `logger.warning`。
|
||||
- **sftp_server**:`validate_public_key` 校验异常 `logger.warning`(auth 路径,避免静默失败);`_close_tunnel_dao` / 读会话用户名失败 `logger.debug`(尽力清理)。
|
||||
|
||||
## Controller 层(app/controllers/)
|
||||
|
||||
- **pdf_controller**:`_resolve_cookie` 解析用户 cookie,无则生成新值挂 request.state 供响应 set_cookie。
|
||||
- **whiteboard_controller**:`_safe_send` / `_safe_close` 发送/关闭 WS 失败 `logger.debug`(best-effort);WS 主循环异常 `logger.warning` 后正常关闭连接。
|
||||
- 所有管理 API(`/api/admin/*`)经 `require_docs_auth` Basic Auth 守卫(常量时间比较)。
|
||||
|
||||
## 日志位置
|
||||
|
||||
- `logs/app.log`(HTTP)、`logs/sftp.log`(SFTP);pidfile:`app.pid`、`sftp.pid`。
|
||||
- 日志器命名:`zikai.pdf` / `zikai.whiteboard` / `zikai.tunnel` / `sftp`,便于按模块过滤。
|
||||
51
docs/routes.md
Normal file
51
docs/routes.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# 路由与访问入口
|
||||
|
||||
## 路由约定
|
||||
|
||||
**所有 zTools2 托管的入口(页面 / 静态资源 / 健康探针 / WebSocket / REST API)统一挂在 `/api/` 前缀下**,只有元信息/文档例外(`/`、`/docs`、`/redoc`、`/openapi.json`)。这样反向代理与 vite dev proxy 都只需一条 `/api/` 规则即可把请求转给当前环境的 zTools2,前端用**同源相对路径**(如 `/api/pdf/jobs`、`/api/health`)即可,与运行环境(本地 / 测试 / 生产)无关,无需区分 dev/prod 指向。
|
||||
|
||||
> PDF 转换的用户侧 UI 由 zMainPage 的 zPDF_package 组件提供(构建期 import,非 iframe);zTools2 仅提供 `/api/pdf/jobs` 等 REST API。PDF 转换管理(查看全部任务含软删标记、硬删)已并入文件管理页 `/api/files-page`。
|
||||
|
||||
页面类入口为避免与同名 REST API 冲突,统一加 `-page` 后缀:
|
||||
|
||||
| 类型 | 页面入口 | REST API(同名不加后缀) |
|
||||
|------|---------|------------------------|
|
||||
| 文件管理 | `GET /api/files-page` | `GET /api/files`、`/api/files/{id}` 等 |
|
||||
| 记事本 | `GET /api/wb-page/{id}` | `GET /api/wb/{id}` |
|
||||
|
||||
其余入口:`/api/index`(导航页)、`/api/health`(探针)、`/api/static/*`(JS/CSS)、`/api/ws/wb/{id}`(WebSocket)、`/api/upload`、`/api/wb-admin`。
|
||||
|
||||
## 功能一览
|
||||
|
||||
| 模块 | 页面 / 接口 | 鉴权 |
|
||||
|------|------------|------|
|
||||
| 导航页 | `GET /api/index`(卡片式入口索引,各子页可返回) | 公开 |
|
||||
| 文件上传 | `POST /api/files/upload`(流式)/ `POST /api/files/chunk-uploads/*`(分片+断点续传) | 公开 |
|
||||
| 上传页 | `GET /api/upload`(拖拽/多文件/分片/去重) | 公开 |
|
||||
| 文件管理 | `GET /api/files-page`(多选/批量下载删除/分页,合并 PDF 转换管理:状态、软删标记、硬删任务) | Basic Auth |
|
||||
| 文件管理 API | `GET /api/admin/files`、`GET /api/admin/files/with-pdf`(合并视图,附带 pdf_jobs 关联)、`GET/DELETE /api/admin/files/{id}`、`GET /api/admin/files/{id}/download` | Basic Auth |
|
||||
| 共享记事本 | `GET /api/wb-page/{id}`(公开,不存在则新建) | 公开 |
|
||||
| 记事本实时同步 | `WS /api/ws/wb/{id}`(心跳 3s,5 次失活移除) | 公开 |
|
||||
| 记事本管理 | `GET /api/wb-admin`(查看/删除) | Basic Auth |
|
||||
| 记事本管理 API | `GET /api/admin/wb`、`DELETE /api/admin/wb/{id}` | Basic Auth |
|
||||
| PDF 转换 API | `POST /api/pdf/jobs`、`GET /api/pdf/jobs[/{id}]`、`GET /api/pdf/jobs/{id}/download`、`DELETE /api/pdf/jobs/{id}` | 公开(cookie) |
|
||||
| PDF 转换管理 API | `GET /api/admin/pdf/jobs`、`DELETE /api/admin/pdf/jobs/{id}`(管理入口已并入 `/api/files-page`) | Basic Auth |
|
||||
| 主机监控 | `GET /api/system/status`(CPU/内存/磁盘,HTML+JSON 内容协商) | 公开 |
|
||||
| 反向隧道反代 | `ALL /api/userPort/{userName}`(经 SSH 隧道转发到 user 本地服务) | 公开 |
|
||||
| SFTP/SSH | 端口 2022(密码+公钥,chroot 到上传目录,承载隧道转发) | SSH |
|
||||
| API 文档 | `GET /docs`(Swagger)/ `GET /redoc` | Basic Auth |
|
||||
|
||||
## 访问入口
|
||||
|
||||
| 入口 | URL |
|
||||
|------|-----|
|
||||
| 导航页 | https://f.zikai.wang/api/index |
|
||||
| API 文档 | https://f.zikai.wang/docs(Basic Auth) |
|
||||
| 上传页 | https://f.zikai.wang/api/upload |
|
||||
| 文件管理 | https://f.zikai.wang/api/files-page(Basic Auth,含 PDF 转换管理) |
|
||||
| 共享记事本 | https://f.zikai.wang/api/wb-page/{id}(公开,`{id}` 为 `[a-zA-Z0-9_-]{1,64}`) |
|
||||
| 记事本管理 | https://f.zikai.wang/api/wb-admin(Basic Auth) |
|
||||
| PDF 转换 | 经 zMainPage 的 PDF 页签(zPDF_package 组件)调用 `/api/pdf/jobs` 等 REST API(公开,凭 cookie) |
|
||||
| 系统状态 | https://f.zikai.wang/api/system/status(HTML,`?format=json` 切 JSON) |
|
||||
| curl 上传 | `curl -F file=@big.iso https://f.zikai.wang/api/files/upload` |
|
||||
| SFTP | `sftp -P 2022 uploader@f.zikai.wang` |
|
||||
@@ -2,10 +2,13 @@ fastapi==0.115.6
|
||||
uvicorn[standard]==0.34.0
|
||||
python-multipart==0.0.20
|
||||
psutil==6.1.1
|
||||
SQLAlchemy==2.0.36
|
||||
SQLAlchemy==2.0.51
|
||||
PyMySQL==1.1.1
|
||||
pydantic-settings==2.7.0
|
||||
PyYAML==6.0.2
|
||||
asyncssh==2.18.0
|
||||
bcrypt==4.2.1
|
||||
httpx==0.28.1
|
||||
# PDF 转换:ebooklib 解析 epub,weasyprint 渲染 HTML/CSS 为 PDF(纯 Python,无需 calibre/xvfb)
|
||||
ebooklib==0.20
|
||||
weasyprint==69.0
|
||||
|
||||
@@ -53,16 +53,42 @@ CREATE TABLE IF NOT EXISTS `tunnel_session` (
|
||||
KEY `idx_status` (`status`)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
|
||||
|
||||
-- 共享白板表。由 app/services/whiteboard_service.py 使用。
|
||||
-- 白板长期留存,strokes 以 JSON 保存全部笔画;实时同步由 WebSocket hub 在内存维护。
|
||||
-- 共享白板表(文本记事本)。由 app/services/whiteboard_service.py 使用。
|
||||
-- content 存完整文本,version 是乐观锁版本号(每次编辑 +1),edit_count 累计编辑次数。
|
||||
-- 白板长期留存,实时同步由 WebSocket hub 在内存维护在线连接。
|
||||
CREATE TABLE IF NOT EXISTS `whiteboard` (
|
||||
`id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
|
||||
`board_id` VARCHAR(64) NOT NULL COMMENT '用户可读的 url id,[a-zA-Z0-9_-]{1,64}',
|
||||
`strokes` JSON NOT NULL COMMENT '笔画数组 [{points,color,width}, ...]',
|
||||
`stroke_count` INT NOT NULL DEFAULT 0 COMMENT '累计修改次数(新增笔画/清空各 +1)',
|
||||
`content` MEDIUMTEXT NOT NULL COMMENT '白板文本内容',
|
||||
`version` INT NOT NULL DEFAULT 0 COMMENT '乐观锁版本号,每次编辑 +1',
|
||||
`edit_count` INT NOT NULL DEFAULT 0 COMMENT '累计编辑次数(含清空)',
|
||||
`created_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
`updated_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
PRIMARY KEY (`id`),
|
||||
UNIQUE KEY `uq_board_id` (`board_id`),
|
||||
KEY `idx_updated_at` (`updated_at`)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
|
||||
|
||||
-- PDF 转换任务表。由 app/services/pdf_service.py 使用。
|
||||
-- 原始文件与产物 PDF 复用 uploaded_file 存储,本表只记录关系与转换状态。
|
||||
-- 删除两级:user_deleted(用户软删,管理页仍可见)/ admin 硬删(真正删除)。
|
||||
CREATE TABLE IF NOT EXISTS `pdf_job` (
|
||||
`id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
|
||||
`owner_cookie` VARCHAR(64) NOT NULL COMMENT '用户标识(httpOnly cookie 值,uuid4 hex)',
|
||||
`source_file_id` BIGINT UNSIGNED NOT NULL COMMENT '原始上传文件 -> uploaded_file.id',
|
||||
`source_filename` VARCHAR(512) NOT NULL,
|
||||
`source_size` BIGINT UNSIGNED NOT NULL DEFAULT 0,
|
||||
`output_file_id` BIGINT UNSIGNED NULL DEFAULT NULL COMMENT '产物 PDF -> uploaded_file.id,转换完成前 NULL',
|
||||
`status` VARCHAR(16) NOT NULL DEFAULT 'pending' COMMENT 'pending/converting/done/failed',
|
||||
`progress` INT NOT NULL DEFAULT 0 COMMENT '转换进度 0-100',
|
||||
`error_message` VARCHAR(512) NOT NULL DEFAULT '',
|
||||
`user_deleted` TINYINT(1) NOT NULL DEFAULT 0 COMMENT '用户软删标记',
|
||||
`deleted_at` DATETIME NULL DEFAULT NULL COMMENT '用户软删时间',
|
||||
`created_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
`updated_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
PRIMARY KEY (`id`),
|
||||
KEY `idx_owner_cookie` (`owner_cookie`),
|
||||
KEY `idx_status` (`status`),
|
||||
KEY `idx_user_deleted` (`user_deleted`),
|
||||
KEY `idx_created_at` (`created_at`)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
|
||||
|
||||
@@ -3,6 +3,12 @@
|
||||
(function (global) {
|
||||
"use strict";
|
||||
|
||||
// 布尔属性:用属性赋值而非 setAttribute(setAttribute("checked","false") 仍会勾选)
|
||||
const BOOL_PROPS = new Set([
|
||||
"checked", "disabled", "readonly", "selected", "hidden", "multiple", "open",
|
||||
"autofocus", "required", "async", "defer", "controls", "autoplay", "loop", "muted",
|
||||
]);
|
||||
|
||||
function el(tag, attrs, ...children) {
|
||||
const node = document.createElement(tag);
|
||||
if (attrs) {
|
||||
@@ -11,7 +17,8 @@
|
||||
else if (k === "dataset") Object.assign(node.dataset, v);
|
||||
else if (k.startsWith("on") && typeof v === "function")
|
||||
node.addEventListener(k.slice(2).toLowerCase(), v);
|
||||
else if (v !== null && v !== undefined) node.setAttribute(k, v);
|
||||
else if (BOOL_PROPS.has(k)) node[k] = !!v;
|
||||
else if (v !== null && v !== undefined && v !== false) node.setAttribute(k, v);
|
||||
}
|
||||
}
|
||||
for (const c of children) {
|
||||
@@ -54,15 +61,15 @@
|
||||
}
|
||||
|
||||
function fmtBytes(n) {
|
||||
if (n == null) return "-";
|
||||
const x = Number(n);
|
||||
if (!isFinite(x)) return "-";
|
||||
for (const unit of ["B", "KiB", "MiB", "GiB", "TiB", "PiB"]) {
|
||||
if (Math.abs(x) < 1024 || unit === "PiB")
|
||||
return unit === "B" ? `${x} B` : `${x.toFixed(1)} ${unit}`;
|
||||
n = x / 1024;
|
||||
let x = Number(n);
|
||||
if (n == null || !isFinite(x)) return "-";
|
||||
const units = ["B", "KB", "MB", "GB", "TB", "PB"];
|
||||
let i = 0;
|
||||
while (Math.abs(x) >= 1000 && i < units.length - 1) {
|
||||
x /= 1000;
|
||||
i++;
|
||||
}
|
||||
return `${n.toFixed(1)} PiB`;
|
||||
return i === 0 ? `${Math.round(x)} ${units[i]}` : `${x.toFixed(1)} ${units[i]}`;
|
||||
}
|
||||
|
||||
function fmtTime(s) {
|
||||
@@ -76,8 +83,15 @@
|
||||
async function api(path, opts) {
|
||||
const res = await fetch(path, opts);
|
||||
if (res.status === 401) {
|
||||
// 触发浏览器 Basic Auth 弹窗(同源 reload 即可带上凭据)
|
||||
// fetch 不会触发浏览器的 Basic Auth 弹窗(只有导航/form 会)。
|
||||
// 重载当前页:浏览器对页面导航的 401 会弹凭据框,凭据缓存后重试即可带上。
|
||||
toast("需要登录");
|
||||
if (location.href.indexOf("/api/") !== -1) {
|
||||
// 纯 API 调用页(无页面壳),跳转到来源页触发鉴权
|
||||
location.reload();
|
||||
} else {
|
||||
location.reload();
|
||||
}
|
||||
throw new Error("UNAUTHORIZED");
|
||||
}
|
||||
return res;
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
.files-table th.col-size { width: 9em; }
|
||||
.files-table th.col-src { width: 6em; }
|
||||
.files-table th.col-time { width: 11em; }
|
||||
.files-table th.col-pdf { width: 13em; }
|
||||
.files-table th.col-act { width: 9em; text-align: right; }
|
||||
.files-table td.col-act { text-align: right; white-space: nowrap; }
|
||||
.files-table td.col-name .fname { font-weight: 600; word-break: break-all; }
|
||||
@@ -12,9 +13,24 @@
|
||||
.files-table td.col-act .btn { padding: 0.3em 0.8em; font-size: 0.85em; margin-left: 0.3em; }
|
||||
.files-table tbody tr.sel { background: var(--primary-soft); }
|
||||
.files-table tbody tr.sel:hover td { background: var(--primary-soft); }
|
||||
/* 关联到已软删 PDF 任务的行:淡化背景提示 */
|
||||
.files-table tbody tr.row-pdf-deleted td { background: var(--danger-soft); }
|
||||
.files-table tbody tr.row-pdf-deleted:hover td { background: var(--danger-soft); }
|
||||
.files-table input[type="checkbox"] { width: 16px; height: 16px; cursor: pointer; accent-color: var(--primary); }
|
||||
.sha-short { cursor: pointer; }
|
||||
.sha-short:hover { color: var(--primary); }
|
||||
/* PDF 任务列:角色徽标 + 状态徽标堆叠 */
|
||||
.pdf-cell { display: flex; flex-direction: column; gap: 0.2em; font-size: 0.82em; }
|
||||
.pdf-cell .pdf-roles { display: flex; flex-wrap: wrap; gap: 0.3em; align-items: center; }
|
||||
.pdf-cell .role-tag {
|
||||
display: inline-block; padding: 0.05em 0.5em; border-radius: 8px;
|
||||
font-size: 0.78em; background: var(--surface-2); color: var(--text-dim);
|
||||
}
|
||||
.pdf-cell .role-tag.source { background: var(--warn-soft); color: var(--warn); }
|
||||
.pdf-cell .role-tag.output { background: var(--success-soft); color: var(--success); }
|
||||
.pdf-cell .del-mark { color: var(--danger); font-size: 0.78em; }
|
||||
.pdf-cell .pdf-act { display: flex; gap: 0.3em; flex-wrap: wrap; }
|
||||
.pdf-cell .pdf-act .btn { padding: 0.2em 0.6em; font-size: 0.8em; }
|
||||
.row-removed { opacity: 0; transition: opacity 0.25s; }
|
||||
|
||||
.toolbar-spacer { flex: 1; }
|
||||
@@ -47,5 +63,6 @@
|
||||
.files-table th, .files-table td { padding: 0.5em 0.4em; }
|
||||
.files-table th.col-src, .files-table td.col-src { display: none; }
|
||||
.files-table th.col-time, .files-table td.col-time { font-size: 0.78em; }
|
||||
.files-table th.col-pdf, .files-table td.col-pdf { font-size: 0.78em; }
|
||||
.batchbar { font-size: 0.85em; }
|
||||
}
|
||||
|
||||
@@ -3,14 +3,14 @@
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>文件浏览 - zikai</title>
|
||||
<link rel="stylesheet" href="/static/common.css">
|
||||
<link rel="stylesheet" href="/static/file_browser.css">
|
||||
<title>文件管理 - zikai</title>
|
||||
<link rel="stylesheet" href="/api/static/common.css">
|
||||
<link rel="stylesheet" href="/api/static/file_browser.css">
|
||||
</head>
|
||||
<body>
|
||||
<div class="wrap">
|
||||
<h1>文件浏览</h1>
|
||||
<p class="sub">查看已上传的文件、下载或删除。支持多选与分页。删除后不再显示。</p>
|
||||
<h1>文件管理</h1>
|
||||
<p class="sub">浏览 / 下载 / 删除已上传文件,并合并 PDF 转换管理:关联文件会标注转换状态与用户软删标记,可硬删任务。支持多选与分页。</p>
|
||||
|
||||
<div class="toolbar">
|
||||
<button class="btn primary" id="refresh">刷新</button>
|
||||
@@ -39,9 +39,9 @@
|
||||
|
||||
<div id="pager" class="pager hidden"></div>
|
||||
|
||||
<p class="foot"><a class="link" href="/upload">上传文件</a> · zikai file service</p>
|
||||
<p class="foot"><a class="link" href="/api/index">导航</a> · <a class="link" href="/api/upload">上传文件</a> · zikai file service</p>
|
||||
</div>
|
||||
<script src="/static/common.js"></script>
|
||||
<script src="/static/file_browser.js"></script>
|
||||
<script src="/api/static/common.js"></script>
|
||||
<script src="/api/static/file_browser.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
/* 文件浏览页:分页拉取 /api/admin/files、复选框多选 + 全选、批量下载/删除、复制 sha。
|
||||
/* 文件管理页:分页拉取 /api/admin/files-with-pdf、复选框多选 + 全选、批量下载/删除、复制 sha。
|
||||
合并 PDF 转换管理:每个文件附带 pdf_jobs(role=source epub / role=output 产物 PDF),
|
||||
渲染转换状态徽标、用户软删标记,并支持硬删任务(DELETE /api/admin/pdf/jobs/{id})。
|
||||
state.items 缓存当前页数据;切换页/页大小重新拉取;删除后若当前页空则回退一页。 */
|
||||
(function () {
|
||||
"use strict";
|
||||
@@ -53,7 +55,7 @@
|
||||
countEl.textContent = "";
|
||||
pagerEl.classList.add("hidden");
|
||||
try {
|
||||
const res = await api(`/api/admin/files?limit=${limit}&offset=${offset}`);
|
||||
const res = await api(`/api/admin/files/with-pdf?limit=${limit}&offset=${offset}`);
|
||||
if (!res.ok) throw new Error("HTTP " + res.status);
|
||||
const body = await res.json();
|
||||
state.items = body.items || [];
|
||||
@@ -73,7 +75,7 @@
|
||||
|
||||
function render() {
|
||||
if (!state.items.length) {
|
||||
listEl.innerHTML = '<div class="empty">还没有文件。去 <a class="link" href="/upload">上传</a> 一个吧。</div>';
|
||||
listEl.innerHTML = '<div class="empty">还没有文件。去 <a class="link" href="/api/upload">上传</a> 一个吧。</div>';
|
||||
renderSelection();
|
||||
return;
|
||||
}
|
||||
@@ -88,13 +90,15 @@
|
||||
el("th", { class: "col-size" }, "大小"),
|
||||
el("th", { class: "col-src" }, "来源"),
|
||||
el("th", { class: "col-time" }, "上传时间"),
|
||||
el("th", { class: "col-pdf" }, "PDF 任务"),
|
||||
el("th", { class: "col-act" }, "操作")
|
||||
)
|
||||
);
|
||||
const tbody = el("tbody", null);
|
||||
for (const f of state.items) {
|
||||
const checked = state.selected.has(f.id);
|
||||
const row = el("tr", { class: checked ? "sel" : "", dataset: { id: f.id } },
|
||||
const pdfDeleted = (f.pdf_jobs || []).some((j) => j.user_deleted);
|
||||
const row = el("tr", { class: [checked ? "sel" : "", pdfDeleted ? "row-pdf-deleted" : ""].filter(Boolean).join(" "), dataset: { id: f.id } },
|
||||
el("td", { class: "col-sel" },
|
||||
el("input", { type: "checkbox", class: "row-sel", checked, dataset: { id: f.id } })
|
||||
),
|
||||
@@ -105,6 +109,7 @@
|
||||
el("td", { class: "col-size mono" }, fmtBytes(f.size_bytes)),
|
||||
el("td", { class: "col-src" }, el("span", { class: "tag" }, f.source || "-")),
|
||||
el("td", { class: "col-time muted" }, fmtTime(f.uploaded_at)),
|
||||
el("td", { class: "col-pdf" }, renderPdfCell(f)),
|
||||
el("td", { class: "col-act" },
|
||||
el("a", { class: "btn primary", href: `/api/admin/files/${f.id}/download`, download: "" }, "下载"),
|
||||
el("button", { class: "btn danger", onclick: () => removeOne(f) }, "删除")
|
||||
@@ -190,6 +195,58 @@
|
||||
return sha.length > 16 ? sha.slice(0, 12) + "…" + sha.slice(-4) : sha;
|
||||
}
|
||||
|
||||
// 渲染 PDF 任务列:展示角色(源 epub / 产物 PDF)徽标、转换状态、用户软删标记与硬删按钮。
|
||||
// 一个文件可能同时被多个 job 引用(罕见),全列出;无关联则显示 -。
|
||||
function renderPdfCell(f) {
|
||||
const jobs = f.pdf_jobs || [];
|
||||
if (!jobs.length) return el("span", { class: "muted" }, "-");
|
||||
const cell = el("div", { class: "pdf-cell" });
|
||||
const roles = el("div", { class: "pdf-roles" });
|
||||
for (const j of jobs) {
|
||||
roles.appendChild(el("span", { class: "role-tag " + j.role, title: j.role === "source" ? "PDF 转换的 epub 源文件" : "PDF 转换的产物 PDF" },
|
||||
j.role === "source" ? "源" : "产物"));
|
||||
roles.appendChild(statusTag(j));
|
||||
if (j.user_deleted) {
|
||||
roles.appendChild(el("span", { class: "del-mark", title: "用户已软删:" + fmtTime(j.deleted_at) }, "已删"));
|
||||
}
|
||||
roles.appendChild(el("span", { class: "muted" }, "#"+j.job_id));
|
||||
}
|
||||
cell.appendChild(roles);
|
||||
// 硬删按钮:对每个关联 job 提供(source 与 output 共属同一 job,去重后只显示一个)
|
||||
const seenJobIds = new Set();
|
||||
const actBar = el("div", { class: "pdf-act" });
|
||||
for (const j of jobs) {
|
||||
if (seenJobIds.has(j.job_id)) continue;
|
||||
seenJobIds.add(j.job_id);
|
||||
actBar.appendChild(el("button", {
|
||||
class: "btn danger", onclick: () => removePdfJob(j, f)
|
||||
}, "硬删任务"));
|
||||
}
|
||||
cell.appendChild(actBar);
|
||||
return cell;
|
||||
}
|
||||
|
||||
function statusTag(j) {
|
||||
if (j.status === "done") return el("span", { class: "tag ok" }, "完成");
|
||||
if (j.status === "failed") return el("span", { class: "tag err", title: j.error_message || "" }, "失败");
|
||||
if (j.status === "converting") return el("span", { class: "tag warn" }, "转换中 " + j.progress + "%");
|
||||
return el("span", { class: "tag" }, "排队中");
|
||||
}
|
||||
|
||||
// 硬删 PDF 任务:删 job + 其源/产物文件。删除后重载当前页。
|
||||
async function removePdfJob(j, f) {
|
||||
if (!confirm(`确定硬删 PDF 任务 #${j.job_id}?\n将同时删除关联的源文件与产物 PDF,不可恢复。`)) return;
|
||||
try {
|
||||
const res = await api(`/api/admin/pdf/jobs/${j.job_id}`, { method: "DELETE" });
|
||||
if (res.status === 404) { toast("任务已不存在"); }
|
||||
else if (!res.ok) throw new Error("HTTP " + res.status);
|
||||
toast("已硬删任务");
|
||||
await load(state.page);
|
||||
} catch (e) {
|
||||
toast("硬删失败:" + (e.message || e), "err");
|
||||
}
|
||||
}
|
||||
|
||||
async function removeOne(f) {
|
||||
if (!confirm(`确定删除「${f.original_filename}」?\n此操作不可恢复,将同时删除磁盘文件。`)) return;
|
||||
try {
|
||||
|
||||
74
static/index.html
Normal file
74
static/index.html
Normal file
@@ -0,0 +1,74 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>导航 - zikai</title>
|
||||
<link rel="stylesheet" href="/api/static/common.css">
|
||||
<style>
|
||||
/* 导航页:卡片网格,复用 common.css 变量 */
|
||||
.nav-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(220px, 1fr));
|
||||
gap: 1em;
|
||||
margin-top: 1.5em;
|
||||
}
|
||||
.nav-card {
|
||||
display: flex; flex-direction: column; gap: 0.3em;
|
||||
padding: 1.2em 1.3em;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
background: var(--surface);
|
||||
box-shadow: var(--shadow);
|
||||
text-decoration: none; color: var(--text);
|
||||
transition: transform 0.05s, border-color 0.15s;
|
||||
}
|
||||
.nav-card:hover { transform: translateY(-2px); border-color: var(--primary); }
|
||||
.nav-card .ico { font-size: 1.6em; }
|
||||
.nav-card .name { font-size: 1.05em; font-weight: 600; }
|
||||
.nav-card .desc { color: var(--text-dim); font-size: 0.82em; line-height: 1.4; }
|
||||
.nav-card .lock { font-size: 0.75em; color: var(--text-dim); }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="wrap">
|
||||
<h1>zikai 工具箱</h1>
|
||||
<p class="sub">个人 Web 服务入口导航。带 🔒 标记的页面需 Basic Auth(凭据见 config.yaml 的 docs 段)。</p>
|
||||
|
||||
<div class="nav-grid">
|
||||
<a class="nav-card" href="/api/upload">
|
||||
<span class="ico">📤</span>
|
||||
<span class="name">上传文件</span>
|
||||
<span class="desc">拖拽 / 多文件 / 分片(4 MiB)/ 断点续传</span>
|
||||
<span class="lock"></span>
|
||||
</a>
|
||||
<a class="nav-card" href="/api/files-page">
|
||||
<span class="ico">📁</span>
|
||||
<span class="name">文件管理</span>
|
||||
<span class="desc">浏览 / 下载 / 删除已上传文件,合并 PDF 转换管理(状态、软删标记、硬删任务)</span>
|
||||
<span class="lock">🔒 需鉴权</span>
|
||||
</a>
|
||||
<a class="nav-card" href="/api/wb-admin">
|
||||
<span class="ico">📝</span>
|
||||
<span class="name">记事本管理</span>
|
||||
<span class="desc">查看 / 删除共享文本记事本,可新建并打开</span>
|
||||
<span class="lock">🔒 需鉴权</span>
|
||||
</a>
|
||||
<a class="nav-card" href="/api/system/status">
|
||||
<span class="ico">📊</span>
|
||||
<span class="name">系统状态</span>
|
||||
<span class="desc">CPU / 内存 / 磁盘实时使用率(?format=json 切 JSON)</span>
|
||||
<span class="lock"></span>
|
||||
</a>
|
||||
<a class="nav-card" href="/docs">
|
||||
<span class="ico">📚</span>
|
||||
<span class="name">API 文档</span>
|
||||
<span class="desc">Swagger UI,全部 REST 接口在线调试</span>
|
||||
<span class="lock">🔒 需鉴权</span>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<p class="foot">zikai file service · <a class="link" href="/api/health">健康探针</a></p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
@@ -1,5 +1,17 @@
|
||||
/* 白板页专属样式:全屏画布、悬浮工具栏、移动端适配。 */
|
||||
:root { --bar-h: 52px; }
|
||||
/* 记事本页专属样式:全屏 textarea、悬浮工具栏、移动端适配。 */
|
||||
/* 强制浅色:本页常被 iframe 嵌入浅色站点,覆盖 common.css 的
|
||||
color-scheme: light dark 与 prefers-color-scheme: dark,避免深色背景。 */
|
||||
:root {
|
||||
--bar-h: 52px;
|
||||
color-scheme: light;
|
||||
--bg: #f6f7f9;
|
||||
--surface: #ffffff;
|
||||
--surface-2: rgba(0, 0, 0, 0.02);
|
||||
--border: #e0e3e7;
|
||||
--text: #1a1a1a;
|
||||
--text-dim: #6b7280;
|
||||
--shadow: 0 1px 3px rgba(0, 0, 0, 0.04), 0 1px 2px rgba(0, 0, 0, 0.06);
|
||||
}
|
||||
body { overflow: hidden; background: var(--bg); }
|
||||
.wb-app { display: flex; flex-direction: column; height: 100vh; height: 100dvh; }
|
||||
.wb-bar {
|
||||
@@ -14,22 +26,20 @@ body { overflow: hidden; background: var(--bg); }
|
||||
.wb-id { background: var(--surface-2); padding: 0.15em 0.5em; border-radius: 6px; font-size: 0.82em; color: var(--text-dim); }
|
||||
.wb-online { color: var(--success); font-size: 0.7em; }
|
||||
.wb-online.off { color: var(--text-dim); }
|
||||
.wb-tool { display: inline-flex; align-items: center; gap: 0.3em; font-size: 0.85em; color: var(--text-dim); }
|
||||
.wb-tool input[type="color"] { width: 28px; height: 28px; padding: 0; border: 1px solid var(--border); border-radius: 6px; background: transparent; cursor: pointer; }
|
||||
.wb-tool input[type="range"] { width: 80px; accent-color: var(--primary); }
|
||||
.wb-width-val { width: 1.4em; text-align: center; }
|
||||
.wb-bar .btn { padding: 0.4em 0.9em; font-size: 0.86em; }
|
||||
|
||||
.wb-stage { position: relative; flex: 1; overflow: hidden; }
|
||||
#canvas {
|
||||
position: absolute; inset: 0; width: 100%; height: 100%;
|
||||
display: block; touch-action: none; cursor: crosshair;
|
||||
background:
|
||||
linear-gradient(var(--border) 1px, transparent 1px) 0 0 / 24px 24px,
|
||||
linear-gradient(90deg, var(--border) 1px, transparent 1px) 0 0 / 24px 24px,
|
||||
var(--surface);
|
||||
background-blend-mode: normal;
|
||||
.wb-editor {
|
||||
position: absolute; inset: 0;
|
||||
width: 100%; height: 100%;
|
||||
display: block; resize: none; border: none; outline: none;
|
||||
padding: 1em 1.2em;
|
||||
font-family: ui-monospace, "SF Mono", Menlo, Consolas, "JetBrains Mono", monospace;
|
||||
font-size: 14px; line-height: 1.6;
|
||||
background: var(--surface); color: var(--text);
|
||||
touch-action: manipulation;
|
||||
}
|
||||
.wb-editor::placeholder { color: var(--text-dim); }
|
||||
.wb-status {
|
||||
position: absolute; bottom: 0.8em; left: 50%; transform: translateX(-50%);
|
||||
background: var(--surface); border: 1px solid var(--border);
|
||||
@@ -44,7 +54,7 @@ body { overflow: hidden; background: var(--bg); }
|
||||
@media (max-width: 640px) {
|
||||
.wb-bar { padding: 0.4em 0.5em; gap: 0.4em; }
|
||||
.wb-id { max-width: 8em; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.wb-tool input[type="range"] { width: 56px; }
|
||||
.wb-bar .btn { padding: 0.45em 0.7em; }
|
||||
.wb-title { display: none; }
|
||||
.wb-editor { font-size: 15px; padding: 0.8em; }
|
||||
}
|
||||
|
||||
@@ -4,36 +4,34 @@
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
|
||||
<meta name="theme-color" content="#1565c0">
|
||||
<title>白板 - zikai</title>
|
||||
<link rel="stylesheet" href="/static/common.css">
|
||||
<link rel="stylesheet" href="/static/whiteboard.css">
|
||||
<!-- 强制浅色:该页面会被 iframe 嵌入到浅色主题站点(mainPage),
|
||||
禁用 common.css 的 prefers-color-scheme: dark,保持背景与嵌入站一致 -->
|
||||
<meta name="color-scheme" content="light">
|
||||
<title>记事本 - zikai</title>
|
||||
<link rel="stylesheet" href="/api/static/common.css">
|
||||
<link rel="stylesheet" href="/api/static/whiteboard.css">
|
||||
</head>
|
||||
<body>
|
||||
<div class="wb-app">
|
||||
<header class="wb-bar">
|
||||
<div class="wb-bar-left">
|
||||
<span class="wb-title">白板</span>
|
||||
<code class="wb-id mono" id="boardId" title="白板 ID"></code>
|
||||
<span class="wb-title">记事本</span>
|
||||
<code class="wb-id mono" id="boardId" title="记事本 ID"></code>
|
||||
<span class="wb-online" id="online" title="在线人数">●</span>
|
||||
</div>
|
||||
<div class="wb-bar-right">
|
||||
<label class="wb-tool" title="笔画颜色">
|
||||
<input type="color" id="color" value="#1565c0">
|
||||
</label>
|
||||
<label class="wb-tool" title="笔画粗细">
|
||||
<input type="range" id="width" min="1" max="24" value="3">
|
||||
<span class="wb-width-val mono" id="widthVal">3</span>
|
||||
</label>
|
||||
<button class="btn" id="copyBtn" title="复制分享链接">复制链接</button>
|
||||
<button class="btn danger" id="clearBtn" title="清空白板(所有人)">清空</button>
|
||||
<a class="btn" href="/api/index" target="_top" title="返回导航">导航</a>
|
||||
<button class="btn" id="copyLinkBtn" title="复制分享链接">复制链接</button>
|
||||
<button class="btn" id="copyBtn" title="复制全部文本">复制文本</button>
|
||||
<button class="btn danger" id="clearBtn" title="清空全部内容(所有人)">清空</button>
|
||||
</div>
|
||||
</header>
|
||||
<main class="wb-stage">
|
||||
<canvas id="canvas"></canvas>
|
||||
<textarea id="editor" class="wb-editor" placeholder="在此输入文本,所有人会实时看到你的编辑…" spellcheck="false" autocomplete="off"></textarea>
|
||||
<div class="wb-status" id="status">连接中…</div>
|
||||
</main>
|
||||
</div>
|
||||
<script src="/static/common.js"></script>
|
||||
<script src="/static/whiteboard.js"></script>
|
||||
<script src="/api/static/common.js"></script>
|
||||
<script src="/api/static/whiteboard.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -1,153 +1,149 @@
|
||||
/* 白板:Canvas 绘画 + WebSocket 实时同步 + 心跳。
|
||||
- 笔画以 {points:[[x,y],...], color, width} 表示,增量广播。
|
||||
- 心跳 3s 一次 ping;服务端 15s 无心跳判失活会主动断连,前端据此重连。
|
||||
- 收到他人 stroke 增量重绘该笔;收到 cleared 清空本地画布。
|
||||
- 兼容鼠标 + 触摸:统一用 pointer events,touch-action:none 防滚动缩放。 */
|
||||
/* 记事本:textarea + WebSocket 实时同步 + 心跳。
|
||||
- 文本以整文本 debounce 400ms 后发服务端,服务端存为新版本并广播给其他端。
|
||||
- 收到他人 update 时用 diff 应用变更,保留本地光标位置(按相对偏移调整)。
|
||||
- 收到 cleared 清空本地 textarea。
|
||||
- 心跳 3s 一次 ping;服务端 15s 无心跳判失活会主动断连,前端据此重连。 */
|
||||
(function () {
|
||||
"use strict";
|
||||
const { el, toast, copyText } = window.ZK;
|
||||
const { toast, copyText } = window.ZK;
|
||||
|
||||
// ---------- 从 URL 解析 board_id ----------
|
||||
// 路径形如 /whiteboard/{id};id 为 [a-zA-Z0-9_-]{1,64}
|
||||
const m = location.pathname.match(/^\/whiteboard\/([^/]+)\/?$/);
|
||||
const m = location.pathname.match(/^\/api\/wb-page\/([^/]+)\/?$/);
|
||||
let boardId = m ? decodeURIComponent(m[1]) : "default";
|
||||
// 合法性兜底:前端非法字符直接回退到 default,真正校验在服务端
|
||||
if (!/^[a-zA-Z0-9_-]{1,64}$/.test(boardId)) boardId = "default";
|
||||
document.getElementById("boardId").textContent = boardId;
|
||||
|
||||
// ---------- DOM ----------
|
||||
const canvas = document.getElementById("canvas");
|
||||
const ctx = canvas.getContext("2d");
|
||||
const colorInput = document.getElementById("color");
|
||||
const widthInput = document.getElementById("width");
|
||||
const widthVal = document.getElementById("widthVal");
|
||||
const editor = document.getElementById("editor");
|
||||
const clearBtn = document.getElementById("clearBtn");
|
||||
const copyBtn = document.getElementById("copyBtn");
|
||||
const copyLinkBtn = document.getElementById("copyLinkBtn");
|
||||
const statusEl = document.getElementById("status");
|
||||
const onlineEl = document.getElementById("online");
|
||||
|
||||
// ---------- 状态 ----------
|
||||
let strokes = []; // 已确认的笔画
|
||||
let current = null; // 正在画的笔画(本地未提交)
|
||||
let drawing = false;
|
||||
let ws = null;
|
||||
let clientId = localStorage.getItem("wb_cid") || "";
|
||||
// 每个 tab 独立的 client_id(用 sessionStorage 而非 localStorage,避免同浏览器
|
||||
// 多 tab 共享 id 导致收到的 update 被误判为「自己的」而跳过不同步)。
|
||||
// 服务端已用 exclude=conn 排除发送者,前端不再用 client_id 跳过 update。
|
||||
let clientId = sessionStorage.getItem("wb_cid") || "";
|
||||
if (!clientId) {
|
||||
clientId = "c_" + Math.random().toString(36).slice(2, 10);
|
||||
localStorage.setItem("wb_cid", clientId);
|
||||
sessionStorage.setItem("wb_cid", clientId);
|
||||
}
|
||||
let heartbeatTimer = null;
|
||||
let reconnectTimer = null;
|
||||
let connected = false;
|
||||
let lastSentText = ""; // 上次发到服务端的文本(避免无变更时重复发)
|
||||
let suppressInput = false; // 应用远端更新时抑制 input 事件,防回环
|
||||
let debounceTimer = null;
|
||||
|
||||
// ---------- 画布尺寸 ----------
|
||||
function resize() {
|
||||
const dpr = window.devicePixelRatio || 1;
|
||||
const w = canvas.clientWidth;
|
||||
const h = canvas.clientHeight;
|
||||
canvas.width = Math.max(1, Math.floor(w * dpr));
|
||||
canvas.height = Math.max(1, Math.floor(h * dpr));
|
||||
ctx.setTransform(dpr, 0, 0, dpr, 0, 0);
|
||||
redraw();
|
||||
}
|
||||
window.addEventListener("resize", resize);
|
||||
|
||||
// ---------- 绘制 ----------
|
||||
function drawStroke(s) {
|
||||
if (!s || !s.points || s.points.length < 1) return;
|
||||
ctx.strokeStyle = s.color || "#1565c0";
|
||||
ctx.lineWidth = Number(s.width) || 3;
|
||||
ctx.lineCap = "round";
|
||||
ctx.lineJoin = "round";
|
||||
const pts = s.points;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(pts[0][0], pts[0][1]);
|
||||
if (pts.length === 1) {
|
||||
// 单点:画一个小圆点
|
||||
ctx.arc(pts[0][0], pts[0][1], (ctx.lineWidth || 3) / 2, 0, Math.PI * 2);
|
||||
ctx.fillStyle = ctx.strokeStyle;
|
||||
ctx.fill();
|
||||
return;
|
||||
}
|
||||
for (let i = 1; i < pts.length; i++) ctx.lineTo(pts[i][0], pts[i][1]);
|
||||
ctx.stroke();
|
||||
}
|
||||
|
||||
function redraw() {
|
||||
ctx.clearRect(0, 0, canvas.width, canvas.height);
|
||||
for (const s of strokes) drawStroke(s);
|
||||
if (current) drawStroke(current);
|
||||
}
|
||||
|
||||
// ---------- 指针事件 ----------
|
||||
function pos(e) {
|
||||
const r = canvas.getBoundingClientRect();
|
||||
return [e.clientX - r.left, e.clientY - r.top];
|
||||
}
|
||||
|
||||
canvas.addEventListener("pointerdown", (e) => {
|
||||
if (!connected) { flashStatus("未连接,正在重连…"); return; }
|
||||
e.preventDefault();
|
||||
canvas.setPointerCapture(e.pointerId);
|
||||
drawing = true;
|
||||
current = { points: [pos(e)], color: colorInput.value, width: Number(widthInput.value) };
|
||||
drawStroke(current);
|
||||
});
|
||||
canvas.addEventListener("pointermove", (e) => {
|
||||
if (!drawing) return;
|
||||
e.preventDefault();
|
||||
const p = pos(e);
|
||||
const last = current.points[current.points.length - 1];
|
||||
// 跳过过近的点,减少数据量
|
||||
if (Math.hypot(p[0] - last[0], p[1] - last[1]) < 1.5) return;
|
||||
current.points.push(p);
|
||||
// 增量画最后一段
|
||||
ctx.strokeStyle = current.color;
|
||||
ctx.lineWidth = current.width;
|
||||
ctx.lineCap = "round"; ctx.lineJoin = "round";
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(last[0], last[1]);
|
||||
ctx.lineTo(p[0], p[1]);
|
||||
ctx.stroke();
|
||||
});
|
||||
function endStroke(e) {
|
||||
if (!drawing) return;
|
||||
drawing = false;
|
||||
if (e && e.pointerId !== undefined) {
|
||||
try { canvas.releasePointerCapture(e.pointerId); } catch {}
|
||||
}
|
||||
if (current && current.points.length) {
|
||||
strokes.push(current);
|
||||
send({ type: "stroke", stroke: current });
|
||||
}
|
||||
current = null;
|
||||
}
|
||||
canvas.addEventListener("pointerup", endStroke);
|
||||
canvas.addEventListener("pointercancel", endStroke);
|
||||
canvas.addEventListener("pointerleave", (e) => {
|
||||
// 仅在抬起时结束;离开但按住不放不结束(pointer capture 已处理)
|
||||
if (!drawing) return;
|
||||
if (e.buttons === 0) endStroke(e);
|
||||
// ---------- 本地编辑 -> debounce -> 发送 ----------
|
||||
editor.addEventListener("input", () => {
|
||||
if (suppressInput) return;
|
||||
scheduleSend();
|
||||
});
|
||||
|
||||
widthInput.addEventListener("input", () => (widthVal.textContent = widthInput.value));
|
||||
function scheduleSend() {
|
||||
if (debounceTimer) clearTimeout(debounceTimer);
|
||||
debounceTimer = setTimeout(() => {
|
||||
debounceTimer = null;
|
||||
const text = editor.value;
|
||||
if (text === lastSentText) return;
|
||||
lastSentText = text;
|
||||
send({ type: "edit", content: text });
|
||||
}, 400);
|
||||
}
|
||||
|
||||
function flushSend() {
|
||||
if (debounceTimer) {
|
||||
clearTimeout(debounceTimer);
|
||||
debounceTimer = null;
|
||||
const text = editor.value;
|
||||
if (text !== lastSentText) {
|
||||
lastSentText = text;
|
||||
send({ type: "edit", content: text });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
clearBtn.addEventListener("click", () => {
|
||||
if (!connected) { flashStatus("未连接"); return; }
|
||||
if (!confirm("确定清空白板?所有人的内容都会被清除。")) return;
|
||||
if (!confirm("确定清空全部内容?所有人的内容都会被清除。")) return;
|
||||
send({ type: "clear" });
|
||||
});
|
||||
|
||||
copyBtn.addEventListener("click", async () => {
|
||||
const url = `${location.origin}/whiteboard/${boardId}`;
|
||||
const text = editor.value;
|
||||
if (!text) { toast("内容为空"); return; }
|
||||
const ok = await copyText(text);
|
||||
toast(ok ? "已复制全部文本" : "复制失败");
|
||||
});
|
||||
|
||||
copyLinkBtn.addEventListener("click", async () => {
|
||||
const url = `${location.origin}/api/wb-page/${boardId}`;
|
||||
const ok = await copyText(url);
|
||||
toast(ok ? "链接已复制" : "复制失败");
|
||||
});
|
||||
|
||||
// ---------- 应用远端更新(保留光标) ----------
|
||||
// 策略:用最长公共前后缀算出变更区间,仅替换该区间,光标按相对位置调整。
|
||||
// 若本地有未发送的编辑(editor.value !== lastSentText),合并后重新 scheduleSend,
|
||||
// 避免本地编辑被远端覆盖后因 lastSentText 短路而丢弃。
|
||||
function applyRemoteUpdate(newText) {
|
||||
const oldText = editor.value;
|
||||
if (newText === oldText) return;
|
||||
|
||||
const selStart = editor.selectionStart;
|
||||
const selEnd = editor.selectionEnd;
|
||||
|
||||
// 算公共前缀
|
||||
let prefix = 0;
|
||||
const minLen = Math.min(oldText.length, newText.length);
|
||||
while (prefix < minLen && oldText[prefix] === newText[prefix]) prefix++;
|
||||
|
||||
// 算公共后缀(不能与前缀重叠)
|
||||
let suffixOld = oldText.length;
|
||||
let suffixNew = newText.length;
|
||||
while (suffixOld > prefix && suffixNew > prefix && oldText[suffixOld - 1] === newText[suffixNew - 1]) {
|
||||
suffixOld--;
|
||||
suffixNew--;
|
||||
}
|
||||
|
||||
const hadPending = editor.value !== lastSentText;
|
||||
suppressInput = true;
|
||||
// 用 setRangeText 替换 [prefix, suffixOld) 为 newText[prefix, suffixNew)
|
||||
editor.setRangeText(newText.slice(prefix, suffixNew), prefix, suffixOld, "end");
|
||||
suppressInput = false;
|
||||
lastSentText = editor.value;
|
||||
|
||||
// 调整光标:若光标在变更区间之前,不动;在之后,平移差值;在区间内,移到区间末尾
|
||||
const delta = (suffixNew - prefix) - (suffixOld - prefix);
|
||||
let newStart = selStart, newEnd = selEnd;
|
||||
if (selStart <= prefix) {
|
||||
// 光标在变更前,不变
|
||||
} else if (selStart >= suffixOld) {
|
||||
// 光标在变更后,平移
|
||||
newStart = selStart + delta;
|
||||
newEnd = selEnd + delta;
|
||||
} else {
|
||||
// 光标在变更区间内,移到区间末尾
|
||||
newStart = newEnd = suffixNew;
|
||||
}
|
||||
try {
|
||||
editor.setSelectionRange(newStart, newEnd);
|
||||
} catch {}
|
||||
// 仅当编辑器当前有焦点时恢复焦点,避免抢其它控件焦点
|
||||
if (document.activeElement === editor) editor.focus();
|
||||
|
||||
// 本地有未发送编辑被合并了,重新安排发送,避免被丢弃
|
||||
if (hadPending) scheduleSend();
|
||||
}
|
||||
|
||||
// ---------- WebSocket ----------
|
||||
function wsUrl() {
|
||||
const proto = location.protocol === "https:" ? "wss:" : "ws:";
|
||||
return `${proto}//${location.host}/ws/whiteboard/${encodeURIComponent(boardId)}`;
|
||||
return `${proto}//${location.host}/api/ws/wb/${encodeURIComponent(boardId)}`;
|
||||
}
|
||||
|
||||
function connect() {
|
||||
@@ -160,7 +156,7 @@
|
||||
}
|
||||
ws.onopen = () => {
|
||||
connected = true;
|
||||
setStatus("已连接", true);
|
||||
setStatus("已连接", false);
|
||||
onlineEl.classList.remove("off");
|
||||
send({ type: "hello", client_id: clientId });
|
||||
startHeartbeat();
|
||||
@@ -175,23 +171,34 @@
|
||||
try { msg = JSON.parse(raw); } catch { return; }
|
||||
switch (msg.type) {
|
||||
case "init":
|
||||
strokes = Array.isArray(msg.strokes) ? msg.strokes : [];
|
||||
redraw();
|
||||
setStatus(`已同步 ${strokes.length} 笔`, true);
|
||||
// 连接建立时服务端下发当前文本。若本地有未发送编辑(断线期间输入的),
|
||||
// 不直接覆盖,而是把本地编辑作为最新版本发上去(last-writer-wins),
|
||||
// 避免断线期间的编辑被静默丢弃。
|
||||
if (editor.value && editor.value !== lastSentText) {
|
||||
lastSentText = editor.value;
|
||||
send({ type: "edit", content: editor.value });
|
||||
} else {
|
||||
suppressInput = true;
|
||||
editor.value = msg.content || "";
|
||||
lastSentText = editor.value;
|
||||
suppressInput = false;
|
||||
editor.focus();
|
||||
}
|
||||
setStatus("已同步", false);
|
||||
break;
|
||||
case "pong":
|
||||
// 心跳回声,保持连接
|
||||
break;
|
||||
case "stroke":
|
||||
if (msg.client_id === clientId) break; // 自己的,已本地画
|
||||
strokes.push(msg.stroke);
|
||||
drawStroke(msg.stroke);
|
||||
case "update":
|
||||
// 服务端 broadcast 已用 exclude=conn 排除发送者,收到即他人编辑,直接应用。
|
||||
applyRemoteUpdate(msg.content || "");
|
||||
flashStatus("对方有更新");
|
||||
break;
|
||||
case "cleared":
|
||||
strokes = [];
|
||||
current = null;
|
||||
redraw();
|
||||
flashStatus(msg.client_id === clientId ? "已清空" : "对方清空了白板");
|
||||
suppressInput = true;
|
||||
editor.value = "";
|
||||
lastSentText = "";
|
||||
suppressInput = false;
|
||||
flashStatus(msg.client_id === clientId ? "已清空" : "对方清空了内容");
|
||||
break;
|
||||
case "error":
|
||||
flashStatus(msg.msg || "错误", true);
|
||||
@@ -242,28 +249,33 @@
|
||||
}
|
||||
|
||||
// ---------- 启动 ----------
|
||||
// 先 GET /whiteboard/{id} 确保白板存在(不存在则服务端新建),再连 WS
|
||||
fetch(`/whiteboard/${encodeURIComponent(boardId)}`, { headers: { accept: "application/json" } })
|
||||
.then((r) => r.ok ? r.json() : null)
|
||||
// 先 GET /api/wb/{id} 确保白板存在(不存在则服务端新建),再连 WS
|
||||
fetch(`/api/wb/${encodeURIComponent(boardId)}`, { headers: { accept: "application/json" } })
|
||||
.then((r) => (r.ok ? r.json() : null))
|
||||
.then((body) => {
|
||||
if (body && Array.isArray(body.strokes)) {
|
||||
strokes = body.strokes;
|
||||
redraw();
|
||||
if (body && typeof body.content === "string") {
|
||||
editor.value = body.content;
|
||||
lastSentText = body.content;
|
||||
}
|
||||
resize();
|
||||
connect();
|
||||
})
|
||||
.catch(() => { resize(); connect(); });
|
||||
.catch(() => { connect(); });
|
||||
|
||||
// 页面隐藏时不发心跳可被服务端判失活;重新可见时主动重连
|
||||
// 页面可见性:重新可见时若已断连则主动重连;隐藏时 flush 未发送的编辑
|
||||
document.addEventListener("visibilitychange", () => {
|
||||
if (document.visibilityState === "visible" && (!ws || ws.readyState !== WebSocket.OPEN)) {
|
||||
if (reconnectTimer) { clearTimeout(reconnectTimer); reconnectTimer = null; }
|
||||
connect();
|
||||
if (document.visibilityState === "visible") {
|
||||
if (!ws || ws.readyState !== WebSocket.OPEN) {
|
||||
if (reconnectTimer) { clearTimeout(reconnectTimer); reconnectTimer = null; }
|
||||
connect();
|
||||
}
|
||||
} else {
|
||||
flushSend();
|
||||
}
|
||||
});
|
||||
|
||||
// 离开前 flush 未发送编辑
|
||||
window.addEventListener("beforeunload", () => {
|
||||
flushSend();
|
||||
stopHeartbeat();
|
||||
if (reconnectTimer) clearTimeout(reconnectTimer);
|
||||
try { ws && ws.close(); } catch {}
|
||||
|
||||
@@ -4,8 +4,8 @@
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>白板管理 - zikai</title>
|
||||
<link rel="stylesheet" href="/static/common.css">
|
||||
<link rel="stylesheet" href="/static/whiteboard_admin.css">
|
||||
<link rel="stylesheet" href="/api/static/common.css">
|
||||
<link rel="stylesheet" href="/api/static/whiteboard_admin.css">
|
||||
</head>
|
||||
<body>
|
||||
<div class="wrap">
|
||||
@@ -22,9 +22,9 @@
|
||||
<div class="skel">加载中…</div>
|
||||
</div>
|
||||
|
||||
<p class="foot"><a class="link" href="/upload">上传文件</a> · <a class="link" href="/files">文件浏览</a> · zikai</p>
|
||||
<p class="foot"><a class="link" href="/api/index">导航</a> · <a class="link" href="/api/upload">上传文件</a> · <a class="link" href="/api/files-page">文件管理</a> · zikai</p>
|
||||
</div>
|
||||
<script src="/static/common.js"></script>
|
||||
<script src="/static/whiteboard_admin.js"></script>
|
||||
<script src="/api/static/common.js"></script>
|
||||
<script src="/api/static/whiteboard_admin.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/* 白板管理页:拉取 /api/admin/whiteboards、渲染表格、删除、新建并跳转。 */
|
||||
/* 白板管理页:拉取 /api/admin/wb、渲染表格、删除、新建并跳转。 */
|
||||
(function () {
|
||||
"use strict";
|
||||
const { el, toast, fmtTime, api } = window.ZK;
|
||||
@@ -11,14 +11,14 @@
|
||||
newBtn.addEventListener("click", () => {
|
||||
// 生成一个随机 board_id 并打开(访问即创建)
|
||||
const id = "b_" + Math.random().toString(36).slice(2, 10);
|
||||
window.open(`/whiteboard/${id}`, "_blank");
|
||||
window.open(`/api/wb-page/${id}`, "_blank");
|
||||
});
|
||||
|
||||
async function load() {
|
||||
listEl.innerHTML = '<div class="skel">加载中…</div>';
|
||||
countEl.textContent = "";
|
||||
try {
|
||||
const res = await api("/api/admin/whiteboards?limit=500&offset=0");
|
||||
const res = await api("/api/admin/wb?limit=500&offset=0");
|
||||
if (!res.ok) throw new Error("HTTP " + res.status);
|
||||
const body = await res.json();
|
||||
render(body.items || []);
|
||||
@@ -47,9 +47,9 @@
|
||||
for (const b of items) {
|
||||
const row = el("tr", null,
|
||||
el("td", { class: "col-id" },
|
||||
el("a", { class: "bid link", href: `/whiteboard/${b.board_id}`, target: "_blank" }, b.board_id)
|
||||
el("a", { class: "bid link", href: `/api/wb-page/${b.board_id}`, target: "_blank" }, b.board_id)
|
||||
),
|
||||
el("td", { class: "col-mods mono" }, String(b.stroke_count ?? 0)),
|
||||
el("td", { class: "col-mods mono" }, String(b.edit_count ?? 0)),
|
||||
el("td", { class: "col-created muted" }, fmtTime(b.created_at)),
|
||||
el("td", { class: "col-updated muted" }, fmtTime(b.updated_at)),
|
||||
el("td", { class: "col-act" },
|
||||
@@ -67,7 +67,7 @@
|
||||
async function remove(b, row) {
|
||||
if (!confirm(`确定删除白板「${b.board_id}」?\n所有在线协作者会被断开,内容不可恢复。`)) return;
|
||||
try {
|
||||
const res = await api(`/api/admin/whiteboards/${encodeURIComponent(b.board_id)}`, { method: "DELETE" });
|
||||
const res = await api(`/api/admin/wb/${encodeURIComponent(b.board_id)}`, { method: "DELETE" });
|
||||
if (res.status === 404) { toast("白板已不存在"); }
|
||||
else if (!res.ok) throw new Error("HTTP " + res.status);
|
||||
row.classList.add("row-removed");
|
||||
|
||||
@@ -8,7 +8,7 @@ import urllib.request
|
||||
|
||||
import websockets
|
||||
|
||||
BASE_WS = "ws://127.0.0.1:6890/ws/whiteboard"
|
||||
BASE_WS = "ws://127.0.0.1:6867/api/ws/wb"
|
||||
BOARD = "kicktest"
|
||||
AUTH = "Basic YTo2NjUxMTMxNQ==" # a:66511315
|
||||
|
||||
@@ -22,7 +22,7 @@ async def main() -> None:
|
||||
|
||||
# 通过 REST 删除白板(带 Basic Auth)
|
||||
req = urllib.request.Request(
|
||||
f"http://127.0.0.1:6890/api/admin/whiteboards/{BOARD}",
|
||||
f"http://127.0.0.1:6867/api/admin/wb/{BOARD}",
|
||||
method="DELETE",
|
||||
headers={"Authorization": AUTH},
|
||||
)
|
||||
|
||||
@@ -1,22 +1,24 @@
|
||||
"""白板 WebSocket 端到端烟测:两客户端实时同步 + 心跳 + 清空。
|
||||
"""白板 WebSocket 端到端烟测(文本记事本):两客户端实时同步 + 心跳 + 清空。
|
||||
|
||||
验证:
|
||||
1. 客户端 A 连入 -> 收到 init
|
||||
1. 客户端 A 连入 -> 收到 init(含 content/version)
|
||||
2. 客户端 B 连入 -> 收到 init
|
||||
3. A 画一笔 -> B 收到 stroke 广播(A 不收自己的)
|
||||
3. A 编辑文本 -> B 收到 update(A 不收自己的)
|
||||
4. 心跳 ping -> pong
|
||||
5. A 清空 -> A、B 都收到 cleared
|
||||
6. 停发心跳的连接会被服务端 reaper 移除(15s,这里只验证 ping/pong 即可,reaper 已在 hub 单测覆盖)
|
||||
6. 持久化:重连后 init 应返回清空后的内容
|
||||
7. edit_count 累计
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import urllib.request
|
||||
|
||||
import websockets
|
||||
|
||||
BASE_WS = "ws://127.0.0.1:6890/ws/whiteboard"
|
||||
BASE_WS = "ws://127.0.0.1:6867/api/ws/wb"
|
||||
BOARD = "e2etest"
|
||||
|
||||
|
||||
@@ -31,7 +33,6 @@ async def recv_msg(ws, timeout=2.0) -> dict | None:
|
||||
async def main() -> None:
|
||||
async with websockets.connect(f"{BASE_WS}/{BOARD}") as a, \
|
||||
websockets.connect(f"{BASE_WS}/{BOARD}") as b:
|
||||
# hello
|
||||
await a.send(json.dumps({"type": "hello", "client_id": "A"}))
|
||||
await b.send(json.dumps({"type": "hello", "client_id": "B"}))
|
||||
|
||||
@@ -42,18 +43,18 @@ async def main() -> None:
|
||||
assert init_a and init_a["type"] == "init"
|
||||
assert init_b and init_b["type"] == "init"
|
||||
|
||||
# A 画一笔
|
||||
stroke = {"points": [[10, 10], [20, 20]], "color": "#1565c0", "width": 3}
|
||||
await a.send(json.dumps({"type": "stroke", "stroke": stroke}))
|
||||
# A 不应收到自己的(排除发送者)
|
||||
# A 编辑文本
|
||||
text = "Hello, this is a shared note.\nSecond line."
|
||||
await a.send(json.dumps({"type": "edit", "content": text}))
|
||||
# A 不应收到自己的 update
|
||||
echo = await recv_msg(a, timeout=1.0)
|
||||
print("A self-echo (expect None):", echo)
|
||||
assert echo is None, "发送者不应收到自己的 stroke"
|
||||
# B 应收到
|
||||
assert echo is None, "发送者不应收到自己的 update"
|
||||
# B 应收到 update
|
||||
got = await recv_msg(b)
|
||||
print("B recv:", got.get("type") if got else None, "client_id=", got.get("client_id") if got else None)
|
||||
assert got and got["type"] == "stroke" and got["client_id"] == "A"
|
||||
assert got["stroke"] == stroke
|
||||
print("B recv:", got.get("type") if got else None, "content=", repr(got.get("content")) if got else None)
|
||||
assert got and got["type"] == "update" and got["client_id"] == "A"
|
||||
assert got["content"] == text
|
||||
|
||||
# 心跳
|
||||
await a.send(json.dumps({"type": "ping"}))
|
||||
@@ -70,20 +71,19 @@ async def main() -> None:
|
||||
assert cleared_b and cleared_b["type"] == "cleared" and cleared_b["client_id"] == "B"
|
||||
assert cleared_a and cleared_a["type"] == "cleared" and cleared_a["client_id"] == "B"
|
||||
|
||||
# 验证持久化:重连后 init 应 strokes 为空(已清空)
|
||||
# 验证持久化:重连后 init 应 content 为空(已清空)
|
||||
async with websockets.connect(f"{BASE_WS}/{BOARD}") as c:
|
||||
await c.send(json.dumps({"type": "hello", "client_id": "C"}))
|
||||
init_c = await recv_msg(c)
|
||||
print("C init after clear, strokes=", init_c.get("strokes") if init_c else None)
|
||||
print("C init after clear, content=", repr(init_c.get("content")) if init_c else None)
|
||||
assert init_c and init_c["type"] == "init"
|
||||
assert init_c["strokes"] == [], "清空后重连应得到空 strokes"
|
||||
assert init_c["content"] == "", "清空后重连应得到空 content"
|
||||
|
||||
# 验证 stroke_count 累计(之前 1 笔 + 1 次清空 = 2)
|
||||
import urllib.request
|
||||
with urllib.request.urlopen(f"http://127.0.0.1:6890/whiteboard/{BOARD}") as r:
|
||||
# 验证 edit_count 累计(1 次编辑 + 1 次清空 = 2)
|
||||
with urllib.request.urlopen(f"http://127.0.0.1:6867/api/wb/{BOARD}") as r:
|
||||
meta = json.load(r)
|
||||
print("stroke_count after ops:", meta["stroke_count"])
|
||||
assert meta["stroke_count"] == 2, "1 笔 + 1 清空 = 2 次修改"
|
||||
print("edit_count after ops:", meta["edit_count"])
|
||||
assert meta["edit_count"] == 2, "1 编辑 + 1 清空 = 2 次"
|
||||
|
||||
print("\nWS 端到端全部通过 ✅")
|
||||
|
||||
|
||||
284
tests/test_pdf_service.py
Normal file
284
tests/test_pdf_service.py
Normal file
@@ -0,0 +1,284 @@
|
||||
"""PDF 转换服务测试(pytest + TestClient)。
|
||||
|
||||
用 SQLite 内存库覆盖 get_db,免依赖 MySQL;用临时目录覆盖上传根目录。
|
||||
覆盖完整链路:上传 epub -> 轮询至 done -> 下载有效 PDF -> 用户软删 ->
|
||||
管理页仍可见 -> 管理员硬删 -> 真正删除。另测大小超限与格式校验。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import time
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
from sqlalchemy import create_engine, text
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from app import config as config_module
|
||||
from app import database as db_module
|
||||
from app.database import Base, get_db
|
||||
from app.models.uploaded_file import UploadedFile # noqa: F401 注册映射
|
||||
|
||||
|
||||
# ---------------- fixtures ----------------
|
||||
|
||||
|
||||
def _make_epub_bytes() -> bytes:
|
||||
"""构造一个最小的合法 epub(含 1 章节 + 1 PNG 图片 + CSS)。"""
|
||||
from PIL import Image # 已是 weasyprint 依赖间接项,环境可用
|
||||
|
||||
png = io.BytesIO()
|
||||
Image.new("RGBA", (8, 8), (26, 95, 180, 255)).save(png, format="PNG")
|
||||
png_bytes = png.getvalue()
|
||||
|
||||
ch = (
|
||||
'<?xml version="1.0" encoding="UTF-8"?>'
|
||||
'<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Ch1</title>'
|
||||
'<link rel="stylesheet" type="text/css" href="style.css"/></head>'
|
||||
'<body><h1>Hello PDF</h1><p>Test paragraph for conversion.</p>'
|
||||
'<p><img src="img.png" alt="pic"/></p></body></html>'
|
||||
)
|
||||
opf = (
|
||||
'<?xml version="1.0" encoding="UTF-8"?>'
|
||||
'<package xmlns="http://www.idpf.org/2007/opf" version="2.0" unique-identifier="Bid">'
|
||||
'<metadata xmlns:dc="http://purl.org/dc/elements/1.1/">'
|
||||
'<dc:title>T</dc:title><dc:identifier id="Bid">urn:uuid:t</dc:identifier>'
|
||||
'<dc:language>en</dc:language></metadata>'
|
||||
'<manifest>'
|
||||
'<item id="ncx" href="toc.ncx" media-type="application/x-dtbncx+xml"/>'
|
||||
'<item id="css" href="style.css" media-type="text/css"/>'
|
||||
'<item id="img" href="img.png" media-type="image/png"/>'
|
||||
'<item id="ch1" href="ch1.xhtml" media-type="application/xhtml+xml"/>'
|
||||
'</manifest><spine toc="ncx"><itemref idref="ch1"/></spine></package>'
|
||||
)
|
||||
ncx = (
|
||||
'<?xml version="1.0" encoding="UTF-8"?>'
|
||||
'<ncx xmlns="http://www.daisy.org/z3986/2005/ncx/" version="2005-1">'
|
||||
'<head><meta name="dtb:uid" content="urn:uuid:t"/></head>'
|
||||
'<docTitle><text>T</text></docTitle>'
|
||||
'<navMap><navPoint id="n1" playOrder="1"><navLabel><text>Ch1</text></navLabel>'
|
||||
'<content src="ch1.xhtml"/></navPoint></navMap></ncx>'
|
||||
)
|
||||
css = "h1{color:#1a5fb4} p{line-height:1.6}"
|
||||
|
||||
buf = io.BytesIO()
|
||||
with zipfile.ZipFile(buf, "w") as z:
|
||||
z.writestr("mimetype", "application/epub+zip", compress_type=zipfile.ZIP_STORED)
|
||||
z.writestr("OEBPS/content.opf", opf, compress_type=zipfile.ZIP_DEFLATED)
|
||||
z.writestr("OEBPS/toc.ncx", ncx, compress_type=zipfile.ZIP_DEFLATED)
|
||||
z.writestr("OEBPS/style.css", css, compress_type=zipfile.ZIP_DEFLATED)
|
||||
z.writestr("OEBPS/img.png", png_bytes, compress_type=zipfile.ZIP_DEFLATED)
|
||||
z.writestr("OEBPS/ch1.xhtml", ch, compress_type=zipfile.ZIP_DEFLATED)
|
||||
z.writestr("META-INF/container.xml",
|
||||
'<?xml version="1.0"?><container version="1.0" '
|
||||
'xmlns="urn:oasis:names:tc:opendocument:xmlns:container">'
|
||||
'<rootfiles><rootfile full-path="OEBPS/content.opf" '
|
||||
'media-type="application/oebps-package+xml"/></rootfiles></container>',
|
||||
compress_type=zipfile.ZIP_DEFLATED)
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client(tmp_path, monkeypatch):
|
||||
"""构造用隔离 MySQL 测试库 + 临时上传目录的 TestClient。
|
||||
|
||||
用独立库 zikai_filesvc_test(与生产库隔离),每个 fixture 重建表,准确测试生产
|
||||
行为(MySQL BigInteger 自增、线程安全连接)。后台转换经 asyncio.to_thread 在
|
||||
独立线程执行,MySQL 连接池天然支持跨线程。
|
||||
"""
|
||||
real_settings = config_module.get_settings()
|
||||
# 用独立测试库 zikai_filesvc_test(与生产库隔离),准确测试生产行为
|
||||
test_settings = real_settings.model_copy(deep=True)
|
||||
test_settings.database.database = "zikai_filesvc_test"
|
||||
test_settings.pdf.convert_timeout_seconds = 60
|
||||
|
||||
# 全局 get_settings 被 lru_cache;config 与 database 两个模块各自 import 了它,
|
||||
# 都需 patch,使后台线程(经 database.get_session_local)也读到测试库
|
||||
monkeypatch.setattr(config_module, "get_settings", lambda: test_settings)
|
||||
monkeypatch.setattr(db_module, "get_settings", lambda: test_settings)
|
||||
# 清掉已建的全局引擎/SessionLocal,下次 get_session_local() 用测试库重建
|
||||
db_module.dispose_engine()
|
||||
|
||||
upload_dir = tmp_path / "uploads"
|
||||
upload_dir.mkdir()
|
||||
|
||||
# 复用全局 SessionLocal(指向测试库),保证请求路径与后台线程用同一库
|
||||
SessionLocal = db_module.get_session_local()
|
||||
|
||||
# 覆盖 get_db(仍用全局 SessionLocal,但确保请求结束关闭)
|
||||
def override_get_db():
|
||||
db = SessionLocal()
|
||||
try:
|
||||
yield db
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
# 上传目录指向临时目录
|
||||
monkeypatch.setattr(test_settings, "storage", test_settings.storage)
|
||||
test_settings.storage.upload_dir = str(upload_dir)
|
||||
# upload_service / pdf_service 内部 import 了 get_settings,需 patch 其引用
|
||||
import app.services.upload_service as us
|
||||
import app.services.pdf_service as ps
|
||||
import app.services.pdf_converter as pc # noqa: F401
|
||||
monkeypatch.setattr(us, "get_settings", lambda: test_settings)
|
||||
monkeypatch.setattr(ps, "get_settings", lambda: test_settings)
|
||||
|
||||
# 每次测试前清空重建表,保证隔离
|
||||
Base.metadata.drop_all(bind=db_module.get_engine())
|
||||
Base.metadata.create_all(bind=db_module.get_engine())
|
||||
|
||||
from app.main import app
|
||||
app.dependency_overrides[get_db] = override_get_db
|
||||
with TestClient(app) as c:
|
||||
yield c
|
||||
app.dependency_overrides.clear()
|
||||
Base.metadata.drop_all(bind=db_module.get_engine())
|
||||
db_module.dispose_engine()
|
||||
|
||||
|
||||
def _wait_done(client, job_id, cookie, timeout=60):
|
||||
"""轮询任务状态直到 done/failed 或超时。"""
|
||||
deadline = time.time() + timeout
|
||||
last = None
|
||||
while time.time() < deadline:
|
||||
r = client.get(f"/api/pdf/jobs/{job_id}", cookies=cookie)
|
||||
assert r.status_code == 200, r.text
|
||||
last = r.json()
|
||||
if last["status"] in ("done", "failed"):
|
||||
return last
|
||||
time.sleep(0.3)
|
||||
raise AssertionError(f"任务未在 {timeout}s 内完成,最后状态: {last}")
|
||||
|
||||
|
||||
# 管理页 Basic Auth 凭据(与 config.yaml 的 docs 段保持一致)
|
||||
ADMIN_AUTH = ("a", "66511315")
|
||||
|
||||
|
||||
# ---------------- 测试 ----------------
|
||||
|
||||
|
||||
class TestSubmitAndConvert:
|
||||
def test_upload_epub_converts_and_downloads(self, client):
|
||||
epub = _make_epub_bytes()
|
||||
# 首次上传:无 cookie,应下发新 cookie
|
||||
r = client.post(
|
||||
"/api/pdf/jobs",
|
||||
files={"file": ("test.epub", epub, "application/epub+zip")},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
body = r.json()
|
||||
assert body["set_cookie"] is True
|
||||
assert "zk_pdf" in r.headers.get("set-cookie", "")
|
||||
job = body["job"]
|
||||
assert job["status"] == "pending"
|
||||
job_id = job["id"]
|
||||
cookie = {"zk_pdf": r.cookies.get("zk_pdf")}
|
||||
|
||||
# 轮询至完成
|
||||
final = _wait_done(client, job_id, cookie)
|
||||
assert final["status"] == "done", final
|
||||
assert final["progress"] == 100
|
||||
|
||||
# 下载产物:应为有效 PDF
|
||||
d = client.get(f"/api/pdf/jobs/{job_id}/download", cookies=cookie)
|
||||
assert d.status_code == 200, d.text
|
||||
assert d.headers["content-type"] == "application/pdf"
|
||||
assert d.content[:5] == b"%PDF-"
|
||||
assert len(d.content) > 100
|
||||
|
||||
def test_user_list_shows_own_jobs(self, client):
|
||||
epub = _make_epub_bytes()
|
||||
r = client.post("/api/pdf/jobs", files={"file": ("a.epub", epub, "application/epub+zip")})
|
||||
cookie = {"zk_pdf": r.cookies.get("zk_pdf")}
|
||||
lst = client.get("/api/pdf/jobs", cookies=cookie)
|
||||
assert lst.status_code == 200
|
||||
assert lst.json()["total"] == 1
|
||||
|
||||
# 另一用户(清空 client cookie jar 模拟全新浏览器,应下发新 cookie)
|
||||
client.cookies.clear()
|
||||
r2 = client.post("/api/pdf/jobs", files={"file": ("b.epub", epub, "application/epub+zip")})
|
||||
cookie2 = {"zk_pdf": r2.cookies.get("zk_pdf")}
|
||||
assert cookie2["zk_pdf"] != cookie["zk_pdf"] # 确是不同用户
|
||||
lst2 = client.get("/api/pdf/jobs", cookies=cookie2)
|
||||
assert lst2.json()["total"] == 1 # 只有自己的 1 个
|
||||
|
||||
def test_rejects_non_epub(self, client):
|
||||
r = client.post(
|
||||
"/api/pdf/jobs",
|
||||
files={"file": ("note.txt", b"hello world", "text/plain")},
|
||||
)
|
||||
assert r.status_code == 400
|
||||
|
||||
def test_rejects_oversize(self, client, monkeypatch):
|
||||
# 把上限调到极小,避免真的构造 250MB 文件
|
||||
import app.services.pdf_service as ps
|
||||
real = ps.get_settings()
|
||||
small = real.model_copy(deep=True)
|
||||
small.pdf.max_size_bytes = 100
|
||||
monkeypatch.setattr(ps, "get_settings", lambda: small)
|
||||
monkeypatch.setattr(ps, "get_settings", lambda: small)
|
||||
# controller 的 _resolve_cookie 也读 get_settings,但 submit 内 service 用 ps.get_settings
|
||||
epub = _make_epub_bytes()
|
||||
r = client.post("/api/pdf/jobs", files={"file": ("big.epub", epub, "application/epub+zip")})
|
||||
assert r.status_code == 413
|
||||
|
||||
|
||||
class TestDeleteSemantics:
|
||||
def test_user_soft_delete_admin_still_visible_then_admin_hard_delete(self, client):
|
||||
epub = _make_epub_bytes()
|
||||
r = client.post("/api/pdf/jobs", files={"file": ("del.epub", epub, "application/epub+zip")})
|
||||
cookie = {"zk_pdf": r.cookies.get("zk_pdf")}
|
||||
job_id = r.json()["job"]["id"]
|
||||
_wait_done(client, job_id, cookie)
|
||||
|
||||
# 用户软删
|
||||
d = client.delete(f"/api/pdf/jobs/{job_id}", cookies=cookie)
|
||||
assert d.status_code == 200
|
||||
assert d.json()["deleted"] is True
|
||||
|
||||
# 用户列表不再可见
|
||||
lst = client.get("/api/pdf/jobs", cookies=cookie)
|
||||
assert lst.json()["total"] == 0
|
||||
|
||||
# 管理页仍可见且标注已删除
|
||||
al = client.get("/api/admin/pdf/jobs", auth=ADMIN_AUTH)
|
||||
assert al.status_code == 200
|
||||
items = al.json()["items"]
|
||||
assert any(it["id"] == job_id and it["user_deleted"] is True for it in items)
|
||||
assert any(it["id"] == job_id and it["deleted_at"] is not None for it in items)
|
||||
|
||||
# 用户已无法下载(已软删)
|
||||
dl = client.get(f"/api/pdf/jobs/{job_id}/download", cookies=cookie)
|
||||
assert dl.status_code == 404
|
||||
|
||||
# 管理员硬删
|
||||
ad = client.delete(f"/api/admin/pdf/jobs/{job_id}", auth=ADMIN_AUTH)
|
||||
assert ad.status_code == 200
|
||||
assert ad.json()["deleted"] is True
|
||||
|
||||
# 管理页也不再可见
|
||||
al2 = client.get("/api/admin/pdf/jobs", auth=ADMIN_AUTH)
|
||||
assert not any(it["id"] == job_id for it in al2.json()["items"])
|
||||
|
||||
def test_admin_requires_auth(self, client):
|
||||
r = client.get("/api/admin/pdf/jobs")
|
||||
assert r.status_code == 401
|
||||
r = client.get("/api/admin/pdf/jobs", auth=("a", "wrong"))
|
||||
assert r.status_code == 401
|
||||
r = client.get("/api/admin/pdf/jobs", auth=ADMIN_AUTH)
|
||||
assert r.status_code == 200
|
||||
|
||||
def test_user_cannot_access_others_job(self, client):
|
||||
epub = _make_epub_bytes()
|
||||
r1 = client.post("/api/pdf/jobs", files={"file": ("x.epub", epub, "application/epub+zip")})
|
||||
job_id = r1.json()["job"]["id"]
|
||||
# 另一用户访问(清空 cookie jar 模拟全新浏览器)
|
||||
client.cookies.clear()
|
||||
r2 = client.post("/api/pdf/jobs", files={"file": ("y.epub", epub, "application/epub+zip")})
|
||||
cookie2 = {"zk_pdf": r2.cookies.get("zk_pdf")}
|
||||
assert client.get(f"/api/pdf/jobs/{job_id}", cookies=cookie2).status_code == 404
|
||||
assert client.get(f"/api/pdf/jobs/{job_id}/download", cookies=cookie2).status_code == 404
|
||||
assert client.delete(f"/api/pdf/jobs/{job_id}", cookies=cookie2).status_code == 404
|
||||
Reference in New Issue
Block a user