commit 80b96d236f8267d53ed3c407f01442fc52d57eb6 Author: zikai Date: Tue Jun 23 16:17:31 2026 +0000 init: 从 /root/zikai 根目录迁入 把原本散落在 /root/zikai 的 FastAPI 服务整理到 server/ 子目录， 开启独立 git 与 venv。运行时数据（config.yaml / keys / uploads） 按 .gitignore 留在工作目录但不入仓。 diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..2df6790 --- /dev/null +++ b/.gitignore @@ -0,0 +1,17 @@ +# Python +.venv/ +__pycache__/ +*.py[cod] +*.egg-info/ +.pytest_cache/ + +# Secrets / runtime +config.yaml +keys/ +uploads/ +logs/ +*.pid + +# Editor +.vscode/ +.idea/ diff --git a/README.md b/README.md new file mode 100644 index 0000000..c12ddb0 --- /dev/null +++ b/README.md @@ -0,0 +1,135 @@ +# zikai file service + +A Python web service (FastAPI) for **f.zikai.wang** that provides host +monitoring and large-file upload over **both HTTP and SFTP**. It follows a +Spring-style layered architecture (`controllers` → `services` → `dao`, plus +`models` and `schemas`), ships with auto-generated API docs, and runs entirely +from a self-contained `.venv`. + +## Features + +- `GET /api/system/status` — CPU, memory, and per-disk usage (via `psutil`). +- `POST /api/files/upload` — **streamed** multipart upload (flat memory, multi-GB friendly), with SHA-256. +- `GET /api/files`, `GET /api/files/{id}`, `GET /api/files/{id}/download`. +- **Embedded SFTP server** (asyncssh) supporting **password + public-key** auth, sharing the same storage as HTTP. +- `/docs` (Swagger UI) and `/redoc` — interactive, auto-lists all APIs. +- Metadata persisted in an **independent MySQL database** (`zikai_filesvc`). +- `start.sh` / `stop.sh` lifecycle; `setup.sh` for one-time provisioning. + +## Architecture (Spring-style layers) + +``` +app/ +├── controllers/ # FastAPI routers — HTTP boundary (like @RestController) +├── services/ # business logic (SystemService, UploadService, SFTP server) +├── dao/ # data access objects — the only layer that issues SQL/ORM +├── models/ # SQLAlchemy ORM entities +├── schemas/ # pydantic DTOs (request/response validation) +├── database.py # engine, session, Base, get_db() dependency +├── config.py # typed Settings loaded from config.yaml +└── scripts/ # init_db.py — DB provisioning +``` + +Request flow: **controller** → **service** → **dao** → **ORM model** → MySQL. +The DB session is injected by FastAPI's `get_db` dependency and passed down. + +## Quick start + +```bash +cd /root/zikai +./setup.sh # one-time: venv, deps, provision DB + user, SFTP host key +./start.sh # start HTTP (127.0.0.1:6867) + SFTP (0.0.0.0:2022) +./stop.sh # stop both +``` + +`setup.sh` is re-runnable. It creates the `.venv`, installs `requirements.txt`, +copies `config.example.yaml` → `config.yaml` (if absent), provisions a **new +independent MySQL database and app user** via the local root socket, and +generates the SFTP host key. + +## Access + +| Where | URL | +|------|-----| +| Status page (HTML) | https://f.zikai.wang/api/system/status | +| Status page (JSON) | https://f.zikai.wang/api/system/status?format=json (or `Accept: application/json`) | +| API docs (Swagger) | https://f.zikai.wang/docs **(HTTP Basic Auth — see `docs:` in config.yaml)** | +| API docs (ReDoc) | https://f.zikai.wang/redoc (same auth) | +| Upload | `curl -F file=@big.iso https://f.zikai.wang/api/files/upload` | +| SFTP | `sftp -P 2022 uploader@f.zikai.wang` | + +`/api/system/status` content-negotiates: browsers (`Accept: text/html`) get a +human-readable page with progress bars; API clients get JSON. Force one with +`?format=html` or `?format=json`. + +`/docs`, `/redoc`, and `/openapi.json` require HTTP Basic Auth — the browser +will prompt you. Username + plaintext password live in `config.yaml` under +`docs:`. `/health` and `/` remain public. + +Apache (`/etc/apache2/sites-available/f.zikai.wang-le-ssl.conf`) proxies +`f.zikai.wang` → `127.0.0.1:6867` with `ProxyPreserveHost On`, so the service +only binds the loopback. + +> **Large/slow HTTP uploads:** Apache's proxy leg inherits the global `Timeout 300`. +> For multi-GB transfers over a slow link, prefer **SFTP** (it bypasses the HTTP +> proxy entirely). To raise the HTTP ceiling you can add `ProxyTimeout`/`Timeout` +> in the Apache vhost. + +## Configuration + +All runtime config lives in **`config.yaml`** (git-ignored). See +`config.example.yaml` for the full schema. Notable keys: + +- `server` — bind host/port (keep `127.0.0.1:6867` to match Apache). +- `database` — host/port/user/password/database. The password is auto-generated + and written here by `setup.sh`/`init_db.py`. +- `storage.upload_dir`, `storage.chunk_bytes` (default 1 MiB streaming chunk). +- `sftp` — enabled, host/port, host key + authorized_keys paths, and `users`. + +### Setting the /docs admin password + +Edit `config.yaml` directly — no hashing required: + +```yaml +docs: + enabled: true + username: admin + password: "your-plaintext-password" + realm: "zikai docs" +``` + +Then `./stop.sh && ./start.sh`. The file is root-owned and stays on this +host; comparison is constant-time (`secrets.compare_digest`). + +### Setting SFTP credentials + +**Password auth** — generate a bcrypt hash and put it in `config.yaml`: + +```bash +.venv/bin/python -c "import bcrypt;print(bcrypt.hashpw(b'yourpass',bcrypt.gensalt()).decode())" +# paste the output into sftp.users[].password_hash, then ./stop.sh && ./start.sh +``` + +**Public-key auth** — append each client's public key (OpenSSH format) to +`keys/authorized_keys` (one per line). Clients in `sftp.users[]` may then log +in with either method. + +### Regenerating the DB password + +```bash +.venv/bin/python -m app.scripts.init_db # new random password +KEEP_DB_PASSWORD=1 .venv/bin/python -m app.scripts.init_db # keep current +``` + +## SFTP notes + +- The SFTP server can't traverse Apache's HTTP proxy, so it binds `0.0.0.0:2022` + directly. **Open port 2022** in your firewall for external clients + (FileZilla/WinSCP/scp). +- Sessions are chrooted to the upload root (`uploads/`), shared with HTTP. +- Only users listed in `sftp.users` may connect; only SFTP (no shell/exec) is allowed. + +## Logs & pidfiles + +- HTTP logs → `logs/app.log`; SFTP logs → `logs/sftp.log`. +- Pidfiles: `app.pid`, `sftp.pid` (used by `stop.sh`). diff --git a/app/__init__.py b/app/__init__.py new file mode 100644 index 0000000..920d8fe --- /dev/null +++ b/app/__init__.py @@ -0,0 +1 @@ +"""zikai 文件服务 - FastAPI 应用包。""" diff --git a/app/config.py b/app/config.py new file mode 100644 index 0000000..8fc02ed --- /dev/null +++ b/app/config.py @@ -0,0 +1,103 @@ +"""运行时配置：所有参数与凭据均从项目根目录的 config.yaml 读取。""" + +from __future__ import annotations + +import os +from functools import lru_cache +from pathlib import Path + +import yaml +from pydantic import BaseModel, Field, field_validator + +PROJECT_ROOT = Path(__file__).resolve().parent.parent +DEFAULT_CONFIG_PATH = PROJECT_ROOT / "config.yaml" + + +class ServerConfig(BaseModel): + host: str = "127.0.0.1" + port: int = 6867 + workers: int = 1 + + +class DatabaseConfig(BaseModel): + host: str = "127.0.0.1" + port: int = 3306 + user: str = "zikai_filesvc" + password: str = "" + database: str = "zikai_filesvc" + pool_size: int = 5 + pool_recycle: int = 1800 + + +class StorageConfig(BaseModel): + upload_dir: str = "./uploads" + chunk_bytes: int = 1024 * 1024 + sha256_on_upload: bool = True + + +class SftpUser(BaseModel): + username: str + password_hash: str = "" # bcrypt + public_key: str | None = None # 可选的内联 OpenSSH 公钥 + + +class SftpConfig(BaseModel): + enabled: bool = True + host: str = "0.0.0.0" + port: int = 2022 + host_key_path: str = "./keys/ssh_host_ed25519_key" + authorized_keys_path: str = "./keys/authorized_keys" + users: list[SftpUser] = Field(default_factory=list) + + +class DocsConfig(BaseModel): + """/docs、/redoc、/openapi.json 的 Basic Auth 凭据（明文）。""" + + enabled: bool = True + username: str = "admin" + password: str = "" # 留空 → /docs 返回 503，避免误暴露 + realm: str = "zikai docs" + + @field_validator("password", mode="before") + @classmethod + def _coerce(cls, v): + # 兼容 YAML 把纯数字密码解析成 int 的情况。 + return "" if v is None else str(v) + + +class Settings(BaseModel): + server: ServerConfig = ServerConfig() + database: DatabaseConfig = DatabaseConfig() + storage: StorageConfig = StorageConfig() + sftp: SftpConfig = SftpConfig() + docs: DocsConfig = DocsConfig() + + def db_url(self) -> str: + c = self.database + return ( + f"mysql+pymysql://{c.user}:{c.password}@{c.host}:{c.port}/{c.database}" + "?charset=utf8mb4" + ) + + def resolved_upload_dir(self) -> Path: + return (PROJECT_ROOT / self.storage.upload_dir).resolve() + + +def _load_yaml(path: Path) -> dict: + if not path.exists(): + raise FileNotFoundError( + f"未找到配置文件 {path}；请先 cp config.example.yaml config.yaml 并填值。" + ) + return yaml.safe_load(path.read_text(encoding="utf-8")) or {} + + +@lru_cache(maxsize=1) +def get_settings() -> Settings: + path = Path(os.getenv("CONFIG_PATH", str(DEFAULT_CONFIG_PATH))) + return Settings.model_validate(_load_yaml(path)) + + +def reload_settings() -> Settings: + """清缓存并重新读取，供脚本与测试使用。""" + get_settings.cache_clear() + return get_settings() diff --git a/app/controllers/__init__.py b/app/controllers/__init__.py new file mode 100644 index 0000000..a21b3e2 --- /dev/null +++ b/app/controllers/__init__.py @@ -0,0 +1,6 @@ +"""Controller 层：API 路由。""" + +from .file_controller import router as file_router +from .system_controller import router as system_router + +__all__ = ["file_router", "system_router"] diff --git a/app/controllers/file_controller.py b/app/controllers/file_controller.py new file mode 100644 index 0000000..433aed6 --- /dev/null +++ b/app/controllers/file_controller.py @@ -0,0 +1,72 @@ +"""文件上传 / 列表 / 下载接口。""" + +from __future__ import annotations + +from pathlib import Path + +from fastapi import APIRouter, Depends, File, HTTPException, UploadFile +from fastapi.responses import FileResponse +from sqlalchemy.orm import Session + +from ..database import get_db +from ..dao.uploaded_file_dao import UploadedFileDAO +from ..schemas.file import FileListResponse, FileUploadResponse, UploadedFileOut +from ..services.upload_service import UploadService + +router = APIRouter(prefix="/api/files", tags=["files"]) + + +def _service(db: Session = Depends(get_db)) -> UploadService: + return UploadService(UploadedFileDAO(db)) + + +@router.post( + "/upload", + response_model=FileUploadResponse, + summary="上传单个文件（流式，支持大文件）", + description=( + "multipart/form-data 上传，按 1 MiB 分片流式落盘，内存占用恒定；" + "落盘过程中计算 SHA-256 并入库。\n\n" + "极大或极慢的传输建议改用 SFTP（详见 README），HTTP 链路受 Apache 代理 300s 超时限制。" + ), +) +async def upload_file( + file: UploadFile = File(..., description="要上传的文件"), + service: UploadService = Depends(_service), +) -> FileUploadResponse: + if not file.filename: + raise HTTPException(400, "请求缺少 'file' 字段") + return service.stream_to_disk(file, source="http", uploaded_by="anonymous") + + +@router.get("", response_model=FileListResponse, summary="列出已上传的文件") +def list_files( + limit: int = 100, + offset: int = 0, + service: UploadService = Depends(_service), +) -> FileListResponse: + total, items = service.list_files(limit=limit, offset=offset) + return FileListResponse(total=total, items=items) + + +@router.get("/{file_id}", response_model=UploadedFileOut, summary="查询单个文件元数据") +def get_file(file_id: int, service: UploadService = Depends(_service)) -> UploadedFileOut: + out = service.get_out(file_id) + if out is None: + raise HTTPException(404, "文件不存在") + return out + + +@router.get("/{file_id}/download", summary="下载文件") +def download_file(file_id: int, service: UploadService = Depends(_service)) -> FileResponse: + out = service.get_out(file_id) + if out is None: + raise HTTPException(404, "文件不存在") + path: Path | None = service.resolve_disk_path(file_id) + if path is None or not path.exists(): + raise HTTPException(410, "文件实体已不在磁盘上") + return FileResponse( + path=str(path), + media_type=out.content_type or "application/octet-stream", + filename=out.original_filename, + ) diff --git a/app/controllers/system_controller.py b/app/controllers/system_controller.py new file mode 100644 index 0000000..6adb963 --- /dev/null +++ b/app/controllers/system_controller.py @@ -0,0 +1,39 @@ +"""系统监控接口。""" + +from __future__ import annotations + +from fastapi import APIRouter, Request +from fastapi.responses import HTMLResponse, JSONResponse, Response + +from ..schemas.system import SystemStatus +from ..services.system_service import SystemService +from ..views.system_status_html import render as render_status_html + +router = APIRouter(prefix="/api/system", tags=["system"]) + + +def _wants_html(request: Request, fmt: str | None) -> bool: + if fmt == "json": + return False + if fmt == "html": + return True + accept = request.headers.get("accept", "").lower() + return "text/html" in accept and "application/json" not in accept + + +@router.get( + "/status", + summary="主机资源状态（HTML 或 JSON）", + description=( + "返回 CPU、内存、磁盘的实时使用情况（数据来源 psutil）。\n\n" + "**内容协商**：浏览器（`Accept: text/html`）返回 HTML 页面，API 客户端返回 JSON。" + "可用 `?format=html` / `?format=json` 强制指定。" + ), + response_model=SystemStatus, + responses={200: {"content": {"application/json": {}, "text/html": {"schema": {"type": "string"}}}}}, +) +def get_status(request: Request, format: str | None = None) -> Response: + status = SystemService().get_status() + if _wants_html(request, format): + return HTMLResponse(render_status_html(status)) + return JSONResponse(status.model_dump(mode="json")) diff --git a/app/dao/__init__.py b/app/dao/__init__.py new file mode 100644 index 0000000..211aac4 --- /dev/null +++ b/app/dao/__init__.py @@ -0,0 +1,5 @@ +"""DAO 层：数据库访问的唯一入口。""" + +from .uploaded_file_dao import UploadedFileDAO + +__all__ = ["UploadedFileDAO"] diff --git a/app/dao/uploaded_file_dao.py b/app/dao/uploaded_file_dao.py new file mode 100644 index 0000000..e8227c6 --- /dev/null +++ b/app/dao/uploaded_file_dao.py @@ -0,0 +1,39 @@ +"""UploadedFile 的 DAO。""" + +from __future__ import annotations + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from ..models.uploaded_file import UploadedFile + + +class UploadedFileDAO: + def __init__(self, db: Session) -> None: + self.db = db + + def create(self, file: UploadedFile) -> UploadedFile: + self.db.add(file) + self.db.commit() + self.db.refresh(file) + return file + + def get_by_id(self, file_id: int) -> UploadedFile | None: + return self.db.get(UploadedFile, file_id) + + def list(self, limit: int = 100, offset: int = 0) -> list[UploadedFile]: + stmt = ( + select(UploadedFile) + .order_by(UploadedFile.uploaded_at.desc()) + .limit(limit) + .offset(offset) + ) + return list(self.db.scalars(stmt).all()) + + def delete(self, file_id: int) -> bool: + file = self.get_by_id(file_id) + if file is None: + return False + self.db.delete(file) + self.db.commit() + return True diff --git a/app/database.py b/app/database.py new file mode 100644 index 0000000..431f81f --- /dev/null +++ b/app/database.py @@ -0,0 +1,41 @@ +"""数据库引擎、Session 与 Declarative Base。""" + +from __future__ import annotations + +from collections.abc import Generator + +from sqlalchemy import create_engine +from sqlalchemy.orm import DeclarativeBase, Session, sessionmaker + +from .config import get_settings + +_settings = get_settings() + +engine = create_engine( + _settings.db_url(), + pool_pre_ping=True, + pool_size=_settings.database.pool_size, + pool_recycle=_settings.database.pool_recycle, + future=True, +) + +SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False, future=True) + + +class Base(DeclarativeBase): + pass + + +def get_db() -> Generator[Session, None, None]: + """FastAPI 依赖：为每个请求产出一个 Session。""" + db = SessionLocal() + try: + yield db + finally: + db.close() + + +def init_db_schema() -> None: + """按需建表（幂等）。先导入 models 以注册映射。""" + from . import models # noqa: F401 + Base.metadata.create_all(bind=engine) diff --git a/app/main.py b/app/main.py new file mode 100644 index 0000000..81240fa --- /dev/null +++ b/app/main.py @@ -0,0 +1,87 @@ +"""FastAPI 应用工厂。 + +路由概览： + GET / -> 仅返回版本号 + GET /docs -> Swagger UI（Basic Auth） + GET /redoc -> ReDoc （Basic Auth） + GET /openapi.json -> OpenAPI 文档（Basic Auth） + GET /health -> 存活探针（公开） + GET /api/... -> 业务接口 +""" + +from __future__ import annotations + +import logging +from contextlib import asynccontextmanager + +from fastapi import Depends, FastAPI +from fastapi.openapi.docs import get_redoc_html, get_swagger_ui_html +from fastapi.responses import JSONResponse, PlainTextResponse + +from .controllers import file_router, system_router +from .database import init_db_schema +from .security import require_docs_auth + +logger = logging.getLogger("zikai") + + +@asynccontextmanager +async def lifespan(app: FastAPI): + try: + init_db_schema() + logger.info("数据库表已就绪。") + except Exception as exc: # pragma: no cover + logger.error("初始化数据库失败：%s", exc) + yield + + +def create_app() -> FastAPI: + app = FastAPI( + title="zikai file service", + description=( + "f.zikai.wang 的文件上传与主机监控服务。\n\n" + "- `GET /api/system/status` — CPU / 内存 / 磁盘（HTML 或 JSON）\n" + "- `POST /api/files/upload` — 大文件流式上传\n" + "- 内置 SFTP 服务（详见 README）\n\n" + "/docs 等接口需 Basic Auth，凭据见 config.yaml 的 docs 段。" + ), + version="1.0.0", + docs_url=None, + redoc_url=None, + openapi_url=None, + lifespan=lifespan, + ) + + app.include_router(system_router) + app.include_router(file_router) + + # 受 Basic Auth 保护的文档接口 + @app.get("/openapi.json", include_in_schema=False) + def protected_openapi(_: str = Depends(require_docs_auth)) -> JSONResponse: + return JSONResponse(app.openapi()) + + @app.get("/docs", include_in_schema=False) + def protected_docs(_: str = Depends(require_docs_auth)): + return get_swagger_ui_html( + openapi_url="/openapi.json", title="zikai docs", swagger_favicon_url="" + ) + + @app.get("/redoc", include_in_schema=False) + def protected_redoc(_: str = Depends(require_docs_auth)): + return get_redoc_html( + openapi_url="/openapi.json", title="zikai docs", redoc_favicon_url="" + ) + + # 公开元信息（不暴露接口列表） + @app.get("/", include_in_schema=False, response_class=PlainTextResponse) + def root() -> PlainTextResponse: + return PlainTextResponse(f"zikai {app.version}\n") + + @app.get("/health", include_in_schema=False) + def health() -> dict: + return {"status": "ok"} + + return app + + +app = create_app() diff --git a/app/models/__init__.py b/app/models/__init__.py new file mode 100644 index 0000000..fa254e9 --- /dev/null +++ b/app/models/__init__.py @@ -0,0 +1,5 @@ +"""ORM 模型包；import 本包即把所有实体注册到 Base.metadata。""" + +from .uploaded_file import UploadedFile + +__all__ = ["UploadedFile"] diff --git a/app/models/uploaded_file.py b/app/models/uploaded_file.py new file mode 100644 index 0000000..112942d --- /dev/null +++ b/app/models/uploaded_file.py @@ -0,0 +1,30 @@ +"""UploadedFile 实体。""" + +from __future__ import annotations + +from datetime import datetime + +from sqlalchemy import BigInteger, DateTime, String, func +from sqlalchemy.orm import Mapped, mapped_column + +from ..database import Base + + +class UploadedFile(Base): + __tablename__ = "uploaded_file" + + id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True) + # upload_dir 下的相对路径，例如 2026/06/.bin + storage_path: Mapped[str] = mapped_column(String(512), nullable=False) + original_filename: Mapped[str] = mapped_column(String(512), nullable=False) + content_type: Mapped[str] = mapped_column(String(255), nullable=False, default="") + size_bytes: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + sha256: Mapped[str] = mapped_column(String(64), nullable=False, default="") + source: Mapped[str] = mapped_column(String(32), nullable=False, default="http") # http / sftp + uploaded_by: Mapped[str] = mapped_column(String(128), nullable=False, default="") + uploaded_at: Mapped[datetime] = mapped_column( + DateTime, server_default=func.now(), nullable=False + ) + + def __repr__(self) -> str: # pragma: no cover + return f"" diff --git a/app/schemas/__init__.py b/app/schemas/__init__.py new file mode 100644 index 0000000..b02c0ab --- /dev/null +++ b/app/schemas/__init__.py @@ -0,0 +1,13 @@ +"""请求 / 响应的 Pydantic DTO。""" + +from .file import FileListResponse, FileUploadResponse, UploadedFileOut +from .system import DiskUsage, MemoryUsage, SystemStatus + +__all__ = [ + "DiskUsage", + "FileListResponse", + "FileUploadResponse", + "MemoryUsage", + "SystemStatus", + "UploadedFileOut", +] diff --git a/app/schemas/file.py b/app/schemas/file.py new file mode 100644 index 0000000..136b97d --- /dev/null +++ b/app/schemas/file.py @@ -0,0 +1,35 @@ +"""文件接口 DTO。""" + +from __future__ import annotations + +from datetime import datetime + +from pydantic import BaseModel, Field + + +class UploadedFileOut(BaseModel): + id: int + storage_path: str + original_filename: str + content_type: str + size_bytes: int + sha256: str + source: str + uploaded_by: str + uploaded_at: datetime + + model_config = {"from_attributes": True} + + +class FileUploadResponse(BaseModel): + id: int = Field(..., description="新写入的数据库行 ID") + filename: str = Field(..., description="客户端原始文件名") + size_bytes: int + sha256: str + storage_path: str + uploaded_at: datetime + + +class FileListResponse(BaseModel): + total: int + items: list[UploadedFileOut] diff --git a/app/schemas/system.py b/app/schemas/system.py new file mode 100644 index 0000000..b2ec179 --- /dev/null +++ b/app/schemas/system.py @@ -0,0 +1,31 @@ +"""系统状态接口 DTO。""" + +from __future__ import annotations + +from pydantic import BaseModel, Field + + +class MemoryUsage(BaseModel): + total: int = Field(..., description="物理内存总量（字节）") + available: int = Field(..., description="可用内存（字节）") + used: int = Field(..., description="已用内存（字节）") + percent: float = Field(..., description="使用率 0-100") + + +class DiskUsage(BaseModel): + device: str + mountpoint: str + fstype: str + total: int + used: int + free: int + percent: float + + +class SystemStatus(BaseModel): + hostname: str + cpu_percent: float = Field(..., description="CPU 使用率（所有核平均）") + cpu_count: int + memory: MemoryUsage + disks: list[DiskUsage] + uptime_seconds: float diff --git a/app/scripts/__init__.py b/app/scripts/__init__.py new file mode 100644 index 0000000..55f93e6 --- /dev/null +++ b/app/scripts/__init__.py @@ -0,0 +1 @@ +"""一次性脚本。""" diff --git a/app/scripts/init_db.py b/app/scripts/init_db.py new file mode 100644 index 0000000..0011826 --- /dev/null +++ b/app/scripts/init_db.py @@ -0,0 +1,85 @@ +"""一次性建库脚本。 + +通过本机 socket 以 root 调用 mysql： + 1. 创建独立数据库； + 2. 创建独立账户，生成随机密码并授权； + 3. 把新密码写回 config.yaml； + 4. 导入 sql/schema.sql。 + +幂等。设置环境变量 KEEP_DB_PASSWORD=1 可复用 config.yaml 中的现有密码。 + + python -m app.scripts.init_db +""" + +from __future__ import annotations + +import os +import secrets +import subprocess +import sys + +import yaml + +from ..config import PROJECT_ROOT, reload_settings + +SCHEMA_PATH = PROJECT_ROOT / "sql" / "schema.sql" +CFG_PATH = PROJECT_ROOT / "config.yaml" + + +def _mysql_root(args: list[str], input_str: str | None = None) -> str: + cmd = ["sudo", "-n", "mysql", "--protocol=socket", "-uroot", *args] + proc = subprocess.run(cmd, input=input_str, text=True, capture_output=True) + if proc.returncode != 0: + sys.stderr.write(proc.stderr) + raise SystemExit(f"mysql 执行失败 (exit {proc.returncode})") + return proc.stdout + + +def _save_password(db: str, user: str, password: str) -> None: + if not CFG_PATH.exists(): + raise SystemExit("config.yaml 不存在；请先从 config.example.yaml 复制。") + data = yaml.safe_load(CFG_PATH.read_text(encoding="utf-8")) or {} + data.setdefault("database", {}) + data["database"].update(database=db, user=user, password=password) + CFG_PATH.write_text( + yaml.safe_dump(data, sort_keys=False, default_flow_style=False), + encoding="utf-8", + ) + print(f"[init_db] 凭据已写入 {CFG_PATH}") + + +def main() -> None: + settings = reload_settings() + db = settings.database.database + user = settings.database.user + + keep = os.getenv("KEEP_DB_PASSWORD") == "1" + if keep and settings.database.password and settings.database.password != "CHANGE_ME": + password = settings.database.password + print(f"[init_db] KEEP_DB_PASSWORD=1，沿用 '{user}' 的现有密码") + else: + password = secrets.token_urlsafe(24) + print(f"[init_db] 为 '{user}'@'localhost' 生成新密码") + + sql = ( + f"CREATE DATABASE IF NOT EXISTS `{db}` " + f" DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;" + f"CREATE USER IF NOT EXISTS '{user}'@'localhost' IDENTIFIED BY '{password}';" + f"ALTER USER '{user}'@'localhost' IDENTIFIED BY '{password}';" + f"GRANT ALL PRIVILEGES ON `{db}`.* TO '{user}'@'localhost';" + f"FLUSH PRIVILEGES;" + ) + _mysql_root(["-e", sql]) + print(f"[init_db] 数据库 '{db}' 与账户 '{user}'@'localhost' 已就绪") + + _mysql_root([db], input_str=SCHEMA_PATH.read_text(encoding="utf-8")) + print(f"[init_db] 已导入 {SCHEMA_PATH}") + + if not keep: + _save_password(db, user, password) + + print("[init_db] 完成。") + + +if __name__ == "__main__": + main() diff --git a/app/security.py b/app/security.py new file mode 100644 index 0000000..5b11b92 --- /dev/null +++ b/app/security.py @@ -0,0 +1,49 @@ +"""文档接口（/docs、/redoc、/openapi.json）的 HTTP Basic Auth 依赖。 + +凭据明文存放于 config.yaml 的 docs 段；该文件仅在本机以 root 持有，比较使用 +secrets.compare_digest 以保证常量时间。 +""" + +from __future__ import annotations + +import secrets + +from fastapi import Depends, HTTPException, status +from fastapi.security import HTTPBasic, HTTPBasicCredentials + +from .config import get_settings + +_security = HTTPBasic(auto_error=False) + + +def require_docs_auth( + credentials: HTTPBasicCredentials | None = Depends(_security), +) -> str: + cfg = get_settings().docs + realm = f'Basic realm="{cfg.realm}"' + + if not cfg.enabled: + raise HTTPException(status.HTTP_404_NOT_FOUND, "docs 已禁用") + + if not cfg.password: + raise HTTPException( + status.HTTP_503_SERVICE_UNAVAILABLE, + "docs 鉴权未配置：请在 config.yaml 的 docs.password 填密码后重启", + ) + + if credentials is None: + raise HTTPException( + status.HTTP_401_UNAUTHORIZED, + "需要认证", + headers={"WWW-Authenticate": realm}, + ) + + user_ok = secrets.compare_digest(credentials.username, cfg.username) + pw_ok = secrets.compare_digest(credentials.password, cfg.password) + if not (user_ok and pw_ok): + raise HTTPException( + status.HTTP_401_UNAUTHORIZED, + "用户名或密码错误", + headers={"WWW-Authenticate": realm}, + ) + return credentials.username diff --git a/app/services/__init__.py b/app/services/__init__.py new file mode 100644 index 0000000..dd1b519 --- /dev/null +++ b/app/services/__init__.py @@ -0,0 +1,6 @@ +"""业务逻辑层（Service）。""" + +from .system_service import SystemService +from .upload_service import UploadService + +__all__ = ["SystemService", "UploadService"] diff --git a/app/services/sftp_server.py b/app/services/sftp_server.py new file mode 100644 index 0000000..899d173 --- /dev/null +++ b/app/services/sftp_server.py @@ -0,0 +1,161 @@ +"""嵌入式 SFTP 服务，与 HTTP API 共用 uploads/ 目录。 + +启动方式： + python -m app.services.sftp_server +""" + +from __future__ import annotations + +import asyncio +import logging +import os +import sys +from pathlib import Path + +import asyncssh +import bcrypt + +from ..config import PROJECT_ROOT, get_settings + +logger = logging.getLogger("sftp") + + +class ZikaiSFTPServer(asyncssh.SFTPServer): + """会话期把客户端 chroot 到 upload_root。""" + + def __init__(self, chan, *, upload_root: Path) -> None: + super().__init__(chan, chroot=str(upload_root).encode()) + try: + self._username = chan.get_extra_info("username") or "unknown" + except Exception: # pragma: no cover + self._username = "unknown" + logger.info("SFTP 会话开始 user=%s chroot=%s", self._username, upload_root) + + def exit(self) -> None: + logger.info("SFTP 会话结束 user=%s", self._username) + + +class ZikaiSSHServer(asyncssh.SSHServer): + """支持密码（bcrypt）与公钥两种鉴权。""" + + def __init__(self, settings, authorized_keys: asyncssh.SSHAuthorizedKeys | None) -> None: + self._settings = settings + self._authorized_keys = authorized_keys + self._conn: asyncssh.SSHServerConnection | None = None + + def connection_made(self, conn: asyncssh.SSHServerConnection) -> None: # type: ignore[override] + self._conn = conn + + def begin_auth(self, username: str) -> bool: + # 始终要求鉴权；用户不存在时所有方法会失败 → 干净的 permission denied + return True + + # 密码 + + def password_auth_supported(self) -> bool: + return True + + def validate_password(self, username: str, password: str) -> bool: + user = next((u for u in self._settings.sftp.users if u.username == username), None) + if user is None or not user.password_hash or user.password_hash == "CHANGE_ME_BCRYPT_HASH": + return False + try: + ok = bcrypt.checkpw(password.encode(), user.password_hash.encode()) + except (ValueError, TypeError): + ok = False + logger.info("SFTP 密码鉴权 %s user=%s", "OK" if ok else "FAIL", username) + return ok + + # 公钥 + + def public_key_auth_supported(self) -> bool: + return self._authorized_keys is not None + + def validate_public_key(self, username: str, key: asyncssh.SSHKey) -> bool: + if self._authorized_keys is None: + return False + if not any(u.username == username for u in self._settings.sftp.users): + return False + addr = "" + if self._conn: + peer = self._conn.get_extra_info("peername") + addr = peer[0] if isinstance(peer, tuple) and peer else "" + try: + # asyncssh 命中返回 dict（可能为空），未命中返回 None + result = self._authorized_keys.validate(key, client_host=addr, client_addr=addr) + except Exception: + result = None + ok = result is not None + logger.info("SFTP 公钥鉴权 %s user=%s", "OK" if ok else "FAIL", username) + return ok + + +def _load_authorized_keys(path: Path) -> asyncssh.SSHAuthorizedKeys | None: + if not path.exists() or path.stat().st_size == 0: + return None + try: + return asyncssh.read_authorized_keys(str(path)) + except Exception as exc: # pragma: no cover + logger.warning("解析 %s 失败：%s", path, exc) + return None + + +def _ensure_host_key(path: Path) -> None: + if path.exists(): + return + path.parent.mkdir(parents=True, exist_ok=True) + logger.info("生成 ed25519 SFTP 主机密钥：%s", path) + key = asyncssh.generate_private_key("ssh-ed25519") + key.write_private_key(str(path)) + key.write_public_key(str(path) + ".pub") + os.chmod(path, 0o600) + + +async def _run() -> None: + settings = get_settings() + if not settings.sftp.enabled: + logger.info("SFTP 在配置中已禁用，进程退出。") + return + + upload_root = settings.resolved_upload_dir() + upload_root.mkdir(parents=True, exist_ok=True) + + host_key_path = (PROJECT_ROOT / settings.sftp.host_key_path).resolve() + _ensure_host_key(host_key_path) + + authorized_keys = _load_authorized_keys( + (PROJECT_ROOT / settings.sftp.authorized_keys_path).resolve() + ) + + def sftp_factory(chan) -> ZikaiSFTPServer: + return ZikaiSFTPServer(chan, upload_root=upload_root) + + logger.info( + "启动 SFTP 服务 %s:%d (root=%s)", + settings.sftp.host, settings.sftp.port, upload_root, + ) + await asyncssh.create_server( + lambda: ZikaiSSHServer(settings, authorized_keys), + settings.sftp.host, + settings.sftp.port, + server_host_keys=[str(host_key_path)], + sftp_factory=sftp_factory, + allow_scp=False, + ) + await asyncio.Event().wait() + + +def main() -> None: + logging.basicConfig( + level=logging.INFO, + format="%(asctime)s %(levelname)s [%(name)s] %(message)s", + stream=sys.stdout, + ) + try: + asyncio.run(_run()) + except (KeyboardInterrupt, SystemExit): + logger.info("SFTP 服务退出。") + + +if __name__ == "__main__": + main() diff --git a/app/services/system_service.py b/app/services/system_service.py new file mode 100644 index 0000000..4d3a8f3 --- /dev/null +++ b/app/services/system_service.py @@ -0,0 +1,45 @@ +"""主机 CPU / 内存 / 磁盘信息采集（基于 psutil）。""" + +from __future__ import annotations + +import platform +import time + +import psutil + +from ..schemas.system import DiskUsage, MemoryUsage, SystemStatus + +# 跳过的伪文件系统 +_IGNORED_FSTYPES = {"squashfs", "tmpfs", "devtmpfs", "overlay"} + + +class SystemService: + def get_status(self) -> SystemStatus: + vm = psutil.virtual_memory() + memory = MemoryUsage( + total=vm.total, available=vm.available, used=vm.used, percent=vm.percent, + ) + + disks: list[DiskUsage] = [] + seen: set[str] = set() + for part in psutil.disk_partitions(all=False): + if part.fstype in _IGNORED_FSTYPES or part.mountpoint in seen: + continue + try: + u = psutil.disk_usage(part.mountpoint) + except (PermissionError, OSError): + continue + seen.add(part.mountpoint) + disks.append(DiskUsage( + device=part.device, mountpoint=part.mountpoint, fstype=part.fstype, + total=u.total, used=u.used, free=u.free, percent=u.percent, + )) + + return SystemStatus( + hostname=platform.node(), + cpu_percent=psutil.cpu_percent(interval=0.5), + cpu_count=psutil.cpu_count(logical=True) or 0, + memory=memory, + disks=disks, + uptime_seconds=max(0.0, time.time() - psutil.boot_time()), + ) diff --git a/app/services/upload_service.py b/app/services/upload_service.py new file mode 100644 index 0000000..acc1a6e --- /dev/null +++ b/app/services/upload_service.py @@ -0,0 +1,122 @@ +"""上传服务：流式落盘 + 原子改名 + 元数据入库。""" + +from __future__ import annotations + +import hashlib +import os +import uuid +from datetime import datetime, timezone +from pathlib import Path + +from fastapi import UploadFile + +from ..config import get_settings +from ..dao.uploaded_file_dao import UploadedFileDAO +from ..models.uploaded_file import UploadedFile +from ..schemas.file import FileUploadResponse, UploadedFileOut + + +class UploadService: + def __init__(self, dao: UploadedFileDAO) -> None: + s = get_settings() + self.dao = dao + self.upload_root = s.resolved_upload_dir() + self.chunk_bytes = s.storage.chunk_bytes + self.hash_on_upload = s.storage.sha256_on_upload + + # ---------------- 上传 ---------------- + + def stream_to_disk( + self, file: UploadFile, source: str, uploaded_by: str, + ) -> FileUploadResponse: + """流式落盘并写元数据。 + + 先写 ``.part``，DB 行提交成功后再 ``os.replace`` 成正式名。 + 任何失败仅会留下可识别的 ``.part``，由 start.sh 启动时自动清理。 + """ + rel_dir = self._relative_dir() + (self.upload_root / rel_dir).mkdir(parents=True, exist_ok=True) + + ext = self._safe_ext(file.filename or "") + rel_path = rel_dir / f"{uuid.uuid4().hex}{ext}" + abs_path = self.upload_root / rel_path + part_path = abs_path.with_name(abs_path.name + ".part") + + size, digest = self._write_part(file, part_path) + + entity = UploadedFile( + storage_path=str(rel_path), + original_filename=os.path.basename(file.filename or rel_path.name), + content_type=file.content_type or "", + size_bytes=size, + sha256=digest, + source=source, + uploaded_by=uploaded_by, + ) + try: + saved = self.dao.create(entity) + except Exception: + part_path.unlink(missing_ok=True) + raise + + try: + os.replace(part_path, abs_path) + except Exception: + try: + self.dao.delete(saved.id) + finally: + part_path.unlink(missing_ok=True) + raise + + return FileUploadResponse( + id=saved.id, + filename=saved.original_filename, + size_bytes=saved.size_bytes, + sha256=saved.sha256, + storage_path=saved.storage_path, + uploaded_at=saved.uploaded_at, + ) + + # ---------------- 查询 ---------------- + + def list_files(self, limit: int = 100, offset: int = 0) -> tuple[int, list[UploadedFileOut]]: + rows = self.dao.list(limit=limit, offset=offset) + items = [UploadedFileOut.model_validate(r) for r in rows] + return len(items), items + + def get_out(self, file_id: int) -> UploadedFileOut | None: + row = self.dao.get_by_id(file_id) + return UploadedFileOut.model_validate(row) if row else None + + def resolve_disk_path(self, file_id: int) -> Path | None: + row = self.dao.get_by_id(file_id) + return (self.upload_root / row.storage_path).resolve() if row else None + + # ---------------- 内部 ---------------- + + @staticmethod + def _relative_dir() -> Path: + now = datetime.now(timezone.utc) + return Path(f"{now:%Y}/{now:%m}") + + @staticmethod + def _safe_ext(filename: str) -> str: + return os.path.splitext(os.path.basename(filename))[1] + + def _write_part(self, file: UploadFile, part_path: Path) -> tuple[int, str]: + """流式写到 part_path 并 fsync；返回 (size, sha256)。失败时清理残品。""" + hasher = hashlib.sha256() if self.hash_on_upload else None + size = 0 + try: + with part_path.open("wb") as out: + while chunk := file.file.read(self.chunk_bytes): + out.write(chunk) + size += len(chunk) + if hasher is not None: + hasher.update(chunk) + out.flush() + os.fsync(out.fileno()) + except Exception: + part_path.unlink(missing_ok=True) + raise + return size, (hasher.hexdigest() if hasher is not None else "") diff --git a/app/views/__init__.py b/app/views/__init__.py new file mode 100644 index 0000000..1d34c46 --- /dev/null +++ b/app/views/__init__.py @@ -0,0 +1,5 @@ +"""HTML 视图（纯字符串模板）。""" + +from . import system_status_html + +__all__ = ["system_status_html"] diff --git a/app/views/system_status_html.py b/app/views/system_status_html.py new file mode 100644 index 0000000..127326f --- /dev/null +++ b/app/views/system_status_html.py @@ -0,0 +1,134 @@ +"""系统状态页面的 HTML 渲染。""" + +from __future__ import annotations + +from html import escape + +from ..schemas.system import SystemStatus + + +def _fmt_bytes(n: int) -> str: + x = float(n) + for unit in ("B", "KiB", "MiB", "GiB", "TiB", "PiB"): + if abs(x) < 1024.0 or unit == "PiB": + return f"{int(x):,} B" if unit == "B" else f"{x:,.1f} {unit}" + x /= 1024.0 + return f"{x:,.1f} PiB" + + +def _fmt_duration(seconds: float) -> str: + s = int(seconds) + d, s = divmod(s, 86400) + h, s = divmod(s, 3600) + m, s = divmod(s, 60) + parts: list[str] = [] + if d: + parts.append(f"{d}天") + if h or d: + parts.append(f"{h}小时") + if m or h or d: + parts.append(f"{m}分") + parts.append(f"{s}秒") + return " ".join(parts) + + +def _bar(pct: float) -> str: + """带颜色阈值的进度条：>85% 红，>60% 黄，否则绿。""" + pct = max(0.0, min(100.0, pct)) + color = "#e53935" if pct >= 85 else "#fb8c00" if pct >= 60 else "#43a047" + return ( + '
' + f'
' + f'{pct:.1f}%' + '
' + ) + + +_CSS = """ +:root { color-scheme: light dark; } +body { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; + max-width: 880px; margin: 2em auto; padding: 0 1em; line-height: 1.5; } +h1 { margin-bottom: 0.1em; } +.sub { color: #777; margin-top: 0; font-size: 0.95em; } +.card { border: 1px solid #ddd; border-radius: 8px; padding: 1em 1.2em; + margin: 1em 0; background: rgba(0,0,0,0.02); } +.card h2 { margin: 0 0 0.6em; font-size: 1.1em; } +.row { display: grid; grid-template-columns: 160px 1fr; gap: 0.25em 1em; + align-items: center; padding: 0.15em 0; } +.row .k { color: #666; } +.bar { position: relative; background: #e6e6e6; border-radius: 4px; + height: 18px; width: 100%; overflow: hidden; } +.bar .fill { height: 100%; transition: width 0.3s; } +.bar .pct { position: absolute; top: 0; left: 0; width: 100%; height: 100%; + text-align: center; font-size: 12px; line-height: 18px; + color: #fff; mix-blend-mode: difference; } +table.disks { border-collapse: collapse; width: 100%; } +table.disks th, table.disks td { padding: 0.4em 0.6em; text-align: left; + border-bottom: 1px solid #eee; vertical-align: middle; } +table.disks th { font-weight: 600; color: #555; font-size: 0.9em; } +.foot { color: #888; font-size: 0.85em; margin-top: 2em; text-align: center; } +a.json { font-size: 0.85em; color: #1565c0; text-decoration: none; } +@media (prefers-color-scheme: dark) { + body { background: #1a1a1a; color: #e0e0e0; } + .card { background: rgba(255,255,255,0.04); border-color: #333; } + .bar { background: #333; } + table.disks th, table.disks td { border-bottom-color: #2a2a2a; } +} +""" + + +def render(status: SystemStatus) -> str: + rows = "".join( + "" + f"{escape(d.mountpoint)}" + f"{escape(d.device)}" + f"{escape(d.fstype)}" + f"{_fmt_bytes(d.used)} / {_fmt_bytes(d.total)}" + f"{_fmt_bytes(d.free)}" + f"{_bar(d.percent)}" + "" + for d in status.disks + ) + + return f""" + + + + +系统状态 — {escape(status.hostname)} + + + +

系统状态

+

主机 {escape(status.hostname)} · 运行 {_fmt_duration(status.uptime_seconds)} + · 查看 JSON

+ +
+

CPU

+
逻辑核心数
{status.cpu_count}
+
使用率
{_bar(status.cpu_percent)}
+
+ +
+

内存

+
总量
{_fmt_bytes(status.memory.total)}
+
已用
{_fmt_bytes(status.memory.used)}
+
可用
{_fmt_bytes(status.memory.available)}
+
使用率
{_bar(status.memory.percent)}
+
+ +
+

磁盘

+ + + + + + {rows} +
挂载点设备文件系统已用 / 总量剩余使用率
+
+ +

zikai file service · 数据来源 psutil

+ + +""" diff --git a/config.example.yaml b/config.example.yaml new file mode 100644 index 0000000..dc6e2e2 --- /dev/null +++ b/config.example.yaml @@ -0,0 +1,44 @@ +# Application runtime configuration. Copy from config.example.yaml and fill in real values. +# All secrets (DB password, SFTP user hashes/keys) live here - keep it out of version control. + +server: + host: 127.0.0.1 # bind address; Apache proxies f.zikai.wang -> here + port: 6867 # internal port; MUST match the Apache ProxyPass target + workers: 1 # uvicorn worker count + +database: + host: 127.0.0.1 + port: 3306 + user: zikai_filesvc + password: "CHANGE_ME" # generated/overwritten by setup.sh provisioning + database: zikai_filesvc # independent new database (not shared with other apps) + pool_size: 5 + pool_recycle: 1800 + +storage: + upload_dir: ./uploads # where uploaded files are written (shared with SFTP) + chunk_bytes: 1048576 # 1 MiB streaming chunk for HTTP upload (keeps RAM flat) + sha256_on_upload: true # compute sha256 while streaming to disk + +sftp: + enabled: true + host: 0.0.0.0 # SFTP cannot go through Apache's HTTP proxy, expose directly + port: 2022 # open this port in the firewall for external SFTP clients + host_key_path: ./keys/ssh_host_ed25519_key + authorized_keys_path: ./keys/authorized_keys + # Each user may authenticate by password (bcrypt hash) and/or by a public key listed + # in authorized_keys_path. Generate a bcrypt hash with: + # python -c "import bcrypt;print(bcrypt.hashpw(b'yourpass',bcrypt.gensalt()).decode())" + users: + - username: uploader + password_hash: "CHANGE_ME_BCRYPT_HASH" + # public keys for this user go in keys/authorized_keys (one key per line, OpenSSH format) + +docs: + # /docs, /redoc and /openapi.json are protected with HTTP Basic Auth. + # Password is stored as plaintext here -- this file is root-owned and lives + # only on this server; comparison is constant-time. No hashing needed. + enabled: true + username: admin + password: "CHANGE_ME" + realm: "zikai docs" diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..0a853fa --- /dev/null +++ b/requirements.txt @@ -0,0 +1,10 @@ +fastapi==0.115.6 +uvicorn[standard]==0.34.0 +python-multipart==0.0.20 +psutil==6.1.1 +SQLAlchemy==2.0.36 +PyMySQL==1.1.1 +pydantic-settings==2.7.0 +PyYAML==6.0.2 +asyncssh==2.18.0 +bcrypt==4.2.1 diff --git a/setup.sh b/setup.sh new file mode 100755 index 0000000..d973751 --- /dev/null +++ b/setup.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# 一次性安装：venv、依赖、数据库、SFTP 主机密钥。可重复执行。 +set -euo pipefail + +cd "$(dirname "$0")" +ROOT="$(pwd)" +VENV="$ROOT/.venv" + +echo "==> [1/5] Python 虚拟环境" +if [ ! -x "$VENV/bin/python" ]; then + python3 -m venv "$VENV" || { + echo "venv 创建失败，尝试安装 python3-venv ..." >&2 + sudo apt-get update -y >/dev/null && sudo apt-get install -y python3-venv >/dev/null + python3 -m venv "$VENV" + } +fi +echo " 使用：$($VENV/bin/python --version)" + +echo "==> [2/5] pip 依赖" +"$VENV/bin/python" -m pip install --upgrade pip >/dev/null +"$VENV/bin/python" -m pip install -r "$ROOT/requirements.txt" + +echo "==> [3/5] config.yaml" +if [ ! -f "$ROOT/config.yaml" ]; then + cp "$ROOT/config.example.yaml" "$ROOT/config.yaml" + echo " 已从 config.example.yaml 生成 config.yaml" +else + echo " config.yaml 已存在，保留原内容" +fi + +echo "==> [4/5] MySQL 数据库（独立 DB + 账户，通过 root socket 建立）" +"$VENV/bin/python" -m app.scripts.init_db + +echo "==> [5/5] 工作目录与 SFTP 主机密钥" +mkdir -p "$ROOT/uploads" "$ROOT/logs" "$ROOT/keys" +if [ ! -f "$ROOT/keys/ssh_host_ed25519_key" ]; then + ssh-keygen -q -t ed25519 -N "" -f "$ROOT/keys/ssh_host_ed25519_key" \ + -C "zikai-sftp-host" || echo " （主机密钥将在 SFTP 首次启动时自动生成）" +fi +if [ ! -f "$ROOT/keys/authorized_keys" ]; then + : > "$ROOT/keys/authorized_keys" + chmod 600 "$ROOT/keys/authorized_keys" + echo " 已创建空的 keys/authorized_keys（在此追加客户端公钥）" +fi + +echo +echo "安装完成。" +echo " - 数据库：$(grep -A1 'database:' "$ROOT/config.yaml" | tail -1 | tr -d ' ')" +echo " - HTTP： 监听 127.0.0.1:6867（由 apache2 在 https://f.zikai.wang 反代）" +echo " - SFTP： 监听 0.0.0.0:2022（请在防火墙放开 2022 端口）" +echo +echo "下一步：在 config.yaml 中设置 SFTP 密码 / 追加客户端公钥，然后 ./start.sh" diff --git a/sql/schema.sql b/sql/schema.sql new file mode 100644 index 0000000..0fc5568 --- /dev/null +++ b/sql/schema.sql @@ -0,0 +1,20 @@ +-- zikai_filesvc 数据库的表定义。 +-- 由 app/scripts/init_db.py 通过本机 socket 以 root 身份执行；表级幂等。 + +SET NAMES utf8mb4; + +CREATE TABLE IF NOT EXISTS `uploaded_file` ( + `id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, + `storage_path` VARCHAR(512) NOT NULL COMMENT 'upload_dir 下相对路径，例如 2026/06/.bin', + `original_filename` VARCHAR(512) NOT NULL, + `content_type` VARCHAR(255) NOT NULL DEFAULT '', + `size_bytes` BIGINT UNSIGNED NOT NULL DEFAULT 0, + `sha256` CHAR(64) NOT NULL DEFAULT '', + `source` VARCHAR(32) NOT NULL DEFAULT 'http' COMMENT 'http / sftp / ...', + `uploaded_by` VARCHAR(128) NOT NULL DEFAULT '', + `uploaded_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, + PRIMARY KEY (`id`), + KEY `idx_uploaded_at` (`uploaded_at`), + KEY `idx_source` (`source`), + KEY `idx_sha256` (`sha256`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; diff --git a/start.sh b/start.sh new file mode 100755 index 0000000..8b6c97e --- /dev/null +++ b/start.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +# 启动 HTTP（uvicorn）与 SFTP；已在运行则跳过（幂等）。 +set -euo pipefail + +cd "$(dirname "$0")" +ROOT="$(pwd)" +VENV="$ROOT/.venv" +PY="$VENV/bin/python" + +[ -x "$PY" ] || { echo "未找到 venv：$VENV，请先运行 ./setup.sh" >&2; exit 1; } +[ -f "$ROOT/config.yaml" ] || { echo "未找到 config.yaml，请先运行 ./setup.sh" >&2; exit 1; } + +mkdir -p "$ROOT/logs" "$ROOT/uploads" + +# 启动前清理：upload_service.py 用 .part 后缀做原子写，进程已停时残留必为半成品。 +PART_COUNT=$(find "$ROOT/uploads" -type f -name '*.part' -printf '.' 2>/dev/null | wc -c) +if [ "$PART_COUNT" -gt 0 ]; then + find "$ROOT/uploads" -type f -name '*.part' -delete + echo "清理：删除残留 .part 文件 $PART_COUNT 个" +fi + +is_running() { [ -f "$1" ] && kill -0 "$(cat "$1")" 2>/dev/null; } + +# --- HTTP --- +APP_PID="$ROOT/app.pid" +if is_running "$APP_PID"; then + echo "HTTP 已在运行 (pid $(cat "$APP_PID"))" +else + read -r HOST PORT <<<"$("$PY" - <<'PY' +import yaml +c = yaml.safe_load(open("config.yaml")) +print(c["server"]["host"], c["server"]["port"]) +PY +)" + nohup "$VENV/bin/uvicorn" app.main:app \ + --host "$HOST" --port "$PORT" \ + >>"$ROOT/logs/app.log" 2>&1 & + echo $! > "$APP_PID" + echo "HTTP 启动 pid=$(cat "$APP_PID") 监听 ${HOST}:${PORT} 日志 logs/app.log" +fi + +# --- SFTP --- +SFTP_PID="$ROOT/sftp.pid" +SFTP_ENABLED="$("$PY" - <<'PY' +import yaml +print("true" if yaml.safe_load(open("config.yaml")).get("sftp", {}).get("enabled", True) else "false") +PY +)" +if [ "$SFTP_ENABLED" = "true" ]; then + if is_running "$SFTP_PID"; then + echo "SFTP 已在运行 (pid $(cat "$SFTP_PID"))" + else + nohup "$PY" -m app.services.sftp_server >>"$ROOT/logs/sftp.log" 2>&1 & + echo $! > "$SFTP_PID" + echo "SFTP 启动 pid=$(cat "$SFTP_PID") 日志 logs/sftp.log" + fi +else + echo "SFTP 已在配置中禁用，跳过。" +fi + +echo "docs: http://127.0.0.1:6867/docs (对外 https://f.zikai.wang/docs)" diff --git a/stop.sh b/stop.sh new file mode 100755 index 0000000..a150c03 --- /dev/null +++ b/stop.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash +# 通过 pidfile 优雅停止 SFTP 与 HTTP，超时则 SIGKILL。 +set -uo pipefail + +cd "$(dirname "$0")" +ROOT="$(pwd)" + +stop_pidfile() { + local name="$1" pidfile="$2" + if [ ! -f "$pidfile" ]; then + echo "$name: 无 pidfile ($pidfile)，未运行。" + return 0 + fi + local pid; pid="$(cat "$pidfile" 2>/dev/null || true)" + if [ -z "$pid" ] || ! kill -0 "$pid" 2>/dev/null; then + rm -f "$pidfile" + echo "$name: 进程已退出，清理 pidfile。" + return 0 + fi + echo "$name: SIGTERM -> $pid ..." + kill -TERM "$pid" 2>/dev/null || true + for _ in $(seq 1 20); do + kill -0 "$pid" 2>/dev/null || break + sleep 0.25 + done + if kill -0 "$pid" 2>/dev/null; then + echo "$name: 仍在运行，发送 SIGKILL..." + kill -KILL "$pid" 2>/dev/null || true + fi + rm -f "$pidfile" + echo "$name: 已停止。" +} + +# 先停 SFTP（避免拆解过程中又有新文件落盘），再停 HTTP。 +stop_pidfile "SFTP" "$ROOT/sftp.pid" +stop_pidfile "HTTP" "$ROOT/app.pid" +echo "全部服务已停止。"