zTools2/README.md

# zikai file service

A Python web service (FastAPI) for **f.zikai.wang** that provides host
monitoring and large-file upload over **both HTTP and SFTP**. It follows a
Spring-style layered architecture (`controllers` → `services` → `dao`, plus
`models` and `schemas`), ships with auto-generated API docs, and runs entirely
from a self-contained `.venv`.

## Features

- `GET /api/system/status` — CPU, memory, and per-disk usage (via `psutil`).
- `POST /api/files/upload` — **streamed** multipart upload (flat memory, multi-GB friendly), with SHA-256.
- `GET /api/files`, `GET /api/files/{id}`, `GET /api/files/{id}/download`.
- **Embedded SFTP server** (asyncssh) supporting **password + public-key** auth, sharing the same storage as HTTP.
- `/docs` (Swagger UI) and `/redoc` — interactive, auto-lists all APIs.
- Metadata persisted in an **independent MySQL database** (`zikai_filesvc`).
- `start.sh` / `stop.sh` lifecycle; `setup.sh` for one-time provisioning.

## Architecture (Spring-style layers)

```
app/
├── controllers/   # FastAPI routers — HTTP boundary (like @RestController)
├── services/      # business logic (SystemService, UploadService, SFTP server)
├── dao/           # data access objects — the only layer that issues SQL/ORM
├── models/        # SQLAlchemy ORM entities
├── schemas/       # pydantic DTOs (request/response validation)
├── database.py    # engine, session, Base, get_db() dependency
├── config.py      # typed Settings loaded from config.yaml
└── scripts/       # init_db.py — DB provisioning
```

Request flow: **controller** → **service** → **dao** → **ORM model** → MySQL.
The DB session is injected by FastAPI's `get_db` dependency and passed down.

## Quick start

```bash
cd /root/zikai
./setup.sh     # one-time: venv, deps, provision DB + user, SFTP host key
./start.sh     # start HTTP (127.0.0.1:6867) + SFTP (0.0.0.0:2022)
./stop.sh      # stop both
```

`setup.sh` is re-runnable. It creates the `.venv`, installs `requirements.txt`,
copies `config.example.yaml` → `config.yaml` (if absent), provisions a **new
independent MySQL database and app user** via the local root socket, and
generates the SFTP host key.

## Access

| Where | URL |
|------|-----|
| Status page (HTML)  | https://f.zikai.wang/api/system/status |
| Status page (JSON)  | https://f.zikai.wang/api/system/status?format=json (or `Accept: application/json`) |
| API docs (Swagger)  | https://f.zikai.wang/docs **(HTTP Basic Auth — see `docs:` in config.yaml)** |
| API docs (ReDoc)    | https://f.zikai.wang/redoc (same auth) |
| Upload              | `curl -F file=@big.iso https://f.zikai.wang/api/files/upload` |
| SFTP                | `sftp -P 2022 uploader@f.zikai.wang` |

`/api/system/status` content-negotiates: browsers (`Accept: text/html`) get a
human-readable page with progress bars; API clients get JSON. Force one with
`?format=html` or `?format=json`.

`/docs`, `/redoc`, and `/openapi.json` require HTTP Basic Auth — the browser
will prompt you. Username + plaintext password live in `config.yaml` under
`docs:`. `/health` and `/` remain public.

Apache (`/etc/apache2/sites-available/f.zikai.wang-le-ssl.conf`) proxies
`f.zikai.wang` → `127.0.0.1:6867` with `ProxyPreserveHost On`, so the service
only binds the loopback.

> **Large/slow HTTP uploads:** Apache's proxy leg inherits the global `Timeout 300`.
> For multi-GB transfers over a slow link, prefer **SFTP** (it bypasses the HTTP
> proxy entirely). To raise the HTTP ceiling you can add `ProxyTimeout`/`Timeout`
> in the Apache vhost.

## Configuration

All runtime config lives in **`config.yaml`** (git-ignored). See
`config.example.yaml` for the full schema. Notable keys:

- `server` — bind host/port (keep `127.0.0.1:6867` to match Apache).
- `database` — host/port/user/password/database. The password is auto-generated
  and written here by `setup.sh`/`init_db.py`.
- `storage.upload_dir`, `storage.chunk_bytes` (default 1 MiB streaming chunk).
- `sftp` — enabled, host/port, host key + authorized_keys paths, and `users`.

### Setting the /docs admin password

Edit `config.yaml` directly — no hashing required:

```yaml
docs:
  enabled: true
  username: admin
  password: "your-plaintext-password"
  realm: "zikai docs"
```

Then `./stop.sh && ./start.sh`. The file is root-owned and stays on this
host; comparison is constant-time (`secrets.compare_digest`).

### Setting SFTP credentials

**Password auth** — generate a bcrypt hash and put it in `config.yaml`:

```bash
.venv/bin/python -c "import bcrypt;print(bcrypt.hashpw(b'yourpass',bcrypt.gensalt()).decode())"
# paste the output into sftp.users[].password_hash, then ./stop.sh && ./start.sh
```

**Public-key auth** — append each client's public key (OpenSSH format) to
`keys/authorized_keys` (one per line). Clients in `sftp.users[]` may then log
in with either method.

### Regenerating the DB password

```bash
.venv/bin/python -m app.scripts.init_db          # new random password
KEEP_DB_PASSWORD=1 .venv/bin/python -m app.scripts.init_db   # keep current
```

## SFTP notes

- The SFTP server can't traverse Apache's HTTP proxy, so it binds `0.0.0.0:2022`
  directly. **Open port 2022** in your firewall for external clients
  (FileZilla/WinSCP/scp).
- Sessions are chrooted to the upload root (`uploads/`), shared with HTTP.
- Only users listed in `sftp.users` may connect; only SFTP (no shell/exec) is allowed.

## Logs & pidfiles

- HTTP logs → `logs/app.log`; SFTP logs → `logs/sftp.log`.
- Pidfiles: `app.pid`, `sftp.pid` (used by `stop.sh`).