ec921dd89779075130a888e4cabf2d260a7e4b09
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jiangbowen
|
4be1ac4713 |
feat: 解析合并转发的聊天记录消息（appmsg type=19）+ 文件本地路径查找工具 (#65)
* feat: 解析合并转发的聊天记录消息（appmsg type=19）+ 新增文件路径查找工具
mcp_server.py:
- _format_app_message_text 增加 app_type=19 分支，解析 <recorditem> 内嵌
XML，把"[链接/文件] xxx的聊天记录"展开成多行 datalist 内容（含发送者/
时间/数据类型）。覆盖 datatype 1/2/3/4/5/6/7/8/17/19/22/23/29/36/37
共 14 种类型；超过 50 条自动截断；空 datalist fallback 到"（待加载）"
- 新增 decode_file_message 工具：从 type=49+sub=6 消息找本地副本路径
（~/Library/.../msg/file/{YYYY-MM}/原文件名.{ext}），返回精确路径
+ size 二次确认，处理同名 (1)(2) 后缀
- 新增 decode_record_item 工具：从 type=49+sub=19 合并记录的第 N 个
dataitem 找本地副本（msg/attach/{table_hash}/*/Rec/*/F/{idx}/{name}），
未下载时给精确"在 wechat 点击哪一项"指引
回归测试：在数千条真实合并转发消息上 ~83% 完美展开 datalist 内容，
剩余的 content 缺失/消息被撤回情况下行为与改前一致（fallback 到原 [链接/文件]）。
* fix: hoist subdir_map in decode_record_item to avoid UnboundLocalError
When the chat's attach directory does not exist (no merged-record
attachment ever downloaded for that chat), the if-block defining
`subdir_map` was skipped, so the not-found branch's reference to
`subdir_map.get(datatype, '?')` raised UnboundLocalError instead of
returning the intended guidance message.
Hoist the dict definition above the if-block so both branches can
safely reference it.
Caught by Codex review on PR #65.
* fix: address Codex P2 — large recorditem XML + glob escape
Two issues caught by Codex review on PR #65:
P2-1: _parse_xml_root rejects payloads >20KB, which silently dropped
~330 large merged-record cards (max observed 418KB with 99 dataitems)
back to "[链接/文件]" fallback. Add a dedicated _parse_record_xml with
a 500KB limit for embedded recorditem XML; routes both call sites in
_format_record_message_text and decode_record_item to it. Boosts
overall parse coverage from 83% to ~87% on real-world data.
P2-2: decode_record_item passed datatitle directly to glob, so file
names containing [ ] * ? would be treated as glob patterns rather
than literals — leading to wrong candidates or missed real files. Wrap
datatitle with glob.escape() before the exact-match query.
Full unit-test suite (35 tests) still passes.
* perf+style: speed up decode_file_message + minor consistency fixes
Self-review findings on top of the Codex P1+P2 fixes:
- perf: decode_file_message previously os.walk-ed `msg/file/` and
`msg/attach/` from scratch on every call, scanning ~185k files /
17GB on a real-world install (~6.3s per call). Now first reads
`create_time` from the message and globs only the matching
`msg/file/{YYYY-MM}/` (plus ±1 month for cross-month edge cases),
with the original walk preserved as a fallback. Measured speed-up
~10x on first call, ~750x on warm cache.
- decode_record_item: extend `type_label` to cover datatype 23
(视频号直播) and 36 (小程序/H5) so the not-found message matches
the labels emitted by _format_record_dataitem instead of falling
back to a raw `datatype=23` string.
- decode_record_item: replace unused `sub_type_packed` with `_` to
silence the "name assigned but unused" smell.
- _parse_record_xml: comment now states the empirically observed
~418KB upper bound (was "~50KB"), making the 500KB ceiling
obviously sufficient.
All 35 existing tests still pass.
* fix: address Codex round-3 P2 — multi-shard lookup + size-validate month scan
Two more issues caught by Codex review on PR #65 that I missed during
self-review:
P2-3 (multi-shard local_id lookup): both `decode_file_message` and
`decode_record_item` were resolving a single message-table via the
singular `_find_msg_table_for_user`, but a chat's messages can span
multiple message_N.db shards (search_messages and history-iteration
already use `_find_msg_tables_for_user`). When the requested local_id
lived in a different shard the tools incorrectly returned "找不到
local_id" or — if IDs collide across shards — picked the wrong row.
Now both tools iterate all shards and stop at the first hit; the
not-found message reports how many shards were scanned.
P2-4 (size validation in month-scan fast path): the perf-fix in the
prior commit collected `msg/file/{YYYY-MM}/` matches without verifying
size, so when a same-named-but-different-size copy existed in the
target month the candidate list was non-empty, the walk-fallback was
skipped, and the later `size_match` filter could end up empty —
returning a wrong-size file. Now the month-scan filters by `totallen`
upfront when known, so unmatched candidates don't poison the fallback.
Same one-shot size validation applied to `decode_record_item`'s
exact-name glob branch for symmetry.
These were both "I should have caught" issues — Codex did the
cross-tool consistency check (singular vs plural shard helper) that I
skipped, and stress-tested an edge case (month-scan finds same-name
wrong-size) that I didn't think through when writing the perf fix.
35/35 existing tests still pass. Real-data smoke: decode_file_message
0.96s end-to-end (multi-shard scan + size validation),
decode_record_item 0.03s.
* refactor: reuse _parse_message_content helper for group prefix stripping
Self-review found that decode_file_message and decode_record_item
hand-rolled their own heuristic for stripping group-chat sender
prefixes ("wxid_xxx:\n<xml...>") via a string-startswith check, while
the rest of the project already uses the canonical
`_parse_message_content(content, local_type, is_group)` helper for
exactly this purpose.
Wired both tools to that helper, deriving is_group from the username
suffix `@chatroom`. Existing edge cases (private chat content with
literal "<...>", group content with "wxid_xxx:\n", etc.) still pass.
35/35 tests still pass; 6/6 edge-case smokes still pass.
* fix: address Codex adversarial-review high+medium findings
Adversarial review caught four issues that the surface-level passes
missed. All four are now fixed end-to-end (validated against real
data, not just helper-level smoke):
[high] Large recorditem outer XML actually parsed:
Previous P2 fix added _parse_record_xml(500KB) for the inner CDATA
but the outer appmsg was still gated by _parse_xml_root(20KB), so
any merged-record card whose outer XML exceeded 20KB silently fell
back to "[链接/文件]" and never reached the inner expansion. Now
_parse_xml_root accepts a max_len kwarg, _format_app_message_text
retries with _RECORD_XML_PARSE_MAX_LEN when the default cap rejects
the outer XML, and _format_record_message_text passes the wider cap
for inner parses. Real-data check: a 34KB outer / 67-dataitem card
now expands fully via the get_chat_history → _format_message_text
→ _format_app_message_text → _format_record_message_text chain.
[high] Multi-shard local_id ambiguity:
decode_file_message and decode_record_item previously broke on the
first shard match. Empirically confirmed local_id 171 in the test
account exists in TWO shards as TWO different messages (one type=1
text, one type=6 file at different create_times). Now both tools
scan all shards, fail with an explicit ambiguity error when more
than one row matches, and accept an optional create_time arg from
the user to disambiguate uniquely.
[medium] History output now exposes (local_id, ts) for file and
record cards, and record dataitem rows are prefixed with their
0-based [item_index]. Without these, callers had no way to feed
decode_file_message / decode_record_item a stable identifier.
[medium] decode_file_message now requires appmsg type=6 and an
appattach node, refusing to search the local cache by title/size for
unrelated app messages (links, miniapps, record cards) that happen
to share a title with a real file.
35/35 existing tests still pass. Real-data smokes:
- 34KB outer XML / 67 dataitems expanded end-to-end
- multi-shard ambiguity correctly raised + resolved by ts kwarg
- history output now contains "(local_id=N, ts=T)" suffixes and
"[N]" dataitem prefixes
* fix: address Codex adversarial round-2 high findings
Round-2 adversarial review caught two issues my self-review missed
again. Both are now fixed end-to-end:
[high] decode_record_item also rejects large outer XML (mcp_server.py:2124-2126)
Round-1 high #1 was fixed by adding a wider-limit retry inside
_format_app_message_text, but decode_record_item itself still
parsed the outer appmsg with `_parse_xml_root(xml_text)` at the
default 20KB cap. Same root cause: I patched one caller, missed
the other — exactly the kind of cross-tool inconsistency that
cost two rounds already.
Extracted a shared `_parse_app_message_outer(content)` helper that
encapsulates the "try default cap, fall back to wider limit when
default rejects" pattern. Now used by all three call sites:
_format_app_message_text, decode_file_message, decode_record_item.
Real-data check: a 34KB outer (67 dataitems) parses through every
caller path, not just history rendering.
[high] Record attachment lookup silently picks wrong cached file
Previous lookup had three fallback tiers (filename+size → size only
→ cross-subdir size only) and on multiple matches sorted by mtime
and took newest. Two failure modes:
1. Different forwarded-record cards in the same chat may produce
paths with identical (filename, item_index, datasize), and the
mtime tiebreak lets the tool return another record's file while
reporting "找到本地文件: ✅".
2. Cross-subdir size-only fallback can match files belonging to
unrelated dataitem types entirely.
Now fail-closed:
- Strict filename + size match only when datatitle is known.
- Size-only fallback now ONLY when datatitle is missing
(e.g. datatype=2 thumbnails) AND scoped to the same sub-dir +
item_index — no more cross-Rec leakage.
- Removed the cross-subdir terminal fallback entirely.
- Multiple candidates after strict matching → ambiguity error
listing all candidates with mtime, no silent pick.
35/35 existing tests still pass. Real-data smokes:
- 大 outer 34KB 卡片 _parse_app_message_outer 解析成功
- decode_record_item(local_id, ts) 正确命中 Lec 4 PDF
- 多分片冲突 + 不传 ts → 报歧义错误并提示加 create_time
- 未下载 dataitem → 精确指引"在 wechat 点第 N 项"
* fix: address Codex round-3 adversarial high+medium findings
Round-3 caught two more cross-tool inconsistency issues, both in the
same family I keep missing (修一处忘另一处):
[high] decode_file_message also needs to fail-closed on ambiguity
Round-2 high #2 forced decode_record_item to fail-closed when
multiple cached candidates remain after strict matching, but I
forgot to apply the same change to decode_file_message — it still
silently sorted by mtime and returned candidates[0]. Same root
cause as round-1 high #1: Codex catches what I miss when the same
pattern needs fixing in two places.
Now decode_file_message: strict size filter when totallen is known,
and ambiguity error (not mtime sort) when more than one candidate
remains. Behavioral change: previously returned 逻辑审计论文(1).pdf
on a real test case; now reports both candidates and asks user to
disambiguate. UX regression but safety-correct.
[medium] decode_record_item rejects non-downloadable datatypes
upfront. Previously, dataitems with unknown datatype fell through
to a wildcard `sub='*'` glob over all attach subdirs (F/Img/V/A),
which could match unrelated files for links/locations/cards/
miniapps/nested-record dataitems that have only metadata, no
binary payload. Now reject non-{2,4,5,8} datatypes with a clear
"no local binary, look at history output instead" message before
any filesystem lookup.
35/35 existing tests still pass.
* fix: address Codex adversarial round-4 high findings
Round-4 found three security/correctness issues. All addressed:
[high] Path traversal via untrusted XML titles
title (decode_file_message) and datatitle (decode_record_item) come
from message XML — attacker-controlled in the "malicious chat
partner" threat model. glob.escape does NOT strip path separators
or normalize absolute paths, so e.g. title="/etc/passwd" makes
os.path.join(month_dir, "/etc/passwd") == "/etc/passwd" (POSIX
rule: join drops left when right is absolute), and glob then walks
outside msg/file. If size also matches, the tool returns an
arbitrary system path as a "found wechat file".
Added _safe_basename(name) helper with strict-reject semantics
(per Codex: reject, don't normalize) — any name containing path
separators, .. components, NUL, or absolute-path prefix is
rejected outright. Both decoders sanitize their XML-derived names
before any filesystem operation. Added _path_under_root realpath
check after candidate selection as a second-line defense against
symlink escapes.
[high] decode_file_message and decode_record_item can return cached
files belonging to a DIFFERENT message even when len(candidates)==1
Both tools rely on (filename + size + optional item_index)
heuristic matching against the cache — they have no way to derive
a record-bound or message-bound path from wechat metadata, so
exactly one matching cached file from an unrelated message looks
identical to a correct hit. This is a design limitation: wechat
does not expose record_hash or attach-uuid in the message XML in
any form derivable from outside the client.
Acknowledged in tool output with an explicit ⚠️ "this path is
heuristic, please verify mtime/context/content" warning attached
to every "found local file" response. The match itself is still
the same heuristic — closing this fully would require either
removing the tools or reverse-engineering wechat's path hashing.
Documented the limitation in the warning so callers can manually
verify before trusting downstream Read/PDF results.
35/35 tests still pass; 12/12 path-sanitize edge cases pass.
* fix: address Codex round-5 adversarial findings + md5-strong binding
Codex round 5 caught two more high issues plus a perf/correctness
concern. All real and addressed:
[high] decode_file_message scanned msg/attach in fallback, picking
up unrelated forwarded-record cached files. Outer files only ever
live in msg/file/{YYYY-MM}/; restricted the slow-path walk to that
subtree only. msg/attach holds merged-card and image attachments
whose presence here is a different message's payload, not ours.
[high] **真正根治** record/file 路径绑定问题：用 md5 强校验
Both decode_file_message (`<md5>` in appmsg) and decode_record_item
(`<fullmd5>` in dataitem) now extract the WeChat-supplied md5 and
hash candidate files locally to compare. If md5 doesn't match, the
tool fails closed with an explicit md5-mismatch error rather than
returning a path. The candidate that *does* match is uniquely
bound to the selected message — md5 collisions of distinct files
are cryptographically negligible. This fixes the heuristic-only
warning paths from rounds 3-4 with cryptographic evidence rather
than just user-facing notes.
As a side benefit, md5 dedup also lets decode_file_message return
a result when WeChat creates "(1)/(2)" copies of the same file:
same-md5 candidates are真同一文件副本 (user re-sent or auto-rename),
any one of them is correct.
When XML doesn't ship md5 (rare but possible), behavior reverts to
the previous fail-closed-on-multiple-candidates path with an
explicit "no md5 available, treating as heuristic" note.
[medium] _parse_app_message_outer was retrying every appmsg under
the 500K cap on initial 20K rejection, which made history rendering
O(content_size) on big non-record appmsgs. Added a substring
`<type>19</type>` short-circuit so only true type=19 records pay
the wider parser cost. Verified non-type=19 big XML now returns
None in <0.01ms instead of doing a 500K parse.
35/35 existing tests still pass. Real-data smokes:
- decode_record_item 142,1 → "✅ md5 校验通过，路径与 dataitem 唯一绑定"
- decode_file_message 171 (with same-name (1).pdf copy in cache) →
md5 dedup recognizes both as same content, returns one with
"✅ md5 校验通过"
- non-type=19 big appmsg parses in <1ms (substring short-circuit)
* fix: address Codex round-6 adversarial findings — strict md5 binding + chunked hash
[high] decode_file_message / decode_record_item now fail-closed when
the message XML has no md5/fullmd5 field — instead of returning a
heuristic single-candidate path with a warning. The previous
warning-only approach (rounds 4-5) didn't actually stop downstream
Read/PDF callers from using the wrong path. Now: no md5 = no path
returned, period. The error message lists the heuristic candidates
with mtime so the user can manually pick if absolutely needed,
but the tool itself does not commit to any of them.
Behavioral consequence: messages where wechat omits md5 (rare but
possible — e.g. some image/voice dataitems lack fullmd5) become
not-resolvable via these tools. Acceptable safety/utility tradeoff
per Codex's recommendation.
[medium] md5 verification was reading the entire candidate file into
memory via `_hashlib.md5(_f.read()).hexdigest()`. For 100MB+
attachments (videos in merged-record cards, large PDFs) this could
spike RSS or stall the MCP process. Replaced with a streaming
helper `_md5_file_chunked` (64KB chunks) plus a 500MB hard cap that
returns an explicit error rather than attempting verification on
oversized files.
35/35 existing tests still pass. Real-data smokes:
- decode_file_message 171 (with md5) → "✅ md5 校验通过"
- decode_record_item 142,1 (with fullmd5) → "✅ md5 校验通过"
- _md5_file_chunked size cap 1KB rejection works correctly
* fix: round-7 + revert round-6 over-strict — match real threat model
Two real bugs from Codex round-7 plus a partial revert of round-6
over-strictness that doesn't match this tool's actual threat model.
[high] Group type=19 with 'sender:<?xml...' (no newline) prefix not
stripped (Codex round-7 high #1)
_parse_message_content only split on ':\n', missing real-world
group rows where wechat writes 'wxid_xxx:<?xml ...' or
'wxid_xxx:<msg ...' inline. _format_app_message_text and
decode_record_item both received the prefixed content, parsed it
as raw XML, and failed. Now also strips on regex match against
'<?xml|<msg|<msglist|<voipmsg|<sysmsg' immediately after a sender
token. Verified with 5 prefix shape variants; legacy ':\n' still
works.
[medium] Record images use flat 'Img/0_t' filenames, not 'Img/0/*'
(Codex round-7 medium #2)
decode_record_item's datatype=2 (image) branch globbed for
'*/Rec/*/Img/{idx}/*' but real wechat caches store record images
as flat files: '*/Rec/<id>/Img/0_t', '*/Rec/<id>/Img/0', or
'*/Rec/<id>/Img/0.{ext}'. Added flat-pattern matching for
datatype=2 with the four observed filename shapes. File/voice/
video classes still use the F|A|V/{idx}/{filename} shape they
always did.
[revert] Round-6's "no md5 → fail-closed" is too strict for this
tool's actual usage
This MCP server is invoked locally by the user, paths surface
only in the local Claude conversation, and contacts are not
hostile. Codex round-6's hard fail-closed-on-missing-md5 broke
ergonomics for real wechat messages that lack md5 (some image
and voice dataitems) without a corresponding security gain in
this scenario. Reverted to round-5 behavior:
- md5 present → cryptographic verification, mismatch fails
- md5 absent → heuristic + ⚠️ warning, multiple-candidate
ambiguity still fails closed.
Kept all other round-6 hardening: streaming chunked md5, 500MB
cap, _safe_basename strict reject, _path_under_root realpath
check, multi-shard ambiguity, substring short-circuit for
non-type=19 big XML.
35/35 existing tests still pass; 5/5 group-prefix variants pass;
real-data smokes for both decoders still hit md5-verified paths.
* fix: round-8 — defer ambiguity until after md5 dedup + tighten file fallback
Two more findings, both real:
[high] decode_file_message no-md5 fallback was using `stem in f`
substring matching — `stem='论文'` would happily accept
`某老师论文.pdf`. Tightened to: exact match OR strict `(N)` copy
variant (`xxx(1).pdf`, `xxx (1).pdf`) per wechat's auto-rename
convention. 7/7 unit cases verify legitimate accept and false-
positive reject behavior.
[medium/P1 from GitHub Codex] decode_record_item had a stale early
`len(candidates) > 1 → ambiguity` check left over from round-7
refactor — it ran BEFORE the fullmd5 filter, making the md5
disambiguation block unreachable for the exact case where md5
could safely pick the right file. Removed the early check; md5
filter now runs first (and the post-md5 ambiguity check at line
~2467 still fails closed when md5 is missing AND multi-candidate).
35/35 tests pass. Real-data smokes (decode_file_message and
decode_record_item with their corresponding md5/fullmd5) still hit
the ✅ md5-verified path.
* test: add 29 helper-level regression tests for record-decoder helpers
Locks in the bugs fixed across PR #65's many review rounds so they
don't silently regress:
- _safe_basename (7 cases): strict reject of absolute paths,
parent-dir components, path separators, NUL — round-4 high #1.
- _md5_file_chunked (3 cases): streaming hash equals stdlib hashlib,
size cap rejects oversized files, missing file → error — round-6.
- _parse_message_content (5 cases): both legacy `:\n` and round-7
`:<?xml`/`:<msg` group-prefix shapes strip correctly; private
chat does not strip; bytes content returns the binary marker.
- _parse_app_message_outer (3 cases): small XML uses default cap,
oversized non-type=19 short-circuits (no 500K parse), oversized
type=19 retries successfully — round-5 medium #3 + round-2 P2-1.
- _format_record_dataitem (7 cases): text / file / image / 视频号 /
音乐 fall-through render correctly; unknown datatype falls back
to datadesc or [未知类型 N].
- _format_record_message_text (4 cases): >20KB outer XML expands
via _format_app_message_text end-to-end (regression for the
"P2-1 was a fake fix because I tested helper in isolation" miss);
empty datalist shows 待加载; chatroom marker appended; overflow
produces "…还有 N 条未显示" line.
The two MCP-tool wrappers (decode_file_message / decode_record_item)
lean on module globals + the real wechat cache layout. They are
exercised by real-data smoke runs in the PR description rather than
mocked here — mocking the entire wechat tree would dwarf the actual
logic under test.
64/64 total tests pass (35 existing + 29 new).
* refactor: simplify per /simplify code review (no behavior change)
Three review agents (reuse / quality / efficiency) flagged the
following high-confidence cleanups. All applied; all 64 tests still
pass; real-data smokes still hit md5-verified paths.
[quality] Remove PR-history references in comments
CLAUDE.md is explicit about this: comments should explain
non-obvious WHY, not narrate which Codex round caught what.
Cleared "round-2 high #2", "Codex round-3 medium #1", "round-5",
"round-6 强制", "round-7 实测", "round-8 high #1" from helper
docstrings and inline comments. Kept the substantive WHY (e.g.
"Reject 而不是 normalize because intent is suspicious").
[reuse + quality] Module-level datatype constants
Three places maintained their own copy of the datatype → label /
subdir mapping (_format_record_dataitem if-cascade, decode_record_
item type_label dict, subdir_map literal). Extracted
_RECORD_DATATYPE_LABEL and _RECORD_BINARY_SUBDIR to module top.
Single source of truth.
[quality] Hoist local imports to module top
Removed 7 inline `import glob as glob_mod` / `from datetime import
datetime as _dt` / `from datetime import datetime as _dt, timedelta
as _td` / `import hashlib as _hashlib` calls inside hot paths and
helpers. Aliases collapsed to plain names (datetime, timedelta,
glob, hashlib).
[efficiency] xpath: drop `.//` recursive descent for known-direct children
_format_record_dataitem was using `.//appbranditem/sourcedisplayname`
and `.//finderFeed/desc` even though both are direct children of
the dataitem. Changed to direct-child paths — meaningful for big
cards (50 items × subtree-walk per render).
[efficiency] md5 verification short-circuits on first match
Multiple candidates sharing the same md5 are wechat re-named copies
of the same file (e.g. `xxx (1).pdf`); any one is correct. Added
`break` after the first md5 match to skip hashing remaining
candidates (which can each be 100+ MB).
[quality] Compress _format_record_dataitem if-cascade
Datatypes that just emit `[label]` (2/3/4/5/7/23/37) and the link/
H5 pair (6/36) collapsed into membership checks against
_RECORD_DATATYPE_LABEL.
64/64 tests still pass.
---------
Co-authored-by: jiangbowen <robin@jiangbowendeMacBook-Air.local>
|
||
|
Belugary
|
c29e8dd868 |
fix: ImageResolver 支持微信 4.0+ V2 加密图片格式 (#61)
ImageResolver.decode_image 之前只调 xor_decrypt_file（老格式 XOR-only 路径），微信 4.0+（2025-08+）已经改用 V2 AES-128-ECB + XOR 混合加密， 导致 mcp_server.py 注册的 decode_image MCP 工具对 V2 .dat 文件返回的 "解密"内容是错的——Claude AI 通过 MCP 调用看不到 V2 时代的图片。 monitor_web.py 早已正确处理 V2（line 41-42, 791-795：从 _cfg 读 image_aes_key / image_xor_key 后调 decrypt_dat_file 自动 magic 分发）， 本次把 MCP 路径补齐，行为与 monitor_web.py 对齐。 改动： - ImageResolver.__init__ 增加 aes_key=None, xor_key=0x88 关键字参数 （默认值保持向后兼容，老调用方无需改动） - ImageResolver.decode_image 把 xor_decrypt_file 换成 decrypt_dat_file， 按 magic 自动分发 V2 / V1 / 老 XOR - V2 文件 + 缺 aes_key 时早期返回结构化错误信息，避免在 v2_decrypt_file 内静默失败成笼统的"解密失败" - v2_decrypt_file 入口接受 xor_key 字符串形式（int(_, 0) 解析）， 与 aes_key 已有的 str→bytes 处理对称，允许 config.json 写 "0x88" - mcp_server.py 实例化时从 _cfg 读 image_aes_key / image_xor_key 注入 兼容性： - ImageResolver 老调用方（不传 keys）继续走老 XOR 路径，零 breaking - V1 magic（\x07\x08V1）不会被 is_v2_format 拦截，走 decrypt_dat_file 内置固定 key，所以 aes_key=None 也能解 V1 文件 - 整 repo 只有 mcp_server.py 一处生产调用 ImageResolver(...)，已 grep 确认 测试覆盖（11 个新测试，tests/test_decode_image_v2.py）： - v2_decrypt_file 合成数据 round-trip 字节级相等 - decrypt_dat_file 按 magic 自动分发 V2 / V1 / 老 XOR 三条路径 - aes_key 接受 str（来自 config.json）和 bytes 两种形式 - xor_key 接受 str（如 "0x88"）和 int 两种形式 - V2 wxgf 裸流返回 fmt='hevc'（HEVC→JPEG 转换是 monitor_web 职责， 不在 ImageResolver 内做，保留 .hevc 输出） - ImageResolver 端到端：from local_id to decrypted file - ImageResolver(aes_key=None) + V1 文件走固定 key 路径 - ImageResolver(aes_key=None) + V2 文件返回 success=False + 友好错误 - ImageResolver 默认参数 + 老 XOR .dat 保持向后兼容 测试 46 个全部通过（11 新 + 35 旧）。 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Belugary
|
49356e1692 |
feat: macOS 图片 AES key 从磁盘 kvcomm 缓存派生（解决 #23） (#60)
* feat: macOS 图片 AES key 从磁盘 kvcomm 缓存派生（issue #23）
macOS 用户长期无法用 C 版 find_image_key_macos 从微信进程内存提取
V2 图片密钥（issue #23 报告 197K 候选全部失败）。新增
find_image_key_macos.py 走完全不同的路径：从磁盘 kvcomm 缓存
文件名派生密钥，无需扫描内存、无需 root、无需重签名。
派生算法
--------
- 扫 ~/.../app_data/net/kvcomm/key_<code>_*.statistic 文件名
- 对每个 (code, wxid) 候选：
xor_key = code & 0xFF
aes_key = MD5(str(code) + cleaned_wxid).hex()[:16] # ASCII 字符串
- 用 V2 _t.dat 文件 [0xF:0x1F] 16 字节做 AES-128-ECB 模板验证：
解出来必须是图像 magic（JPEG / PNG / GIF / WebP / wxgf）
- 为防短 magic 偶然命中，要求多个不同模板都通过验证才算成功
- 命中后写回 config.json 的 image_aes_key / image_xor_key，
monitor_web.py 自动加载
致谢
----
派生算法源自 @hicccc77 在 issue #23 的评论；参考实现见其 WeFlow
项目 (CC BY-NC-SA 4.0)。本模块是独立的 Python clean-room 实现，
未复制其 TypeScript 源码；函数边界与变量命名沿用算法的自然结构
（regex 模式 / MD5 调用顺序 / magic 字节表等不可避免地相同）。
健壮性细节
----------
- 多候选 kvcomm 路径：枚举 5 个不同的 macOS 微信版本路径布局
- 多模板交叉验证：默认收集 3 个不同密文，全部通过才算命中
- 已有 image_aes_key 仍有效时短路返回，不重写 config
- 原子写 config.json：tmp + os.replace + finally 清理 .tmp
- 多 wxid 候选：同时试 raw 和归一化后的 wxid（A_Hare_626a → A_Hare）
- print(flush=True) 逐次显式（与 find_image_key.py 风格一致）
测试
----
新增 tests/test_find_image_key_macos.py，53 个测试覆盖：
派生算法 / wxid 归一化 / kvcomm 路径推算（含多候选）/ 模板收集
（去重 / 子目录 / max_files 边界）/ AES 验证（5 种 magic / 短输入
/ 空 key）/ 多模板交叉验证 / 端到端集成（命中 / 各种失败分支）/
原子写 / main 短路（已有有效 key 不重写 / 已有错 key 落到派生）。
全部通过：python -m unittest discover tests → 88/88。
兼容性
------
- 无新增依赖（pycryptodome 已在 requirements.txt）
- 不改任何现有 Python 文件，零回归风险
- 现有 Windows / Linux 路径 (find_image_key.py / find_image_key_monitor.py) 不受影响
* feat: macOS 图片 AES key 加方案2 fallback (issue #68 思路)
PR #60 的方案1 (kvcomm 缓存派生) 在 kvcomm 缺失 / 多账号歧义 / 首次
启动等场景下会失败。@H3CoF6 在 issue #68 提出关键洞察:
wxid 目录后 4 位 hex == md5(str(uin))[:4]
意味着不需要 kvcomm，可以从 wxid 目录名 + 任意 V2 .dat 反推 uin。
本 commit 在保留 PR #60 方案1 不变的前提下，加方案2 作为 dispatcher
fallback。
方案2 算法
----------
1. 从 db_dir 提 wxid 后 4 位 hex 作为 md5 前缀目标
2. 扫多个 V2 .dat 末字节投票反推 xor_key (假设 JPG EOI 0xD9,
默认至少 3 个样本投票)
3. 枚举 0~2^32 中 (uin & 0xff == xor_key) 的 2^24 个候选,
md5(str(uin))[:4] 匹配 wxid 后缀 → 得 ~256 个 uin 候选
4. 对每个候选算 aes_key, 用 PR #60 的 verify_aes_key_against_all
做 AES 模板交叉验证, 唯一定位 uin
实现
----
- find_image_key_macos 重构为 dispatcher: 先方案1 (kvcomm),
失败 fallback 方案2 (候选搜索); 模板收集移到 dispatcher 共享
- 新增 helper: extract_wxid_parts, derive_xor_key_from_v2_dat,
bruteforce_uin_candidates
- 模块顶部 docstring 加方案2 算法说明 + @H3CoF6 致谢
(保留 PR #60 对 @hicccc77 的方案1 致谢)
clean-room 声明
---------------
方案2 按 issue #68 的算法描述独立实现，未引用 @H3CoF6 任何代码。
方案1 仍沿用 PR #60 实现 (其 clean-room 声明对 @hicccc77 / WeFlow
保持不变)。
健壮性细节
----------
- xor_key 反推默认 min_samples=3, 样本不足直接放弃方案2 (避免
1-2 个样本时一旦撞到非 JPG 就 lock 错 xor_key)
- wxid 后缀正则收紧为 [0-9a-fA-F]{4} (md5 hex), 非 hex 后缀直接
返回 None 而非误导用户跑空候选搜索
- 投票分歧时打印 warning, 但仍试取多数 (兼容 attach 含少量非 JPG)
- 删除重构后未用的 import glob; Counter 统一在模块顶部 import
测试
----
新增 17 个测试 (53 → 70), 全部 7.4s 内通过:
- ExtractWxidPartsTests (5)
- DeriveXorKeyFromV2DatTests (7, 含新增 below_min_samples 边界)
- BruteforceUinCandidatesTests (1, 真跑全空间金标准验证)
- FindViaBruteforceTests (3)
- DispatcherFallbackTests (1, mock 加速)
顺手修复 2 个 pre-existing 测试 fail
------------------------------------
test_account_with_4char_alnum_suffix_stripped 与
test_returns_raw_and_normalized_when_different 用 6-char 后缀
your_wxid_a1b2c3, 但 normalize_wxid 只去 4-char 后缀 (匹配真实
macOS 路径) → 测试期望与代码不一致, 长期 fail。统一改用 4-char
后缀让测试与 macOS 现实对齐。
兼容性
------
- API 不变: find_image_key_macos(db_dir) 签名 / 返回值不变
- 现有 53 个测试全部仍通过 (含 happy path / 各种返回 None 分支 /
main 短路 / 原子写)
- 真实数据验证: 在本地 macOS 微信 4.x 上方案2 端到端跑通, 结果
与方案1 完全一致
* fix: replace test fixture with synthetic uin/wxid (privacy hardening)
PR #60 测试 fixture 与 docstring 示例之前用了真实 uin (8 位十进制)
作为 golden value，并在 docstring 里把 wxid 后缀作为示例展示。虽然
单独的 uin/suffix 不直接 unlock 任何资产 (需要配合真实 wxid + 物理
访问加密文件)，但行业最佳实践 (yt-dlp / openssl / Linux kernel test
fixture) 都明确要求用合成确定性值, 不绑定任何真实账号。
合成方案
--------
- uin: 12345678 (8 位, 一目了然 placeholder)
- suffix: md5("12345678")[:4] = "25d5" (派生, self-consistent)
- wxid_full 示例: your_wxid_25d5
- wxid_norm 示例: your_wxid
- aes_key_test_value: a0c093edddc98490 = md5("12345678your_wxid")[:16]
- xor_key: 0x4E (= 12345678 & 0xFF)
改动范围
--------
- tests/test_find_image_key_macos.py: 全部 fixture 改用合成值,
bruteforce 测试的 xor 也对应更新 (0x7F → 0x4E)
- find_image_key_macos.py:260 docstring 示例: 真实 wxid 字符串
替换为 placeholder
- 长 kvcomm 缓存文件名 fixture 同步合成 (避免暴露真实时间戳 / 内部 ID)
测试
----
70/70 仍通过 (7.1s), 合成 fixture self-consistent。
非范围 (历史 commit
|
||
|
btc-z
|
66eddaff0e |
feat: transcribe_voice 新增 OpenAI Whisper API 后端 (#66)
默认 local，零行为变化。opt-in 双因素：transcription_backend=openai 且 openai_api_key 都齐才生效；任一缺失静默回退 local + stderr 一行警告。 首次进入云路径会 stderr 警告"语音将上传至 OpenAI 服务器"。 新增 config.json 字段： - transcription_backend: "local" (默认) | "openai" - local_whisper_model: "base" (替换 mcp_server.py 里硬编码 DEFAULT_WHISPER_MODEL) - openai_api_key: "" (默认空；openai 包为 optional，按需 pip install) 关键技术选择： - _transcribe(wav, backend) 单一 if/else 分发，不引入插件/工厂层 (Rule of Three —— 只有一个云后端时不值得抽象) - 文件 > 25MB 在 OpenAI() 实例化之前提前拒绝，避免无谓上传 - 错误分类清晰: 缺 key / 缺 openai 包 / 401 / 429 / APIError 各自的提示 - PR #58 缓存 schema 自然扩展: 条目加 backend 字段，命中需 backend+model_size 都匹配 - 旧条目缺 backend 字段视为 "local"，向前兼容 PR #58 已落盘的所有数据 - transcribe_chat.py 批量 CLI 与 MCP 工具共享同一份配置，保持一致 新增 2 个测试 (tests/test_openai_backend.py)，只覆盖回归风险最高的两条： - 文件 > 25MB 必须在 SDK 实例化前拒绝（隐私契约的防线） - backend 不匹配的旧条目不命中（避免切后端时返回错后端结果） 其余路径要么琐碎（默认值读取）、要么坏掉时声音很大（SDK 错误、ImportError）， 要么已被 PR #58 现有测试隐式覆盖（缺 backend 字段的旧条目），不再单独写测试。 顺手把 README 里 PR #53 漏掉的 voice 三件套（get_voice_messages / decode_voice / transcribe_voice）补进 MCP 工具表，并新增"⚠️ 语音转录隐私" 章节说清数据流向、成本（约 \$0.006/分钟）、25MB 上限、回退行为。 Closes ylytdeng/wechat-decrypt#59 |
||
|
Belugary
|
989badd14f |
feat: 给 transcribe_voice 工具加持久化缓存 (#58)
Whisper 本地推理在 CPU 下每条语音数秒到数十秒，且同一段 voice_data 产出相同 text，非常适合缓存。新增 voice_transcriptions.json 持久化 存储，命中时跳过 DB 查询、SILK 解码和 Whisper 推理全链路。 关键技术选择： - 缓存 key 用 json.dumps([username, local_id])，即使 username 含 分隔符也不冲突 - 写入走 tmp + os.replace 原子替换，进程中断不会损坏主文件 - 条目记录 model_size，Whisper 默认模型升级后旧条目自动失效 - 空转录也缓存（配合 model_size 失效），避免静音片段每次重跑 - threading.Lock 防御并发 load/save 竞态 - 首次 OSError 写 stderr 警告一次，后续静默避免刷屏 小的行为改进：resolve_username 移到 whisper/pysilk 导入探测之前， bad chat_name 情况下不再需要 whisper 已安装也能给出"找不到聊天对象" 的错误提示。 15 个新测试：持久化 roundtrip、UTF-8 保留、corrupt JSON 容错、原子 写、写前失败不污染主文件、并发 load/save、缓存命中跳过重活、model 不匹配视为 miss、key 对含分隔符 username 的防御。全部通过。 |
||
|
dsjzazs
|
7c42ff5d38 | Investigate get_chat_history limit | ||
|
dsjzazs
|
9ae558a31e | Fix global search pagination | ||
|
dsjzazs
|
b623711410 | Add MCP search unit tests |