Require valid JSON with escaped quotes in the definition files

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

CLAUDE.md

@@ -9,14 +9,15 @@
   `prompts/<track>-<step>-<task>.md` (e.g. 01-01-tag.md; no
   version suffix -- versions live in git history) and are
   passed verbatim as the system prompt.
-- LLM steps whose outputs are item-by-item comparable
-  (convergence, coding) run the same definition file twice,
-  then a separate arbitration step settles only the
-  script-computed disagreements ("2 runs + 1 arbitration").
-  Free-generation steps run twice and both outputs are pooled,
-  unarbitrated.  If arbitration output is unexpected, revise
-  the definition file and repeat that cycle; never patch
-  results by hand.
+- Per-song LLM judgments (coding, screening) run the same
+  definition file twice, then a separate arbitration step
+  settles only the script-computed disagreements
+  ("2 runs + 1 arbitration").  Free-generation steps run
+  twice and both outputs are pooled, unarbitrated.
+  Vocabulary-building steps (convergence) run once as a
+  recorded pass.  If an arbitration or validation outcome is
+  unexpected, revise the definition file and repeat that
+  cycle; never patch results by hand.
 - Each run of a step is archived self-contained under the
   destination directory given explicitly on the `run-llm`
   command line (by convention `runs/<definition-file>/run<N>/`):

docs/decision-log.md

@@ -336,3 +336,63 @@
   考」敘述同步改為考據結果。附帶認清：先導第一步並非零
   語意提示——此即正式研究「只定格式、不定語意」設計所
   矯正者。
+- **定義檔輸出形狀統一與去署名**：LLM 輸出統一為單層
+  dict／array——tag、code：關鍵字→引述；merge、cap：
+  組名→成員；code-arb：保留關鍵字→仲裁者自己的引述
+  （剔除即不列，剔除集合由程式以鍵差推得）；screen 系：
+  引述陣列，非空即「有」（present 布林刪除，判斷與依據
+  合一）。仲裁輸入一律不含執行別——標示何方主張會誘使
+  仲裁者揣測「哪次較可信」，而非就文本裁決。收斂組名採
+  自由命名（不限取自成員詞），取命名貼合度；可追溯性由
+  軌跡歸檔承擔。
+- **定義檔改三層編號**：
+  `prompts/<軌>-<步>-<次步>-<task>.md`——次步為步內執行
+  順序，明定讀者依循的先後（如 01-04-01-code →
+  01-04-02-code-arb）；仲裁檔後綴 `-arb`。
+- **收斂演算法：檢視而棄用的方案**：詞彙表建構（merge、
+  cap）的重複執行與仲裁，歷經四個方案後全數棄用——
+  ①整條管線獨立跑兩遍、於終點仲裁兩份最終詞彙表：兩套
+  分類系統互不可比，仲裁淪為第三次建構，一致率無從
+  定義。②組對組匹配（以相似度門檻判定兩組是否「同一
+  組」）：無原則性答案，門檻任意。③tag 步驟仲裁：自由
+  詞彙兩次輸出不共享比對單位，無物可裁（tag 改為兩次
+  進池，沿用至今）。④逐對仲裁鏈：交集細分出共識塊、
+  分歧塊對三票多數決、union-find 遞移重組、仲裁後命名
+  2+1——機械上可行且逐項可驗（共 15 份定義檔，全版
+  保存於分支 `tag-algo-13` 備考），但其變異縮減未經證實
+  （逐對多數決降低對層變異，遞移閉包卻放大結構層變異，
+  淨效果不明），複雜度成本則屬確定，且縮減的是儀器變異
+  ——對量測無關緊要的量（見次條）。
+- **詞彙表建構改為單次記錄性程序（演算法簡化）**：
+  merge、cap 各單次執行、執行內自行命名（具名分組
+  輸出），定義檔減為 7 份。2+1 原則改寫：語料層逐首
+  判斷（code、screen）一律 2+1；自由生成（tag）兩次
+  進池；詞彙表建構單次、全程歸檔。理由：(1) 影響量測的
+  是編碼層——詞彙表屬揭露的儀器選擇，凍結後下游同尺量
+  到底，其抽樣變異不污染量測；(2) 對齊領域慣行——
+  codebook 建構本為單次詮釋程序，信度檢驗施於編碼應用
+  層；(3) 複雜仲裁機械無以自證其益（見前條）。驗證改為
+  確定性格式檢查（完整分割、組名唯一、cap ≤ 50），違規
+  依協定修訂定義檔重跑。
+
+## 2026-08-05
+
+- **歌詞捕捉加浮水印／誤碼正規化**：全語料普查（程式窮舉
+  非 ASCII）發現歌詞站浮水印與 cp1252 誤碼——同形字（西里爾
+  е 160 處／78 檔、希臘 ό 6 處）、異體空格（U+2005 211 處、
+  U+205F 80 處、U+200A 6 處）、零寬字元（U+200B 3 處）、C1
+  誤碼（U+0091/92/93/94/97 共 58 處；先前手修之 U+0085 同
+  類）；受影響 81/883 檔全部來自 pilot 匯入，新抓取檔乾淨。
+  正規化規則四條（C1 依 cp1252 本義還原、同形字還原、異體
+  空格→ASCII 空格、零寬刪除）實作於 fetch-lyrics 的
+  normalize_lyrics()，寫入時套用；既有語料以同一函式一次性
+  套用。倒放行等歌曲本體內容不動——策展界線：清傳輸雜質，
+  不改內容。
+- **Song 775 拒答機制查明與裁定為無法分析**：tag 首跑僅
+  此一首遭模型拒答。以隔離子代理做消融測試（四變體）
+  證實觸發源是歌詞中的倒放拼寫行（觸發混淆文字防護），
+  與歌詞中的種族蔑稱無關、與零寬字元亦無關（移除
+  U+200B 而保留倒放行仍拒答）。裁定：倒放行是歌曲的
+  忠實內容，保留不動——語料忠實優先於可分析性；該首
+  視為無法分析，缺漏歸因於 Anthropic 平台防護，非本
+  研究語料的問題。

docs/methodology.md

@@ -0,0 +1,123 @@
+# 方法細節
+
+（全文方法節底稿。演算法在執行前寫定；任何修訂記入
+`decision-log.md`。定義檔全文見 `prompts/`，執行紀錄見
+`runs/`。）
+
+## 自然編碼管線總覽
+
+四步驟：自由標註（tag，兩次進池）→ 自然收斂（merge，
+單次）→ 強制收斂（cap，單次）→ 全量編碼（code，2+1），
+另設「女性力量」單目標篩選（screen，2+1）作黃金標準取樣
+的補漏網。歌詞只出現在 tag、code、screen 與其仲裁步驟；
+merge、cap 不接觸歌詞。設計原則見 `research-plan.md`；
+本檔記載可重現的演算法細節。
+
+## 詞彙表建構（merge、cap）——單次記錄性程序
+
+merge 與 cap 各以單次執行完成，執行內自行為各組命名，
+輸出具名分組（組名 → 成員詞）。不做重複執行與仲裁。
+
+- **定位**：詞彙表是研究的儀器，不是量測。詞彙表凍結
+  入庫後，下游全部量測（編碼 vs 黃金標準）以同一把尺
+  進行，內部一致；建構過程的抽樣變異屬「揭露的儀器
+  選擇」，不污染量測。此與質性研究慣行一致——codebook
+  建構本為單次的詮釋程序，信度檢驗施於編碼應用層。
+- **取捨紀錄**：曾設計逐對仲裁鏈（共識塊、分歧塊對三票
+  多數、命名 2+1；完整版本保存於分支 `tag-algo-13`）。
+  棄用理由：逐對多數決雖降低單對變異，但遞移閉包會放大
+  結構層變異，淨縮減未經證實；其複雜度成本卻是確定的；
+  且縮減的是儀器變異——對論文主張無關緊要的量。
+- **確定性驗證**（違規即依協定修訂定義檔重跑）：輸出須
+  為輸入詞集的完整分割（缺詞、多詞、重複即失敗）；組名
+  唯一且符合格式；cap 組數 ≤ 50。
+- **收斂軌跡**：原始關鍵字 → merge 組名 → cap 組名，
+  兩份執行輸出本身即完整記錄。
+
+## 編碼步驟（code、screen）的 2+1 比對與仲裁
+
+- **code**：逐首比對兩次執行的標籤集合（引述不參與
+  比對）。兩次皆有的標籤為共識保留、兩次皆無為共識
+  不標；單邊標籤送仲裁（`01-04-02-code-arb.md`）——仲裁者
+  看歌詞全文與該標籤的引述（不含執行別），裁決保留者
+  附仲裁者自己的引述，剔除者不列。剔除集合＝送裁鍵減
+  輸出鍵，由程式推得。
+- **screen**：輸出即引述陣列，非空＝有、空＝無。僅
+  「一有一無」的歌送仲裁（`02-01-02-screen-arb.md`），
+  輸入為歌詞加主張「有」方的引述（不記名），輸出同為
+  引述陣列。
+- 仲裁者的引述可能與原引述不同：仲裁是對歌詞的重新
+  判讀，其引述是該裁決自身的依據，非轉抄。
+- 送入仲裁的標籤即模型自身不穩定的邊界判斷，其裁決為
+  單次記錄性決定，不宣稱可再生；可重現性依計畫定義為
+  「程序透明＋可稽核」，裁決與其輸入全程歸檔。
+
+## 軌跡對映（診斷用）
+
+沿收斂軌跡的機械對映：原始關鍵字 → merge 組 → cap 組，
+純程式查表，決定性。以其結果與 code 直接編碼的差異率
+作為「收斂軌跡扭曲」的診斷量，不作主結果。
+
+## 全管線的交接契約
+
+每一步的輸出如何變成下一步的輸入，皆為確定性程序，規則
+明定如下：
+
+- **歌詞輸入檔（tag、code、screen 共用）**：
+  `export-llm-input` 自工作儲存產出，每筆
+  `{"id": "song-<ID>", "content": <歌詞>}`，依歌曲 ID
+  升序。三個讀歌詞的步驟共用同一檔，SHA-256 記入各步
+  meta。
+- **tag → merge**：兩次執行的全部關鍵字取聯集、逐字串
+  精確去重、字典序排列成 JSON 陣列，即 merge 的輸入。
+  進池同時寫出處記錄（關鍵字 →（執行別，歌曲 ID）
+  清單），供軌跡對映回到歌曲；出處記錄不進任何 LLM
+  輸入。
+- **merge → cap**：merge 輸出的組名以字典序排成 JSON
+  陣列，即 cap 的輸入；原始關鍵字不下傳。
+- **cap → code 定義檔**：cap 輸出的組名以字典序逐行填入
+  `01-04-01-code.md` 的詞彙表節（逐字），檔案隨 git
+  commit 後方可執行——code 的定義檔因此自我完備，
+  論文附錄可直接引用。
+- **code 兩次執行 → code-arb**：逐首比對標籤集合
+  （鍵集合，引述不參與比對）；僅有分歧的歌入仲裁輸入
+  JSONL，依歌曲 ID 升序，每筆 `id` 沿用 `song-<ID>`、
+  `content` 為固定鍵序序列化的
+  `{"lyrics": …, "disagreements": …}`，disagreements
+  鍵按字典序。
+- **code 定案**：每首歌的最終標籤＝共識標籤 ∪ 仲裁保留
+  標籤，寫入逐首紀錄檔（歌依 ID 升序、標籤按字典序，
+  各標籤附其定案時的引述與來源層——共識或仲裁）。
+- **screen 兩次執行 → screen-arb**：僅「一有一無」的歌
+  入仲裁輸入 JSONL（依 ID 升序），`content` 為
+  `{"lyrics": …, "evidence": <肯定方引述>}`。
+- **screen 定案**：命中集合＝兩次皆有 ∪ 仲裁裁定有。
+- **女性力量候選集**：於 cap 詞彙表定案後、黃金標準
+  編碼開始前，由研究者指認詞彙表中屬「女性力量」概念
+  域的組（指認及理由記入決策日誌），候選集＝code 定案
+  標籤含該等組者 ∪ screen 命中者。
+- **序列化通則**：所有中間檔為 UTF-8 JSON，鍵序與元素
+  序皆依上列規則明定，無時間戳、無隨機成分；解析一律
+  偵測重複鍵，違規即失敗。JSONL 一律以換行字元（\n）
+  切行——歌詞含 U+0085 等控制字元時，
+  `str.splitlines()` 類的通用切行會截斷 JSON 字串
+  （實測踩中）。給定相同的 LLM 執行輸出，全部交接產物
+  逐位元組可再生。
+
+## 執行與稽核
+
+- 每一步驟以 `run-llm <定義檔> <輸入檔> <歸檔目錄>`
+  執行；2+1 步驟的兩次執行＝重現命令清單上的兩行命令，
+  各自歸檔（`runs/<定義檔名>/run1`、`run2`），仲裁為
+  獨立步驟、獨立歸檔；單次步驟（merge、cap）一行命令、
+  一份歸檔。
+- 進池、比對、裁決套用、對映皆為確定性程式（子命令），
+  其輸入輸出檔隨 runs/ 歸檔。
+- Batch API 的每筆請求自含全部脈絡且互不可見（平台
+  契約），歌與歌之間的獨立性由此成立；兩次執行的獨立
+  性由「兩次呼叫、兩個批次、兩份歸檔」的執行結構自明。
+
+## 映射分析方法
+
+（依 2026-07-30 決策，於看到結果前寫定；待黃金標準
+編碼展開前補入。）

docs/project-structure.md

@@ -24,7 +24,8 @@ pop-fem-audit/
 │       ├── songs.csv          # 歌曲報表（人讀；進 git）
 │       └── artists.csv        # 歌手報表（人讀；進 git）
 ├── prompts/                   # LLM 定義檔（逐字作為 system prompt）
-│   └── <軌>-<步>-<task>.md    #   01-01-tag.md、02-01-screen.md…
+│   └── <軌>-<步>-<次步>-<task>.md  # 01-01-01-tag.md、
+│                              #   01-04-02-code-arb.md…
 │                              #   不帶版本號，版本即 git 歷史
 ├── tools/                     # 輔助工具子專案（src-layout）
 │   ├── pyproject.toml         #   發行名 pop-fem-audit-tools；

docs/research-plan.md

@@ -13,11 +13,14 @@
 - **執行原則**：主會話只做討論；所有分析由 deterministic script
   執行。LLM 步驟以 Python script 呼叫 Anthropic Messages API
   （個人 Console 帳號、Batch API 五折），定義檔逐字作為 system
-  prompt。2+1 協定依輸出可比性適用：輸出可逐項機械比對的步驟
-  （收斂、編碼）「同一定義檔獨立執行兩次＋一次仲裁」，仲裁
-  只裁程式算出的分歧清單；自由生成步驟（首步自由標註）兩次
-  執行全數進池、不仲裁——自由詞彙兩次輸出不共享比對單位，
-  無物可裁。仲裁結果不符預期則修訂定義檔重跑該循環。
+  prompt。2+1 協定適用於語料層的逐首判斷（編碼、篩選）：
+  「同一定義檔獨立執行兩次＋一次仲裁」，仲裁只裁程式算出
+  的分歧。自由生成步驟（自由標註）兩次執行全數進池、不
+  仲裁——自由詞彙兩次輸出不共享比對單位，無物可裁。詞彙
+  表建構（收斂）為單次記錄性程序，不重複執行——詞彙表是
+  揭露的儀器選擇，非量測；信度檢驗施於編碼層（詳見
+  `methodology.md`）。仲裁或驗證結果不符預期則修訂定義檔
+  重跑該循環。
 - **提示詞只定格式、不定語意**：研究對象是通用 LLM 以其
   網路語料知識背景所做的自然編碼，編碼結果本身是批判對象。
   LLM 定義檔只規定任務形狀（輸入、數量範圍、輸出格式），
@@ -95,11 +98,12 @@
    themes 過早抽象）之繼承，於執行前鎖定，防止事後擇優。
 2. **自然收斂（merge）**：輸入為池中純去重關鍵字清單——
    無歌詞、無頻次、無歌曲出處——模型按自身理解合併近似
-   概念。2+1：兩次收斂處理同一批輸入詞，程式先算出共識核
-   （兩次都同組的詞塊，機械凍結）與分歧清單，仲裁者只裁
-   分歧清單，無權動共識核、無權引入新概念。
-3. **強制收斂（cap）**：同第 2 步形態，限制併至 50 個
-   以內。頻次不入收斂：頻率的分析角色由第 4 步編碼承擔；
+   概念，執行內自行為各組命名。單次執行，不仲裁；確定性
+   驗證（完整分割、組名唯一）違規即修訂重跑（詳見
+   `methodology.md`）。
+3. **強制收斂（cap）**：同第 2 步形態，輸入為 merge
+   輸出的組名清單（原始關鍵字不下傳），限制併至 50 個
+   以內。單次執行，不仲裁。頻次不入收斂：頻率的分析角色由第 4 步編碼承擔；
    池中頻次含跨執行噪音；頻次會誘使模型以頻率剪枝（喪失
    稀有主題）並把高頻大主題切細。代價（特異主題佔名額）
    已知並接受，換取主題多樣性與純語意歸併的可辯護性。
@@ -120,15 +124,17 @@
 召回之用，永不進任何統計的分子分母。標籤詞是 screen
 提示中唯一的語意種子，屬研究者的儀器選擇，據實揭露。
 
-定義檔命名 `prompts/<軌>-<步>-<task>.md`——軌 01＝由下
-而上自然編碼、02＝預先決定的 women-power
-篩選，步為軌內步驟序：01-01-tag.md、01-02-merge.md、
-01-03-cap.md、01-04-code.md、02-01-screen.md；仲裁定義檔
-同 prefix 加 `-arb`（如 01-02-merge-arb.md）。檔名不帶
-版本號——版本即 git 歷史，失敗的版本不保留，需要回看的
-舊版都在 git history；每次執行的定義檔快照隨 `runs/`
-自我完備。四次收斂執行（merge ×2、cap ×2）各將合併記錄
-（哪些詞併入哪組）存成 JSON，隨該次執行入 `runs/`。
+定義檔命名 `prompts/<軌>-<步>-<次步>-<task>.md`——軌
+01＝由下而上自然編碼、02＝預先決定的 women-power 篩選；
+步為軌內步驟序，次步為步內執行順序，讀者依編號先後依循：
+01-01-01-tag.md、01-02-01-merge.md、01-03-01-cap.md、
+01-04-01-code.md、01-04-02-code-arb.md、
+02-01-01-screen.md、02-01-02-screen-arb.md；仲裁定義檔
+同 prefix 加 `-arb`。檔名不帶版本號——版本即 git 歷史，
+失敗的版本不保留，需要回看的舊版都在 git history；每次
+執行的定義檔快照隨 `runs/` 自我完備。收斂執行的輸出本身
+即分組記錄（具名分組 JSON），與全部中間交接檔同隨
+`runs/` 歸檔（交接契約見 `methodology.md`）。
 
 ## 階段與時程（全文截稿 2026-08-15）
 
@@ -136,7 +142,7 @@
 |---|---|---|---|
 | 0 | 基礎建設：git init、目錄結構、.gitignore、決策日誌、runner script（含 Batch API）、codebook v0 骨架 | script + 討論 | 7/30–7/31 |
 | 1 | 資料準備：`run_llm` 改走統一設定 → `build-db`（解析榜單成 songs/chart_entries/artists/song_artists）→ `import-lyrics`（pilot 2018–2025）→ `fetch-lyrics`（2016–17 與缺漏，Lyrics.ovh / LRCLIB）→ `fetch-artists`（Wikidata 快照）→ `export-llm-input` | 子命令 | 7/31–8/3 |
-| 2 | 自然編碼管線：tag ×2 進池 → merge 2+1 → cap 2+1 → 詞彙表定稿 → code 2+1（全 883 首，附引述）；另跑 screen 補漏網 | API + script | 8/4–8/7 |
+| 2 | 自然編碼管線：tag ×2 進池 → merge ×1 → cap ×1 → 詞彙表定稿 → code 2+1（全 883 首，附引述）；另跑 screen 補漏網 | API + script | 8/4–8/7 |
 | 3 | 黃金標準：依 codebook 人工逐首判定 genuine/peripheral/fake，附引用歌詞證據表（LLM 只做摘錄，不給判定建議）；先以 10–15 首校準樣本試編並修訂 codebook 後凍結；同批校準樣本實測 Sonnet 4.6 vs Opus 5 一致率 | 人工 + script 輔助 | 8/5–8/9 |
 | 4 | 受控比較（盲點實驗）：條件 A（詞彙層提示）vs 條件 B（框架感知提示），各 2+1，對照黃金標準計算假陽／假陰率 | API | 8/8–8/11 |
 | 4' | 映射分析：自然編碼結果（第 4 步）與黃金標準交叉表；軌跡對映 vs 直接編碼的扭曲診斷（分析方法先寫入 methodology.md 再看結果） | script | 與 4 並行 |

prompts/01-01-01-tag.md

@@ -0,0 +1,27 @@
+You are annotating song lyrics for a content-analysis corpus.
+
+Input: the complete lyrics of one song.
+
+Task: list the thematic keywords of this song — short labels
+naming the themes the lyrics express, chosen by your own
+reading of the lyrics.
+
+Rules:
+
+- Give between 5 and 10 thematic keywords.
+- Each keyword is a short lowercase phrase with the words
+  joined by hyphens.
+- For each keyword, quote 1 to 3 verbatim lines of the lyrics
+  that ground it; do not give a keyword you cannot ground.
+
+Do not wrap the output in a Markdown code fence.
+The output must be valid JSON; escape any double quote
+inside a string as \".
+
+Output a single JSON object mapping each keyword to its list
+of quotes, and nothing else:
+
+{
+  "first-keyword": ["quoted line", "another quoted line"],
+  "second-keyword": ["quoted line"]
+}

prompts/01-02-01-merge.md

@@ -0,0 +1,30 @@
+You are consolidating a vocabulary of thematic keywords that
+were independently annotated on the songs of a corpus.
+
+Input: a JSON array of thematic keywords, deduplicated and
+unordered.
+
+Task: merge keywords that express the same or nearly the same
+theme, by your own understanding of what the keywords mean.
+
+Rules:
+
+- Every input keyword must appear in exactly one group.
+- Do not invent keywords that are not in the input.
+- Merge only what you consider the same theme; a group may be
+  a single keyword.
+- Name each group with a short lowercase phrase, words joined
+  by hyphens, that best names the theme it gathers.  Group
+  names must be unique.
+
+Do not wrap the output in a Markdown code fence.
+The output must be valid JSON; escape any double quote
+inside a string as \".
+
+Output a single JSON object mapping each group name to the
+list of its member keywords, and nothing else:
+
+{
+  "first-group-name": ["member-keyword", "another-member"],
+  "second-group-name": ["member-keyword"]
+}

prompts/01-03-01-cap.md

@@ -0,0 +1,33 @@
+You are consolidating a vocabulary of thematic keywords
+from a song corpus further, to a fixed maximum number of
+themes.
+
+Input: a JSON array of thematic keywords, deduplicated and
+unordered.
+
+Task: merge keywords that express the same or a similar
+theme, by your own understanding of what the keywords mean,
+until at most 50 groups remain.
+
+Rules:
+
+- Every input keyword must appear in exactly one group.
+- Do not invent keywords that are not in the input.
+- Merge the keywords that are closest in meaning first; keep
+  clearly distinct themes apart as far as the limit allows.
+  A group may be a single keyword.
+- Name each group with a short lowercase phrase, words joined
+  by hyphens, that best names the theme it gathers.  Group
+  names must be unique.
+
+Do not wrap the output in a Markdown code fence.
+The output must be valid JSON; escape any double quote
+inside a string as \".
+
+Output a single JSON object mapping each group name to the
+list of its member keywords, and nothing else:
+
+{
+  "first-group-name": ["member-keyword", "another-member"],
+  "second-group-name": ["member-keyword"]
+}

prompts/01-04-01-code.md

@@ -0,0 +1,35 @@
+You are coding song lyrics against a fixed vocabulary of
+thematic keywords.
+
+Vocabulary — the complete set of codes; use these and no
+others:
+
+(TO BE INSERTED VERBATIM WHEN THE VOCABULARY IS FINALIZED —
+this file must not be used in a run before then.)
+
+Input: the complete lyrics of one song.
+
+Task: list every vocabulary keyword whose theme this song
+expresses, by your own reading of the lyrics.
+
+Rules:
+
+- Use only keywords from the vocabulary, spelled exactly as
+  given.
+- A song may match any number of keywords, including none.
+- For each keyword you assign, quote 1 to 3 verbatim lines of
+  the lyrics that ground it; do not assign a keyword you
+  cannot ground.
+
+Do not wrap the output in a Markdown code fence.
+The output must be valid JSON; escape any double quote
+inside a string as \".
+
+Output a single JSON object mapping each assigned keyword to
+its list of quotes (an empty object when no keyword applies),
+and nothing else:
+
+{
+  "first-keyword": ["quoted line", "another quoted line"],
+  "second-keyword": ["quoted line"]
+}

prompts/01-04-02-code-arb.md

@@ -0,0 +1,43 @@
+You are arbitrating between two independent codings of the
+same song against the same fixed vocabulary of thematic
+keywords.
+
+The keywords the two codings agree on have been settled by
+script and are not shown.  You rule only on the
+disagreements: the keywords assigned by one coding but not
+the other.
+
+Input: a JSON object with the lyrics of the song and the
+disagreements — each disagreement maps a keyword to the lyric
+quotes the assigning coding gave as evidence:
+
+{
+  "lyrics": "...",
+  "disagreements": {
+    "first-keyword": ["quoted line", "another quoted line"],
+    "second-keyword": ["quoted line"]
+  }
+}
+
+Task: for each disagreement keyword, decide whether the song
+expresses that theme, by your own reading of the lyrics and
+of the quoted evidence.
+
+Rules:
+
+- Rule on every listed keyword: keep it or drop it.
+- Keep a keyword only when you can ground it in verbatim
+  lines of the lyrics; quote them.
+- Do not add any keyword that is not listed.
+
+Do not wrap the output in a Markdown code fence.
+The output must be valid JSON; escape any double quote
+inside a string as \".
+
+Output a single JSON object mapping each kept keyword to your
+list of quotes, and nothing else.  A dropped keyword is simply
+left out; output an empty object when nothing is kept:
+
+{
+  "kept-keyword": ["quoted line", "another quoted line"]
+}

prompts/02-01-01-screen.md

@@ -0,0 +1,23 @@
+You are screening song lyrics for one specific theme.
+
+Input: the complete lyrics of one song.
+
+Task: judge whether this song expresses the theme
+"women-power", by your own understanding of
+that label.
+
+Rules:
+
+- If the song expresses the theme, quote 1 to 3 verbatim
+  lines of the lyrics that ground the judgment; if you cannot
+  ground it in a quote, the answer is no.
+
+Do not wrap the output in a Markdown code fence.
+The output must be valid JSON; escape any double quote
+inside a string as \".
+
+Output a single JSON array and nothing else: the quoted
+lines when the song expresses the theme, or an empty array
+when it does not:
+
+["quoted line", "another quoted line"]

prompts/02-01-02-screen-arb.md

@@ -0,0 +1,32 @@
+You are arbitrating a screening of song lyrics for one
+specific theme: two independent screenings of the same song
+disagreed on whether it expresses the theme
+"women-power".
+
+Input: a JSON object with the lyrics of the song and the
+lyric quotes that the affirming screening gave as evidence:
+
+{
+  "lyrics": "...",
+  "evidence": ["quoted line", "another quoted line"]
+}
+
+Task: decide whether the song expresses the theme, by your
+own understanding of the label, reading the lyrics and the
+quoted evidence.
+
+Rules:
+
+- If the song expresses the theme, quote 1 to 3 verbatim
+  lines of the lyrics that ground the judgment; if you cannot
+  ground it in a quote, the answer is no.
+
+Do not wrap the output in a Markdown code fence.
+The output must be valid JSON; escape any double quote
+inside a string as \".
+
+Output a single JSON array and nothing else: the quoted
+lines when the song expresses the theme, or an empty array
+when it does not:
+
+["quoted line", "another quoted line"]

tools/src/pop_fem_audit_tools/commands/fetch_lyrics.py

@@ -57,6 +57,49 @@ SLEEP_SECONDS: float = 1.0
 """The delay between consecutive HTTP requests, in seconds."""
 
 
+def _build_normalization() -> dict[int, str | None]:
+    """Build the lyrics normalization translation table.
+
+    :return: The codepoint-to-replacement mapping, a replacement
+        of None meaning removal.
+    """
+    table: dict[int, str | None] = {}
+    codepoint: int
+    for codepoint in range(0x80, 0xa0):
+        try:
+            table[codepoint] = bytes([codepoint]).decode("cp1252")
+        except UnicodeDecodeError:
+            table[codepoint] = None
+    table[0x0435] = "e"
+    table[0x03cc] = "ó"
+    for codepoint in (0x2005, 0x205f, 0x200a):
+        table[codepoint] = " "
+    for codepoint in (0x200b, 0x200c, 0x200d, 0xfeff):
+        table[codepoint] = None
+    return table
+
+
+NORMALIZATION: dict[int, str | None] = _build_normalization()
+"""The codepoint-to-replacement mapping applied to fetched
+lyrics: cp1252-mojibake restoration for U+0080-U+009F (with the
+five byte values undefined in cp1252 removed), homoglyph
+restoration for the Cyrillic "e" and the Greek "o" with tonos,
+ASCII-space restoration for exotic space variants, and removal
+of zero-width characters.  A replacement of None removes the
+codepoint."""
+
+
+def normalize_lyrics(text: str) -> str:
+    """Restore or remove watermark and mojibake characters.
+
+    :param text: The lyrics text as fetched from an API.
+    :return: The text with the codepoints in
+        :data:`NORMALIZATION` replaced or removed; every other
+        character is unchanged.
+    """
+    return text.translate(NORMALIZATION)
+
+
 def parse_args(argv: list[str] | None) -> argparse.Namespace:
     """Parse the command-line arguments.
 
@@ -187,6 +230,9 @@ def save_lyrics(lyrics_dir: Path, song_id: int,
 
     The cache directory is created when missing.
 
+    The lyrics text is normalized with :func:`normalize_lyrics`
+    before being written.
+
     :param lyrics_dir: The lyrics cache directory.
     :param song_id: The song ID.
     :param lyrics: The lyrics text.
@@ -195,7 +241,7 @@ def save_lyrics(lyrics_dir: Path, song_id: int,
     """
     lyrics_dir.mkdir(parents=True, exist_ok=True)
     (lyrics_dir / f"{song_id}.txt").write_text(
-        lyrics, encoding="utf-8")
+        normalize_lyrics(lyrics), encoding="utf-8")
 
 
 def append_provenance(path: Path, song_id: int,

tools/tests/test_fetch_lyrics.py

@@ -317,6 +317,79 @@ class TestFetchLyrics(unittest.TestCase):
         self.assertEqual([x[0] for x in rows[1:]],
                          ["1", "2", "2"])
 
+    def test_normalize_cp1252_mojibake(self) -> None:
+        """Test that cp1252 mojibake codepoints are restored."""
+        self.assertEqual(
+            fetch_lyrics.normalize_lyrics("wait…"),
+            "wait…")
+        self.assertEqual(
+            fetch_lyrics.normalize_lyrics(
+                "‘quote’"),
+            "‘quote’")
+        self.assertEqual(
+            fetch_lyrics.normalize_lyrics(
+                "“quote”"),
+            "“quote”")
+        self.assertEqual(
+            fetch_lyrics.normalize_lyrics("dash—line"),
+            "dash—line")
+
+    def test_normalize_undefined_cp1252_removed(self) -> None:
+        """Test that undefined cp1252 byte values are removed."""
+        text: str = ("abcdef")
+        self.assertEqual(
+            fetch_lyrics.normalize_lyrics(text), "abcdef")
+
+    def test_normalize_homoglyphs(self) -> None:
+        """Test that watermark homoglyphs are restored."""
+        self.assertEqual(
+            fetch_lyrics.normalize_lyrics("likе that"),
+            "like that")
+        self.assertEqual(
+            fetch_lyrics.normalize_lyrics("lό que soy"),
+            "ló que soy")
+
+    def test_normalize_space_variants(self) -> None:
+        """Test that exotic space variants become ASCII space."""
+        self.assertEqual(
+            fetch_lyrics.normalize_lyrics(
+                "a b c d"),
+            "a b c d")
+
+    def test_normalize_zero_width_removed(self) -> None:
+        """Test that zero-width characters are removed."""
+        text: str = (
+            "a​b‌c‍de")
+        self.assertEqual(
+            fetch_lyrics.normalize_lyrics(text), "abcde")
+
+    def test_normalize_ascii_unchanged(self) -> None:
+        """Test that plain ASCII text passes through unchanged."""
+        text: str = "Hello, it's me\n"
+        self.assertEqual(
+            fetch_lyrics.normalize_lyrics(text), text)
+
+    def test_normalize_legitimate_non_ascii_unchanged(self) -> None:
+        """Test that legitimate non-ASCII content is unchanged."""
+        text: str = "¿cómo estás? 안녕하세요\n"
+        self.assertEqual(
+            fetch_lyrics.normalize_lyrics(text), text)
+
+    def test_fetched_lyrics_saved_normalized(self) -> None:
+        """Test that a fetched lyric is normalized before saving."""
+        self.__seed([("Hello", "Adele")])
+        with mock.patch(
+                "urllib.request.urlopen",
+                side_effect=[self.__response(
+                    {"lyrics": "wait… likе"
+                               " that now​\n"})]):
+            status: int = self.__run_fetch()[0]
+        self.assertEqual(status, 0)
+        self.assertEqual(
+            (self.__lyrics / "1.txt")
+            .read_text(encoding="utf-8"),
+            "wait… like that now\n")
+
     def test_no_store_fails(self) -> None:
         """Test that a missing working store fails the run."""
         urlopen: mock.Mock