From 964006ec6d2b38d149c8841eec9c0dcac65c1454 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=BE=9D=E7=91=AA=E8=B2=93?= Date: Tue, 4 Aug 2026 22:38:13 +0800 Subject: [PATCH] Add the natural-coding definition files and the convergence algorithms Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 7 +- docs/decision-log.md | 45 +++++++ docs/methodology.md | 197 +++++++++++++++++++++++++++++ docs/project-structure.md | 3 +- docs/research-plan.md | 30 +++-- prompts/01-01-01-tag.md | 23 ++++ prompts/01-02-01-merge.md | 23 ++++ prompts/01-02-02-merge-arb.md | 36 ++++++ prompts/01-02-03-merge-arb-arb.md | 34 +++++ prompts/01-02-04-merge-name.md | 28 ++++ prompts/01-02-05-merge-name-arb.md | 37 ++++++ prompts/01-03-01-cap.md | 26 ++++ prompts/01-03-02-cap-arb.md | 37 ++++++ prompts/01-03-03-cap-arb-arb.md | 35 +++++ prompts/01-03-04-cap-name.md | 29 +++++ prompts/01-03-05-cap-name-arb.md | 38 ++++++ prompts/01-04-01-code.md | 31 +++++ prompts/01-04-02-code-arb.md | 39 ++++++ prompts/02-01-01-screen.md | 19 +++ prompts/02-01-02-screen-arb.md | 28 ++++ 20 files changed, 729 insertions(+), 16 deletions(-) create mode 100644 docs/methodology.md create mode 100644 prompts/01-01-01-tag.md create mode 100644 prompts/01-02-01-merge.md create mode 100644 prompts/01-02-02-merge-arb.md create mode 100644 prompts/01-02-03-merge-arb-arb.md create mode 100644 prompts/01-02-04-merge-name.md create mode 100644 prompts/01-02-05-merge-name-arb.md create mode 100644 prompts/01-03-01-cap.md create mode 100644 prompts/01-03-02-cap-arb.md create mode 100644 prompts/01-03-03-cap-arb-arb.md create mode 100644 prompts/01-03-04-cap-name.md create mode 100644 prompts/01-03-05-cap-name-arb.md create mode 100644 prompts/01-04-01-code.md create mode 100644 prompts/01-04-02-code-arb.md create mode 100644 prompts/02-01-01-screen.md create mode 100644 prompts/02-01-02-screen-arb.md diff --git a/CLAUDE.md b/CLAUDE.md index ee072be..ec854e1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,9 +6,10 @@ Messages API: model `claude-sonnet-4-6`, `temperature=0`, thinking disabled, Batch API where possible. - Prompt definition files live in - `prompts/--.md` (e.g. 01-01-tag.md; no - version suffix -- versions live in git history) and are - passed verbatim as the system prompt. + `prompts/---.md` (e.g. + 01-01-01-tag.md; the substep number fixes the execution + order; no version suffix -- versions live in git history) + and are passed verbatim as the system prompt. - LLM steps whose outputs are item-by-item comparable (convergence, coding) run the same definition file twice, then a separate arbitration step settles only the diff --git a/docs/decision-log.md b/docs/decision-log.md index 97f4114..6b16e0e 100644 --- a/docs/decision-log.md +++ b/docs/decision-log.md @@ -323,6 +323,51 @@ 式化、消滅隱性推導」原則的貫徹。比對子命令驗證兩 run 的定義檔與 輸入 SHA 一致、缺 run 即失敗。目標目錄已存在即拒絕執行，重跑須明 示 `--replace`。 +- **收斂仲裁演算法定案（逐對裁決）**：不做組對組匹配 + （兩組是否「同一組」無原則性答案），比較化約為詞對的 + 共組關係——共識塊（交集細分）→ 分歧塊對枚舉 → 仲裁 + 逐對二元裁決（輸入匿名，不含執行別、共識與歌詞）→ + union-find 遞移重組（固定邊集的連通元件唯一，與順序 + 無關，決定性成立）。仲裁自身亦 2+1：獨立執行兩次， + 逐對比對，兩次相同即定案，相異者送 `-arb-arb` 終局票 + （單次，依協定終局）——每個分歧對等同三票多數決， + 仲裁鏈至此終止。邊界判斷之裁決為單次記錄性決定，不 + 宣稱可再生（可重現性依計畫定義＝程序透明＋可稽核）。 + 遞移導出的合併（共識「分開」的塊經第三塊橋接併組）為 + 等價關係語意的邏輯結論，照單接受並逐筆記錄供稽核。 + 一致率低於門檻（暫訂 50%，首輪實跑後校準）即不仲裁 + ——視為定義檔約束不足，修訂後重跑該循環。cap 仲裁後 + 組數略超 50 可接受，如實記錄。演算法明文於 + `methodology.md`，供讀者依循重現。 +- **收斂執行不命名，組名全權歸命名鏈（命名 2+1）**： + merge、cap 的執行輸出改為無名分組（陣列之陣列）—— + 既然命名鏈存在，收斂執行自取的名字毫無用處，徒增 + 兩次執行間「同組不同名」的假分歧。最終分組定案後， + `-name` 命名步驟獨立執行兩次對全部組自由命名——逐組 + 比對，同名即定案（單次執行內名字唯一，故定名間必無 + 撞名）；異名送 `-name-arb` 擇一仲裁（匿名呈現兩候選， + 輸出限於候選並避讓已定名，機械可驗證）。理由：組名 + 承重——merge 名為 cap 的輸入詞、cap 名為 code 的編碼 + 類目——凡承重的語意判斷皆受 2+1。cap 的輸入即 merge + 定案後的組名清單，原始關鍵字不下傳——如同證據不進 + 收斂，上游細節不污染下游步驟。 +- **定義檔輸出形狀統一與去署名**：LLM 輸出統一為單層 + dict／array——tag、code：關鍵字→引述；merge、cap： + 無名分組（陣列之陣列）；code-arb：保留關鍵字→仲裁者自己的引述 + （剔除即不列，剔除集合由程式以鍵差推得）；screen 系： + 引述陣列，非空即「有」（present 布林刪除，判斷與依據 + 合一）。仲裁輸入一律不含執行別（assigned_by、 + run1/run2 刪除）——標示何方主張會誘使仲裁者揣測 + 「哪次較可信」，而非就文本裁決。收斂組名採自由命名 + （不限取自成員詞），取命名貼合度；可追溯性由軌跡歸檔 + 承擔。 +- **定義檔改三層編號**： + `prompts/<軌>-<步>-<次步>-.md`——次步為步內執行 + 順序，明定讀者依循的先後（如 01-02-01-merge → + 01-02-02-merge-arb → 01-02-03-merge-arb-arb → + 01-02-04-merge-name → 01-02-05-merge-name-arb）。後綴 + `-arb` 仲裁、`-arb-arb` 終局票、`-name` 命名、 + `-name-arb` 命名擇一仲裁。 - **screen 標籤詞改為 `women-power`（來歷考據定案）**： 考據先導研究的 local agent 存檔：其第一步指令含數十個 範例 thematic keywords，其中即有 women-power——為當時 diff --git a/docs/methodology.md b/docs/methodology.md new file mode 100644 index 0000000..8f3b7ea --- /dev/null +++ b/docs/methodology.md @@ -0,0 +1,197 @@ +# 方法細節 + +（全文方法節底稿。演算法在執行前寫定；任何修訂記入 +`decision-log.md`。定義檔全文見 `prompts/`，執行紀錄見 +`runs/`。） + +## 自然編碼管線總覽 + +四步驟：自由標註（tag）→ 自然收斂（merge）→ 強制收斂 +（cap）→ 全量編碼（code），另設「女性力量」單目標篩選 +（screen）作黃金標準取樣的補漏網。歌詞只出現在 tag、code、 +screen 與各仲裁步驟；merge、cap 及其仲裁、命名皆不接觸 +歌詞。設計原則見 `research-plan.md`；本檔記載可重現的 +演算法細節。 + +## 收斂步驟（merge、cap）的 2+1 比對與仲裁 + +兩次獨立執行對同一批輸入詞各產生一個分組。比對不做 +「組對組」的匹配——兩個組是否為「同一組的變體」無原則性 +答案——而是把比較化約為「詞對的共組關係」： + +1. **共識塊（交集細分）**：兩次執行都放在同組的詞歸為 + 同一塊。即以「(第一次的組, 第二次的組)」二元組為鍵 + 分桶，一桶一塊。此步為純集合運算。 +2. **分歧塊對枚舉**：每個塊完整落在各次執行的恰一組內， + 故「兩塊在某次執行中是否同組」定義良好。逐塊對檢查： + 兩次執行答案相同者為共識（同組或分開，直接定案）； + 不同者列入分歧清單。 +3. **逐對仲裁（自身 2+1）**：分歧塊對送 LLM 仲裁 + （`01-02-02-merge-arb.md`、`01-03-02-cap-arb.md`）。 + 仲裁者只看兩塊的內容詞，逐對二元裁決「是否同一 + 主題」；輸入不含執行別（避免「猜哪一次較可信」的 + 偏誤），不含共識部份，不含歌詞。仲裁自身獨立執行 + 兩次，逐對比對：兩次裁決相同即定案；相異的塊對送 + 終局票（`01-02-03-merge-arb-arb.md`、 + `01-03-03-cap-arb-arb.md`，單次執行，依協定為終局） + ——每個分歧塊對等同三票多數決，仲裁鏈至此終止。 + 送入仲裁的塊對即模型自身不穩定的邊界判斷，其裁決為 + 單次記錄性決定，不宣稱可再生；可重現性依計畫定義為 + 「程序透明＋可稽核」，裁決與其輸入全程歸檔。 +4. **確定性重組**：以每個「同組」裁決為一條邊，最終 + 分組＝圖的連通元件（union-find）。固定邊集的連通 + 元件唯一，與處理順序無關，故重組決定性成立。 + 遞移性後果照單全收：兩個共識「分開」的塊可能經第三 + 塊橋接而併入同組——此為等價關係語意的邏輯結論， + 比對程式將此類「遞移導出的合併」逐筆記錄於歸檔， + 供稽核。仲裁的結果空間因此大於「兩次執行擇一」： + 可能比兩次都粗（多對皆裁同組），也可能比兩次都細 + （多對皆裁分開）。 +5. **一致率與保險絲**：比對程式計算塊對層級的一致率並 + 記入歸檔。低於門檻（暫訂 50%，首輪實跑後校準）即 + 不進行仲裁——兩次分組面目全非說明定義檔約束不足， + 依協定修訂定義檔並重跑該循環。 +6. **cap 的上限容忍**：仲裁後組數可能略超 50（多對裁 + 「分開」時）。略微超過可接受，如實記錄，不強行 + 壓縮。 +7. **組名定案（收斂不命名＋命名 2+1）**：merge、cap 的 + 執行輸出不含組名（無名分組，陣列之陣列）——命名鏈 + 既然存在，收斂執行自取的名字只會製造兩次執行間 + 「同組不同名」的假分歧。組名承重——merge 的組名是 + cap 的輸入詞，cap 的組名是 code 對歌詞編碼的類目—— + 故與其他語意判斷同受 2+1。程序：(1) 最終分組定案後， + 命名步驟（`01-02-04-merge-name.md`、 + `01-03-04-cap-name.md`）獨立執行兩次，輸入為不透明 + 組 ID 對成員詞，對全部組自由命名，僅受格式約束 + （小寫連字號、單次執行內不重複）；(2) 逐組比對：兩次 + 同名即定案（單次執行內名字唯一，故定名間必無撞名）； + (3) 異名的組送擇一仲裁（`01-02-05-merge-name-arb.md`、 + `01-03-05-cap-name-arb.md`）——匿名呈現兩候選，逐組 + 擇一，輸出限於候選並避讓已定名，機械可驗證。cap 的 + 輸入即 merge 定案後的組名清單。 + +### 收斂步驟的資料流（比對子命令的輸入輸出契約） + +以 merge 為例（cap 完全同構，檔名換為 01-03 系）： + +1. 兩次執行的原始輸出：`runs/01-02-01-merge/run1/output.jsonl` + 與 `run2/output.jsonl`，各含一個無名分組（陣列之陣列）。 +2. 比對子命令讀入兩份分組，先驗證兩者為同一輸入詞集的 + 完整分割（缺詞、多詞、重複即失敗），再計算共識塊與 + 分歧塊對，產出仲裁輸入檔——即 + `01-02-02-merge-arb.md` 所收的 JSON： + - `blocks`：塊 ID → 成員詞。**塊 ID 的指派決定性**： + 全部塊先按「各塊字典序最小的成員詞」排序，依序編為 + b1、b2、…；塊內成員詞亦按字典序排列。 + - `pairs`：分歧塊對清單，每對內部按塊 ID 序、清單 + 整體按 (第一元素, 第二元素) 字典序排列。 + - 僅分歧塊對入列；共識（同組或分開）不送仲裁，由 + 比對子命令直接寫入共識紀錄檔。 + 同時產出：共識紀錄（共識塊、共識同組對、共識分開對） + 與塊對一致率（含保險絲判定）。 +3. 仲裁輸入檔以 run-llm 跑兩次 + （`runs/01-02-02-merge-arb/run1`、`run2`）；比對 + 子命令逐對比對兩份裁決，兩票相同即定案，相異的塊對 + 依同一契約組成終局票輸入檔（`blocks` 僅含涉事塊、 + ID 沿用原編號），跑 + `runs/01-02-03-merge-arb-arb/run1`。 +4. 裁決套用子命令彙整三票結果，以 union-find 重組出 + 最終分組，並寫出：最終分組檔（無名，陣列之陣列， + 組序與組內成員皆字典序）、遞移導出合併的紀錄、 + 收斂軌跡（原始詞 → 最終組）。 +5. 最終分組轉為不透明組 ID（依組序編 g1、g2、…）進入 + 命名鏈（見第 7 條）；命名定案後，組名清單（字典序） + 即下一步的輸入。 +6. 以上中間檔全部隨 `runs/` 歸檔；一切排序規則固定， + 故給定相同的兩份執行輸出與相同的裁決，全流程輸出 + 逐位元組可再生。 + +## 編碼步驟（code、screen）的 2+1 比對與仲裁 + +- **code**：逐首比對兩次執行的標籤集合（引述不參與 + 比對）。兩次皆有的標籤為共識保留、兩次皆無為共識 + 不標；單邊標籤送仲裁（`01-04-02-code-arb.md`）——仲裁者 + 看歌詞全文與該標籤的引述（不含執行別），裁決保留者 + 附仲裁者自己的引述，剔除者不列。剔除集合＝送裁鍵減 + 輸出鍵，由程式推得。 +- **screen**：輸出即引述陣列，非空＝有、空＝無。僅 + 「一有一無」的歌送仲裁（`02-01-02-screen-arb.md`）， + 輸入為歌詞加主張「有」方的引述（不記名），輸出同為 + 引述陣列。 +- 仲裁者的引述可能與原引述不同：仲裁是對歌詞的重新 + 判讀，其引述是該裁決自身的依據，非轉抄。 + +## 軌跡對映（診斷用） + +沿收斂軌跡的機械對映：原始關鍵字 → merge 組 → cap 組， +純程式查表，決定性。以其結果與 code 直接編碼的差異率 +作為「收斂軌跡扭曲」的診斷量，不作主結果。 + +## 全管線的交接契約 + +每一步的輸出如何變成下一步的輸入，皆為確定性程序，規則 +明定如下（收斂步驟內部的交接見上節）： + +- **歌詞輸入檔（tag、code、screen 共用）**： + `export-llm-input` 自工作儲存產出，每筆 + `{"id": "song-", "content": <歌詞>}`，依歌曲 ID + 升序。三個讀歌詞的步驟共用同一檔，SHA-256 記入各步 + meta。 +- **tag → merge**：兩次執行的全部關鍵字取聯集、逐字串 + 精確去重、字典序排列成 JSON 陣列，即 merge 的輸入。 + 進池同時寫出處記錄（關鍵字 →（執行別，歌曲 ID） + 清單），供軌跡對映回到歌曲；出處記錄不進任何 LLM + 輸入。 +- **merge 定案 → cap**：merge 定案組名以字典序排成 + JSON 陣列，即 cap 的輸入（見上節第 5 點）。 +- **cap 定案 → code 定義檔**：定案組名以字典序逐行填入 + `01-04-01-code.md` 的詞彙表節（逐字），檔案隨 git + commit 後方可執行——code 的定義檔因此自我完備， + 論文附錄可直接引用。 +- **code 兩次執行 → code-arb**：逐首比對標籤集合 + （鍵集合，引述不參與比對）；僅有分歧的歌入仲裁輸入 + JSONL，依歌曲 ID 升序，每筆 `id` 沿用 `song-`、 + `content` 為固定鍵序序列化的 + `{"lyrics": …, "disagreements": …}`，disagreements + 鍵按字典序。 +- **code 定案**：每首歌的最終標籤＝共識標籤 ∪ 仲裁保留 + 標籤，寫入逐首紀錄檔（歌依 ID 升序、標籤按字典序， + 各標籤附其定案時的引述與來源層——共識或仲裁）。 +- **screen 兩次執行 → screen-arb**：僅「一有一無」的歌 + 入仲裁輸入 JSONL（依 ID 升序），`content` 為 + `{"lyrics": …, "evidence": <肯定方引述>}`。 +- **screen 定案**：命中集合＝兩次皆有 ∪ 仲裁裁定有。 +- **命名鏈的輸入構成**：`-name` 輸入的組 ID 依最終分組 + 之組序（上節第 5 點）編 g1、g2、…，組內成員字典序； + `-name-arb` 輸入中每組的兩個候選名**按字典序排列** + ——不按執行別，避免順序洩漏何方所取；`taken` 為已 + 定案名的字典序清單。 +- **女性力量候選集**：於 cap 詞彙表定案後、黃金標準 + 編碼開始前，由研究者指認詞彙表中屬「女性力量」概念 + 域的組（指認及理由記入決策日誌），候選集＝code 定案 + 標籤含該等組者 ∪ screen 命中者。 +- **序列化通則**：所有中間檔為 UTF-8 JSON，鍵序與元素 + 序皆依上列規則明定，無時間戳、無隨機成分；解析一律 + 偵測重複鍵，違規即失敗。JSONL 一律以換行字元（\n） + 切行——歌詞含 U+2028 等 Unicode 行分隔符， + `str.splitlines()` 類的通用切行會截斷 JSON 字串 + （實測踩中）。給定相同的 LLM 執行輸出， + 全部交接產物逐位元組可再生。 + +## 執行與稽核 + +- 每一步驟以 `run-llm <定義檔> <輸入檔> <歸檔目錄>` + 執行；獨立執行兩次＝重現命令清單上的兩行命令，各自 + 歸檔（`runs/<定義檔名>/run1`、`run2`），仲裁與命名 + 各為獨立步驟、獨立歸檔。 +- 比對、裁決套用、重組、對映皆為確定性程式（子命令）， + 其輸入輸出檔隨 runs/ 歸檔，JSON 解析一律偵測重複鍵， + 違規即失敗。 +- Batch API 的每筆請求自含全部脈絡且互不可見（平台 + 契約），歌與歌之間的獨立性由此成立；兩次執行的獨立 + 性由「兩次呼叫、兩個批次、兩份歸檔」的執行結構自明。 + +## 映射分析方法 + +（依 2026-07-30 決策，於看到結果前寫定；待黃金標準 +編碼展開前補入。） diff --git a/docs/project-structure.md b/docs/project-structure.md index 3e6b07e..e6c8c53 100644 --- a/docs/project-structure.md +++ b/docs/project-structure.md @@ -24,7 +24,8 @@ pop-fem-audit/ │ ├── songs.csv # 歌曲報表（人讀；進 git） │ └── artists.csv # 歌手報表（人讀；進 git） ├── prompts/ # LLM 定義檔（逐字作為 system prompt） -│ └── <軌>-<步>-.md # 01-01-tag.md、02-01-screen.md… +│ └── <軌>-<步>-<次步>-.md # 01-01-01-tag.md、 +│ # 01-02-02-merge-arb.md… │ # 不帶版本號，版本即 git 歷史 ├── tools/ # 輔助工具子專案（src-layout） │ ├── pyproject.toml # 發行名 pop-fem-audit-tools； diff --git a/docs/research-plan.md b/docs/research-plan.md index 1d18fe4..46a9605 100644 --- a/docs/research-plan.md +++ b/docs/research-plan.md @@ -95,11 +95,13 @@ themes 過早抽象）之繼承，於執行前鎖定，防止事後擇優。 2. **自然收斂（merge）**：輸入為池中純去重關鍵字清單—— 無歌詞、無頻次、無歌曲出處——模型按自身理解合併近似 - 概念。2+1：兩次收斂處理同一批輸入詞，程式先算出共識核 - （兩次都同組的詞塊，機械凍結）與分歧清單，仲裁者只裁 - 分歧清單，無權動共識核、無權引入新概念。 -3. **強制收斂（cap）**：同第 2 步形態，限制併至 50 個 - 以內。頻次不入收斂：頻率的分析角色由第 4 步編碼承擔； + 概念。2+1：兩次收斂處理同一批輸入詞，程式算出共識塊 + 與分歧塊對，逐對仲裁（仲裁自身亦 2+1，第三票終局）後 + 以連通元件決定性重組；收斂執行不命名，組名由命名步驟 + 2+1 定案（演算法詳見 `methodology.md`）。 +3. **強制收斂（cap）**：同第 2 步形態，輸入為 merge + 定案後的組名清單（原始關鍵字不下傳），限制併至 50 個 + 以內；仲裁後略超 50 可接受，如實記錄。頻次不入收斂：頻率的分析角色由第 4 步編碼承擔； 池中頻次含跨執行噪音；頻次會誘使模型以頻率剪枝（喪失 稀有主題）並把高頻大主題切細。代價（特異主題佔名額） 已知並接受，換取主題多樣性與純語意歸併的可辯護性。 @@ -120,15 +122,19 @@ 召回之用，永不進任何統計的分子分母。標籤詞是 screen 提示中唯一的語意種子，屬研究者的儀器選擇，據實揭露。 -定義檔命名 `prompts/<軌>-<步>-.md`——軌 01＝由下 -而上自然編碼、02＝預先決定的 women-power -篩選，步為軌內步驟序：01-01-tag.md、01-02-merge.md、 -01-03-cap.md、01-04-code.md、02-01-screen.md；仲裁定義檔 -同 prefix 加 `-arb`（如 01-02-merge-arb.md）。檔名不帶 +定義檔命名 `prompts/<軌>-<步>-<次步>-.md`——軌 +01＝由下而上自然編碼、02＝預先決定的 women-power +篩選；步為軌內步驟序，次步為 +步內執行順序，讀者依編號先後依循（如 01-02-01-merge.md → +01-02-02-merge-arb.md → 01-02-03-merge-arb-arb.md → +01-02-04-merge-name.md → 01-02-05-merge-name-arb.md）。 +後綴 `-arb` 為仲裁、`-arb-arb` 為仲裁之終局票、`-name` +為命名、`-name-arb` 為命名擇一仲裁。檔名不帶 版本號——版本即 git 歷史，失敗的版本不保留，需要回看的 舊版都在 git history；每次執行的定義檔快照隨 `runs/` -自我完備。四次收斂執行（merge ×2、cap ×2）各將合併記錄 -（哪些詞併入哪組）存成 JSON，隨該次執行入 `runs/`。 +自我完備。收斂執行的輸出本身即分組記錄（無名分組 +JSON），與全部中間交接檔同隨 `runs/` 歸檔（交接契約見 +`methodology.md`）。 ## 階段與時程（全文截稿 2026-08-15） diff --git a/prompts/01-01-01-tag.md b/prompts/01-01-01-tag.md new file mode 100644 index 0000000..17e97c0 --- /dev/null +++ b/prompts/01-01-01-tag.md @@ -0,0 +1,23 @@ +You are annotating song lyrics for a content-analysis corpus. + +Input: the complete lyrics of one song. + +Task: list the thematic keywords of this song — short labels +naming the themes the lyrics express, chosen by your own +reading of the lyrics. + +Rules: + +- Give between 5 and 10 thematic keywords. +- Each keyword is a short lowercase phrase with the words + joined by hyphens. +- For each keyword, quote 1 to 3 verbatim lines of the lyrics + that ground it; do not give a keyword you cannot ground. + +Output a single JSON object mapping each keyword to its list +of quotes, and nothing else: + +{ + "first-keyword": ["quoted line", "another quoted line"], + "second-keyword": ["quoted line"] +} diff --git a/prompts/01-02-01-merge.md b/prompts/01-02-01-merge.md new file mode 100644 index 0000000..ff16f75 --- /dev/null +++ b/prompts/01-02-01-merge.md @@ -0,0 +1,23 @@ +You are consolidating a vocabulary of thematic keywords that +were independently annotated on the songs of a corpus. + +Input: a JSON array of thematic keywords, deduplicated and +unordered. + +Task: merge keywords that express the same or nearly the same +theme, by your own understanding of what the keywords mean. + +Rules: + +- Every input keyword must appear in exactly one group. +- Do not invent keywords that are not in the input. +- Merge only what you consider the same theme; a group may be + a single keyword. + +Output a single JSON array of groups — each group an array of +its member keywords — and nothing else: + +[ + ["keyword", "another-keyword"], + ["keyword"] +] diff --git a/prompts/01-02-02-merge-arb.md b/prompts/01-02-02-merge-arb.md new file mode 100644 index 0000000..ede6519 --- /dev/null +++ b/prompts/01-02-02-merge-arb.md @@ -0,0 +1,36 @@ +You are arbitrating between two independent consolidations of +the same vocabulary of thematic keywords from a song corpus. + +Where the two consolidations agree, the result has been +settled by script and is not shown. You rule on the +disagreements: pairs of keyword blocks that one consolidation +merged into one theme and the other kept apart. + +Input: a JSON object. "blocks" maps a block id to the +keywords of that block; "pairs" lists the block-id pairs in +question: + +{ + "blocks": { + "b1": ["keyword", "another-keyword"], + "b2": ["keyword"], + "b3": ["keyword"] + }, + "pairs": [["b1", "b2"], ["b1", "b3"]] +} + +Task: for each listed pair, decide whether the two blocks +express the same theme, by your own understanding of what the +keywords mean. + +Rules: + +- Judge every listed pair, each on its own merits from the + block contents alone. +- Judge only the listed pairs. + +Output a single JSON array holding the pairs whose two blocks +express the same theme, written exactly as given in "pairs"; +an empty array when none do: + +[["b1", "b2"]] diff --git a/prompts/01-02-03-merge-arb-arb.md b/prompts/01-02-03-merge-arb-arb.md new file mode 100644 index 0000000..f9d425a --- /dev/null +++ b/prompts/01-02-03-merge-arb-arb.md @@ -0,0 +1,34 @@ +You are casting the deciding judgment on disagreements about +a consolidated vocabulary of thematic keywords from a song +corpus: for each pair of keyword blocks in question, earlier +independent judgments disagreed on whether the two blocks +express the same theme. + +Input: a JSON object. "blocks" maps a block id to the +keywords of that block; "pairs" lists the block-id pairs in +question: + +{ + "blocks": { + "b1": ["keyword", "another-keyword"], + "b2": ["keyword"], + "b3": ["keyword"] + }, + "pairs": [["b1", "b2"], ["b1", "b3"]] +} + +Task: for each listed pair, decide whether the two blocks +express the same theme, by your own understanding of what the +keywords mean. + +Rules: + +- Judge every listed pair, each on its own merits from the + block contents alone. +- Judge only the listed pairs. + +Output a single JSON array holding the pairs whose two blocks +express the same theme, written exactly as given in "pairs"; +an empty array when none do: + +[["b1", "b2"]] diff --git a/prompts/01-02-04-merge-name.md b/prompts/01-02-04-merge-name.md new file mode 100644 index 0000000..685d8b8 --- /dev/null +++ b/prompts/01-02-04-merge-name.md @@ -0,0 +1,28 @@ +You are naming the theme groups of a consolidated vocabulary +of thematic keywords from a song corpus. + +Input: a JSON object mapping an opaque group id to the member +keywords of that group: + +{ + "g1": ["keyword", "another-keyword"], + "g2": ["keyword"] +} + +Task: give each group a name that best names the theme its +members gather, by your own understanding of what the +keywords mean. + +Rules: + +- Each name is a short lowercase phrase with the words joined + by hyphens. +- Names must be unique across the groups. + +Output a single JSON object mapping each group id to its +name, and nothing else: + +{ + "g1": "group-name", + "g2": "another-group-name" +} diff --git a/prompts/01-02-05-merge-name-arb.md b/prompts/01-02-05-merge-name-arb.md new file mode 100644 index 0000000..89e511c --- /dev/null +++ b/prompts/01-02-05-merge-name-arb.md @@ -0,0 +1,37 @@ +You are arbitrating between two independent namings of the +theme groups of a consolidated vocabulary of thematic +keywords from a song corpus. + +For each group in question, the two namings proposed +different names. You choose between them. + +Input: a JSON object. "groups" maps an opaque group id to +its member keywords and its two candidate names; "taken" +lists names that are already in use: + +{ + "groups": { + "g1": { + "members": ["keyword", "another-keyword"], + "candidates": ["one-name", "other-name"] + } + }, + "taken": ["existing-name"] +} + +Task: for each group, choose the candidate that better names +the theme its members gather, by your own understanding of +what the keywords mean. + +Rules: + +- Choose only from that group's two candidates. +- Choices must be unique across the groups and must not + reuse any name in "taken". + +Output a single JSON object mapping each group id to the +chosen name, and nothing else: + +{ + "g1": "one-name" +} diff --git a/prompts/01-03-01-cap.md b/prompts/01-03-01-cap.md new file mode 100644 index 0000000..db0ede8 --- /dev/null +++ b/prompts/01-03-01-cap.md @@ -0,0 +1,26 @@ +You are consolidating a vocabulary of thematic keywords +from a song corpus further, to a fixed maximum number of +themes. + +Input: a JSON array of thematic keywords, deduplicated and +unordered. + +Task: merge keywords that express the same or a similar +theme, by your own understanding of what the keywords mean, +until at most 50 groups remain. + +Rules: + +- Every input keyword must appear in exactly one group. +- Do not invent keywords that are not in the input. +- Merge the keywords that are closest in meaning first; keep + clearly distinct themes apart as far as the limit allows. + A group may be a single keyword. + +Output a single JSON array of groups — each group an array of +its member keywords — and nothing else: + +[ + ["keyword", "another-keyword"], + ["keyword"] +] diff --git a/prompts/01-03-02-cap-arb.md b/prompts/01-03-02-cap-arb.md new file mode 100644 index 0000000..4e8fd91 --- /dev/null +++ b/prompts/01-03-02-cap-arb.md @@ -0,0 +1,37 @@ +You are arbitrating between two independent consolidations +that each reduced the same vocabulary of thematic keywords +from a song corpus to a fixed maximum number of themes. + +Where the two consolidations agree, the result has been +settled by script and is not shown. You rule on the +disagreements: pairs of keyword blocks that one consolidation +merged into one theme and the other kept apart. + +Input: a JSON object. "blocks" maps a block id to the +keywords of that block; "pairs" lists the block-id pairs in +question: + +{ + "blocks": { + "b1": ["keyword", "another-keyword"], + "b2": ["keyword"], + "b3": ["keyword"] + }, + "pairs": [["b1", "b2"], ["b1", "b3"]] +} + +Task: for each listed pair, decide whether the two blocks +express the same theme, by your own understanding of what the +keywords mean. + +Rules: + +- Judge every listed pair, each on its own merits from the + block contents alone. +- Judge only the listed pairs. + +Output a single JSON array holding the pairs whose two blocks +express the same theme, written exactly as given in "pairs"; +an empty array when none do: + +[["b1", "b2"]] diff --git a/prompts/01-03-03-cap-arb-arb.md b/prompts/01-03-03-cap-arb-arb.md new file mode 100644 index 0000000..49ce5a5 --- /dev/null +++ b/prompts/01-03-03-cap-arb-arb.md @@ -0,0 +1,35 @@ +You are casting the deciding judgment on disagreements about +a vocabulary of thematic keywords from a song corpus that was +consolidated to a fixed maximum number of themes: for each +pair of keyword blocks in question, earlier independent +judgments disagreed on whether the two blocks express the +same theme. + +Input: a JSON object. "blocks" maps a block id to the +keywords of that block; "pairs" lists the block-id pairs in +question: + +{ + "blocks": { + "b1": ["keyword", "another-keyword"], + "b2": ["keyword"], + "b3": ["keyword"] + }, + "pairs": [["b1", "b2"], ["b1", "b3"]] +} + +Task: for each listed pair, decide whether the two blocks +express the same theme, by your own understanding of what the +keywords mean. + +Rules: + +- Judge every listed pair, each on its own merits from the + block contents alone. +- Judge only the listed pairs. + +Output a single JSON array holding the pairs whose two blocks +express the same theme, written exactly as given in "pairs"; +an empty array when none do: + +[["b1", "b2"]] diff --git a/prompts/01-03-04-cap-name.md b/prompts/01-03-04-cap-name.md new file mode 100644 index 0000000..f94a481 --- /dev/null +++ b/prompts/01-03-04-cap-name.md @@ -0,0 +1,29 @@ +You are naming the theme groups of a vocabulary of thematic +keywords from a song corpus that was consolidated to a fixed +maximum number of themes. + +Input: a JSON object mapping an opaque group id to the member +keywords of that group: + +{ + "g1": ["keyword", "another-keyword"], + "g2": ["keyword"] +} + +Task: give each group a name that best names the theme its +members gather, by your own understanding of what the +keywords mean. + +Rules: + +- Each name is a short lowercase phrase with the words joined + by hyphens. +- Names must be unique across the groups. + +Output a single JSON object mapping each group id to its +name, and nothing else: + +{ + "g1": "group-name", + "g2": "another-group-name" +} diff --git a/prompts/01-03-05-cap-name-arb.md b/prompts/01-03-05-cap-name-arb.md new file mode 100644 index 0000000..defe1e6 --- /dev/null +++ b/prompts/01-03-05-cap-name-arb.md @@ -0,0 +1,38 @@ +You are arbitrating between two independent namings of the +theme groups of a vocabulary of thematic keywords from a song +corpus that was consolidated to a fixed maximum number of +themes. + +For each group in question, the two namings proposed +different names. You choose between them. + +Input: a JSON object. "groups" maps an opaque group id to +its member keywords and its two candidate names; "taken" +lists names that are already in use: + +{ + "groups": { + "g1": { + "members": ["keyword", "another-keyword"], + "candidates": ["one-name", "other-name"] + } + }, + "taken": ["existing-name"] +} + +Task: for each group, choose the candidate that better names +the theme its members gather, by your own understanding of +what the keywords mean. + +Rules: + +- Choose only from that group's two candidates. +- Choices must be unique across the groups and must not + reuse any name in "taken". + +Output a single JSON object mapping each group id to the +chosen name, and nothing else: + +{ + "g1": "one-name" +} diff --git a/prompts/01-04-01-code.md b/prompts/01-04-01-code.md new file mode 100644 index 0000000..3a161c5 --- /dev/null +++ b/prompts/01-04-01-code.md @@ -0,0 +1,31 @@ +You are coding song lyrics against a fixed vocabulary of +thematic keywords. + +Vocabulary — the complete set of codes; use these and no +others: + +(TO BE INSERTED VERBATIM WHEN THE VOCABULARY IS FINALIZED — +this file must not be used in a run before then.) + +Input: the complete lyrics of one song. + +Task: list every vocabulary keyword whose theme this song +expresses, by your own reading of the lyrics. + +Rules: + +- Use only keywords from the vocabulary, spelled exactly as + given. +- A song may match any number of keywords, including none. +- For each keyword you assign, quote 1 to 3 verbatim lines of + the lyrics that ground it; do not assign a keyword you + cannot ground. + +Output a single JSON object mapping each assigned keyword to +its list of quotes (an empty object when no keyword applies), +and nothing else: + +{ + "first-keyword": ["quoted line", "another quoted line"], + "second-keyword": ["quoted line"] +} diff --git a/prompts/01-04-02-code-arb.md b/prompts/01-04-02-code-arb.md new file mode 100644 index 0000000..92133cd --- /dev/null +++ b/prompts/01-04-02-code-arb.md @@ -0,0 +1,39 @@ +You are arbitrating between two independent codings of the +same song against the same fixed vocabulary of thematic +keywords. + +The keywords the two codings agree on have been settled by +script and are not shown. You rule only on the +disagreements: the keywords assigned by one coding but not +the other. + +Input: a JSON object with the lyrics of the song and the +disagreements — each disagreement maps a keyword to the lyric +quotes the assigning coding gave as evidence: + +{ + "lyrics": "...", + "disagreements": { + "first-keyword": ["quoted line", "another quoted line"], + "second-keyword": ["quoted line"] + } +} + +Task: for each disagreement keyword, decide whether the song +expresses that theme, by your own reading of the lyrics and +of the quoted evidence. + +Rules: + +- Rule on every listed keyword: keep it or drop it. +- Keep a keyword only when you can ground it in verbatim + lines of the lyrics; quote them. +- Do not add any keyword that is not listed. + +Output a single JSON object mapping each kept keyword to your +list of quotes, and nothing else. A dropped keyword is simply +left out; output an empty object when nothing is kept: + +{ + "kept-keyword": ["quoted line", "another quoted line"] +} diff --git a/prompts/02-01-01-screen.md b/prompts/02-01-01-screen.md new file mode 100644 index 0000000..1938733 --- /dev/null +++ b/prompts/02-01-01-screen.md @@ -0,0 +1,19 @@ +You are screening song lyrics for one specific theme. + +Input: the complete lyrics of one song. + +Task: judge whether this song expresses the theme +"women-power", by your own understanding of +that label. + +Rules: + +- If the song expresses the theme, quote 1 to 3 verbatim + lines of the lyrics that ground the judgment; if you cannot + ground it in a quote, the answer is no. + +Output a single JSON array and nothing else: the quoted +lines when the song expresses the theme, or an empty array +when it does not: + +["quoted line", "another quoted line"] diff --git a/prompts/02-01-02-screen-arb.md b/prompts/02-01-02-screen-arb.md new file mode 100644 index 0000000..0bab97a --- /dev/null +++ b/prompts/02-01-02-screen-arb.md @@ -0,0 +1,28 @@ +You are arbitrating a screening of song lyrics for one +specific theme: two independent screenings of the same song +disagreed on whether it expresses the theme +"women-power". + +Input: a JSON object with the lyrics of the song and the +lyric quotes that the affirming screening gave as evidence: + +{ + "lyrics": "...", + "evidence": ["quoted line", "another quoted line"] +} + +Task: decide whether the song expresses the theme, by your +own understanding of the label, reading the lyrics and the +quoted evidence. + +Rules: + +- If the song expresses the theme, quote 1 to 3 verbatim + lines of the lyrics that ground the judgment; if you cannot + ground it in a quote, the answer is no. + +Output a single JSON array and nothing else: the quoted +lines when the song expresses the theme, or an empty array +when it does not: + +["quoted line", "another quoted line"]