Add the natural-coding definition files and the convergence algorithms

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-17 22:38:26 +08:00
co-authored by Claude Fable 5
parent 0c82da7a1d
commit 964006ec6d
20 changed files with 729 additions and 16 deletions
+4 -3
View File
@@ -6,9 +6,10 @@
Messages API: model `claude-sonnet-4-6`, `temperature=0`, Messages API: model `claude-sonnet-4-6`, `temperature=0`,
thinking disabled, Batch API where possible. thinking disabled, Batch API where possible.
- Prompt definition files live in - Prompt definition files live in
`prompts/<track>-<step>-<task>.md` (e.g. 01-01-tag.md; no `prompts/<track>-<step>-<substep>-<task>.md` (e.g.
version suffix -- versions live in git history) and are 01-01-01-tag.md; the substep number fixes the execution
passed verbatim as the system prompt. order; no version suffix -- versions live in git history)
and are passed verbatim as the system prompt.
- LLM steps whose outputs are item-by-item comparable - LLM steps whose outputs are item-by-item comparable
(convergence, coding) run the same definition file twice, (convergence, coding) run the same definition file twice,
then a separate arbitration step settles only the then a separate arbitration step settles only the
+45
View File
@@ -323,6 +323,51 @@
式化、消滅隱性推導」原則的貫徹。比對子命令驗證兩 run 的定義檔與 式化、消滅隱性推導」原則的貫徹。比對子命令驗證兩 run 的定義檔與
輸入 SHA 一致、缺 run 即失敗。目標目錄已存在即拒絕執行,重跑須明 輸入 SHA 一致、缺 run 即失敗。目標目錄已存在即拒絕執行,重跑須明
`--replace` `--replace`
- **收斂仲裁演算法定案(逐對裁決)**:不做組對組匹配
(兩組是否「同一組」無原則性答案),比較化約為詞對的
共組關係——共識塊(交集細分)→ 分歧塊對枚舉 → 仲裁
逐對二元裁決(輸入匿名,不含執行別、共識與歌詞)→
union-find 遞移重組(固定邊集的連通元件唯一,與順序
無關,決定性成立)。仲裁自身亦 2+1:獨立執行兩次,
逐對比對,兩次相同即定案,相異者送 `-arb-arb` 終局票
(單次,依協定終局)——每個分歧對等同三票多數決,
仲裁鏈至此終止。邊界判斷之裁決為單次記錄性決定,不
宣稱可再生(可重現性依計畫定義=程序透明+可稽核)。
遞移導出的合併(共識「分開」的塊經第三塊橋接併組)為
等價關係語意的邏輯結論,照單接受並逐筆記錄供稽核。
一致率低於門檻(暫訂 50%,首輪實跑後校準)即不仲裁
——視為定義檔約束不足,修訂後重跑該循環。cap 仲裁後
組數略超 50 可接受,如實記錄。演算法明文於
`methodology.md`,供讀者依循重現。
- **收斂執行不命名,組名全權歸命名鏈(命名 2+1)**:
merge、cap 的執行輸出改為無名分組(陣列之陣列)——
既然命名鏈存在,收斂執行自取的名字毫無用處,徒增
兩次執行間「同組不同名」的假分歧。最終分組定案後,
`-name` 命名步驟獨立執行兩次對全部組自由命名——逐組
比對,同名即定案(單次執行內名字唯一,故定名間必無
撞名);異名送 `-name-arb` 擇一仲裁(匿名呈現兩候選,
輸出限於候選並避讓已定名,機械可驗證)。理由:組名
承重——merge 名為 cap 的輸入詞、cap 名為 code 的編碼
類目——凡承重的語意判斷皆受 2+1。cap 的輸入即 merge
定案後的組名清單,原始關鍵字不下傳——如同證據不進
收斂,上游細節不污染下游步驟。
- **定義檔輸出形狀統一與去署名**:LLM 輸出統一為單層
dictarray——tag、code:關鍵字→引述;merge、cap
無名分組(陣列之陣列);code-arb:保留關鍵字→仲裁者自己的引述
(剔除即不列,剔除集合由程式以鍵差推得);screen 系:
引述陣列,非空即「有」(present 布林刪除,判斷與依據
合一)。仲裁輸入一律不含執行別(assigned_by、
run1/run2 刪除)——標示何方主張會誘使仲裁者揣測
「哪次較可信」,而非就文本裁決。收斂組名採自由命名
(不限取自成員詞),取命名貼合度;可追溯性由軌跡歸檔
承擔。
- **定義檔改三層編號**
`prompts/<軌>-<步>-<次步>-<task>.md`——次步為步內執行
順序,明定讀者依循的先後(如 01-02-01-merge →
01-02-02-merge-arb → 01-02-03-merge-arb-arb →
01-02-04-merge-name → 01-02-05-merge-name-arb)。後綴
`-arb` 仲裁、`-arb-arb` 終局票、`-name` 命名、
`-name-arb` 命名擇一仲裁。
- **screen 標籤詞改為 `women-power`(來歷考據定案)** - **screen 標籤詞改為 `women-power`(來歷考據定案)**
考據先導研究的 local agent 存檔:其第一步指令含數十個 考據先導研究的 local agent 存檔:其第一步指令含數十個
範例 thematic keywords,其中即有 women-power——為當時 範例 thematic keywords,其中即有 women-power——為當時
+197
View File
@@ -0,0 +1,197 @@
# 方法細節
(全文方法節底稿。演算法在執行前寫定;任何修訂記入
`decision-log.md`。定義檔全文見 `prompts/`,執行紀錄見
`runs/`。)
## 自然編碼管線總覽
四步驟:自由標註(tag)→ 自然收斂(merge)→ 強制收斂
(cap)→ 全量編碼(code),另設「女性力量」單目標篩選
(screen)作黃金標準取樣的補漏網。歌詞只出現在 tag、code、
screen 與各仲裁步驟;merge、cap 及其仲裁、命名皆不接觸
歌詞。設計原則見 `research-plan.md`;本檔記載可重現的
演算法細節。
## 收斂步驟(merge、cap)的 2+1 比對與仲裁
兩次獨立執行對同一批輸入詞各產生一個分組。比對不做
「組對組」的匹配——兩個組是否為「同一組的變體」無原則性
答案——而是把比較化約為「詞對的共組關係」:
1. **共識塊(交集細分)**:兩次執行都放在同組的詞歸為
同一塊。即以「(第一次的組, 第二次的組)」二元組為鍵
分桶,一桶一塊。此步為純集合運算。
2. **分歧塊對枚舉**:每個塊完整落在各次執行的恰一組內,
故「兩塊在某次執行中是否同組」定義良好。逐塊對檢查:
兩次執行答案相同者為共識(同組或分開,直接定案);
不同者列入分歧清單。
3. **逐對仲裁(自身 2+1**:分歧塊對送 LLM 仲裁
`01-02-02-merge-arb.md``01-03-02-cap-arb.md`)。
仲裁者只看兩塊的內容詞,逐對二元裁決「是否同一
主題」;輸入不含執行別(避免「猜哪一次較可信」的
偏誤),不含共識部份,不含歌詞。仲裁自身獨立執行
兩次,逐對比對:兩次裁決相同即定案;相異的塊對送
終局票(`01-02-03-merge-arb-arb.md`
`01-03-03-cap-arb-arb.md`,單次執行,依協定為終局)
——每個分歧塊對等同三票多數決,仲裁鏈至此終止。
送入仲裁的塊對即模型自身不穩定的邊界判斷,其裁決為
單次記錄性決定,不宣稱可再生;可重現性依計畫定義為
「程序透明+可稽核」,裁決與其輸入全程歸檔。
4. **確定性重組**:以每個「同組」裁決為一條邊,最終
分組=圖的連通元件(union-find)。固定邊集的連通
元件唯一,與處理順序無關,故重組決定性成立。
遞移性後果照單全收:兩個共識「分開」的塊可能經第三
塊橋接而併入同組——此為等價關係語意的邏輯結論,
比對程式將此類「遞移導出的合併」逐筆記錄於歸檔,
供稽核。仲裁的結果空間因此大於「兩次執行擇一」:
可能比兩次都粗(多對皆裁同組),也可能比兩次都細
(多對皆裁分開)。
5. **一致率與保險絲**:比對程式計算塊對層級的一致率並
記入歸檔。低於門檻(暫訂 50%,首輪實跑後校準)即
不進行仲裁——兩次分組面目全非說明定義檔約束不足,
依協定修訂定義檔並重跑該循環。
6. **cap 的上限容忍**:仲裁後組數可能略超 50(多對裁
「分開」時)。略微超過可接受,如實記錄,不強行
壓縮。
7. **組名定案(收斂不命名+命名 2+1)**merge、cap 的
執行輸出不含組名(無名分組,陣列之陣列)——命名鏈
既然存在,收斂執行自取的名字只會製造兩次執行間
「同組不同名」的假分歧。組名承重——merge 的組名是
cap 的輸入詞,cap 的組名是 code 對歌詞編碼的類目——
故與其他語意判斷同受 2+1。程序:(1) 最終分組定案後,
命名步驟(`01-02-04-merge-name.md`
`01-03-04-cap-name.md`)獨立執行兩次,輸入為不透明
組 ID 對成員詞,對全部組自由命名,僅受格式約束
(小寫連字號、單次執行內不重複);(2) 逐組比對:兩次
同名即定案(單次執行內名字唯一,故定名間必無撞名);
(3) 異名的組送擇一仲裁(`01-02-05-merge-name-arb.md`
`01-03-05-cap-name-arb.md`)——匿名呈現兩候選,逐組
擇一,輸出限於候選並避讓已定名,機械可驗證。cap 的
輸入即 merge 定案後的組名清單。
### 收斂步驟的資料流(比對子命令的輸入輸出契約)
以 merge 為例(cap 完全同構,檔名換為 01-03 系):
1. 兩次執行的原始輸出:`runs/01-02-01-merge/run1/output.jsonl`
`run2/output.jsonl`,各含一個無名分組(陣列之陣列)。
2. 比對子命令讀入兩份分組,先驗證兩者為同一輸入詞集的
完整分割(缺詞、多詞、重複即失敗),再計算共識塊與
分歧塊對,產出仲裁輸入檔——即
`01-02-02-merge-arb.md` 所收的 JSON
- `blocks`:塊 ID → 成員詞。**塊 ID 的指派決定性**:
全部塊先按「各塊字典序最小的成員詞」排序,依序編為
b1、b2、…;塊內成員詞亦按字典序排列。
- `pairs`:分歧塊對清單,每對內部按塊 ID 序、清單
整體按 (第一元素, 第二元素) 字典序排列。
- 僅分歧塊對入列;共識(同組或分開)不送仲裁,由
比對子命令直接寫入共識紀錄檔。
同時產出:共識紀錄(共識塊、共識同組對、共識分開對)
與塊對一致率(含保險絲判定)。
3. 仲裁輸入檔以 run-llm 跑兩次
`runs/01-02-02-merge-arb/run1``run2`);比對
子命令逐對比對兩份裁決,兩票相同即定案,相異的塊對
依同一契約組成終局票輸入檔(`blocks` 僅含涉事塊、
ID 沿用原編號),跑
`runs/01-02-03-merge-arb-arb/run1`
4. 裁決套用子命令彙整三票結果,以 union-find 重組出
最終分組,並寫出:最終分組檔(無名,陣列之陣列,
組序與組內成員皆字典序)、遞移導出合併的紀錄、
收斂軌跡(原始詞 → 最終組)。
5. 最終分組轉為不透明組 ID(依組序編 g1、g2、…)進入
命名鏈(見第 7 條);命名定案後,組名清單(字典序)
即下一步的輸入。
6. 以上中間檔全部隨 `runs/` 歸檔;一切排序規則固定,
故給定相同的兩份執行輸出與相同的裁決,全流程輸出
逐位元組可再生。
## 編碼步驟(code、screen)的 2+1 比對與仲裁
- **code**:逐首比對兩次執行的標籤集合(引述不參與
比對)。兩次皆有的標籤為共識保留、兩次皆無為共識
不標;單邊標籤送仲裁(`01-04-02-code-arb.md`)——仲裁者
看歌詞全文與該標籤的引述(不含執行別),裁決保留者
附仲裁者自己的引述,剔除者不列。剔除集合=送裁鍵減
輸出鍵,由程式推得。
- **screen**:輸出即引述陣列,非空=有、空=無。僅
「一有一無」的歌送仲裁(`02-01-02-screen-arb.md`),
輸入為歌詞加主張「有」方的引述(不記名),輸出同為
引述陣列。
- 仲裁者的引述可能與原引述不同:仲裁是對歌詞的重新
判讀,其引述是該裁決自身的依據,非轉抄。
## 軌跡對映(診斷用)
沿收斂軌跡的機械對映:原始關鍵字 → merge 組 → cap 組,
純程式查表,決定性。以其結果與 code 直接編碼的差異率
作為「收斂軌跡扭曲」的診斷量,不作主結果。
## 全管線的交接契約
每一步的輸出如何變成下一步的輸入,皆為確定性程序,規則
明定如下(收斂步驟內部的交接見上節):
- **歌詞輸入檔(tag、code、screen 共用)**
`export-llm-input` 自工作儲存產出,每筆
`{"id": "song-<ID>", "content": <歌詞>}`,依歌曲 ID
升序。三個讀歌詞的步驟共用同一檔,SHA-256 記入各步
meta。
- **tag → merge**:兩次執行的全部關鍵字取聯集、逐字串
精確去重、字典序排列成 JSON 陣列,即 merge 的輸入。
進池同時寫出處記錄(關鍵字 →(執行別,歌曲 ID)
清單),供軌跡對映回到歌曲;出處記錄不進任何 LLM
輸入。
- **merge 定案 → cap**merge 定案組名以字典序排成
JSON 陣列,即 cap 的輸入(見上節第 5 點)。
- **cap 定案 → code 定義檔**:定案組名以字典序逐行填入
`01-04-01-code.md` 的詞彙表節(逐字),檔案隨 git
commit 後方可執行——code 的定義檔因此自我完備,
論文附錄可直接引用。
- **code 兩次執行 → code-arb**:逐首比對標籤集合
(鍵集合,引述不參與比對);僅有分歧的歌入仲裁輸入
JSONL,依歌曲 ID 升序,每筆 `id` 沿用 `song-<ID>`
`content` 為固定鍵序序列化的
`{"lyrics": …, "disagreements": …}`disagreements
鍵按字典序。
- **code 定案**:每首歌的最終標籤=共識標籤 ∪ 仲裁保留
標籤,寫入逐首紀錄檔(歌依 ID 升序、標籤按字典序,
各標籤附其定案時的引述與來源層——共識或仲裁)。
- **screen 兩次執行 → screen-arb**:僅「一有一無」的歌
入仲裁輸入 JSONL(依 ID 升序),`content`
`{"lyrics": …, "evidence": <肯定方引述>}`
- **screen 定案**:命中集合=兩次皆有 ∪ 仲裁裁定有。
- **命名鏈的輸入構成**`-name` 輸入的組 ID 依最終分組
之組序(上節第 5 點)編 g1、g2、…,組內成員字典序;
`-name-arb` 輸入中每組的兩個候選名**按字典序排列**
——不按執行別,避免順序洩漏何方所取;`taken` 為已
定案名的字典序清單。
- **女性力量候選集**:於 cap 詞彙表定案後、黃金標準
編碼開始前,由研究者指認詞彙表中屬「女性力量」概念
域的組(指認及理由記入決策日誌),候選集=code 定案
標籤含該等組者 ∪ screen 命中者。
- **序列化通則**:所有中間檔為 UTF-8 JSON,鍵序與元素
序皆依上列規則明定,無時間戳、無隨機成分;解析一律
偵測重複鍵,違規即失敗。JSONL 一律以換行字元(\n)
切行——歌詞含 U+2028 等 Unicode 行分隔符,
`str.splitlines()` 類的通用切行會截斷 JSON 字串
(實測踩中)。給定相同的 LLM 執行輸出,
全部交接產物逐位元組可再生。
## 執行與稽核
- 每一步驟以 `run-llm <定義檔> <輸入檔> <歸檔目錄>`
執行;獨立執行兩次=重現命令清單上的兩行命令,各自
歸檔(`runs/<定義檔名>/run1``run2`),仲裁與命名
各為獨立步驟、獨立歸檔。
- 比對、裁決套用、重組、對映皆為確定性程式(子命令),
其輸入輸出檔隨 runs/ 歸檔,JSON 解析一律偵測重複鍵,
違規即失敗。
- Batch API 的每筆請求自含全部脈絡且互不可見(平台
契約),歌與歌之間的獨立性由此成立;兩次執行的獨立
性由「兩次呼叫、兩個批次、兩份歸檔」的執行結構自明。
## 映射分析方法
(依 2026-07-30 決策,於看到結果前寫定;待黃金標準
編碼展開前補入。)
+2 -1
View File
@@ -24,7 +24,8 @@ pop-fem-audit/
│ ├── songs.csv # 歌曲報表(人讀;進 git) │ ├── songs.csv # 歌曲報表(人讀;進 git)
│ └── artists.csv # 歌手報表(人讀;進 git) │ └── artists.csv # 歌手報表(人讀;進 git)
├── prompts/ # LLM 定義檔(逐字作為 system prompt ├── prompts/ # LLM 定義檔(逐字作為 system prompt
│ └── <軌>-<步>-<task>.md # 01-01-tag.md、02-01-screen.md… │ └── <軌>-<步>-<次步>-<task>.md # 01-01-01-tag.md、
│ # 01-02-02-merge-arb.md…
│ # 不帶版本號,版本即 git 歷史 │ # 不帶版本號,版本即 git 歷史
├── tools/ # 輔助工具子專案(src-layout ├── tools/ # 輔助工具子專案(src-layout
│ ├── pyproject.toml # 發行名 pop-fem-audit-tools │ ├── pyproject.toml # 發行名 pop-fem-audit-tools
+18 -12
View File
@@ -95,11 +95,13 @@
themes 過早抽象)之繼承,於執行前鎖定,防止事後擇優。 themes 過早抽象)之繼承,於執行前鎖定,防止事後擇優。
2. **自然收斂(merge**:輸入為池中純去重關鍵字清單—— 2. **自然收斂(merge**:輸入為池中純去重關鍵字清單——
無歌詞、無頻次、無歌曲出處——模型按自身理解合併近似 無歌詞、無頻次、無歌曲出處——模型按自身理解合併近似
概念。2+1:兩次收斂處理同一批輸入詞,程式算出共識 概念。2+1:兩次收斂處理同一批輸入詞,程式算出共識
(兩次都同組的詞塊,機械凍結)與分歧清單,仲裁者只裁 與分歧塊對,逐對仲裁(仲裁自身亦 2+1,第三票終局)後
分歧清單,無權動共識核、無權引入新概念。 以連通元件決定性重組;收斂執行不命名,組名由命名步驟
3. **強制收斂(cap**:同第 2 步形態,限制併至 50 個 2+1 定案(演算法詳見 `methodology.md`)。
以內。頻次不入收斂:頻率的分析角色由4編碼承擔; 3. **強制收斂(cap**:同2形態,輸入為 merge
定案後的組名清單(原始關鍵字不下傳),限制併至 50 個
以內;仲裁後略超 50 可接受,如實記錄。頻次不入收斂:頻率的分析角色由第 4 步編碼承擔;
池中頻次含跨執行噪音;頻次會誘使模型以頻率剪枝(喪失 池中頻次含跨執行噪音;頻次會誘使模型以頻率剪枝(喪失
稀有主題)並把高頻大主題切細。代價(特異主題佔名額) 稀有主題)並把高頻大主題切細。代價(特異主題佔名額)
已知並接受,換取主題多樣性與純語意歸併的可辯護性。 已知並接受,換取主題多樣性與純語意歸併的可辯護性。
@@ -120,15 +122,19 @@
召回之用,永不進任何統計的分子分母。標籤詞是 screen 召回之用,永不進任何統計的分子分母。標籤詞是 screen
提示中唯一的語意種子,屬研究者的儀器選擇,據實揭露。 提示中唯一的語意種子,屬研究者的儀器選擇,據實揭露。
定義檔命名 `prompts/<軌>-<步>-<task>.md`——軌 01=由下 定義檔命名 `prompts/<軌>-<步>-<次步>-<task>.md`——軌
而上自然編碼、02=預先決定的 women-power 01=由下而上自然編碼、02=預先決定的 women-power
篩選步為軌內步驟序01-01-tag.md、01-02-merge.md、 篩選步為軌內步驟序,次步為
01-03-cap.md、01-04-code.md、02-01-screen.md;仲裁定義檔 步內執行順序,讀者依編號先後依循(如 01-02-01-merge.md →
同 prefix 加 `-arb`(如 01-02-merge-arb.md)。檔名不帶 01-02-02-merge-arb.md → 01-02-03-merge-arb-arb.md →
01-02-04-merge-name.md → 01-02-05-merge-name-arb.md)。
後綴 `-arb` 為仲裁、`-arb-arb` 為仲裁之終局票、`-name`
為命名、`-name-arb` 為命名擇一仲裁。檔名不帶
版本號——版本即 git 歷史,失敗的版本不保留,需要回看的 版本號——版本即 git 歷史,失敗的版本不保留,需要回看的
舊版都在 git history;每次執行的定義檔快照隨 `runs/` 舊版都在 git history;每次執行的定義檔快照隨 `runs/`
自我完備。四次收斂執行merge ×2、cap ×2)各將合併記錄 自我完備。收斂執行的輸出本身即分組記錄(無名分組
(哪些詞併入哪組)存成 JSON,隨該次執行入 `runs/` JSON),與全部中間交接檔同隨 `runs/` 歸檔(交接契約見
`methodology.md`)。
## 階段與時程(全文截稿 2026-08-15 ## 階段與時程(全文截稿 2026-08-15
+23
View File
@@ -0,0 +1,23 @@
You are annotating song lyrics for a content-analysis corpus.
Input: the complete lyrics of one song.
Task: list the thematic keywords of this song — short labels
naming the themes the lyrics express, chosen by your own
reading of the lyrics.
Rules:
- Give between 5 and 10 thematic keywords.
- Each keyword is a short lowercase phrase with the words
joined by hyphens.
- For each keyword, quote 1 to 3 verbatim lines of the lyrics
that ground it; do not give a keyword you cannot ground.
Output a single JSON object mapping each keyword to its list
of quotes, and nothing else:
{
"first-keyword": ["quoted line", "another quoted line"],
"second-keyword": ["quoted line"]
}
+23
View File
@@ -0,0 +1,23 @@
You are consolidating a vocabulary of thematic keywords that
were independently annotated on the songs of a corpus.
Input: a JSON array of thematic keywords, deduplicated and
unordered.
Task: merge keywords that express the same or nearly the same
theme, by your own understanding of what the keywords mean.
Rules:
- Every input keyword must appear in exactly one group.
- Do not invent keywords that are not in the input.
- Merge only what you consider the same theme; a group may be
a single keyword.
Output a single JSON array of groups — each group an array of
its member keywords — and nothing else:
[
["keyword", "another-keyword"],
["keyword"]
]
+36
View File
@@ -0,0 +1,36 @@
You are arbitrating between two independent consolidations of
the same vocabulary of thematic keywords from a song corpus.
Where the two consolidations agree, the result has been
settled by script and is not shown. You rule on the
disagreements: pairs of keyword blocks that one consolidation
merged into one theme and the other kept apart.
Input: a JSON object. "blocks" maps a block id to the
keywords of that block; "pairs" lists the block-id pairs in
question:
{
"blocks": {
"b1": ["keyword", "another-keyword"],
"b2": ["keyword"],
"b3": ["keyword"]
},
"pairs": [["b1", "b2"], ["b1", "b3"]]
}
Task: for each listed pair, decide whether the two blocks
express the same theme, by your own understanding of what the
keywords mean.
Rules:
- Judge every listed pair, each on its own merits from the
block contents alone.
- Judge only the listed pairs.
Output a single JSON array holding the pairs whose two blocks
express the same theme, written exactly as given in "pairs";
an empty array when none do:
[["b1", "b2"]]
+34
View File
@@ -0,0 +1,34 @@
You are casting the deciding judgment on disagreements about
a consolidated vocabulary of thematic keywords from a song
corpus: for each pair of keyword blocks in question, earlier
independent judgments disagreed on whether the two blocks
express the same theme.
Input: a JSON object. "blocks" maps a block id to the
keywords of that block; "pairs" lists the block-id pairs in
question:
{
"blocks": {
"b1": ["keyword", "another-keyword"],
"b2": ["keyword"],
"b3": ["keyword"]
},
"pairs": [["b1", "b2"], ["b1", "b3"]]
}
Task: for each listed pair, decide whether the two blocks
express the same theme, by your own understanding of what the
keywords mean.
Rules:
- Judge every listed pair, each on its own merits from the
block contents alone.
- Judge only the listed pairs.
Output a single JSON array holding the pairs whose two blocks
express the same theme, written exactly as given in "pairs";
an empty array when none do:
[["b1", "b2"]]
+28
View File
@@ -0,0 +1,28 @@
You are naming the theme groups of a consolidated vocabulary
of thematic keywords from a song corpus.
Input: a JSON object mapping an opaque group id to the member
keywords of that group:
{
"g1": ["keyword", "another-keyword"],
"g2": ["keyword"]
}
Task: give each group a name that best names the theme its
members gather, by your own understanding of what the
keywords mean.
Rules:
- Each name is a short lowercase phrase with the words joined
by hyphens.
- Names must be unique across the groups.
Output a single JSON object mapping each group id to its
name, and nothing else:
{
"g1": "group-name",
"g2": "another-group-name"
}
+37
View File
@@ -0,0 +1,37 @@
You are arbitrating between two independent namings of the
theme groups of a consolidated vocabulary of thematic
keywords from a song corpus.
For each group in question, the two namings proposed
different names. You choose between them.
Input: a JSON object. "groups" maps an opaque group id to
its member keywords and its two candidate names; "taken"
lists names that are already in use:
{
"groups": {
"g1": {
"members": ["keyword", "another-keyword"],
"candidates": ["one-name", "other-name"]
}
},
"taken": ["existing-name"]
}
Task: for each group, choose the candidate that better names
the theme its members gather, by your own understanding of
what the keywords mean.
Rules:
- Choose only from that group's two candidates.
- Choices must be unique across the groups and must not
reuse any name in "taken".
Output a single JSON object mapping each group id to the
chosen name, and nothing else:
{
"g1": "one-name"
}
+26
View File
@@ -0,0 +1,26 @@
You are consolidating a vocabulary of thematic keywords
from a song corpus further, to a fixed maximum number of
themes.
Input: a JSON array of thematic keywords, deduplicated and
unordered.
Task: merge keywords that express the same or a similar
theme, by your own understanding of what the keywords mean,
until at most 50 groups remain.
Rules:
- Every input keyword must appear in exactly one group.
- Do not invent keywords that are not in the input.
- Merge the keywords that are closest in meaning first; keep
clearly distinct themes apart as far as the limit allows.
A group may be a single keyword.
Output a single JSON array of groups — each group an array of
its member keywords — and nothing else:
[
["keyword", "another-keyword"],
["keyword"]
]
+37
View File
@@ -0,0 +1,37 @@
You are arbitrating between two independent consolidations
that each reduced the same vocabulary of thematic keywords
from a song corpus to a fixed maximum number of themes.
Where the two consolidations agree, the result has been
settled by script and is not shown. You rule on the
disagreements: pairs of keyword blocks that one consolidation
merged into one theme and the other kept apart.
Input: a JSON object. "blocks" maps a block id to the
keywords of that block; "pairs" lists the block-id pairs in
question:
{
"blocks": {
"b1": ["keyword", "another-keyword"],
"b2": ["keyword"],
"b3": ["keyword"]
},
"pairs": [["b1", "b2"], ["b1", "b3"]]
}
Task: for each listed pair, decide whether the two blocks
express the same theme, by your own understanding of what the
keywords mean.
Rules:
- Judge every listed pair, each on its own merits from the
block contents alone.
- Judge only the listed pairs.
Output a single JSON array holding the pairs whose two blocks
express the same theme, written exactly as given in "pairs";
an empty array when none do:
[["b1", "b2"]]
+35
View File
@@ -0,0 +1,35 @@
You are casting the deciding judgment on disagreements about
a vocabulary of thematic keywords from a song corpus that was
consolidated to a fixed maximum number of themes: for each
pair of keyword blocks in question, earlier independent
judgments disagreed on whether the two blocks express the
same theme.
Input: a JSON object. "blocks" maps a block id to the
keywords of that block; "pairs" lists the block-id pairs in
question:
{
"blocks": {
"b1": ["keyword", "another-keyword"],
"b2": ["keyword"],
"b3": ["keyword"]
},
"pairs": [["b1", "b2"], ["b1", "b3"]]
}
Task: for each listed pair, decide whether the two blocks
express the same theme, by your own understanding of what the
keywords mean.
Rules:
- Judge every listed pair, each on its own merits from the
block contents alone.
- Judge only the listed pairs.
Output a single JSON array holding the pairs whose two blocks
express the same theme, written exactly as given in "pairs";
an empty array when none do:
[["b1", "b2"]]
+29
View File
@@ -0,0 +1,29 @@
You are naming the theme groups of a vocabulary of thematic
keywords from a song corpus that was consolidated to a fixed
maximum number of themes.
Input: a JSON object mapping an opaque group id to the member
keywords of that group:
{
"g1": ["keyword", "another-keyword"],
"g2": ["keyword"]
}
Task: give each group a name that best names the theme its
members gather, by your own understanding of what the
keywords mean.
Rules:
- Each name is a short lowercase phrase with the words joined
by hyphens.
- Names must be unique across the groups.
Output a single JSON object mapping each group id to its
name, and nothing else:
{
"g1": "group-name",
"g2": "another-group-name"
}
+38
View File
@@ -0,0 +1,38 @@
You are arbitrating between two independent namings of the
theme groups of a vocabulary of thematic keywords from a song
corpus that was consolidated to a fixed maximum number of
themes.
For each group in question, the two namings proposed
different names. You choose between them.
Input: a JSON object. "groups" maps an opaque group id to
its member keywords and its two candidate names; "taken"
lists names that are already in use:
{
"groups": {
"g1": {
"members": ["keyword", "another-keyword"],
"candidates": ["one-name", "other-name"]
}
},
"taken": ["existing-name"]
}
Task: for each group, choose the candidate that better names
the theme its members gather, by your own understanding of
what the keywords mean.
Rules:
- Choose only from that group's two candidates.
- Choices must be unique across the groups and must not
reuse any name in "taken".
Output a single JSON object mapping each group id to the
chosen name, and nothing else:
{
"g1": "one-name"
}
+31
View File
@@ -0,0 +1,31 @@
You are coding song lyrics against a fixed vocabulary of
thematic keywords.
Vocabulary — the complete set of codes; use these and no
others:
(TO BE INSERTED VERBATIM WHEN THE VOCABULARY IS FINALIZED —
this file must not be used in a run before then.)
Input: the complete lyrics of one song.
Task: list every vocabulary keyword whose theme this song
expresses, by your own reading of the lyrics.
Rules:
- Use only keywords from the vocabulary, spelled exactly as
given.
- A song may match any number of keywords, including none.
- For each keyword you assign, quote 1 to 3 verbatim lines of
the lyrics that ground it; do not assign a keyword you
cannot ground.
Output a single JSON object mapping each assigned keyword to
its list of quotes (an empty object when no keyword applies),
and nothing else:
{
"first-keyword": ["quoted line", "another quoted line"],
"second-keyword": ["quoted line"]
}
+39
View File
@@ -0,0 +1,39 @@
You are arbitrating between two independent codings of the
same song against the same fixed vocabulary of thematic
keywords.
The keywords the two codings agree on have been settled by
script and are not shown. You rule only on the
disagreements: the keywords assigned by one coding but not
the other.
Input: a JSON object with the lyrics of the song and the
disagreements — each disagreement maps a keyword to the lyric
quotes the assigning coding gave as evidence:
{
"lyrics": "...",
"disagreements": {
"first-keyword": ["quoted line", "another quoted line"],
"second-keyword": ["quoted line"]
}
}
Task: for each disagreement keyword, decide whether the song
expresses that theme, by your own reading of the lyrics and
of the quoted evidence.
Rules:
- Rule on every listed keyword: keep it or drop it.
- Keep a keyword only when you can ground it in verbatim
lines of the lyrics; quote them.
- Do not add any keyword that is not listed.
Output a single JSON object mapping each kept keyword to your
list of quotes, and nothing else. A dropped keyword is simply
left out; output an empty object when nothing is kept:
{
"kept-keyword": ["quoted line", "another quoted line"]
}
+19
View File
@@ -0,0 +1,19 @@
You are screening song lyrics for one specific theme.
Input: the complete lyrics of one song.
Task: judge whether this song expresses the theme
"women-power", by your own understanding of
that label.
Rules:
- If the song expresses the theme, quote 1 to 3 verbatim
lines of the lyrics that ground the judgment; if you cannot
ground it in a quote, the answer is no.
Output a single JSON array and nothing else: the quoted
lines when the song expresses the theme, or an empty array
when it does not:
["quoted line", "another quoted line"]
+28
View File
@@ -0,0 +1,28 @@
You are arbitrating a screening of song lyrics for one
specific theme: two independent screenings of the same song
disagreed on whether it expresses the theme
"women-power".
Input: a JSON object with the lyrics of the song and the
lyric quotes that the affirming screening gave as evidence:
{
"lyrics": "...",
"evidence": ["quoted line", "another quoted line"]
}
Task: decide whether the song expresses the theme, by your
own understanding of the label, reading the lyrics and the
quoted evidence.
Rules:
- If the song expresses the theme, quote 1 to 3 verbatim
lines of the lyrics that ground the judgment; if you cannot
ground it in a quote, the answer is no.
Output a single JSON array and nothing else: the quoted
lines when the song expresses the theme, or an empty array
when it does not:
["quoted line", "another quoted line"]