Add the natural-coding definition files and the convergence algorithms
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -6,9 +6,10 @@
|
||||
Messages API: model `claude-sonnet-4-6`, `temperature=0`,
|
||||
thinking disabled, Batch API where possible.
|
||||
- Prompt definition files live in
|
||||
`prompts/<track>-<step>-<task>.md` (e.g. 01-01-tag.md; no
|
||||
version suffix -- versions live in git history) and are
|
||||
passed verbatim as the system prompt.
|
||||
`prompts/<track>-<step>-<substep>-<task>.md` (e.g.
|
||||
01-01-01-tag.md; the substep number fixes the execution
|
||||
order; no version suffix -- versions live in git history)
|
||||
and are passed verbatim as the system prompt.
|
||||
- LLM steps whose outputs are item-by-item comparable
|
||||
(convergence, coding) run the same definition file twice,
|
||||
then a separate arbitration step settles only the
|
||||
|
||||
@@ -323,6 +323,51 @@
|
||||
式化、消滅隱性推導」原則的貫徹。比對子命令驗證兩 run 的定義檔與
|
||||
輸入 SHA 一致、缺 run 即失敗。目標目錄已存在即拒絕執行,重跑須明
|
||||
示 `--replace`。
|
||||
- **收斂仲裁演算法定案(逐對裁決)**:不做組對組匹配
|
||||
(兩組是否「同一組」無原則性答案),比較化約為詞對的
|
||||
共組關係——共識塊(交集細分)→ 分歧塊對枚舉 → 仲裁
|
||||
逐對二元裁決(輸入匿名,不含執行別、共識與歌詞)→
|
||||
union-find 遞移重組(固定邊集的連通元件唯一,與順序
|
||||
無關,決定性成立)。仲裁自身亦 2+1:獨立執行兩次,
|
||||
逐對比對,兩次相同即定案,相異者送 `-arb-arb` 終局票
|
||||
(單次,依協定終局)——每個分歧對等同三票多數決,
|
||||
仲裁鏈至此終止。邊界判斷之裁決為單次記錄性決定,不
|
||||
宣稱可再生(可重現性依計畫定義=程序透明+可稽核)。
|
||||
遞移導出的合併(共識「分開」的塊經第三塊橋接併組)為
|
||||
等價關係語意的邏輯結論,照單接受並逐筆記錄供稽核。
|
||||
一致率低於門檻(暫訂 50%,首輪實跑後校準)即不仲裁
|
||||
——視為定義檔約束不足,修訂後重跑該循環。cap 仲裁後
|
||||
組數略超 50 可接受,如實記錄。演算法明文於
|
||||
`methodology.md`,供讀者依循重現。
|
||||
- **收斂執行不命名,組名全權歸命名鏈(命名 2+1)**:
|
||||
merge、cap 的執行輸出改為無名分組(陣列之陣列)——
|
||||
既然命名鏈存在,收斂執行自取的名字毫無用處,徒增
|
||||
兩次執行間「同組不同名」的假分歧。最終分組定案後,
|
||||
`-name` 命名步驟獨立執行兩次對全部組自由命名——逐組
|
||||
比對,同名即定案(單次執行內名字唯一,故定名間必無
|
||||
撞名);異名送 `-name-arb` 擇一仲裁(匿名呈現兩候選,
|
||||
輸出限於候選並避讓已定名,機械可驗證)。理由:組名
|
||||
承重——merge 名為 cap 的輸入詞、cap 名為 code 的編碼
|
||||
類目——凡承重的語意判斷皆受 2+1。cap 的輸入即 merge
|
||||
定案後的組名清單,原始關鍵字不下傳——如同證據不進
|
||||
收斂,上游細節不污染下游步驟。
|
||||
- **定義檔輸出形狀統一與去署名**:LLM 輸出統一為單層
|
||||
dict/array——tag、code:關鍵字→引述;merge、cap:
|
||||
無名分組(陣列之陣列);code-arb:保留關鍵字→仲裁者自己的引述
|
||||
(剔除即不列,剔除集合由程式以鍵差推得);screen 系:
|
||||
引述陣列,非空即「有」(present 布林刪除,判斷與依據
|
||||
合一)。仲裁輸入一律不含執行別(assigned_by、
|
||||
run1/run2 刪除)——標示何方主張會誘使仲裁者揣測
|
||||
「哪次較可信」,而非就文本裁決。收斂組名採自由命名
|
||||
(不限取自成員詞),取命名貼合度;可追溯性由軌跡歸檔
|
||||
承擔。
|
||||
- **定義檔改三層編號**:
|
||||
`prompts/<軌>-<步>-<次步>-<task>.md`——次步為步內執行
|
||||
順序,明定讀者依循的先後(如 01-02-01-merge →
|
||||
01-02-02-merge-arb → 01-02-03-merge-arb-arb →
|
||||
01-02-04-merge-name → 01-02-05-merge-name-arb)。後綴
|
||||
`-arb` 仲裁、`-arb-arb` 終局票、`-name` 命名、
|
||||
`-name-arb` 命名擇一仲裁。
|
||||
- **screen 標籤詞改為 `women-power`(來歷考據定案)**:
|
||||
考據先導研究的 local agent 存檔:其第一步指令含數十個
|
||||
範例 thematic keywords,其中即有 women-power——為當時
|
||||
|
||||
@@ -0,0 +1,197 @@
|
||||
# 方法細節
|
||||
|
||||
(全文方法節底稿。演算法在執行前寫定;任何修訂記入
|
||||
`decision-log.md`。定義檔全文見 `prompts/`,執行紀錄見
|
||||
`runs/`。)
|
||||
|
||||
## 自然編碼管線總覽
|
||||
|
||||
四步驟:自由標註(tag)→ 自然收斂(merge)→ 強制收斂
|
||||
(cap)→ 全量編碼(code),另設「女性力量」單目標篩選
|
||||
(screen)作黃金標準取樣的補漏網。歌詞只出現在 tag、code、
|
||||
screen 與各仲裁步驟;merge、cap 及其仲裁、命名皆不接觸
|
||||
歌詞。設計原則見 `research-plan.md`;本檔記載可重現的
|
||||
演算法細節。
|
||||
|
||||
## 收斂步驟(merge、cap)的 2+1 比對與仲裁
|
||||
|
||||
兩次獨立執行對同一批輸入詞各產生一個分組。比對不做
|
||||
「組對組」的匹配——兩個組是否為「同一組的變體」無原則性
|
||||
答案——而是把比較化約為「詞對的共組關係」:
|
||||
|
||||
1. **共識塊(交集細分)**:兩次執行都放在同組的詞歸為
|
||||
同一塊。即以「(第一次的組, 第二次的組)」二元組為鍵
|
||||
分桶,一桶一塊。此步為純集合運算。
|
||||
2. **分歧塊對枚舉**:每個塊完整落在各次執行的恰一組內,
|
||||
故「兩塊在某次執行中是否同組」定義良好。逐塊對檢查:
|
||||
兩次執行答案相同者為共識(同組或分開,直接定案);
|
||||
不同者列入分歧清單。
|
||||
3. **逐對仲裁(自身 2+1)**:分歧塊對送 LLM 仲裁
|
||||
(`01-02-02-merge-arb.md`、`01-03-02-cap-arb.md`)。
|
||||
仲裁者只看兩塊的內容詞,逐對二元裁決「是否同一
|
||||
主題」;輸入不含執行別(避免「猜哪一次較可信」的
|
||||
偏誤),不含共識部份,不含歌詞。仲裁自身獨立執行
|
||||
兩次,逐對比對:兩次裁決相同即定案;相異的塊對送
|
||||
終局票(`01-02-03-merge-arb-arb.md`、
|
||||
`01-03-03-cap-arb-arb.md`,單次執行,依協定為終局)
|
||||
——每個分歧塊對等同三票多數決,仲裁鏈至此終止。
|
||||
送入仲裁的塊對即模型自身不穩定的邊界判斷,其裁決為
|
||||
單次記錄性決定,不宣稱可再生;可重現性依計畫定義為
|
||||
「程序透明+可稽核」,裁決與其輸入全程歸檔。
|
||||
4. **確定性重組**:以每個「同組」裁決為一條邊,最終
|
||||
分組=圖的連通元件(union-find)。固定邊集的連通
|
||||
元件唯一,與處理順序無關,故重組決定性成立。
|
||||
遞移性後果照單全收:兩個共識「分開」的塊可能經第三
|
||||
塊橋接而併入同組——此為等價關係語意的邏輯結論,
|
||||
比對程式將此類「遞移導出的合併」逐筆記錄於歸檔,
|
||||
供稽核。仲裁的結果空間因此大於「兩次執行擇一」:
|
||||
可能比兩次都粗(多對皆裁同組),也可能比兩次都細
|
||||
(多對皆裁分開)。
|
||||
5. **一致率與保險絲**:比對程式計算塊對層級的一致率並
|
||||
記入歸檔。低於門檻(暫訂 50%,首輪實跑後校準)即
|
||||
不進行仲裁——兩次分組面目全非說明定義檔約束不足,
|
||||
依協定修訂定義檔並重跑該循環。
|
||||
6. **cap 的上限容忍**:仲裁後組數可能略超 50(多對裁
|
||||
「分開」時)。略微超過可接受,如實記錄,不強行
|
||||
壓縮。
|
||||
7. **組名定案(收斂不命名+命名 2+1)**:merge、cap 的
|
||||
執行輸出不含組名(無名分組,陣列之陣列)——命名鏈
|
||||
既然存在,收斂執行自取的名字只會製造兩次執行間
|
||||
「同組不同名」的假分歧。組名承重——merge 的組名是
|
||||
cap 的輸入詞,cap 的組名是 code 對歌詞編碼的類目——
|
||||
故與其他語意判斷同受 2+1。程序:(1) 最終分組定案後,
|
||||
命名步驟(`01-02-04-merge-name.md`、
|
||||
`01-03-04-cap-name.md`)獨立執行兩次,輸入為不透明
|
||||
組 ID 對成員詞,對全部組自由命名,僅受格式約束
|
||||
(小寫連字號、單次執行內不重複);(2) 逐組比對:兩次
|
||||
同名即定案(單次執行內名字唯一,故定名間必無撞名);
|
||||
(3) 異名的組送擇一仲裁(`01-02-05-merge-name-arb.md`、
|
||||
`01-03-05-cap-name-arb.md`)——匿名呈現兩候選,逐組
|
||||
擇一,輸出限於候選並避讓已定名,機械可驗證。cap 的
|
||||
輸入即 merge 定案後的組名清單。
|
||||
|
||||
### 收斂步驟的資料流(比對子命令的輸入輸出契約)
|
||||
|
||||
以 merge 為例(cap 完全同構,檔名換為 01-03 系):
|
||||
|
||||
1. 兩次執行的原始輸出:`runs/01-02-01-merge/run1/output.jsonl`
|
||||
與 `run2/output.jsonl`,各含一個無名分組(陣列之陣列)。
|
||||
2. 比對子命令讀入兩份分組,先驗證兩者為同一輸入詞集的
|
||||
完整分割(缺詞、多詞、重複即失敗),再計算共識塊與
|
||||
分歧塊對,產出仲裁輸入檔——即
|
||||
`01-02-02-merge-arb.md` 所收的 JSON:
|
||||
- `blocks`:塊 ID → 成員詞。**塊 ID 的指派決定性**:
|
||||
全部塊先按「各塊字典序最小的成員詞」排序,依序編為
|
||||
b1、b2、…;塊內成員詞亦按字典序排列。
|
||||
- `pairs`:分歧塊對清單,每對內部按塊 ID 序、清單
|
||||
整體按 (第一元素, 第二元素) 字典序排列。
|
||||
- 僅分歧塊對入列;共識(同組或分開)不送仲裁,由
|
||||
比對子命令直接寫入共識紀錄檔。
|
||||
同時產出:共識紀錄(共識塊、共識同組對、共識分開對)
|
||||
與塊對一致率(含保險絲判定)。
|
||||
3. 仲裁輸入檔以 run-llm 跑兩次
|
||||
(`runs/01-02-02-merge-arb/run1`、`run2`);比對
|
||||
子命令逐對比對兩份裁決,兩票相同即定案,相異的塊對
|
||||
依同一契約組成終局票輸入檔(`blocks` 僅含涉事塊、
|
||||
ID 沿用原編號),跑
|
||||
`runs/01-02-03-merge-arb-arb/run1`。
|
||||
4. 裁決套用子命令彙整三票結果,以 union-find 重組出
|
||||
最終分組,並寫出:最終分組檔(無名,陣列之陣列,
|
||||
組序與組內成員皆字典序)、遞移導出合併的紀錄、
|
||||
收斂軌跡(原始詞 → 最終組)。
|
||||
5. 最終分組轉為不透明組 ID(依組序編 g1、g2、…)進入
|
||||
命名鏈(見第 7 條);命名定案後,組名清單(字典序)
|
||||
即下一步的輸入。
|
||||
6. 以上中間檔全部隨 `runs/` 歸檔;一切排序規則固定,
|
||||
故給定相同的兩份執行輸出與相同的裁決,全流程輸出
|
||||
逐位元組可再生。
|
||||
|
||||
## 編碼步驟(code、screen)的 2+1 比對與仲裁
|
||||
|
||||
- **code**:逐首比對兩次執行的標籤集合(引述不參與
|
||||
比對)。兩次皆有的標籤為共識保留、兩次皆無為共識
|
||||
不標;單邊標籤送仲裁(`01-04-02-code-arb.md`)——仲裁者
|
||||
看歌詞全文與該標籤的引述(不含執行別),裁決保留者
|
||||
附仲裁者自己的引述,剔除者不列。剔除集合=送裁鍵減
|
||||
輸出鍵,由程式推得。
|
||||
- **screen**:輸出即引述陣列,非空=有、空=無。僅
|
||||
「一有一無」的歌送仲裁(`02-01-02-screen-arb.md`),
|
||||
輸入為歌詞加主張「有」方的引述(不記名),輸出同為
|
||||
引述陣列。
|
||||
- 仲裁者的引述可能與原引述不同:仲裁是對歌詞的重新
|
||||
判讀,其引述是該裁決自身的依據,非轉抄。
|
||||
|
||||
## 軌跡對映(診斷用)
|
||||
|
||||
沿收斂軌跡的機械對映:原始關鍵字 → merge 組 → cap 組,
|
||||
純程式查表,決定性。以其結果與 code 直接編碼的差異率
|
||||
作為「收斂軌跡扭曲」的診斷量,不作主結果。
|
||||
|
||||
## 全管線的交接契約
|
||||
|
||||
每一步的輸出如何變成下一步的輸入,皆為確定性程序,規則
|
||||
明定如下(收斂步驟內部的交接見上節):
|
||||
|
||||
- **歌詞輸入檔(tag、code、screen 共用)**:
|
||||
`export-llm-input` 自工作儲存產出,每筆
|
||||
`{"id": "song-<ID>", "content": <歌詞>}`,依歌曲 ID
|
||||
升序。三個讀歌詞的步驟共用同一檔,SHA-256 記入各步
|
||||
meta。
|
||||
- **tag → merge**:兩次執行的全部關鍵字取聯集、逐字串
|
||||
精確去重、字典序排列成 JSON 陣列,即 merge 的輸入。
|
||||
進池同時寫出處記錄(關鍵字 →(執行別,歌曲 ID)
|
||||
清單),供軌跡對映回到歌曲;出處記錄不進任何 LLM
|
||||
輸入。
|
||||
- **merge 定案 → cap**:merge 定案組名以字典序排成
|
||||
JSON 陣列,即 cap 的輸入(見上節第 5 點)。
|
||||
- **cap 定案 → code 定義檔**:定案組名以字典序逐行填入
|
||||
`01-04-01-code.md` 的詞彙表節(逐字),檔案隨 git
|
||||
commit 後方可執行——code 的定義檔因此自我完備,
|
||||
論文附錄可直接引用。
|
||||
- **code 兩次執行 → code-arb**:逐首比對標籤集合
|
||||
(鍵集合,引述不參與比對);僅有分歧的歌入仲裁輸入
|
||||
JSONL,依歌曲 ID 升序,每筆 `id` 沿用 `song-<ID>`、
|
||||
`content` 為固定鍵序序列化的
|
||||
`{"lyrics": …, "disagreements": …}`,disagreements
|
||||
鍵按字典序。
|
||||
- **code 定案**:每首歌的最終標籤=共識標籤 ∪ 仲裁保留
|
||||
標籤,寫入逐首紀錄檔(歌依 ID 升序、標籤按字典序,
|
||||
各標籤附其定案時的引述與來源層——共識或仲裁)。
|
||||
- **screen 兩次執行 → screen-arb**:僅「一有一無」的歌
|
||||
入仲裁輸入 JSONL(依 ID 升序),`content` 為
|
||||
`{"lyrics": …, "evidence": <肯定方引述>}`。
|
||||
- **screen 定案**:命中集合=兩次皆有 ∪ 仲裁裁定有。
|
||||
- **命名鏈的輸入構成**:`-name` 輸入的組 ID 依最終分組
|
||||
之組序(上節第 5 點)編 g1、g2、…,組內成員字典序;
|
||||
`-name-arb` 輸入中每組的兩個候選名**按字典序排列**
|
||||
——不按執行別,避免順序洩漏何方所取;`taken` 為已
|
||||
定案名的字典序清單。
|
||||
- **女性力量候選集**:於 cap 詞彙表定案後、黃金標準
|
||||
編碼開始前,由研究者指認詞彙表中屬「女性力量」概念
|
||||
域的組(指認及理由記入決策日誌),候選集=code 定案
|
||||
標籤含該等組者 ∪ screen 命中者。
|
||||
- **序列化通則**:所有中間檔為 UTF-8 JSON,鍵序與元素
|
||||
序皆依上列規則明定,無時間戳、無隨機成分;解析一律
|
||||
偵測重複鍵,違規即失敗。JSONL 一律以換行字元(\n)
|
||||
切行——歌詞含 U+2028 等 Unicode 行分隔符,
|
||||
`str.splitlines()` 類的通用切行會截斷 JSON 字串
|
||||
(實測踩中)。給定相同的 LLM 執行輸出,
|
||||
全部交接產物逐位元組可再生。
|
||||
|
||||
## 執行與稽核
|
||||
|
||||
- 每一步驟以 `run-llm <定義檔> <輸入檔> <歸檔目錄>`
|
||||
執行;獨立執行兩次=重現命令清單上的兩行命令,各自
|
||||
歸檔(`runs/<定義檔名>/run1`、`run2`),仲裁與命名
|
||||
各為獨立步驟、獨立歸檔。
|
||||
- 比對、裁決套用、重組、對映皆為確定性程式(子命令),
|
||||
其輸入輸出檔隨 runs/ 歸檔,JSON 解析一律偵測重複鍵,
|
||||
違規即失敗。
|
||||
- Batch API 的每筆請求自含全部脈絡且互不可見(平台
|
||||
契約),歌與歌之間的獨立性由此成立;兩次執行的獨立
|
||||
性由「兩次呼叫、兩個批次、兩份歸檔」的執行結構自明。
|
||||
|
||||
## 映射分析方法
|
||||
|
||||
(依 2026-07-30 決策,於看到結果前寫定;待黃金標準
|
||||
編碼展開前補入。)
|
||||
@@ -24,7 +24,8 @@ pop-fem-audit/
|
||||
│ ├── songs.csv # 歌曲報表(人讀;進 git)
|
||||
│ └── artists.csv # 歌手報表(人讀;進 git)
|
||||
├── prompts/ # LLM 定義檔(逐字作為 system prompt)
|
||||
│ └── <軌>-<步>-<task>.md # 01-01-tag.md、02-01-screen.md…
|
||||
│ └── <軌>-<步>-<次步>-<task>.md # 01-01-01-tag.md、
|
||||
│ # 01-02-02-merge-arb.md…
|
||||
│ # 不帶版本號,版本即 git 歷史
|
||||
├── tools/ # 輔助工具子專案(src-layout)
|
||||
│ ├── pyproject.toml # 發行名 pop-fem-audit-tools;
|
||||
|
||||
+18
-12
@@ -95,11 +95,13 @@
|
||||
themes 過早抽象)之繼承,於執行前鎖定,防止事後擇優。
|
||||
2. **自然收斂(merge)**:輸入為池中純去重關鍵字清單——
|
||||
無歌詞、無頻次、無歌曲出處——模型按自身理解合併近似
|
||||
概念。2+1:兩次收斂處理同一批輸入詞,程式先算出共識核
|
||||
(兩次都同組的詞塊,機械凍結)與分歧清單,仲裁者只裁
|
||||
分歧清單,無權動共識核、無權引入新概念。
|
||||
3. **強制收斂(cap)**:同第 2 步形態,限制併至 50 個
|
||||
以內。頻次不入收斂:頻率的分析角色由第 4 步編碼承擔;
|
||||
概念。2+1:兩次收斂處理同一批輸入詞,程式算出共識塊
|
||||
與分歧塊對,逐對仲裁(仲裁自身亦 2+1,第三票終局)後
|
||||
以連通元件決定性重組;收斂執行不命名,組名由命名步驟
|
||||
2+1 定案(演算法詳見 `methodology.md`)。
|
||||
3. **強制收斂(cap)**:同第 2 步形態,輸入為 merge
|
||||
定案後的組名清單(原始關鍵字不下傳),限制併至 50 個
|
||||
以內;仲裁後略超 50 可接受,如實記錄。頻次不入收斂:頻率的分析角色由第 4 步編碼承擔;
|
||||
池中頻次含跨執行噪音;頻次會誘使模型以頻率剪枝(喪失
|
||||
稀有主題)並把高頻大主題切細。代價(特異主題佔名額)
|
||||
已知並接受,換取主題多樣性與純語意歸併的可辯護性。
|
||||
@@ -120,15 +122,19 @@
|
||||
召回之用,永不進任何統計的分子分母。標籤詞是 screen
|
||||
提示中唯一的語意種子,屬研究者的儀器選擇,據實揭露。
|
||||
|
||||
定義檔命名 `prompts/<軌>-<步>-<task>.md`——軌 01=由下
|
||||
而上自然編碼、02=預先決定的 women-power
|
||||
篩選,步為軌內步驟序:01-01-tag.md、01-02-merge.md、
|
||||
01-03-cap.md、01-04-code.md、02-01-screen.md;仲裁定義檔
|
||||
同 prefix 加 `-arb`(如 01-02-merge-arb.md)。檔名不帶
|
||||
定義檔命名 `prompts/<軌>-<步>-<次步>-<task>.md`——軌
|
||||
01=由下而上自然編碼、02=預先決定的 women-power
|
||||
篩選;步為軌內步驟序,次步為
|
||||
步內執行順序,讀者依編號先後依循(如 01-02-01-merge.md →
|
||||
01-02-02-merge-arb.md → 01-02-03-merge-arb-arb.md →
|
||||
01-02-04-merge-name.md → 01-02-05-merge-name-arb.md)。
|
||||
後綴 `-arb` 為仲裁、`-arb-arb` 為仲裁之終局票、`-name`
|
||||
為命名、`-name-arb` 為命名擇一仲裁。檔名不帶
|
||||
版本號——版本即 git 歷史,失敗的版本不保留,需要回看的
|
||||
舊版都在 git history;每次執行的定義檔快照隨 `runs/`
|
||||
自我完備。四次收斂執行(merge ×2、cap ×2)各將合併記錄
|
||||
(哪些詞併入哪組)存成 JSON,隨該次執行入 `runs/`。
|
||||
自我完備。收斂執行的輸出本身即分組記錄(無名分組
|
||||
JSON),與全部中間交接檔同隨 `runs/` 歸檔(交接契約見
|
||||
`methodology.md`)。
|
||||
|
||||
## 階段與時程(全文截稿 2026-08-15)
|
||||
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
You are annotating song lyrics for a content-analysis corpus.
|
||||
|
||||
Input: the complete lyrics of one song.
|
||||
|
||||
Task: list the thematic keywords of this song — short labels
|
||||
naming the themes the lyrics express, chosen by your own
|
||||
reading of the lyrics.
|
||||
|
||||
Rules:
|
||||
|
||||
- Give between 5 and 10 thematic keywords.
|
||||
- Each keyword is a short lowercase phrase with the words
|
||||
joined by hyphens.
|
||||
- For each keyword, quote 1 to 3 verbatim lines of the lyrics
|
||||
that ground it; do not give a keyword you cannot ground.
|
||||
|
||||
Output a single JSON object mapping each keyword to its list
|
||||
of quotes, and nothing else:
|
||||
|
||||
{
|
||||
"first-keyword": ["quoted line", "another quoted line"],
|
||||
"second-keyword": ["quoted line"]
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
You are consolidating a vocabulary of thematic keywords that
|
||||
were independently annotated on the songs of a corpus.
|
||||
|
||||
Input: a JSON array of thematic keywords, deduplicated and
|
||||
unordered.
|
||||
|
||||
Task: merge keywords that express the same or nearly the same
|
||||
theme, by your own understanding of what the keywords mean.
|
||||
|
||||
Rules:
|
||||
|
||||
- Every input keyword must appear in exactly one group.
|
||||
- Do not invent keywords that are not in the input.
|
||||
- Merge only what you consider the same theme; a group may be
|
||||
a single keyword.
|
||||
|
||||
Output a single JSON array of groups — each group an array of
|
||||
its member keywords — and nothing else:
|
||||
|
||||
[
|
||||
["keyword", "another-keyword"],
|
||||
["keyword"]
|
||||
]
|
||||
@@ -0,0 +1,36 @@
|
||||
You are arbitrating between two independent consolidations of
|
||||
the same vocabulary of thematic keywords from a song corpus.
|
||||
|
||||
Where the two consolidations agree, the result has been
|
||||
settled by script and is not shown. You rule on the
|
||||
disagreements: pairs of keyword blocks that one consolidation
|
||||
merged into one theme and the other kept apart.
|
||||
|
||||
Input: a JSON object. "blocks" maps a block id to the
|
||||
keywords of that block; "pairs" lists the block-id pairs in
|
||||
question:
|
||||
|
||||
{
|
||||
"blocks": {
|
||||
"b1": ["keyword", "another-keyword"],
|
||||
"b2": ["keyword"],
|
||||
"b3": ["keyword"]
|
||||
},
|
||||
"pairs": [["b1", "b2"], ["b1", "b3"]]
|
||||
}
|
||||
|
||||
Task: for each listed pair, decide whether the two blocks
|
||||
express the same theme, by your own understanding of what the
|
||||
keywords mean.
|
||||
|
||||
Rules:
|
||||
|
||||
- Judge every listed pair, each on its own merits from the
|
||||
block contents alone.
|
||||
- Judge only the listed pairs.
|
||||
|
||||
Output a single JSON array holding the pairs whose two blocks
|
||||
express the same theme, written exactly as given in "pairs";
|
||||
an empty array when none do:
|
||||
|
||||
[["b1", "b2"]]
|
||||
@@ -0,0 +1,34 @@
|
||||
You are casting the deciding judgment on disagreements about
|
||||
a consolidated vocabulary of thematic keywords from a song
|
||||
corpus: for each pair of keyword blocks in question, earlier
|
||||
independent judgments disagreed on whether the two blocks
|
||||
express the same theme.
|
||||
|
||||
Input: a JSON object. "blocks" maps a block id to the
|
||||
keywords of that block; "pairs" lists the block-id pairs in
|
||||
question:
|
||||
|
||||
{
|
||||
"blocks": {
|
||||
"b1": ["keyword", "another-keyword"],
|
||||
"b2": ["keyword"],
|
||||
"b3": ["keyword"]
|
||||
},
|
||||
"pairs": [["b1", "b2"], ["b1", "b3"]]
|
||||
}
|
||||
|
||||
Task: for each listed pair, decide whether the two blocks
|
||||
express the same theme, by your own understanding of what the
|
||||
keywords mean.
|
||||
|
||||
Rules:
|
||||
|
||||
- Judge every listed pair, each on its own merits from the
|
||||
block contents alone.
|
||||
- Judge only the listed pairs.
|
||||
|
||||
Output a single JSON array holding the pairs whose two blocks
|
||||
express the same theme, written exactly as given in "pairs";
|
||||
an empty array when none do:
|
||||
|
||||
[["b1", "b2"]]
|
||||
@@ -0,0 +1,28 @@
|
||||
You are naming the theme groups of a consolidated vocabulary
|
||||
of thematic keywords from a song corpus.
|
||||
|
||||
Input: a JSON object mapping an opaque group id to the member
|
||||
keywords of that group:
|
||||
|
||||
{
|
||||
"g1": ["keyword", "another-keyword"],
|
||||
"g2": ["keyword"]
|
||||
}
|
||||
|
||||
Task: give each group a name that best names the theme its
|
||||
members gather, by your own understanding of what the
|
||||
keywords mean.
|
||||
|
||||
Rules:
|
||||
|
||||
- Each name is a short lowercase phrase with the words joined
|
||||
by hyphens.
|
||||
- Names must be unique across the groups.
|
||||
|
||||
Output a single JSON object mapping each group id to its
|
||||
name, and nothing else:
|
||||
|
||||
{
|
||||
"g1": "group-name",
|
||||
"g2": "another-group-name"
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
You are arbitrating between two independent namings of the
|
||||
theme groups of a consolidated vocabulary of thematic
|
||||
keywords from a song corpus.
|
||||
|
||||
For each group in question, the two namings proposed
|
||||
different names. You choose between them.
|
||||
|
||||
Input: a JSON object. "groups" maps an opaque group id to
|
||||
its member keywords and its two candidate names; "taken"
|
||||
lists names that are already in use:
|
||||
|
||||
{
|
||||
"groups": {
|
||||
"g1": {
|
||||
"members": ["keyword", "another-keyword"],
|
||||
"candidates": ["one-name", "other-name"]
|
||||
}
|
||||
},
|
||||
"taken": ["existing-name"]
|
||||
}
|
||||
|
||||
Task: for each group, choose the candidate that better names
|
||||
the theme its members gather, by your own understanding of
|
||||
what the keywords mean.
|
||||
|
||||
Rules:
|
||||
|
||||
- Choose only from that group's two candidates.
|
||||
- Choices must be unique across the groups and must not
|
||||
reuse any name in "taken".
|
||||
|
||||
Output a single JSON object mapping each group id to the
|
||||
chosen name, and nothing else:
|
||||
|
||||
{
|
||||
"g1": "one-name"
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
You are consolidating a vocabulary of thematic keywords
|
||||
from a song corpus further, to a fixed maximum number of
|
||||
themes.
|
||||
|
||||
Input: a JSON array of thematic keywords, deduplicated and
|
||||
unordered.
|
||||
|
||||
Task: merge keywords that express the same or a similar
|
||||
theme, by your own understanding of what the keywords mean,
|
||||
until at most 50 groups remain.
|
||||
|
||||
Rules:
|
||||
|
||||
- Every input keyword must appear in exactly one group.
|
||||
- Do not invent keywords that are not in the input.
|
||||
- Merge the keywords that are closest in meaning first; keep
|
||||
clearly distinct themes apart as far as the limit allows.
|
||||
A group may be a single keyword.
|
||||
|
||||
Output a single JSON array of groups — each group an array of
|
||||
its member keywords — and nothing else:
|
||||
|
||||
[
|
||||
["keyword", "another-keyword"],
|
||||
["keyword"]
|
||||
]
|
||||
@@ -0,0 +1,37 @@
|
||||
You are arbitrating between two independent consolidations
|
||||
that each reduced the same vocabulary of thematic keywords
|
||||
from a song corpus to a fixed maximum number of themes.
|
||||
|
||||
Where the two consolidations agree, the result has been
|
||||
settled by script and is not shown. You rule on the
|
||||
disagreements: pairs of keyword blocks that one consolidation
|
||||
merged into one theme and the other kept apart.
|
||||
|
||||
Input: a JSON object. "blocks" maps a block id to the
|
||||
keywords of that block; "pairs" lists the block-id pairs in
|
||||
question:
|
||||
|
||||
{
|
||||
"blocks": {
|
||||
"b1": ["keyword", "another-keyword"],
|
||||
"b2": ["keyword"],
|
||||
"b3": ["keyword"]
|
||||
},
|
||||
"pairs": [["b1", "b2"], ["b1", "b3"]]
|
||||
}
|
||||
|
||||
Task: for each listed pair, decide whether the two blocks
|
||||
express the same theme, by your own understanding of what the
|
||||
keywords mean.
|
||||
|
||||
Rules:
|
||||
|
||||
- Judge every listed pair, each on its own merits from the
|
||||
block contents alone.
|
||||
- Judge only the listed pairs.
|
||||
|
||||
Output a single JSON array holding the pairs whose two blocks
|
||||
express the same theme, written exactly as given in "pairs";
|
||||
an empty array when none do:
|
||||
|
||||
[["b1", "b2"]]
|
||||
@@ -0,0 +1,35 @@
|
||||
You are casting the deciding judgment on disagreements about
|
||||
a vocabulary of thematic keywords from a song corpus that was
|
||||
consolidated to a fixed maximum number of themes: for each
|
||||
pair of keyword blocks in question, earlier independent
|
||||
judgments disagreed on whether the two blocks express the
|
||||
same theme.
|
||||
|
||||
Input: a JSON object. "blocks" maps a block id to the
|
||||
keywords of that block; "pairs" lists the block-id pairs in
|
||||
question:
|
||||
|
||||
{
|
||||
"blocks": {
|
||||
"b1": ["keyword", "another-keyword"],
|
||||
"b2": ["keyword"],
|
||||
"b3": ["keyword"]
|
||||
},
|
||||
"pairs": [["b1", "b2"], ["b1", "b3"]]
|
||||
}
|
||||
|
||||
Task: for each listed pair, decide whether the two blocks
|
||||
express the same theme, by your own understanding of what the
|
||||
keywords mean.
|
||||
|
||||
Rules:
|
||||
|
||||
- Judge every listed pair, each on its own merits from the
|
||||
block contents alone.
|
||||
- Judge only the listed pairs.
|
||||
|
||||
Output a single JSON array holding the pairs whose two blocks
|
||||
express the same theme, written exactly as given in "pairs";
|
||||
an empty array when none do:
|
||||
|
||||
[["b1", "b2"]]
|
||||
@@ -0,0 +1,29 @@
|
||||
You are naming the theme groups of a vocabulary of thematic
|
||||
keywords from a song corpus that was consolidated to a fixed
|
||||
maximum number of themes.
|
||||
|
||||
Input: a JSON object mapping an opaque group id to the member
|
||||
keywords of that group:
|
||||
|
||||
{
|
||||
"g1": ["keyword", "another-keyword"],
|
||||
"g2": ["keyword"]
|
||||
}
|
||||
|
||||
Task: give each group a name that best names the theme its
|
||||
members gather, by your own understanding of what the
|
||||
keywords mean.
|
||||
|
||||
Rules:
|
||||
|
||||
- Each name is a short lowercase phrase with the words joined
|
||||
by hyphens.
|
||||
- Names must be unique across the groups.
|
||||
|
||||
Output a single JSON object mapping each group id to its
|
||||
name, and nothing else:
|
||||
|
||||
{
|
||||
"g1": "group-name",
|
||||
"g2": "another-group-name"
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
You are arbitrating between two independent namings of the
|
||||
theme groups of a vocabulary of thematic keywords from a song
|
||||
corpus that was consolidated to a fixed maximum number of
|
||||
themes.
|
||||
|
||||
For each group in question, the two namings proposed
|
||||
different names. You choose between them.
|
||||
|
||||
Input: a JSON object. "groups" maps an opaque group id to
|
||||
its member keywords and its two candidate names; "taken"
|
||||
lists names that are already in use:
|
||||
|
||||
{
|
||||
"groups": {
|
||||
"g1": {
|
||||
"members": ["keyword", "another-keyword"],
|
||||
"candidates": ["one-name", "other-name"]
|
||||
}
|
||||
},
|
||||
"taken": ["existing-name"]
|
||||
}
|
||||
|
||||
Task: for each group, choose the candidate that better names
|
||||
the theme its members gather, by your own understanding of
|
||||
what the keywords mean.
|
||||
|
||||
Rules:
|
||||
|
||||
- Choose only from that group's two candidates.
|
||||
- Choices must be unique across the groups and must not
|
||||
reuse any name in "taken".
|
||||
|
||||
Output a single JSON object mapping each group id to the
|
||||
chosen name, and nothing else:
|
||||
|
||||
{
|
||||
"g1": "one-name"
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
You are coding song lyrics against a fixed vocabulary of
|
||||
thematic keywords.
|
||||
|
||||
Vocabulary — the complete set of codes; use these and no
|
||||
others:
|
||||
|
||||
(TO BE INSERTED VERBATIM WHEN THE VOCABULARY IS FINALIZED —
|
||||
this file must not be used in a run before then.)
|
||||
|
||||
Input: the complete lyrics of one song.
|
||||
|
||||
Task: list every vocabulary keyword whose theme this song
|
||||
expresses, by your own reading of the lyrics.
|
||||
|
||||
Rules:
|
||||
|
||||
- Use only keywords from the vocabulary, spelled exactly as
|
||||
given.
|
||||
- A song may match any number of keywords, including none.
|
||||
- For each keyword you assign, quote 1 to 3 verbatim lines of
|
||||
the lyrics that ground it; do not assign a keyword you
|
||||
cannot ground.
|
||||
|
||||
Output a single JSON object mapping each assigned keyword to
|
||||
its list of quotes (an empty object when no keyword applies),
|
||||
and nothing else:
|
||||
|
||||
{
|
||||
"first-keyword": ["quoted line", "another quoted line"],
|
||||
"second-keyword": ["quoted line"]
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
You are arbitrating between two independent codings of the
|
||||
same song against the same fixed vocabulary of thematic
|
||||
keywords.
|
||||
|
||||
The keywords the two codings agree on have been settled by
|
||||
script and are not shown. You rule only on the
|
||||
disagreements: the keywords assigned by one coding but not
|
||||
the other.
|
||||
|
||||
Input: a JSON object with the lyrics of the song and the
|
||||
disagreements — each disagreement maps a keyword to the lyric
|
||||
quotes the assigning coding gave as evidence:
|
||||
|
||||
{
|
||||
"lyrics": "...",
|
||||
"disagreements": {
|
||||
"first-keyword": ["quoted line", "another quoted line"],
|
||||
"second-keyword": ["quoted line"]
|
||||
}
|
||||
}
|
||||
|
||||
Task: for each disagreement keyword, decide whether the song
|
||||
expresses that theme, by your own reading of the lyrics and
|
||||
of the quoted evidence.
|
||||
|
||||
Rules:
|
||||
|
||||
- Rule on every listed keyword: keep it or drop it.
|
||||
- Keep a keyword only when you can ground it in verbatim
|
||||
lines of the lyrics; quote them.
|
||||
- Do not add any keyword that is not listed.
|
||||
|
||||
Output a single JSON object mapping each kept keyword to your
|
||||
list of quotes, and nothing else. A dropped keyword is simply
|
||||
left out; output an empty object when nothing is kept:
|
||||
|
||||
{
|
||||
"kept-keyword": ["quoted line", "another quoted line"]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
You are screening song lyrics for one specific theme.
|
||||
|
||||
Input: the complete lyrics of one song.
|
||||
|
||||
Task: judge whether this song expresses the theme
|
||||
"women-power", by your own understanding of
|
||||
that label.
|
||||
|
||||
Rules:
|
||||
|
||||
- If the song expresses the theme, quote 1 to 3 verbatim
|
||||
lines of the lyrics that ground the judgment; if you cannot
|
||||
ground it in a quote, the answer is no.
|
||||
|
||||
Output a single JSON array and nothing else: the quoted
|
||||
lines when the song expresses the theme, or an empty array
|
||||
when it does not:
|
||||
|
||||
["quoted line", "another quoted line"]
|
||||
@@ -0,0 +1,28 @@
|
||||
You are arbitrating a screening of song lyrics for one
|
||||
specific theme: two independent screenings of the same song
|
||||
disagreed on whether it expresses the theme
|
||||
"women-power".
|
||||
|
||||
Input: a JSON object with the lyrics of the song and the
|
||||
lyric quotes that the affirming screening gave as evidence:
|
||||
|
||||
{
|
||||
"lyrics": "...",
|
||||
"evidence": ["quoted line", "another quoted line"]
|
||||
}
|
||||
|
||||
Task: decide whether the song expresses the theme, by your
|
||||
own understanding of the label, reading the lyrics and the
|
||||
quoted evidence.
|
||||
|
||||
Rules:
|
||||
|
||||
- If the song expresses the theme, quote 1 to 3 verbatim
|
||||
lines of the lyrics that ground the judgment; if you cannot
|
||||
ground it in a quote, the answer is no.
|
||||
|
||||
Output a single JSON array and nothing else: the quoted
|
||||
lines when the song expresses the theme, or an empty array
|
||||
when it does not:
|
||||
|
||||
["quoted line", "another quoted line"]
|
||||
Reference in New Issue
Block a user