Changed the natural-coding definition files to single-pass convergence
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -6,18 +6,18 @@
|
|||||||
Messages API: model `claude-sonnet-4-6`, `temperature=0`,
|
Messages API: model `claude-sonnet-4-6`, `temperature=0`,
|
||||||
thinking disabled, Batch API where possible.
|
thinking disabled, Batch API where possible.
|
||||||
- Prompt definition files live in
|
- Prompt definition files live in
|
||||||
`prompts/<track>-<step>-<substep>-<task>.md` (e.g.
|
`prompts/<track>-<step>-<task>.md` (e.g. 01-01-tag.md; no
|
||||||
01-01-01-tag.md; the substep number fixes the execution
|
version suffix -- versions live in git history) and are
|
||||||
order; no version suffix -- versions live in git history)
|
passed verbatim as the system prompt.
|
||||||
and are passed verbatim as the system prompt.
|
- Per-song LLM judgments (coding, screening) run the same
|
||||||
- LLM steps whose outputs are item-by-item comparable
|
definition file twice, then a separate arbitration step
|
||||||
(convergence, coding) run the same definition file twice,
|
settles only the script-computed disagreements
|
||||||
then a separate arbitration step settles only the
|
("2 runs + 1 arbitration"). Free-generation steps run
|
||||||
script-computed disagreements ("2 runs + 1 arbitration").
|
twice and both outputs are pooled, unarbitrated.
|
||||||
Free-generation steps run twice and both outputs are pooled,
|
Vocabulary-building steps (convergence) run once as a
|
||||||
unarbitrated. If arbitration output is unexpected, revise
|
recorded pass. If an arbitration or validation outcome is
|
||||||
the definition file and repeat that cycle; never patch
|
unexpected, revise the definition file and repeat that
|
||||||
results by hand.
|
cycle; never patch results by hand.
|
||||||
- Each run of a step is archived self-contained under the
|
- Each run of a step is archived self-contained under the
|
||||||
destination directory given explicitly on the `run-llm`
|
destination directory given explicitly on the `run-llm`
|
||||||
command line (by convention `runs/<definition-file>/run<N>/`):
|
command line (by convention `runs/<definition-file>/run<N>/`):
|
||||||
|
|||||||
+38
-45
@@ -323,51 +323,6 @@
|
|||||||
式化、消滅隱性推導」原則的貫徹。比對子命令驗證兩 run 的定義檔與
|
式化、消滅隱性推導」原則的貫徹。比對子命令驗證兩 run 的定義檔與
|
||||||
輸入 SHA 一致、缺 run 即失敗。目標目錄已存在即拒絕執行,重跑須明
|
輸入 SHA 一致、缺 run 即失敗。目標目錄已存在即拒絕執行,重跑須明
|
||||||
示 `--replace`。
|
示 `--replace`。
|
||||||
- **收斂仲裁演算法定案(逐對裁決)**:不做組對組匹配
|
|
||||||
(兩組是否「同一組」無原則性答案),比較化約為詞對的
|
|
||||||
共組關係——共識塊(交集細分)→ 分歧塊對枚舉 → 仲裁
|
|
||||||
逐對二元裁決(輸入匿名,不含執行別、共識與歌詞)→
|
|
||||||
union-find 遞移重組(固定邊集的連通元件唯一,與順序
|
|
||||||
無關,決定性成立)。仲裁自身亦 2+1:獨立執行兩次,
|
|
||||||
逐對比對,兩次相同即定案,相異者送 `-arb-arb` 終局票
|
|
||||||
(單次,依協定終局)——每個分歧對等同三票多數決,
|
|
||||||
仲裁鏈至此終止。邊界判斷之裁決為單次記錄性決定,不
|
|
||||||
宣稱可再生(可重現性依計畫定義=程序透明+可稽核)。
|
|
||||||
遞移導出的合併(共識「分開」的塊經第三塊橋接併組)為
|
|
||||||
等價關係語意的邏輯結論,照單接受並逐筆記錄供稽核。
|
|
||||||
一致率低於門檻(暫訂 50%,首輪實跑後校準)即不仲裁
|
|
||||||
——視為定義檔約束不足,修訂後重跑該循環。cap 仲裁後
|
|
||||||
組數略超 50 可接受,如實記錄。演算法明文於
|
|
||||||
`methodology.md`,供讀者依循重現。
|
|
||||||
- **收斂執行不命名,組名全權歸命名鏈(命名 2+1)**:
|
|
||||||
merge、cap 的執行輸出改為無名分組(陣列之陣列)——
|
|
||||||
既然命名鏈存在,收斂執行自取的名字毫無用處,徒增
|
|
||||||
兩次執行間「同組不同名」的假分歧。最終分組定案後,
|
|
||||||
`-name` 命名步驟獨立執行兩次對全部組自由命名——逐組
|
|
||||||
比對,同名即定案(單次執行內名字唯一,故定名間必無
|
|
||||||
撞名);異名送 `-name-arb` 擇一仲裁(匿名呈現兩候選,
|
|
||||||
輸出限於候選並避讓已定名,機械可驗證)。理由:組名
|
|
||||||
承重——merge 名為 cap 的輸入詞、cap 名為 code 的編碼
|
|
||||||
類目——凡承重的語意判斷皆受 2+1。cap 的輸入即 merge
|
|
||||||
定案後的組名清單,原始關鍵字不下傳——如同證據不進
|
|
||||||
收斂,上游細節不污染下游步驟。
|
|
||||||
- **定義檔輸出形狀統一與去署名**:LLM 輸出統一為單層
|
|
||||||
dict/array——tag、code:關鍵字→引述;merge、cap:
|
|
||||||
無名分組(陣列之陣列);code-arb:保留關鍵字→仲裁者自己的引述
|
|
||||||
(剔除即不列,剔除集合由程式以鍵差推得);screen 系:
|
|
||||||
引述陣列,非空即「有」(present 布林刪除,判斷與依據
|
|
||||||
合一)。仲裁輸入一律不含執行別(assigned_by、
|
|
||||||
run1/run2 刪除)——標示何方主張會誘使仲裁者揣測
|
|
||||||
「哪次較可信」,而非就文本裁決。收斂組名採自由命名
|
|
||||||
(不限取自成員詞),取命名貼合度;可追溯性由軌跡歸檔
|
|
||||||
承擔。
|
|
||||||
- **定義檔改三層編號**:
|
|
||||||
`prompts/<軌>-<步>-<次步>-<task>.md`——次步為步內執行
|
|
||||||
順序,明定讀者依循的先後(如 01-02-01-merge →
|
|
||||||
01-02-02-merge-arb → 01-02-03-merge-arb-arb →
|
|
||||||
01-02-04-merge-name → 01-02-05-merge-name-arb)。後綴
|
|
||||||
`-arb` 仲裁、`-arb-arb` 終局票、`-name` 命名、
|
|
||||||
`-name-arb` 命名擇一仲裁。
|
|
||||||
- **screen 標籤詞改為 `women-power`(來歷考據定案)**:
|
- **screen 標籤詞改為 `women-power`(來歷考據定案)**:
|
||||||
考據先導研究的 local agent 存檔:其第一步指令含數十個
|
考據先導研究的 local agent 存檔:其第一步指令含數十個
|
||||||
範例 thematic keywords,其中即有 women-power——為當時
|
範例 thematic keywords,其中即有 women-power——為當時
|
||||||
@@ -381,3 +336,41 @@
|
|||||||
考」敘述同步改為考據結果。附帶認清:先導第一步並非零
|
考」敘述同步改為考據結果。附帶認清:先導第一步並非零
|
||||||
語意提示——此即正式研究「只定格式、不定語意」設計所
|
語意提示——此即正式研究「只定格式、不定語意」設計所
|
||||||
矯正者。
|
矯正者。
|
||||||
|
- **定義檔輸出形狀統一與去署名**:LLM 輸出統一為單層
|
||||||
|
dict/array——tag、code:關鍵字→引述;merge、cap:
|
||||||
|
組名→成員;code-arb:保留關鍵字→仲裁者自己的引述
|
||||||
|
(剔除即不列,剔除集合由程式以鍵差推得);screen 系:
|
||||||
|
引述陣列,非空即「有」(present 布林刪除,判斷與依據
|
||||||
|
合一)。仲裁輸入一律不含執行別——標示何方主張會誘使
|
||||||
|
仲裁者揣測「哪次較可信」,而非就文本裁決。收斂組名採
|
||||||
|
自由命名(不限取自成員詞),取命名貼合度;可追溯性由
|
||||||
|
軌跡歸檔承擔。
|
||||||
|
- **定義檔改三層編號**:
|
||||||
|
`prompts/<軌>-<步>-<次步>-<task>.md`——次步為步內執行
|
||||||
|
順序,明定讀者依循的先後(如 01-04-01-code →
|
||||||
|
01-04-02-code-arb);仲裁檔後綴 `-arb`。
|
||||||
|
- **收斂演算法:檢視而棄用的方案**:詞彙表建構(merge、
|
||||||
|
cap)的重複執行與仲裁,歷經四個方案後全數棄用——
|
||||||
|
①整條管線獨立跑兩遍、於終點仲裁兩份最終詞彙表:兩套
|
||||||
|
分類系統互不可比,仲裁淪為第三次建構,一致率無從
|
||||||
|
定義。②組對組匹配(以相似度門檻判定兩組是否「同一
|
||||||
|
組」):無原則性答案,門檻任意。③tag 步驟仲裁:自由
|
||||||
|
詞彙兩次輸出不共享比對單位,無物可裁(tag 改為兩次
|
||||||
|
進池,沿用至今)。④逐對仲裁鏈:交集細分出共識塊、
|
||||||
|
分歧塊對三票多數決、union-find 遞移重組、仲裁後命名
|
||||||
|
2+1——機械上可行且逐項可驗(共 15 份定義檔,全版
|
||||||
|
保存於分支 `tag-algo-13` 備考),但其變異縮減未經證實
|
||||||
|
(逐對多數決降低對層變異,遞移閉包卻放大結構層變異,
|
||||||
|
淨效果不明),複雜度成本則屬確定,且縮減的是儀器變異
|
||||||
|
——對量測無關緊要的量(見次條)。
|
||||||
|
- **詞彙表建構改為單次記錄性程序(演算法簡化)**:
|
||||||
|
merge、cap 各單次執行、執行內自行命名(具名分組
|
||||||
|
輸出),定義檔減為 7 份。2+1 原則改寫:語料層逐首
|
||||||
|
判斷(code、screen)一律 2+1;自由生成(tag)兩次
|
||||||
|
進池;詞彙表建構單次、全程歸檔。理由:(1) 影響量測的
|
||||||
|
是編碼層——詞彙表屬揭露的儀器選擇,凍結後下游同尺量
|
||||||
|
到底,其抽樣變異不污染量測;(2) 對齊領域慣行——
|
||||||
|
codebook 建構本為單次詮釋程序,信度檢驗施於編碼應用
|
||||||
|
層;(3) 複雜仲裁機械無以自證其益(見前條)。驗證改為
|
||||||
|
確定性格式檢查(完整分割、組名唯一、cap ≤ 50),違規
|
||||||
|
依協定修訂定義檔重跑。
|
||||||
|
|||||||
+40
-114
@@ -6,105 +6,33 @@
|
|||||||
|
|
||||||
## 自然編碼管線總覽
|
## 自然編碼管線總覽
|
||||||
|
|
||||||
四步驟:自由標註(tag)→ 自然收斂(merge)→ 強制收斂
|
四步驟:自由標註(tag,兩次進池)→ 自然收斂(merge,
|
||||||
(cap)→ 全量編碼(code),另設「女性力量」單目標篩選
|
單次)→ 強制收斂(cap,單次)→ 全量編碼(code,2+1),
|
||||||
(screen)作黃金標準取樣的補漏網。歌詞只出現在 tag、code、
|
另設「女性力量」單目標篩選(screen,2+1)作黃金標準取樣
|
||||||
screen 與各仲裁步驟;merge、cap 及其仲裁、命名皆不接觸
|
的補漏網。歌詞只出現在 tag、code、screen 與其仲裁步驟;
|
||||||
歌詞。設計原則見 `research-plan.md`;本檔記載可重現的
|
merge、cap 不接觸歌詞。設計原則見 `research-plan.md`;
|
||||||
演算法細節。
|
本檔記載可重現的演算法細節。
|
||||||
|
|
||||||
## 收斂步驟(merge、cap)的 2+1 比對與仲裁
|
## 詞彙表建構(merge、cap)——單次記錄性程序
|
||||||
|
|
||||||
兩次獨立執行對同一批輸入詞各產生一個分組。比對不做
|
merge 與 cap 各以單次執行完成,執行內自行為各組命名,
|
||||||
「組對組」的匹配——兩個組是否為「同一組的變體」無原則性
|
輸出具名分組(組名 → 成員詞)。不做重複執行與仲裁。
|
||||||
答案——而是把比較化約為「詞對的共組關係」:
|
|
||||||
|
|
||||||
1. **共識塊(交集細分)**:兩次執行都放在同組的詞歸為
|
- **定位**:詞彙表是研究的儀器,不是量測。詞彙表凍結
|
||||||
同一塊。即以「(第一次的組, 第二次的組)」二元組為鍵
|
入庫後,下游全部量測(編碼 vs 黃金標準)以同一把尺
|
||||||
分桶,一桶一塊。此步為純集合運算。
|
進行,內部一致;建構過程的抽樣變異屬「揭露的儀器
|
||||||
2. **分歧塊對枚舉**:每個塊完整落在各次執行的恰一組內,
|
選擇」,不污染量測。此與質性研究慣行一致——codebook
|
||||||
故「兩塊在某次執行中是否同組」定義良好。逐塊對檢查:
|
建構本為單次的詮釋程序,信度檢驗施於編碼應用層。
|
||||||
兩次執行答案相同者為共識(同組或分開,直接定案);
|
- **取捨紀錄**:曾設計逐對仲裁鏈(共識塊、分歧塊對三票
|
||||||
不同者列入分歧清單。
|
多數、命名 2+1;完整版本保存於分支 `tag-algo-13`)。
|
||||||
3. **逐對仲裁(自身 2+1)**:分歧塊對送 LLM 仲裁
|
棄用理由:逐對多數決雖降低單對變異,但遞移閉包會放大
|
||||||
(`01-02-02-merge-arb.md`、`01-03-02-cap-arb.md`)。
|
結構層變異,淨縮減未經證實;其複雜度成本卻是確定的;
|
||||||
仲裁者只看兩塊的內容詞,逐對二元裁決「是否同一
|
且縮減的是儀器變異——對論文主張無關緊要的量。
|
||||||
主題」;輸入不含執行別(避免「猜哪一次較可信」的
|
- **確定性驗證**(違規即依協定修訂定義檔重跑):輸出須
|
||||||
偏誤),不含共識部份,不含歌詞。仲裁自身獨立執行
|
為輸入詞集的完整分割(缺詞、多詞、重複即失敗);組名
|
||||||
兩次,逐對比對:兩次裁決相同即定案;相異的塊對送
|
唯一且符合格式;cap 組數 ≤ 50。
|
||||||
終局票(`01-02-03-merge-arb-arb.md`、
|
- **收斂軌跡**:原始關鍵字 → merge 組名 → cap 組名,
|
||||||
`01-03-03-cap-arb-arb.md`,單次執行,依協定為終局)
|
兩份執行輸出本身即完整記錄。
|
||||||
——每個分歧塊對等同三票多數決,仲裁鏈至此終止。
|
|
||||||
送入仲裁的塊對即模型自身不穩定的邊界判斷,其裁決為
|
|
||||||
單次記錄性決定,不宣稱可再生;可重現性依計畫定義為
|
|
||||||
「程序透明+可稽核」,裁決與其輸入全程歸檔。
|
|
||||||
4. **確定性重組**:以每個「同組」裁決為一條邊,最終
|
|
||||||
分組=圖的連通元件(union-find)。固定邊集的連通
|
|
||||||
元件唯一,與處理順序無關,故重組決定性成立。
|
|
||||||
遞移性後果照單全收:兩個共識「分開」的塊可能經第三
|
|
||||||
塊橋接而併入同組——此為等價關係語意的邏輯結論,
|
|
||||||
比對程式將此類「遞移導出的合併」逐筆記錄於歸檔,
|
|
||||||
供稽核。仲裁的結果空間因此大於「兩次執行擇一」:
|
|
||||||
可能比兩次都粗(多對皆裁同組),也可能比兩次都細
|
|
||||||
(多對皆裁分開)。
|
|
||||||
5. **一致率與保險絲**:比對程式計算塊對層級的一致率並
|
|
||||||
記入歸檔。低於門檻(暫訂 50%,首輪實跑後校準)即
|
|
||||||
不進行仲裁——兩次分組面目全非說明定義檔約束不足,
|
|
||||||
依協定修訂定義檔並重跑該循環。
|
|
||||||
6. **cap 的上限容忍**:仲裁後組數可能略超 50(多對裁
|
|
||||||
「分開」時)。略微超過可接受,如實記錄,不強行
|
|
||||||
壓縮。
|
|
||||||
7. **組名定案(收斂不命名+命名 2+1)**:merge、cap 的
|
|
||||||
執行輸出不含組名(無名分組,陣列之陣列)——命名鏈
|
|
||||||
既然存在,收斂執行自取的名字只會製造兩次執行間
|
|
||||||
「同組不同名」的假分歧。組名承重——merge 的組名是
|
|
||||||
cap 的輸入詞,cap 的組名是 code 對歌詞編碼的類目——
|
|
||||||
故與其他語意判斷同受 2+1。程序:(1) 最終分組定案後,
|
|
||||||
命名步驟(`01-02-04-merge-name.md`、
|
|
||||||
`01-03-04-cap-name.md`)獨立執行兩次,輸入為不透明
|
|
||||||
組 ID 對成員詞,對全部組自由命名,僅受格式約束
|
|
||||||
(小寫連字號、單次執行內不重複);(2) 逐組比對:兩次
|
|
||||||
同名即定案(單次執行內名字唯一,故定名間必無撞名);
|
|
||||||
(3) 異名的組送擇一仲裁(`01-02-05-merge-name-arb.md`、
|
|
||||||
`01-03-05-cap-name-arb.md`)——匿名呈現兩候選,逐組
|
|
||||||
擇一,輸出限於候選並避讓已定名,機械可驗證。cap 的
|
|
||||||
輸入即 merge 定案後的組名清單。
|
|
||||||
|
|
||||||
### 收斂步驟的資料流(比對子命令的輸入輸出契約)
|
|
||||||
|
|
||||||
以 merge 為例(cap 完全同構,檔名換為 01-03 系):
|
|
||||||
|
|
||||||
1. 兩次執行的原始輸出:`runs/01-02-01-merge/run1/output.jsonl`
|
|
||||||
與 `run2/output.jsonl`,各含一個無名分組(陣列之陣列)。
|
|
||||||
2. 比對子命令讀入兩份分組,先驗證兩者為同一輸入詞集的
|
|
||||||
完整分割(缺詞、多詞、重複即失敗),再計算共識塊與
|
|
||||||
分歧塊對,產出仲裁輸入檔——即
|
|
||||||
`01-02-02-merge-arb.md` 所收的 JSON:
|
|
||||||
- `blocks`:塊 ID → 成員詞。**塊 ID 的指派決定性**:
|
|
||||||
全部塊先按「各塊字典序最小的成員詞」排序,依序編為
|
|
||||||
b1、b2、…;塊內成員詞亦按字典序排列。
|
|
||||||
- `pairs`:分歧塊對清單,每對內部按塊 ID 序、清單
|
|
||||||
整體按 (第一元素, 第二元素) 字典序排列。
|
|
||||||
- 僅分歧塊對入列;共識(同組或分開)不送仲裁,由
|
|
||||||
比對子命令直接寫入共識紀錄檔。
|
|
||||||
同時產出:共識紀錄(共識塊、共識同組對、共識分開對)
|
|
||||||
與塊對一致率(含保險絲判定)。
|
|
||||||
3. 仲裁輸入檔以 run-llm 跑兩次
|
|
||||||
(`runs/01-02-02-merge-arb/run1`、`run2`);比對
|
|
||||||
子命令逐對比對兩份裁決,兩票相同即定案,相異的塊對
|
|
||||||
依同一契約組成終局票輸入檔(`blocks` 僅含涉事塊、
|
|
||||||
ID 沿用原編號),跑
|
|
||||||
`runs/01-02-03-merge-arb-arb/run1`。
|
|
||||||
4. 裁決套用子命令彙整三票結果,以 union-find 重組出
|
|
||||||
最終分組,並寫出:最終分組檔(無名,陣列之陣列,
|
|
||||||
組序與組內成員皆字典序)、遞移導出合併的紀錄、
|
|
||||||
收斂軌跡(原始詞 → 最終組)。
|
|
||||||
5. 最終分組轉為不透明組 ID(依組序編 g1、g2、…)進入
|
|
||||||
命名鏈(見第 7 條);命名定案後,組名清單(字典序)
|
|
||||||
即下一步的輸入。
|
|
||||||
6. 以上中間檔全部隨 `runs/` 歸檔;一切排序規則固定,
|
|
||||||
故給定相同的兩份執行輸出與相同的裁決,全流程輸出
|
|
||||||
逐位元組可再生。
|
|
||||||
|
|
||||||
## 編碼步驟(code、screen)的 2+1 比對與仲裁
|
## 編碼步驟(code、screen)的 2+1 比對與仲裁
|
||||||
|
|
||||||
@@ -120,6 +48,9 @@ screen 與各仲裁步驟;merge、cap 及其仲裁、命名皆不接觸
|
|||||||
引述陣列。
|
引述陣列。
|
||||||
- 仲裁者的引述可能與原引述不同:仲裁是對歌詞的重新
|
- 仲裁者的引述可能與原引述不同:仲裁是對歌詞的重新
|
||||||
判讀,其引述是該裁決自身的依據,非轉抄。
|
判讀,其引述是該裁決自身的依據,非轉抄。
|
||||||
|
- 送入仲裁的標籤即模型自身不穩定的邊界判斷,其裁決為
|
||||||
|
單次記錄性決定,不宣稱可再生;可重現性依計畫定義為
|
||||||
|
「程序透明+可稽核」,裁決與其輸入全程歸檔。
|
||||||
|
|
||||||
## 軌跡對映(診斷用)
|
## 軌跡對映(診斷用)
|
||||||
|
|
||||||
@@ -130,7 +61,7 @@ screen 與各仲裁步驟;merge、cap 及其仲裁、命名皆不接觸
|
|||||||
## 全管線的交接契約
|
## 全管線的交接契約
|
||||||
|
|
||||||
每一步的輸出如何變成下一步的輸入,皆為確定性程序,規則
|
每一步的輸出如何變成下一步的輸入,皆為確定性程序,規則
|
||||||
明定如下(收斂步驟內部的交接見上節):
|
明定如下:
|
||||||
|
|
||||||
- **歌詞輸入檔(tag、code、screen 共用)**:
|
- **歌詞輸入檔(tag、code、screen 共用)**:
|
||||||
`export-llm-input` 自工作儲存產出,每筆
|
`export-llm-input` 自工作儲存產出,每筆
|
||||||
@@ -142,9 +73,9 @@ screen 與各仲裁步驟;merge、cap 及其仲裁、命名皆不接觸
|
|||||||
進池同時寫出處記錄(關鍵字 →(執行別,歌曲 ID)
|
進池同時寫出處記錄(關鍵字 →(執行別,歌曲 ID)
|
||||||
清單),供軌跡對映回到歌曲;出處記錄不進任何 LLM
|
清單),供軌跡對映回到歌曲;出處記錄不進任何 LLM
|
||||||
輸入。
|
輸入。
|
||||||
- **merge 定案 → cap**:merge 定案組名以字典序排成
|
- **merge → cap**:merge 輸出的組名以字典序排成 JSON
|
||||||
JSON 陣列,即 cap 的輸入(見上節第 5 點)。
|
陣列,即 cap 的輸入;原始關鍵字不下傳。
|
||||||
- **cap 定案 → code 定義檔**:定案組名以字典序逐行填入
|
- **cap → code 定義檔**:cap 輸出的組名以字典序逐行填入
|
||||||
`01-04-01-code.md` 的詞彙表節(逐字),檔案隨 git
|
`01-04-01-code.md` 的詞彙表節(逐字),檔案隨 git
|
||||||
commit 後方可執行——code 的定義檔因此自我完備,
|
commit 後方可執行——code 的定義檔因此自我完備,
|
||||||
論文附錄可直接引用。
|
論文附錄可直接引用。
|
||||||
@@ -161,11 +92,6 @@ screen 與各仲裁步驟;merge、cap 及其仲裁、命名皆不接觸
|
|||||||
入仲裁輸入 JSONL(依 ID 升序),`content` 為
|
入仲裁輸入 JSONL(依 ID 升序),`content` 為
|
||||||
`{"lyrics": …, "evidence": <肯定方引述>}`。
|
`{"lyrics": …, "evidence": <肯定方引述>}`。
|
||||||
- **screen 定案**:命中集合=兩次皆有 ∪ 仲裁裁定有。
|
- **screen 定案**:命中集合=兩次皆有 ∪ 仲裁裁定有。
|
||||||
- **命名鏈的輸入構成**:`-name` 輸入的組 ID 依最終分組
|
|
||||||
之組序(上節第 5 點)編 g1、g2、…,組內成員字典序;
|
|
||||||
`-name-arb` 輸入中每組的兩個候選名**按字典序排列**
|
|
||||||
——不按執行別,避免順序洩漏何方所取;`taken` 為已
|
|
||||||
定案名的字典序清單。
|
|
||||||
- **女性力量候選集**:於 cap 詞彙表定案後、黃金標準
|
- **女性力量候選集**:於 cap 詞彙表定案後、黃金標準
|
||||||
編碼開始前,由研究者指認詞彙表中屬「女性力量」概念
|
編碼開始前,由研究者指認詞彙表中屬「女性力量」概念
|
||||||
域的組(指認及理由記入決策日誌),候選集=code 定案
|
域的組(指認及理由記入決策日誌),候選集=code 定案
|
||||||
@@ -173,20 +99,20 @@ screen 與各仲裁步驟;merge、cap 及其仲裁、命名皆不接觸
|
|||||||
- **序列化通則**:所有中間檔為 UTF-8 JSON,鍵序與元素
|
- **序列化通則**:所有中間檔為 UTF-8 JSON,鍵序與元素
|
||||||
序皆依上列規則明定,無時間戳、無隨機成分;解析一律
|
序皆依上列規則明定,無時間戳、無隨機成分;解析一律
|
||||||
偵測重複鍵,違規即失敗。JSONL 一律以換行字元(\n)
|
偵測重複鍵,違規即失敗。JSONL 一律以換行字元(\n)
|
||||||
切行——歌詞含 U+2028 等 Unicode 行分隔符,
|
切行——歌詞含 U+0085 等控制字元時,
|
||||||
`str.splitlines()` 類的通用切行會截斷 JSON 字串
|
`str.splitlines()` 類的通用切行會截斷 JSON 字串
|
||||||
(實測踩中)。給定相同的 LLM 執行輸出,
|
(實測踩中)。給定相同的 LLM 執行輸出,全部交接產物
|
||||||
全部交接產物逐位元組可再生。
|
逐位元組可再生。
|
||||||
|
|
||||||
## 執行與稽核
|
## 執行與稽核
|
||||||
|
|
||||||
- 每一步驟以 `run-llm <定義檔> <輸入檔> <歸檔目錄>`
|
- 每一步驟以 `run-llm <定義檔> <輸入檔> <歸檔目錄>`
|
||||||
執行;獨立執行兩次=重現命令清單上的兩行命令,各自
|
執行;2+1 步驟的兩次執行=重現命令清單上的兩行命令,
|
||||||
歸檔(`runs/<定義檔名>/run1`、`run2`),仲裁與命名
|
各自歸檔(`runs/<定義檔名>/run1`、`run2`),仲裁為
|
||||||
各為獨立步驟、獨立歸檔。
|
獨立步驟、獨立歸檔;單次步驟(merge、cap)一行命令、
|
||||||
- 比對、裁決套用、重組、對映皆為確定性程式(子命令),
|
一份歸檔。
|
||||||
其輸入輸出檔隨 runs/ 歸檔,JSON 解析一律偵測重複鍵,
|
- 進池、比對、裁決套用、對映皆為確定性程式(子命令),
|
||||||
違規即失敗。
|
其輸入輸出檔隨 runs/ 歸檔。
|
||||||
- Batch API 的每筆請求自含全部脈絡且互不可見(平台
|
- Batch API 的每筆請求自含全部脈絡且互不可見(平台
|
||||||
契約),歌與歌之間的獨立性由此成立;兩次執行的獨立
|
契約),歌與歌之間的獨立性由此成立;兩次執行的獨立
|
||||||
性由「兩次呼叫、兩個批次、兩份歸檔」的執行結構自明。
|
性由「兩次呼叫、兩個批次、兩份歸檔」的執行結構自明。
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ pop-fem-audit/
|
|||||||
│ └── artists.csv # 歌手報表(人讀;進 git)
|
│ └── artists.csv # 歌手報表(人讀;進 git)
|
||||||
├── prompts/ # LLM 定義檔(逐字作為 system prompt)
|
├── prompts/ # LLM 定義檔(逐字作為 system prompt)
|
||||||
│ └── <軌>-<步>-<次步>-<task>.md # 01-01-01-tag.md、
|
│ └── <軌>-<步>-<次步>-<task>.md # 01-01-01-tag.md、
|
||||||
│ # 01-02-02-merge-arb.md…
|
│ # 01-04-02-code-arb.md…
|
||||||
│ # 不帶版本號,版本即 git 歷史
|
│ # 不帶版本號,版本即 git 歷史
|
||||||
├── tools/ # 輔助工具子專案(src-layout)
|
├── tools/ # 輔助工具子專案(src-layout)
|
||||||
│ ├── pyproject.toml # 發行名 pop-fem-audit-tools;
|
│ ├── pyproject.toml # 發行名 pop-fem-audit-tools;
|
||||||
|
|||||||
+24
-24
@@ -13,11 +13,14 @@
|
|||||||
- **執行原則**:主會話只做討論;所有分析由 deterministic script
|
- **執行原則**:主會話只做討論;所有分析由 deterministic script
|
||||||
執行。LLM 步驟以 Python script 呼叫 Anthropic Messages API
|
執行。LLM 步驟以 Python script 呼叫 Anthropic Messages API
|
||||||
(個人 Console 帳號、Batch API 五折),定義檔逐字作為 system
|
(個人 Console 帳號、Batch API 五折),定義檔逐字作為 system
|
||||||
prompt。2+1 協定依輸出可比性適用:輸出可逐項機械比對的步驟
|
prompt。2+1 協定適用於語料層的逐首判斷(編碼、篩選):
|
||||||
(收斂、編碼)「同一定義檔獨立執行兩次+一次仲裁」,仲裁
|
「同一定義檔獨立執行兩次+一次仲裁」,仲裁只裁程式算出
|
||||||
只裁程式算出的分歧清單;自由生成步驟(首步自由標註)兩次
|
的分歧。自由生成步驟(自由標註)兩次執行全數進池、不
|
||||||
執行全數進池、不仲裁——自由詞彙兩次輸出不共享比對單位,
|
仲裁——自由詞彙兩次輸出不共享比對單位,無物可裁。詞彙
|
||||||
無物可裁。仲裁結果不符預期則修訂定義檔重跑該循環。
|
表建構(收斂)為單次記錄性程序,不重複執行——詞彙表是
|
||||||
|
揭露的儀器選擇,非量測;信度檢驗施於編碼層(詳見
|
||||||
|
`methodology.md`)。仲裁或驗證結果不符預期則修訂定義檔
|
||||||
|
重跑該循環。
|
||||||
- **提示詞只定格式、不定語意**:研究對象是通用 LLM 以其
|
- **提示詞只定格式、不定語意**:研究對象是通用 LLM 以其
|
||||||
網路語料知識背景所做的自然編碼,編碼結果本身是批判對象。
|
網路語料知識背景所做的自然編碼,編碼結果本身是批判對象。
|
||||||
LLM 定義檔只規定任務形狀(輸入、數量範圍、輸出格式),
|
LLM 定義檔只規定任務形狀(輸入、數量範圍、輸出格式),
|
||||||
@@ -95,13 +98,12 @@
|
|||||||
themes 過早抽象)之繼承,於執行前鎖定,防止事後擇優。
|
themes 過早抽象)之繼承,於執行前鎖定,防止事後擇優。
|
||||||
2. **自然收斂(merge)**:輸入為池中純去重關鍵字清單——
|
2. **自然收斂(merge)**:輸入為池中純去重關鍵字清單——
|
||||||
無歌詞、無頻次、無歌曲出處——模型按自身理解合併近似
|
無歌詞、無頻次、無歌曲出處——模型按自身理解合併近似
|
||||||
概念。2+1:兩次收斂處理同一批輸入詞,程式算出共識塊
|
概念,執行內自行為各組命名。單次執行,不仲裁;確定性
|
||||||
與分歧塊對,逐對仲裁(仲裁自身亦 2+1,第三票終局)後
|
驗證(完整分割、組名唯一)違規即修訂重跑(詳見
|
||||||
以連通元件決定性重組;收斂執行不命名,組名由命名步驟
|
`methodology.md`)。
|
||||||
2+1 定案(演算法詳見 `methodology.md`)。
|
|
||||||
3. **強制收斂(cap)**:同第 2 步形態,輸入為 merge
|
3. **強制收斂(cap)**:同第 2 步形態,輸入為 merge
|
||||||
定案後的組名清單(原始關鍵字不下傳),限制併至 50 個
|
輸出的組名清單(原始關鍵字不下傳),限制併至 50 個
|
||||||
以內;仲裁後略超 50 可接受,如實記錄。頻次不入收斂:頻率的分析角色由第 4 步編碼承擔;
|
以內。單次執行,不仲裁。頻次不入收斂:頻率的分析角色由第 4 步編碼承擔;
|
||||||
池中頻次含跨執行噪音;頻次會誘使模型以頻率剪枝(喪失
|
池中頻次含跨執行噪音;頻次會誘使模型以頻率剪枝(喪失
|
||||||
稀有主題)並把高頻大主題切細。代價(特異主題佔名額)
|
稀有主題)並把高頻大主題切細。代價(特異主題佔名額)
|
||||||
已知並接受,換取主題多樣性與純語意歸併的可辯護性。
|
已知並接受,換取主題多樣性與純語意歸併的可辯護性。
|
||||||
@@ -123,18 +125,16 @@
|
|||||||
提示中唯一的語意種子,屬研究者的儀器選擇,據實揭露。
|
提示中唯一的語意種子,屬研究者的儀器選擇,據實揭露。
|
||||||
|
|
||||||
定義檔命名 `prompts/<軌>-<步>-<次步>-<task>.md`——軌
|
定義檔命名 `prompts/<軌>-<步>-<次步>-<task>.md`——軌
|
||||||
01=由下而上自然編碼、02=預先決定的 women-power
|
01=由下而上自然編碼、02=預先決定的 women-power 篩選;
|
||||||
篩選;步為軌內步驟序,次步為
|
步為軌內步驟序,次步為步內執行順序,讀者依編號先後依循:
|
||||||
步內執行順序,讀者依編號先後依循(如 01-02-01-merge.md →
|
01-01-01-tag.md、01-02-01-merge.md、01-03-01-cap.md、
|
||||||
01-02-02-merge-arb.md → 01-02-03-merge-arb-arb.md →
|
01-04-01-code.md、01-04-02-code-arb.md、
|
||||||
01-02-04-merge-name.md → 01-02-05-merge-name-arb.md)。
|
02-01-01-screen.md、02-01-02-screen-arb.md;仲裁定義檔
|
||||||
後綴 `-arb` 為仲裁、`-arb-arb` 為仲裁之終局票、`-name`
|
同 prefix 加 `-arb`。檔名不帶版本號——版本即 git 歷史,
|
||||||
為命名、`-name-arb` 為命名擇一仲裁。檔名不帶
|
失敗的版本不保留,需要回看的舊版都在 git history;每次
|
||||||
版本號——版本即 git 歷史,失敗的版本不保留,需要回看的
|
執行的定義檔快照隨 `runs/` 自我完備。收斂執行的輸出本身
|
||||||
舊版都在 git history;每次執行的定義檔快照隨 `runs/`
|
即分組記錄(具名分組 JSON),與全部中間交接檔同隨
|
||||||
自我完備。收斂執行的輸出本身即分組記錄(無名分組
|
`runs/` 歸檔(交接契約見 `methodology.md`)。
|
||||||
JSON),與全部中間交接檔同隨 `runs/` 歸檔(交接契約見
|
|
||||||
`methodology.md`)。
|
|
||||||
|
|
||||||
## 階段與時程(全文截稿 2026-08-15)
|
## 階段與時程(全文截稿 2026-08-15)
|
||||||
|
|
||||||
@@ -142,7 +142,7 @@ JSON),與全部中間交接檔同隨 `runs/` 歸檔(交接契約見
|
|||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| 0 | 基礎建設:git init、目錄結構、.gitignore、決策日誌、runner script(含 Batch API)、codebook v0 骨架 | script + 討論 | 7/30–7/31 |
|
| 0 | 基礎建設:git init、目錄結構、.gitignore、決策日誌、runner script(含 Batch API)、codebook v0 骨架 | script + 討論 | 7/30–7/31 |
|
||||||
| 1 | 資料準備:`run_llm` 改走統一設定 → `build-db`(解析榜單成 songs/chart_entries/artists/song_artists)→ `import-lyrics`(pilot 2018–2025)→ `fetch-lyrics`(2016–17 與缺漏,Lyrics.ovh / LRCLIB)→ `fetch-artists`(Wikidata 快照)→ `export-llm-input` | 子命令 | 7/31–8/3 |
|
| 1 | 資料準備:`run_llm` 改走統一設定 → `build-db`(解析榜單成 songs/chart_entries/artists/song_artists)→ `import-lyrics`(pilot 2018–2025)→ `fetch-lyrics`(2016–17 與缺漏,Lyrics.ovh / LRCLIB)→ `fetch-artists`(Wikidata 快照)→ `export-llm-input` | 子命令 | 7/31–8/3 |
|
||||||
| 2 | 自然編碼管線:tag ×2 進池 → merge 2+1 → cap 2+1 → 詞彙表定稿 → code 2+1(全 883 首,附引述);另跑 screen 補漏網 | API + script | 8/4–8/7 |
|
| 2 | 自然編碼管線:tag ×2 進池 → merge ×1 → cap ×1 → 詞彙表定稿 → code 2+1(全 883 首,附引述);另跑 screen 補漏網 | API + script | 8/4–8/7 |
|
||||||
| 3 | 黃金標準:依 codebook 人工逐首判定 genuine/peripheral/fake,附引用歌詞證據表(LLM 只做摘錄,不給判定建議);先以 10–15 首校準樣本試編並修訂 codebook 後凍結;同批校準樣本實測 Sonnet 4.6 vs Opus 5 一致率 | 人工 + script 輔助 | 8/5–8/9 |
|
| 3 | 黃金標準:依 codebook 人工逐首判定 genuine/peripheral/fake,附引用歌詞證據表(LLM 只做摘錄,不給判定建議);先以 10–15 首校準樣本試編並修訂 codebook 後凍結;同批校準樣本實測 Sonnet 4.6 vs Opus 5 一致率 | 人工 + script 輔助 | 8/5–8/9 |
|
||||||
| 4 | 受控比較(盲點實驗):條件 A(詞彙層提示)vs 條件 B(框架感知提示),各 2+1,對照黃金標準計算假陽/假陰率 | API | 8/8–8/11 |
|
| 4 | 受控比較(盲點實驗):條件 A(詞彙層提示)vs 條件 B(框架感知提示),各 2+1,對照黃金標準計算假陽/假陰率 | API | 8/8–8/11 |
|
||||||
| 4' | 映射分析:自然編碼結果(第 4 步)與黃金標準交叉表;軌跡對映 vs 直接編碼的扭曲診斷(分析方法先寫入 methodology.md 再看結果) | script | 與 4 並行 |
|
| 4' | 映射分析:自然編碼結果(第 4 步)與黃金標準交叉表;軌跡對映 vs 直接編碼的扭曲診斷(分析方法先寫入 methodology.md 再看結果) | script | 與 4 並行 |
|
||||||
|
|||||||
@@ -13,11 +13,14 @@ Rules:
|
|||||||
- Do not invent keywords that are not in the input.
|
- Do not invent keywords that are not in the input.
|
||||||
- Merge only what you consider the same theme; a group may be
|
- Merge only what you consider the same theme; a group may be
|
||||||
a single keyword.
|
a single keyword.
|
||||||
|
- Name each group with a short lowercase phrase, words joined
|
||||||
|
by hyphens, that best names the theme it gathers. Group
|
||||||
|
names must be unique.
|
||||||
|
|
||||||
Output a single JSON array of groups — each group an array of
|
Output a single JSON object mapping each group name to the
|
||||||
its member keywords — and nothing else:
|
list of its member keywords, and nothing else:
|
||||||
|
|
||||||
[
|
{
|
||||||
["keyword", "another-keyword"],
|
"first-group-name": ["member-keyword", "another-member"],
|
||||||
["keyword"]
|
"second-group-name": ["member-keyword"]
|
||||||
]
|
}
|
||||||
|
|||||||
@@ -1,36 +0,0 @@
|
|||||||
You are arbitrating between two independent consolidations of
|
|
||||||
the same vocabulary of thematic keywords from a song corpus.
|
|
||||||
|
|
||||||
Where the two consolidations agree, the result has been
|
|
||||||
settled by script and is not shown. You rule on the
|
|
||||||
disagreements: pairs of keyword blocks that one consolidation
|
|
||||||
merged into one theme and the other kept apart.
|
|
||||||
|
|
||||||
Input: a JSON object. "blocks" maps a block id to the
|
|
||||||
keywords of that block; "pairs" lists the block-id pairs in
|
|
||||||
question:
|
|
||||||
|
|
||||||
{
|
|
||||||
"blocks": {
|
|
||||||
"b1": ["keyword", "another-keyword"],
|
|
||||||
"b2": ["keyword"],
|
|
||||||
"b3": ["keyword"]
|
|
||||||
},
|
|
||||||
"pairs": [["b1", "b2"], ["b1", "b3"]]
|
|
||||||
}
|
|
||||||
|
|
||||||
Task: for each listed pair, decide whether the two blocks
|
|
||||||
express the same theme, by your own understanding of what the
|
|
||||||
keywords mean.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
|
|
||||||
- Judge every listed pair, each on its own merits from the
|
|
||||||
block contents alone.
|
|
||||||
- Judge only the listed pairs.
|
|
||||||
|
|
||||||
Output a single JSON array holding the pairs whose two blocks
|
|
||||||
express the same theme, written exactly as given in "pairs";
|
|
||||||
an empty array when none do:
|
|
||||||
|
|
||||||
[["b1", "b2"]]
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
You are casting the deciding judgment on disagreements about
|
|
||||||
a consolidated vocabulary of thematic keywords from a song
|
|
||||||
corpus: for each pair of keyword blocks in question, earlier
|
|
||||||
independent judgments disagreed on whether the two blocks
|
|
||||||
express the same theme.
|
|
||||||
|
|
||||||
Input: a JSON object. "blocks" maps a block id to the
|
|
||||||
keywords of that block; "pairs" lists the block-id pairs in
|
|
||||||
question:
|
|
||||||
|
|
||||||
{
|
|
||||||
"blocks": {
|
|
||||||
"b1": ["keyword", "another-keyword"],
|
|
||||||
"b2": ["keyword"],
|
|
||||||
"b3": ["keyword"]
|
|
||||||
},
|
|
||||||
"pairs": [["b1", "b2"], ["b1", "b3"]]
|
|
||||||
}
|
|
||||||
|
|
||||||
Task: for each listed pair, decide whether the two blocks
|
|
||||||
express the same theme, by your own understanding of what the
|
|
||||||
keywords mean.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
|
|
||||||
- Judge every listed pair, each on its own merits from the
|
|
||||||
block contents alone.
|
|
||||||
- Judge only the listed pairs.
|
|
||||||
|
|
||||||
Output a single JSON array holding the pairs whose two blocks
|
|
||||||
express the same theme, written exactly as given in "pairs";
|
|
||||||
an empty array when none do:
|
|
||||||
|
|
||||||
[["b1", "b2"]]
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
You are naming the theme groups of a consolidated vocabulary
|
|
||||||
of thematic keywords from a song corpus.
|
|
||||||
|
|
||||||
Input: a JSON object mapping an opaque group id to the member
|
|
||||||
keywords of that group:
|
|
||||||
|
|
||||||
{
|
|
||||||
"g1": ["keyword", "another-keyword"],
|
|
||||||
"g2": ["keyword"]
|
|
||||||
}
|
|
||||||
|
|
||||||
Task: give each group a name that best names the theme its
|
|
||||||
members gather, by your own understanding of what the
|
|
||||||
keywords mean.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
|
|
||||||
- Each name is a short lowercase phrase with the words joined
|
|
||||||
by hyphens.
|
|
||||||
- Names must be unique across the groups.
|
|
||||||
|
|
||||||
Output a single JSON object mapping each group id to its
|
|
||||||
name, and nothing else:
|
|
||||||
|
|
||||||
{
|
|
||||||
"g1": "group-name",
|
|
||||||
"g2": "another-group-name"
|
|
||||||
}
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
You are arbitrating between two independent namings of the
|
|
||||||
theme groups of a consolidated vocabulary of thematic
|
|
||||||
keywords from a song corpus.
|
|
||||||
|
|
||||||
For each group in question, the two namings proposed
|
|
||||||
different names. You choose between them.
|
|
||||||
|
|
||||||
Input: a JSON object. "groups" maps an opaque group id to
|
|
||||||
its member keywords and its two candidate names; "taken"
|
|
||||||
lists names that are already in use:
|
|
||||||
|
|
||||||
{
|
|
||||||
"groups": {
|
|
||||||
"g1": {
|
|
||||||
"members": ["keyword", "another-keyword"],
|
|
||||||
"candidates": ["one-name", "other-name"]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"taken": ["existing-name"]
|
|
||||||
}
|
|
||||||
|
|
||||||
Task: for each group, choose the candidate that better names
|
|
||||||
the theme its members gather, by your own understanding of
|
|
||||||
what the keywords mean.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
|
|
||||||
- Choose only from that group's two candidates.
|
|
||||||
- Choices must be unique across the groups and must not
|
|
||||||
reuse any name in "taken".
|
|
||||||
|
|
||||||
Output a single JSON object mapping each group id to the
|
|
||||||
chosen name, and nothing else:
|
|
||||||
|
|
||||||
{
|
|
||||||
"g1": "one-name"
|
|
||||||
}
|
|
||||||
@@ -16,11 +16,14 @@ Rules:
|
|||||||
- Merge the keywords that are closest in meaning first; keep
|
- Merge the keywords that are closest in meaning first; keep
|
||||||
clearly distinct themes apart as far as the limit allows.
|
clearly distinct themes apart as far as the limit allows.
|
||||||
A group may be a single keyword.
|
A group may be a single keyword.
|
||||||
|
- Name each group with a short lowercase phrase, words joined
|
||||||
|
by hyphens, that best names the theme it gathers. Group
|
||||||
|
names must be unique.
|
||||||
|
|
||||||
Output a single JSON array of groups — each group an array of
|
Output a single JSON object mapping each group name to the
|
||||||
its member keywords — and nothing else:
|
list of its member keywords, and nothing else:
|
||||||
|
|
||||||
[
|
{
|
||||||
["keyword", "another-keyword"],
|
"first-group-name": ["member-keyword", "another-member"],
|
||||||
["keyword"]
|
"second-group-name": ["member-keyword"]
|
||||||
]
|
}
|
||||||
|
|||||||
@@ -1,37 +0,0 @@
|
|||||||
You are arbitrating between two independent consolidations
|
|
||||||
that each reduced the same vocabulary of thematic keywords
|
|
||||||
from a song corpus to a fixed maximum number of themes.
|
|
||||||
|
|
||||||
Where the two consolidations agree, the result has been
|
|
||||||
settled by script and is not shown. You rule on the
|
|
||||||
disagreements: pairs of keyword blocks that one consolidation
|
|
||||||
merged into one theme and the other kept apart.
|
|
||||||
|
|
||||||
Input: a JSON object. "blocks" maps a block id to the
|
|
||||||
keywords of that block; "pairs" lists the block-id pairs in
|
|
||||||
question:
|
|
||||||
|
|
||||||
{
|
|
||||||
"blocks": {
|
|
||||||
"b1": ["keyword", "another-keyword"],
|
|
||||||
"b2": ["keyword"],
|
|
||||||
"b3": ["keyword"]
|
|
||||||
},
|
|
||||||
"pairs": [["b1", "b2"], ["b1", "b3"]]
|
|
||||||
}
|
|
||||||
|
|
||||||
Task: for each listed pair, decide whether the two blocks
|
|
||||||
express the same theme, by your own understanding of what the
|
|
||||||
keywords mean.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
|
|
||||||
- Judge every listed pair, each on its own merits from the
|
|
||||||
block contents alone.
|
|
||||||
- Judge only the listed pairs.
|
|
||||||
|
|
||||||
Output a single JSON array holding the pairs whose two blocks
|
|
||||||
express the same theme, written exactly as given in "pairs";
|
|
||||||
an empty array when none do:
|
|
||||||
|
|
||||||
[["b1", "b2"]]
|
|
||||||
@@ -1,35 +0,0 @@
|
|||||||
You are casting the deciding judgment on disagreements about
|
|
||||||
a vocabulary of thematic keywords from a song corpus that was
|
|
||||||
consolidated to a fixed maximum number of themes: for each
|
|
||||||
pair of keyword blocks in question, earlier independent
|
|
||||||
judgments disagreed on whether the two blocks express the
|
|
||||||
same theme.
|
|
||||||
|
|
||||||
Input: a JSON object. "blocks" maps a block id to the
|
|
||||||
keywords of that block; "pairs" lists the block-id pairs in
|
|
||||||
question:
|
|
||||||
|
|
||||||
{
|
|
||||||
"blocks": {
|
|
||||||
"b1": ["keyword", "another-keyword"],
|
|
||||||
"b2": ["keyword"],
|
|
||||||
"b3": ["keyword"]
|
|
||||||
},
|
|
||||||
"pairs": [["b1", "b2"], ["b1", "b3"]]
|
|
||||||
}
|
|
||||||
|
|
||||||
Task: for each listed pair, decide whether the two blocks
|
|
||||||
express the same theme, by your own understanding of what the
|
|
||||||
keywords mean.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
|
|
||||||
- Judge every listed pair, each on its own merits from the
|
|
||||||
block contents alone.
|
|
||||||
- Judge only the listed pairs.
|
|
||||||
|
|
||||||
Output a single JSON array holding the pairs whose two blocks
|
|
||||||
express the same theme, written exactly as given in "pairs";
|
|
||||||
an empty array when none do:
|
|
||||||
|
|
||||||
[["b1", "b2"]]
|
|
||||||
@@ -1,29 +0,0 @@
|
|||||||
You are naming the theme groups of a vocabulary of thematic
|
|
||||||
keywords from a song corpus that was consolidated to a fixed
|
|
||||||
maximum number of themes.
|
|
||||||
|
|
||||||
Input: a JSON object mapping an opaque group id to the member
|
|
||||||
keywords of that group:
|
|
||||||
|
|
||||||
{
|
|
||||||
"g1": ["keyword", "another-keyword"],
|
|
||||||
"g2": ["keyword"]
|
|
||||||
}
|
|
||||||
|
|
||||||
Task: give each group a name that best names the theme its
|
|
||||||
members gather, by your own understanding of what the
|
|
||||||
keywords mean.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
|
|
||||||
- Each name is a short lowercase phrase with the words joined
|
|
||||||
by hyphens.
|
|
||||||
- Names must be unique across the groups.
|
|
||||||
|
|
||||||
Output a single JSON object mapping each group id to its
|
|
||||||
name, and nothing else:
|
|
||||||
|
|
||||||
{
|
|
||||||
"g1": "group-name",
|
|
||||||
"g2": "another-group-name"
|
|
||||||
}
|
|
||||||
@@ -1,38 +0,0 @@
|
|||||||
You are arbitrating between two independent namings of the
|
|
||||||
theme groups of a vocabulary of thematic keywords from a song
|
|
||||||
corpus that was consolidated to a fixed maximum number of
|
|
||||||
themes.
|
|
||||||
|
|
||||||
For each group in question, the two namings proposed
|
|
||||||
different names. You choose between them.
|
|
||||||
|
|
||||||
Input: a JSON object. "groups" maps an opaque group id to
|
|
||||||
its member keywords and its two candidate names; "taken"
|
|
||||||
lists names that are already in use:
|
|
||||||
|
|
||||||
{
|
|
||||||
"groups": {
|
|
||||||
"g1": {
|
|
||||||
"members": ["keyword", "another-keyword"],
|
|
||||||
"candidates": ["one-name", "other-name"]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"taken": ["existing-name"]
|
|
||||||
}
|
|
||||||
|
|
||||||
Task: for each group, choose the candidate that better names
|
|
||||||
the theme its members gather, by your own understanding of
|
|
||||||
what the keywords mean.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
|
|
||||||
- Choose only from that group's two candidates.
|
|
||||||
- Choices must be unique across the groups and must not
|
|
||||||
reuse any name in "taken".
|
|
||||||
|
|
||||||
Output a single JSON object mapping each group id to the
|
|
||||||
chosen name, and nothing else:
|
|
||||||
|
|
||||||
{
|
|
||||||
"g1": "one-name"
|
|
||||||
}
|
|
||||||
Reference in New Issue
Block a user