Compare commits
114
Commits
v1.0.0
..
0ae0a90750
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0ae0a90750 | ||
|
|
9a48446129 | ||
|
|
20fa83576c | ||
|
|
5990c15dde | ||
|
|
46835bfb77 | ||
|
|
76a90cb8ae | ||
|
|
e5e323f938 | ||
|
|
60a1803a06 | ||
|
|
71fd8ce9c6 | ||
|
|
5731782fe6 | ||
|
|
9756c07cad | ||
|
|
e9c29e5867 | ||
|
|
171d2f81fe | ||
|
|
df363e14f0 | ||
|
|
d591931101 | ||
|
|
03ee51912d | ||
|
|
87ad31568a | ||
|
|
ed54a1248f | ||
|
|
963504e9c3 | ||
|
|
64dd9b0cc3 | ||
|
|
41d2822063 | ||
|
|
70578b5d77 | ||
|
|
ff2a3a217f | ||
|
|
2c15c0bb8b | ||
|
|
5d10383dcb | ||
|
|
99f5c1c260 | ||
|
|
d370ac51dc | ||
|
|
c7b630c632 | ||
|
|
5194d4246b | ||
|
|
36b3456414 | ||
|
|
511abd0a55 | ||
|
|
047f322bf3 | ||
|
|
2d8a34ea9e | ||
|
|
1c78f4a918 | ||
|
|
71f9c713f0 | ||
|
|
95b843f76e | ||
|
|
76d5df3a87 | ||
|
|
a21e407e2f | ||
|
|
7a5176b8b3 | ||
|
|
045086f9ca | ||
|
|
87352a2530 | ||
|
|
bfb5208133 | ||
|
|
5018f12cd1 | ||
|
|
10fccca108 | ||
|
|
4dd82867fa | ||
|
|
8c06406b3a | ||
|
|
3a2419f43a | ||
|
|
8299906b51 | ||
|
|
adbd9c1a24 | ||
|
|
94971eb07d | ||
|
|
85db4dac12 | ||
|
|
c2ffa9d247 | ||
|
|
c997cc8d74 | ||
|
|
c1e099901d | ||
|
|
787ca10f29 | ||
|
|
1db46039fe | ||
|
|
d4c313734a | ||
|
|
fbb80180af | ||
|
|
def2affa95 | ||
|
|
c5ad09524b | ||
|
|
5b376e001d | ||
|
|
59f62cd4d4 | ||
|
|
864735352a | ||
|
|
1c4d5c5ed7 | ||
|
|
066ae15800 | ||
|
|
bf2b4970fd | ||
|
|
896ee09b21 | ||
|
|
a14e2a878a | ||
|
|
b0a79af1a8 | ||
|
|
0b995e5a51 | ||
|
|
4eb278164b | ||
|
|
d9014cf1f3 | ||
|
|
4a0a4ceead | ||
|
|
2e5ecaaad0 | ||
|
|
e858f7a38f | ||
|
|
5050b16812 | ||
|
|
93b8afc547 | ||
|
|
cba73a6ace | ||
|
|
bc98fdf171 | ||
|
|
9ef84cfe54 | ||
|
|
09a8381519 | ||
|
|
cf80a6f51e | ||
|
|
0b77310d86 | ||
|
|
5092e04d34 | ||
|
|
376916fb8c | ||
|
|
ce8a217e94 | ||
|
|
14cc80e662 | ||
|
|
21c5ae5481 | ||
|
|
383670ab2b | ||
|
|
c59a023095 | ||
|
|
10dc4cb693 | ||
|
|
449675e7a2 | ||
|
|
a7806c0c9d | ||
|
|
105664e93b | ||
|
|
1220c21807 | ||
|
|
86914de850 | ||
|
|
8463cfc906 | ||
|
|
ef4112c61e | ||
|
|
1eaebeb5fb | ||
|
|
6fc4b7db17 | ||
|
|
c94b6f2589 | ||
|
|
b3ac12c88d | ||
|
|
5b2c192827 | ||
|
|
991ab5d93a | ||
|
|
78cf5e6ce0 | ||
|
|
aa78a54d12 | ||
|
|
3d472d5c15 | ||
|
|
2f048dee36 | ||
|
|
a3062e848c | ||
|
|
1b2d46cb42 | ||
|
|
f2ccca60b5 | ||
|
|
db7b5f6679 | ||
|
|
98b665537f | ||
|
|
9a089f6c6b |
@@ -0,0 +1,68 @@
|
|||||||
|
# Project Conventions
|
||||||
|
|
||||||
|
## Analysis pipeline
|
||||||
|
|
||||||
|
- LLM analysis runs via Python scripts calling the Anthropic
|
||||||
|
Messages API, Batch API where possible. Steps 1 and 3 run
|
||||||
|
on `claude-sonnet-4-6` with `temperature=0` and thinking
|
||||||
|
disabled; steps 4 and 5 run on `claude-fable-5`, which
|
||||||
|
accepts neither parameter -- step 4 absorbs its sampling
|
||||||
|
variance by the majority vote, step 5 by consolidating the
|
||||||
|
three readings.
|
||||||
|
- Prompt definition files live in
|
||||||
|
`prompts/<step>-<substep>-<task>.md` (e.g. 01-tag.md; no
|
||||||
|
version suffix -- versions live in git history) and are
|
||||||
|
passed verbatim as the system prompt. The number names a
|
||||||
|
step of the research procedure, not the file: the
|
||||||
|
deterministic vocabulary step (step 2) has no definition
|
||||||
|
file yet holds its own number. Zero padding is for
|
||||||
|
sorting only -- prose says "step 1", "step 3".
|
||||||
|
- Itemwise LLM judgments (per-song coding in step 3,
|
||||||
|
per-keyword group selection in step 4) run the same
|
||||||
|
definition file three times, independently, over the same
|
||||||
|
input; a deterministic tally then assigns an item (a
|
||||||
|
(song, keyword) or (group, keyword) pair) when at least
|
||||||
|
two of the three runs assign it ("3 runs + majority
|
||||||
|
vote"). Free-generation steps run
|
||||||
|
twice and both outputs are pooled. The step-5 qualitative
|
||||||
|
readings are neither: three independent readings per song,
|
||||||
|
consolidated per song and synthesized across songs by
|
||||||
|
their own definition files -- a qualitative protocol, not
|
||||||
|
a vote (see docs/methodology.md). The vocabulary is
|
||||||
|
built by a deterministic subcommand (embedding +
|
||||||
|
clustering), not by an LLM. If a validation outcome is
|
||||||
|
unexpected, revise the definition file and repeat that
|
||||||
|
cycle; never patch results by hand.
|
||||||
|
- Each run of a step is archived self-contained under the
|
||||||
|
destination directory given explicitly on the `run-llm`
|
||||||
|
command line (by convention `runs/<step>/run<N>/`):
|
||||||
|
prompt snapshot, raw output, and `meta.json` (model ID,
|
||||||
|
parameters, timestamps, batch ID). The runs of a step are
|
||||||
|
that many separate invocations of `run-llm`. Replacing
|
||||||
|
an existing run archive requires an explicit flag;
|
||||||
|
superseded runs live in git history. Deterministic steps
|
||||||
|
archive under `runs/<step>/` with no `run<N>` level.
|
||||||
|
- Token usage and cost of every `run-llm` execution are
|
||||||
|
recorded in `docs/run-costs.md` in the same commit as the
|
||||||
|
run archive.
|
||||||
|
- Scripts read the API key from the `ANTHROPIC_API_KEY`
|
||||||
|
environment variable (`.env`, gitignored).
|
||||||
|
|
||||||
|
## Data rules
|
||||||
|
|
||||||
|
- `data/source/` holds the immutable hand-placed raw files;
|
||||||
|
`data/captures/` is written only by the fetch commands and the
|
||||||
|
private import script; `data/manual/` is written only by the
|
||||||
|
user's own hand; `data/derived/` is written only by the
|
||||||
|
`build-db` subcommand.
|
||||||
|
- Full lyrics are copyrighted: they stay in `data/captures/lyrics/`
|
||||||
|
(gitignored) and must never be committed or reproduced in
|
||||||
|
full anywhere in the repo.
|
||||||
|
|
||||||
|
## Documents
|
||||||
|
|
||||||
|
- `results/` holds the final tallied tables (what the paper
|
||||||
|
cites); `runs/` holds raw audit records. The paper cites
|
||||||
|
`results/` only.
|
||||||
|
- Any change to a definition file, the codebook, or the plan is
|
||||||
|
recorded in `docs/decision-log.md` with date and reason.
|
||||||
@@ -1,26 +1,39 @@
|
|||||||
# 流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End Hot 100(2016-2025)為例的內容分析
|
# 流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End Hot 100(2018–2025)為例的內容分析。
|
||||||
|
|
||||||
這是研究論文《流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End Hot 100(2016-2025)為例的內容分析》的專案資料,包括論文本身、文件、紀錄、資料、工具程式、LLM提示詞,等等。
|
流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End
|
||||||
|
Hot 100(2016–2025)為例的內容分析。
|
||||||
|
|
||||||
## 論文和摘要
|
台灣女性學學會 2026 年會論文之研究資料與分析程式。
|
||||||
|
|
||||||
研究論文全文和摘要,請參閱 paper/ 資料夾。
|
## 目錄結構
|
||||||
|
|
||||||
## 文件說明
|
見 `docs/project-structure.md`。研究步驟規劃見
|
||||||
|
`docs/research-plan.md`;方法細節見 `docs/methodology.md`;
|
||||||
|
人工編碼手冊見 `docs/codebook.md`;決策日誌見
|
||||||
|
`docs/decision-log.md`。
|
||||||
|
|
||||||
研究方法、記錄等文件,請參閱 docs/ 資料夾。
|
## 重現方式
|
||||||
|
|
||||||
## LLM提示
|
1. 準備 Python 3.14+ 環境,安裝分析管線套件:
|
||||||
|
`pip install -e tools/`。
|
||||||
|
2. 自 `tools/.env.example` 建立 `tools/.env`,寫入
|
||||||
|
Anthropic API 金鑰。
|
||||||
|
3. 依 `docs/research-plan.md` 的階段順序,於 `tools/` 目錄下
|
||||||
|
執行子命令,必要輸入以位置引數、選擇性輸入以選項給定(如
|
||||||
|
`pop-fem-audit-tools build-db
|
||||||
|
../data/source/yearend_hot100_2016_2025.csv
|
||||||
|
../data/derived
|
||||||
|
--lyrics-dir ../data/captures/lyrics
|
||||||
|
--wikidata-csv ../data/captures/artists-wikidata.csv`)。
|
||||||
|
LLM 步驟使用 `claude-sonnet-4-6`、temperature=0、
|
||||||
|
thinking 關閉;輸出可逐項比對的步驟獨立執行三次,由
|
||||||
|
程式取三票多數決定案(「三次執行+多數決」協定),
|
||||||
|
自由生成步驟兩次執行進池。
|
||||||
|
4. 每次執行的完整紀錄(定義檔快照、原始輸出、參數)存於
|
||||||
|
`runs/`,可逐筆稽核。論文引用的最終資料表在 `results/`。
|
||||||
|
|
||||||
LLM提示請參閱 prompts/ 資料夾。
|
注意:歌詞受版權保護,`data/captures/lyrics/` 不隨 repo 發布,須自行
|
||||||
|
以 `tools/` 中的抓取程式重建。
|
||||||
## 輔助工具程式
|
|
||||||
|
|
||||||
輔助工具程式請參閱 tools/ 資料夾。
|
|
||||||
|
|
||||||
## 研究資料
|
|
||||||
|
|
||||||
研究資料請參閱 data/ 資料夾。
|
|
||||||
|
|
||||||
## 授權
|
## 授權
|
||||||
|
|
||||||
@@ -31,9 +44,3 @@ LLM提示請參閱 prompts/ 資料夾。
|
|||||||
排除物件:
|
排除物件:
|
||||||
1. 本專案引用之第三方歌詞,相應權利歸各該權利人所有;
|
1. 本專案引用之第三方歌詞,相應權利歸各該權利人所有;
|
||||||
2. tools/ 資料夾內之工具程式,依該工具程式授權條款提供。
|
2. tools/ 資料夾內之工具程式,依該工具程式授權條款提供。
|
||||||
|
|
||||||
## 作者
|
|
||||||
|
|
||||||
楊士青<br>
|
|
||||||
imacat@mail.imacat.idv.tw<br>
|
|
||||||
2026/8/18<br>
|
|
||||||
|
|||||||
@@ -1,69 +0,0 @@
|
|||||||
# 常設工作規範
|
|
||||||
|
|
||||||
本專案的常設工作規範。原專案 `CLAUDE.md`;移置於此,
|
|
||||||
使 Claude Code subagent 不將其繼承入 context(盲判型
|
|
||||||
agent 不得見之)。凡於本專案工作的主會話,動手管線、
|
|
||||||
資料或文件之前,先讀本檔。
|
|
||||||
|
|
||||||
## 分析管線
|
|
||||||
|
|
||||||
管線已跑畢;本規範適用於任何重跑或擴充。
|
|
||||||
|
|
||||||
- LLM 分析以 Python 腳本呼叫 Anthropic Messages API
|
|
||||||
執行,能用 Batch API 處即用之。步驟 1 與步驟 3 以
|
|
||||||
`claude-sonnet-4-6` 執行,`temperature=0`、thinking
|
|
||||||
停用;步驟 4 與步驟 5 以 `claude-fable-5` 執行,該
|
|
||||||
模型兩個參數皆不受理——步驟 4 的取樣變異由多數決
|
|
||||||
吸收,步驟 5 由整合三份閱讀吸收。
|
|
||||||
- 定義檔置於 `prompts/<步><次步>-<task>.md`(如
|
|
||||||
1-tag.md、5a-read.md;次步以字母標示,與論文正文的
|
|
||||||
步驟編號一致;不帶版本號——版本即 git 歷史),逐字
|
|
||||||
作為 system prompt。
|
|
||||||
- 逐項的 LLM 判斷(步驟 3 的逐首編碼、步驟 4 的逐碼
|
|
||||||
入群判斷)以同一份定義檔、同一份輸入獨立執行三次;
|
|
||||||
再由確定性計票將三次執行中至少兩次指派的項目(一個
|
|
||||||
(歌,關鍵字)或(群,關鍵字)配對)收入定案
|
|
||||||
(「三次執行+多數決」)。自由生成步驟執行兩次,
|
|
||||||
兩份輸出進池。步驟 5 的質性閱讀兩者皆非:逐首三次
|
|
||||||
獨立閱讀,由各自的定義檔逐首整合、跨首統整——質性
|
|
||||||
協定,不是投票(見 docs/methodology.md)。詞彙表由
|
|
||||||
確定性子命令建構(嵌入+分群),不經 LLM。驗證結果
|
|
||||||
不符預期時,修訂定義檔並重複該循環;絕不手改結果。
|
|
||||||
- 一步的每次執行皆自我完備歸檔於 `run-llm` 命令列上
|
|
||||||
明示指定的目的目錄下(慣例為
|
|
||||||
`data/runs/<步驟>/run<N>/`):定義檔快照、原始輸出與
|
|
||||||
`meta.json`(model ID、參數、時間戳、batch ID)。
|
|
||||||
一步的 N 次執行即 N 次各自的 `run-llm` 呼叫。覆蓋
|
|
||||||
既有執行歸檔須明示旗標;被取代的執行留在 git 歷史。
|
|
||||||
確定性步驟歸檔於 `data/runs/<步驟>/`,不分 `run<N>` 層。
|
|
||||||
- 每次 `run-llm` 執行的 token 用量與費用記入
|
|
||||||
`docs/run-costs.md`,與執行歸檔同一 commit。
|
|
||||||
- 腳本自環境變數 `ANTHROPIC_API_KEY` 讀取 API key
|
|
||||||
(`.env`,gitignored)。
|
|
||||||
|
|
||||||
## 資料規則
|
|
||||||
|
|
||||||
- `data/source/` 存手放後不動的原始檔;
|
|
||||||
`data/captures/` 只由 fetch 命令與私人匯入腳本
|
|
||||||
寫入;`data/manual/` 只由研究者親手寫入;
|
|
||||||
`data/derived/` 只由 `build-db` 子命令寫入;
|
|
||||||
`data/runs/` 存 LLM 執行的原始歸檔,只由執行程序
|
|
||||||
寫入;`data/results/` 存論文引用的定案表,只由
|
|
||||||
計票程序寫入。
|
|
||||||
- 歌詞全文有版權:一律置於 `data/captures/lyrics/`
|
|
||||||
(gitignored),絕不 commit,亦絕不於 repo 任何處
|
|
||||||
全文重現。
|
|
||||||
|
|
||||||
## 文件
|
|
||||||
|
|
||||||
- `data/results/` 存計票後的定案表(論文所引);
|
|
||||||
`data/runs/` 存原始稽核紀錄。論文只引 `data/results/`。
|
|
||||||
- **Commit 判準**:凡能由「committed 的輸入+committed
|
|
||||||
的程式」決定性再生者不 commit;凡不能者一律以文字
|
|
||||||
格式 commit,格式跟著上文「資料規則」一節所定的
|
|
||||||
層次走。例外:`data/results/` 定案表與
|
|
||||||
`data/derived/` 人讀報表雖可再生仍 commit——理由是
|
|
||||||
引用穩定性、審稿人零門檻、撰稿期數字變動可 diff;
|
|
||||||
兩者皆與工作儲存同一動作產出,稽核鏈無中間空缺。
|
|
||||||
- 凡定義檔或研究規劃之更動,皆記入
|
|
||||||
`docs/decision-log.md`,註明日期與原因。
|
|
||||||
+18
-62
@@ -99,8 +99,8 @@
|
|||||||
|
|
||||||
## 2026-08-02
|
## 2026-08-02
|
||||||
|
|
||||||
- **`import-lyrics` 不設為子命令,pilot 歌詞改以私人腳本
|
- **`import-lyrics` 不設為子命令,pilot 歌詞改以 `excludes/`
|
||||||
匯入**。理由:pilot 捕捉檔不隨論文發布,子命令
|
私人腳本匯入**。理由:pilot 捕捉檔不隨論文發布,子命令
|
||||||
形式會在發布的 CLI 裡留下讀者無法執行的死命令——要交待的
|
形式會在發布的 CLI 裡留下讀者無法執行的死命令——要交待的
|
||||||
是「沿用 pilot 捕捉」的事實(記於 lyrics-provenance.csv 與
|
是「沿用 pilot 捕捉」的事實(記於 lyrics-provenance.csv 與
|
||||||
論文方法節),不是工具本身;工具移出專案,發布管線即
|
論文方法節),不是工具本身;工具移出專案,發布管線即
|
||||||
@@ -178,8 +178,8 @@
|
|||||||
來源公開可稽核,快照乾淨、overrides 縮小;論文方法節揭露
|
來源公開可稽核,快照乾淨、overrides 縮小;論文方法節揭露
|
||||||
「捕捉前經研究者查證補完」。性別以公開自我認同為準,
|
「捕捉前經研究者查證補完」。性別以公開自我認同為準,
|
||||||
推測不確定且無佐證者列疑慮清單;查證屬資料策展而非
|
推測不確定且無佐證者列疑慮清單;查證屬資料策展而非
|
||||||
分析,以 Claude Code 輔助、不走分析 API;QS 批次以
|
分析,以 Claude Code 輔助、不走分析 API;QS 批次存
|
||||||
私人工作檔留存,不隨論文發布。
|
`excludes/`(私人工作檔,不隨論文發布)。
|
||||||
|
|
||||||
## 2026-08-04
|
## 2026-08-04
|
||||||
|
|
||||||
@@ -227,7 +227,7 @@
|
|||||||
- **Markdown 檔名一律以 dash 連接**(decision-log.md、
|
- **Markdown 檔名一律以 dash 連接**(decision-log.md、
|
||||||
research-plan.md、project-structure.md 等;與 data/ 層
|
research-plan.md、project-structure.md 等;與 data/ 層
|
||||||
CSV 檔名慣例一致),定義檔命名慣例同步改為
|
CSV 檔名慣例一致),定義檔命名慣例同步改為
|
||||||
`prompts/<task>-v<N>.md`(如 screen-v1.md),版本庫外的
|
`prompts/<task>-v<N>.md`(如 screen-v1.md),`excludes/`
|
||||||
私人工作檔一併改名;`conference_abstract.md` 改名並搬入
|
私人工作檔一併改名;`conference_abstract.md` 改名並搬入
|
||||||
`paper/`——它是本次年會實際送出的摘要,與全文同屬投稿
|
`paper/`——它是本次年會實際送出的摘要,與全文同屬投稿
|
||||||
血脈,不是 `docs/` 的內部工作文件。全 repo 指涉同步更新。
|
血脈,不是 `docs/` 的內部工作文件。全 repo 指涉同步更新。
|
||||||
@@ -583,8 +583,8 @@
|
|||||||
Billboard 署名 Pinkfong 為品牌(Wikidata Q55735607,型態
|
Billboard 署名 Pinkfong 為品牌(Wikidata Q55735607,型態
|
||||||
brand),非演唱者;其唯一上榜曲 Baby Shark(2019#75)
|
brand),非演唱者;其唯一上榜曲 Baby Shark(2019#75)
|
||||||
的實際演唱者為 Hope Segoine(KTVB 2019-03-06 報導、
|
的實際演唱者為 Hope Segoine(KTVB 2019-03-06 報導、
|
||||||
Songfacts、經紀簡介,出處詳私人留存的
|
Songfacts、經紀簡介,出處詳
|
||||||
QuickStatements 草稿)。藉既有的
|
`excludes/quickstatements/hope-segoine.md`)。藉既有的
|
||||||
署名正規化機制(`ArtistImporter.CANONICAL_ARTIST_NAMES`)
|
署名正規化機制(`ArtistImporter.CANONICAL_ARTIST_NAMES`)
|
||||||
指認:歌曲的署名字串維持榜單所印的 "Pinkfong",解析出
|
指認:歌曲的署名字串維持榜單所印的 "Pinkfong",解析出
|
||||||
的演出者實體為 Hope Segoine。品牌無性別可言,人有;
|
的演出者實體為 Hope Segoine。品牌無性別可言,人有;
|
||||||
@@ -637,9 +637,10 @@
|
|||||||
(sentence-transformers/all-mpnet-base-v2, revision
|
(sentence-transformers/all-mpnet-base-v2, revision
|
||||||
e8c3b32e)對 `runs/02-cluster/source-keywords.txt` 的
|
e8c3b32e)對 `runs/02-cluster/source-keywords.txt` 的
|
||||||
5,999 個關鍵字重新計算。組內一致性定義為組內成員兩兩
|
5,999 個關鍵字重新計算。組內一致性定義為組內成員兩兩
|
||||||
餘弦相似度之平均。k=50 直接取自私人留存的分群結果;
|
餘弦相似度之平均。k=50 直接取自留存的分群結果
|
||||||
k=30 以同參數重跑(確定性演算法),重算所得最大組恰為
|
(`excludes/k50/02-cluster/groups.csv`);k=30 以同參數
|
||||||
491 詞,與當時粗算紀錄吻合,佐證重算與原實驗一致。數表:
|
重跑(確定性演算法),重算所得最大組恰為 491 詞,與
|
||||||
|
當時粗算紀錄吻合,佐證重算與原實驗一致。數表:
|
||||||
|
|
||||||
| | k=30 | k=50 | k=100 |
|
| | k=30 | k=50 | k=100 |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
@@ -702,8 +703,8 @@
|
|||||||
力量語意即入選,women-power 群 11 碼,並產詞彙表外
|
力量語意即入選,women-power 群 11 碼,並產詞彙表外
|
||||||
幻覺碼一筆),claude-fable-5 讀為詞彙化概念(要求女性
|
幻覺碼一筆),claude-fable-5 讀為詞彙化概念(要求女性
|
||||||
標記,women-power 群 2 碼),與草稿盲選及深度閱讀輔助
|
標記,women-power 群 2 碼),與草稿盲選及深度閱讀輔助
|
||||||
判讀同讀法;sonnet 對照執行歸檔另行私人備份,不入
|
判讀同讀法;sonnet 對照執行歸檔備份於
|
||||||
版本庫,支出留帳。claude-fable-5 不
|
`excludes/experiments/`,支出留帳。claude-fable-5 不
|
||||||
受理 temperature 與 thinking 參數(均不送出),無法釘
|
受理 temperature 與 thinking 參數(均不送出),無法釘
|
||||||
temperature=0;執行間變異實測存在(run1/run2 於陽剛、
|
temperature=0;執行間變異實測存在(run1/run2 於陽剛、
|
||||||
脆弱邊緣碼分歧),由三票多數決吸收;詞彙表外輸出項
|
脆弱邊緣碼分歧),由三票多數決吸收;詞彙表外輸出項
|
||||||
@@ -750,9 +751,9 @@
|
|||||||
編碼、票數),納入重建摘要與清空範圍,供群層次查詢;
|
編碼、票數),納入重建摘要與清空範圍,供群層次查詢;
|
||||||
重建命令自此帶 `--groups results/groups.csv`。
|
重建命令自此帶 `--groups results/groups.csv`。
|
||||||
|
|
||||||
- **Fable 5 輔助判讀實驗總錄(非正式編碼;結果檔私人
|
- **Fable 5 輔助判讀實驗總錄(非正式編碼;結果檔於
|
||||||
留存、不入版本庫,本條為其版本庫內的程序錨點)**:
|
gitignored 之 `excludes/`,本條為其版本庫內的程序
|
||||||
2026-08-08 起以 Claude Code subagent(模型
|
錨點)**:2026-08-08 起以 Claude Code subagent(模型
|
||||||
Fable 5)進行七項深度閱讀實驗。共同程序:每首歌一個
|
Fable 5)進行七項深度閱讀實驗。共同程序:每首歌一個
|
||||||
獨立會話、提示最少化且逐字統一、判讀者互不知情、不經
|
獨立會話、提示最少化且逐字統一、判讀者互不知情、不經
|
||||||
三票制;定位為研究者的輔助判讀(初篩),不改動任何
|
三票制;定位為研究者的輔助判讀(初篩),不改動任何
|
||||||
@@ -782,7 +783,7 @@
|
|||||||
`male-mixed-wp-fe-feminist-reading.md` 與同名 .ods)。
|
`male-mixed-wp-fe-feminist-reading.md` 與同名 .ods)。
|
||||||
此系列即論文方法節「最後編碼的結果,再由研究者與 LLM
|
此系列即論文方法節「最後編碼的結果,再由研究者與 LLM
|
||||||
(Claude Code)輔助判讀」之所指;結果檔含大量歌詞引文,
|
(Claude Code)輔助判讀」之所指;結果檔含大量歌詞引文,
|
||||||
依著作權紀律私人留置,不入版本庫。
|
依著作權紀律留置 `excludes/`,不入版本庫。
|
||||||
|
|
||||||
- **步驟 5:女性主義問題之質性深讀(設計定案)**:論文
|
- **步驟 5:女性主義問題之質性深讀(設計定案)**:論文
|
||||||
題目「挪用與污染」需要框架層的系統性證據;47 首
|
題目「挪用與污染」需要框架層的系統性證據;47 首
|
||||||
@@ -898,49 +899,4 @@
|
|||||||
performer_gender 手工修正)、四份 results 定案表、時程
|
performer_gender 手工修正)、四份 results 定案表、時程
|
||||||
展延一至二日與剩餘撰寫工作。仍然有效的原則(先導不
|
展延一至二日與剩餘撰寫工作。仍然有效的原則(先導不
|
||||||
比較、提示只定格式、歌手背景防火牆、commit 判準、
|
比較、提示只定格式、歌手背景防火牆、commit 判準、
|
||||||
版權規則)保留。
|
版權規則、positionality)保留。
|
||||||
|
|
||||||
## 2026-08-17
|
|
||||||
|
|
||||||
- **先導研究的地位另立常設紀錄 `docs/pilot-study.md`**:論文
|
|
||||||
交件後重讀投稿摘要,認清先導研究係由當時協作的 Claude Code
|
|
||||||
設計並執行,研究者檢視的是研究問題與答案,未逐步檢視中間
|
|
||||||
過程;摘要中的具體數字未經稽核,亦無歸檔可回溯。此地位
|
|
||||||
先前僅零星散見於本日誌各條(歌詞沿用、women-power 來歷
|
|
||||||
考據、粒度繼承、簿記容量),投稿摘要本身則只是產物,不足以
|
|
||||||
記述過程。裁定:另立 `docs/pilot-study.md`,完整記述先導
|
|
||||||
研究的執行方式、沿用與棄用的成果,以及它如何促成正式研究
|
|
||||||
的可稽核設計;投稿摘要不於樹中另存副本,其內容留在 git
|
|
||||||
歷史(`paper/abstract.md` 的前身)。
|
|
||||||
|
|
||||||
## 2026-08-18
|
|
||||||
|
|
||||||
- **工序編號改與論文正文一致**:論文正文將五個步驟的細分
|
|
||||||
工序以字母標示(表三「步驟3a」、表四「步驟5d」),repo
|
|
||||||
的檔名與歸檔目錄卻沿用 `05-01`、`05-02` 這種數字次步,
|
|
||||||
兩邊對不上。裁定:改為 `1-tag`、`2-cluster`、`3a-code`、
|
|
||||||
`4-group`、`5a-read`、`5b-consolidate`、`5c-synthesize`、
|
|
||||||
`5d-annotate`(`prompts/` 與 `runs/` 同步,以 `git mv`
|
|
||||||
保留歷史),補零一併取消。`runs/*/meta.json`
|
|
||||||
內記的 `prompt_path` 不追改——它是執行當下的實況記錄;
|
|
||||||
本日誌的既有條目同理,維持當時的名稱。`run-costs.md` 則
|
|
||||||
相反:它的「步驟」欄是查閱歸檔的索引,故一律換為現行
|
|
||||||
編號(`01-01-01-tag`→`1-tag`、`03-01-code` 與
|
|
||||||
`03-code`→`3a-code`),並新增末欄「原步驟名」保留執行
|
|
||||||
當時的名稱——索引要指得到現在的歸檔,紀錄要留得住當時
|
|
||||||
的形狀,兩者以兩欄並存解決。已刪除的工序(步驟 2 的
|
|
||||||
LLM 合併嘗試 `01-02-01-merge`、步驟 3 的仲裁
|
|
||||||
`03-02-arbitration`)其歸檔已不存在,無現行編號可指,
|
|
||||||
不另發新名,步驟欄逕標「已廢棄」,原名存於末欄。
|
|
||||||
|
|
||||||
- **CLAUDE.md 移為 `docs/conventions.md`**:專案 CLAUDE.md 會
|
|
||||||
全文自動注入每一個 Claude Code subagent 的 context(2026-07-30
|
|
||||||
實測條),步驟 6 的 883 個盲判 agent 因此都看到了工作規範。
|
|
||||||
規範本身無研究語意(無 women-power、無假說),污染有限,但
|
|
||||||
「防脈絡污染」的原則應貫徹到自家工具鏈:docs/ 底下的檔案
|
|
||||||
不會自動注入,搬移後 subagent 除非主動翻閱否則不會看到,
|
|
||||||
盲判 agent 的定義檔並明令不得查閱專案材料。代價是主會話
|
|
||||||
也不再自動載入,須自行先讀 `docs/conventions.md`。內容
|
|
||||||
順帶微調:註明管線已完成、規範適用於重跑;刪去已不存在
|
|
||||||
的 codebook 一詞。進論文的數據一律走 Anthropic API(無
|
|
||||||
檔案系統,機制上不可能讀到),此層防線不受本次搬移影響。
|
|
||||||
|
|||||||
+202
-101
@@ -1,12 +1,12 @@
|
|||||||
# 方法細節
|
# 方法細節
|
||||||
|
|
||||||
(全文方法節底稿。演算法在執行前寫定;任何修訂記入
|
(全文方法節底稿。演算法在執行前寫定;任何修訂記入
|
||||||
`decision-log.md`。)
|
`decision-log.md`。定義檔全文見 `prompts/`,執行紀錄見
|
||||||
|
`runs/`。)
|
||||||
|
|
||||||
## 自然編碼管線總覽
|
## 自然編碼管線總覽
|
||||||
|
|
||||||
五個步驟:步驟 1 自由標註(`claude-sonnet-4-6`,
|
五個步驟:步驟 1 自由標註(兩次執行進池)→ 步驟 2 詞彙表
|
||||||
temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
|
|
||||||
建構(詞向量分群,確定性)→ 步驟 3 全量編碼(三次執行+
|
建構(詞向量分群,確定性)→ 步驟 3 全量編碼(三次執行+
|
||||||
多數決)→ 步驟 4 語意編碼群(三次執行+多數決)→ 步驟 5
|
多數決)→ 步驟 4 語意編碼群(三次執行+多數決)→ 步驟 5
|
||||||
女性主義問題之質性深讀(三次閱讀+逐首整合+樣態統整)。
|
女性主義問題之質性深讀(三次閱讀+逐首整合+樣態統整)。
|
||||||
@@ -14,6 +14,13 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
|
|||||||
不接觸歌詞,步驟 2 亦不呼叫 LLM。設計原則見
|
不接觸歌詞,步驟 2 亦不呼叫 LLM。設計原則見
|
||||||
`research-plan.md`;本檔記載可重現的演算法細節。
|
`research-plan.md`;本檔記載可重現的演算法細節。
|
||||||
|
|
||||||
|
編號的所指為**研究程序的工序**,不是定義檔:步驟 1、
|
||||||
|
步驟 3、步驟 4 與步驟 5 有定義檔(`prompts/`;步驟 5 依
|
||||||
|
子工序有三份),步驟 2 沒有——它是單一確定性計算,由
|
||||||
|
`cluster-keywords` 一個子命令完成。有無定義檔的區別即
|
||||||
|
「該步是否為 LLM 判斷」,由 `prompts/` 是否存在同號檔案
|
||||||
|
直接可見。
|
||||||
|
|
||||||
## 步驟 2 詞彙表建構——詞向量分群
|
## 步驟 2 詞彙表建構——詞向量分群
|
||||||
|
|
||||||
詞彙表由確定性程序產生,不經 LLM。完整分割(每個關鍵字
|
詞彙表由確定性程序產生,不經 LLM。完整分割(每個關鍵字
|
||||||
@@ -23,7 +30,8 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
|
|||||||
### 進池
|
### 進池
|
||||||
|
|
||||||
兩次標註執行的全部關鍵字取聯集、逐字串精確去重、字典序
|
兩次標註執行的全部關鍵字取聯集、逐字串精確去重、字典序
|
||||||
排列。失敗與拒答的記錄跳過(其歌曲不貢獻關鍵字)。
|
排列。失敗與拒答的記錄跳過(其歌曲不貢獻關鍵字);解析
|
||||||
|
時偵測重複鍵,違規即失敗。
|
||||||
|
|
||||||
### 分群
|
### 分群
|
||||||
|
|
||||||
@@ -31,10 +39,14 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
|
|||||||
(釘定 revision),關鍵字的連字號先還原為空格再編碼,
|
(釘定 revision),關鍵字的連字號先還原為空格再編碼,
|
||||||
輸出 768 維向量並 L2 正規化。
|
輸出 768 維向量並 L2 正規化。
|
||||||
- **分群**:階層式聚合分群(Ward linkage),k=100。
|
- **分群**:階層式聚合分群(Ward linkage),k=100。
|
||||||
向量既已正規化,歐氏距離與餘弦相似度單調對應。
|
向量既已正規化,歐氏距離與餘弦相似度單調對應;三種
|
||||||
|
linkage 實測比較,Ward 於各個 k 的組內一致性均最高
|
||||||
|
(average 與 complete 皆產生吞噬半數語料的巨大異質
|
||||||
|
組)。
|
||||||
- **組數的取捨**:k 太小則壓縮比過高,樹上層被迫併入
|
- **組數的取捨**:k 太小則壓縮比過高,樹上層被迫併入
|
||||||
不相干的詞,組雖大而無主題;k 太大則人工難以通覽。
|
不相干的詞,組雖大而無主題(k=30 最大組 491 詞、
|
||||||
定於 100,理由是實測顯示雜物櫃
|
組內一致性 0.41,成員橫跨籃球、海灘、外星人綁架);
|
||||||
|
k 太大則人工難以通覽。定於 100,理由是實測顯示雜物櫃
|
||||||
組於此始裂解為有主題的組,且編碼實測未見碼數過多的
|
組於此始裂解為有主題的組,且編碼實測未見碼數過多的
|
||||||
副作用——全量三次執行下 101 個碼全數用到;模型另行
|
副作用——全量三次執行下 101 個碼全數用到;模型另行
|
||||||
造出的碼共 13 筆,佔 44,149 筆標籤指派的 0.03%。
|
造出的碼共 13 筆,佔 44,149 筆標籤指派的 0.03%。
|
||||||
@@ -43,23 +55,39 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
|
|||||||
自己產出過的關鍵字,非任何人事後撰寫。已知限制:
|
自己產出過的關鍵字,非任何人事後撰寫。已知限制:
|
||||||
組越大越異質時,medoid 只是折衷詞,可能代表不了組內
|
組越大越異質時,medoid 只是折衷詞,可能代表不了組內
|
||||||
內容(實測 `mutual-individuality` 組內一致 0.70 而
|
內容(實測 `mutual-individuality` 組內一致 0.70 而
|
||||||
編碼從未使用);此類碼於結果中呈現為零使用。
|
編碼從未使用);此類碼於結果中呈現為零使用,據實
|
||||||
- **取捨紀錄**:詞彙表分群曾比較的替代法與棄用理由,見
|
報告,不事後改名。
|
||||||
`decision-log.md` 2026-08-05 條。
|
- **取捨紀錄**:曾以 LLM 單發收斂(merge/cap 兩步)
|
||||||
- **產物**:分群輸出中,只有實際交給模型的碼表——組名
|
實作本步,四種模型六次執行全部無法維持完整分割,
|
||||||
加上先驗主題詞——含研究者的介入;其餘皆為分群過程本身
|
已棄用(詳見 `decision-log.md` 2026-08-05;棄用的
|
||||||
的機械紀錄。
|
定義檔止於 git 歷史,見 `git log -- prompts/`)。
|
||||||
|
- **產物**:五份,前綴分別標示來源與結果。
|
||||||
|
`source-keywords.txt`(進池後的關鍵字,一行一個)
|
||||||
|
記錄進來的是什麼;`result-keywords.txt`(組名,一行
|
||||||
|
一個)與 `groups.csv`(欄位 Group、Keyword,一列一個
|
||||||
|
成員)記錄算出來的分割;`keywords-to-merge.json`
|
||||||
|
(`{"keywords": [...]}`)是實際交給模型的碼,即組名
|
||||||
|
加上先驗主題詞——五份中只有這一份含研究者的介入。
|
||||||
|
`meta.json` 記錄執行本身:進池的兩份執行歸檔與其有效
|
||||||
|
筆數、嵌入模型與釘定 revision、分群參數與組數、外加
|
||||||
|
的先驗詞、關鍵字總數,以及產生數字的套件版本。凡命令
|
||||||
|
列上的選擇與環境事實皆在此,不記時間戳與輸入雜湊
|
||||||
|
——前者使同環境重跑逐位元組可再生,後者只會重述 git
|
||||||
|
已保證的事。
|
||||||
- **可重現性**:同一輸入、同一釘定模型、同一參數逐次
|
- **可重現性**:同一輸入、同一釘定模型、同一參數逐次
|
||||||
重現。不同 CPU/BLAS 實作的浮點尾數差異可能使邊界
|
重現。不同 CPU/BLAS 實作的浮點尾數差異可能使邊界
|
||||||
詞的歸屬翻動,屬已揭露的限制。
|
詞的歸屬翻動,屬已揭露的限制;論文所用碼表逐字
|
||||||
|
commit,引用單位為該份定案檔案。
|
||||||
|
|
||||||
### women-power 的注入
|
### women-power 的注入
|
||||||
|
|
||||||
定案詞彙表為 100 個分群組名再加上 `women-power` 一詞,
|
定案詞彙表為 100 個分群組名再加上 `women-power` 一詞,
|
||||||
共 101 個碼。`women-power` 是研究者任意決定的先驗主題(即本
|
共 101 個碼。`women-power` 是研究者任意決定的先驗主題(即本
|
||||||
論文的主題本身),不由資料產生,屬揭露的儀器介入。該詞
|
論文的主題本身),不由資料產生,屬揭露的儀器介入。該詞
|
||||||
的加入於每次執行皆為明示、可稽核的介入,而非無聲內建於
|
於執行時以 `--extra-keyword` 明示加入,不寫死在程式裏
|
||||||
判斷邏輯;分群結果本身不含它,只有交給模型的碼表含它。
|
——研究者的介入因此每次都出現在重現命令上,而非無聲
|
||||||
|
發生;分群結果的兩份產物不含它,只有交給模型的碼表含
|
||||||
|
它。
|
||||||
|
|
||||||
注入而非另設篩選軌的理由:讓研究者的主題詞與模型自己
|
注入而非另設篩選軌的理由:讓研究者的主題詞與模型自己
|
||||||
收斂出的類別(分群已自行長出 `female-empowerment` 等組)
|
收斂出的類別(分群已自行長出 `female-empowerment` 等組)
|
||||||
@@ -69,17 +97,24 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
|
|||||||
|
|
||||||
## 步驟 3 編碼的三次執行與多數決
|
## 步驟 3 編碼的三次執行與多數決
|
||||||
|
|
||||||
- **模型**:`claude-sonnet-4-6`,temperature=0、thinking 關閉。
|
|
||||||
- **三次執行**:同一份定義檔、同一份輸入檔,獨立執行
|
- **三次執行**:同一份定義檔、同一份輸入檔,獨立執行
|
||||||
三次,三份歸檔並列,彼此無先後主從之別。
|
三次,三份歸檔並列(`runs/03-code/run1`、`run2`、
|
||||||
|
`run3`),彼此無先後主從之別。
|
||||||
- **多數決**:一首歌的一個標籤,三次執行中至少兩次標出
|
- **多數決**:一首歌的一個標籤,三次執行中至少兩次標出
|
||||||
即收入定案編碼。三票不平手,裁決規則因此無例外條款,
|
即收入定案編碼。三票不平手,裁決規則因此無例外條款,
|
||||||
計票由確定性程序完成(見交接契約)。
|
計票由確定性子命令完成(見交接契約)。
|
||||||
- **第三票取全量**:第三次執行同為全量——全部歌曲、
|
- **第三票取全量**:只對前兩次分歧的標籤補問第三票,
|
||||||
全部關鍵字——使三票在同一條件下取得。
|
計票結果相同;仍採全量執行——全部歌曲、全部關鍵字
|
||||||
|
——使三票在同一條件下取得。
|
||||||
- **對邊緣標籤的作用**:兩次執行只分得出「兩次皆標」與
|
- **對邊緣標籤的作用**:兩次執行只分得出「兩次皆標」與
|
||||||
「僅一次標」;三次執行還分得出 3-0 與 2-1,故「定案
|
「僅一次標」;三次執行還分得出 3-0 與 2-1,故「定案
|
||||||
編碼中有多少比例僅以一票之差成立」成為可報告的量。
|
編碼中有多少比例僅以一票之差成立」成為可報告的量。
|
||||||
|
至於判定本身,兩次執行相左的標籤在何種協定下都由第三
|
||||||
|
個判斷定奪,票數不使不確定性消失:某標籤於單次執行被
|
||||||
|
標出的傾向若恰為一半,任何票數皆為擲幣。三票之效在
|
||||||
|
傾向偏離一半處——多數決將判定推向該傾向本身(單次
|
||||||
|
0.7 者為 0.78,0.9 者為 0.97),程序重跑的一致性因而
|
||||||
|
高於單次執行,唯獨恰半處無從改善。
|
||||||
|
|
||||||
## 步驟 4 語意編碼群
|
## 步驟 4 語意編碼群
|
||||||
|
|
||||||
@@ -90,79 +125,103 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
|
|||||||
由 LLM 依編碼名的字面語意判斷。
|
由 LLM 依編碼名的字面語意判斷。
|
||||||
- **任務**:每筆輸入為一個群名加 101 個編碼的字母序
|
- **任務**:每筆輸入為一個群名加 101 個編碼的字母序
|
||||||
清單,輸出為入選編碼的單層 JSON 陣列;定義檔
|
清單,輸出為入選編碼的單層 JSON 陣列;定義檔
|
||||||
只定格式,不含任何群的語意定義。
|
`prompts/04-group.md` 只定格式,不含任何群的語意定義。
|
||||||
- **模型**:`claude-fable-5`;取樣變異由多數決吸收;
|
- **模型**:`claude-fable-5`(步驟 1、3 為
|
||||||
|
`claude-sonnet-4-6`)。該模型不受理 `temperature` 與
|
||||||
|
`thinking` 參數,兩者均不送出;取樣變異由多數決吸收。
|
||||||
模型裁定的理由與對照實驗見決策日誌。
|
模型裁定的理由與對照實驗見決策日誌。
|
||||||
- **三次執行**:同一份定義檔、同一份輸入檔,獨立執行
|
- **三次執行**:同一份定義檔、同一份輸入檔,獨立執行
|
||||||
三次,歸檔並列。
|
三次,歸檔並列(`runs/04-group/run1`、`run2`、`run3`)。
|
||||||
- **多數決**:一個(群,編碼)配對,三次執行中至少兩次
|
- **多數決**:一個(群,編碼)配對,三次執行中至少兩次
|
||||||
入選即屬該群;不在 101 碼詞彙表內的輸出項無效,逐筆
|
入選即屬該群;不在 101 碼詞彙表內的輸出項無效,每筆
|
||||||
記錄後丟棄。計票由確定性計票程序完成,合法碼
|
丟棄印於標準錯誤。計票由確定性子命令 `tally-groups`
|
||||||
清單之產法同步驟 3,結果為定案分群表。
|
完成:`tally-groups <執行歸檔 1> <執行歸檔 2> <執行歸檔
|
||||||
|
3> <合法碼清單> <輸出 CSV>`,合法碼清單之產法同步驟 3。
|
||||||
|
定案分群寫入 `results/groups.csv`,欄位 `Group`、
|
||||||
|
`Keyword`、`Votes`,列序先依群名、再依編碼,一律以
|
||||||
|
Unicode 碼位比較,換行為 CRLF。
|
||||||
|
- **工作儲存**:`build-db --groups <定案分群 CSV>` 將定案
|
||||||
|
分群逐欄照存入 `groups` 資料表(群、編碼、票數),供
|
||||||
|
群層次查詢。
|
||||||
|
|
||||||
## 步驟 5 女性主義問題之質性深讀
|
## 步驟 5 女性主義問題之質性深讀
|
||||||
|
|
||||||
本步驟之 5a 至 5c 為**質性閱讀,非編碼**:輸出為自由
|
本步驟之 5-1 至 5-3 為**質性閱讀,非編碼**:輸出為自由
|
||||||
文字的問題閱讀報告,無可逐項機械比對的單位,故不適用
|
文字的問題閱讀報告,無可逐項機械比對的單位,故不適用
|
||||||
三票多數決與仲裁;
|
三票多數決與仲裁;
|
||||||
三次獨立閱讀為分析者三角檢核,逐首整合為整合而非裁決,
|
三次獨立閱讀為分析者三角檢核,逐首整合為整合而非裁決,
|
||||||
跨首統整之產出為草稿,終審與詮釋由研究者為之。論文引用
|
跨首統整之產出為草稿,終審與詮釋由研究者為之。論文引用
|
||||||
本步驟時不作次數宣稱。
|
本步驟時不作次數宣稱。
|
||||||
|
|
||||||
- **對象**:定案編碼含 `women-power` 或
|
- **對象**:定案編碼含 `women-power` 或
|
||||||
`female-empowerment` 的 145 首歌。
|
`female-empowerment` 的 145 首歌。
|
||||||
- **5a 逐首閱讀**:每筆輸入為一首歌的完整歌詞逐字全文,
|
- **5-1 逐首閱讀**:每筆輸入為一首歌的完整歌詞逐字全文,
|
||||||
不含歌名與演唱者(盲讀)。同一份定義檔、同一份輸入檔,
|
不含歌名與演唱者(盲讀);定義檔
|
||||||
獨立執行三次,歸檔並列。
|
`prompts/05-01-read.md`。同一份定義檔、同一份輸入檔,
|
||||||
- **5b 逐首整合**:每筆輸入為該首歌的三份閱讀報告
|
獨立執行三次,歸檔並列(`runs/05-01-read/run1`、
|
||||||
(不含歌詞);以問題機制為單位保守合併,標收斂註記
|
`run2`、`run3`)。
|
||||||
((3/3)、(2/3)),主清單僅列兩讀以上提出者,單讀發現
|
- **5-2 逐首整合**:每筆輸入為該首歌的三份閱讀報告
|
||||||
以一行存目;定義檔,執行一次。
|
(不含歌詞);以問題機制為單位保守合併,標收斂註記
|
||||||
- **5c 樣態統整**:單筆輸入為一批整合報告;歸納問題
|
((3/3)、(2/3)),主清單僅列兩讀以上提出者,單讀發現
|
||||||
**樣態**——問題呈現與運作的重複形態,非問題分類,
|
以一行存目;定義檔 `prompts/05-02-consolidate.md`,
|
||||||
代表引句僅取自主清單。四種輸入範圍各執行
|
執行一次,歸檔 `runs/05-02-consolidate/run1`。
|
||||||
一次:全 145 首之基底統整,及依演唱聲音之性別切分
|
- **5-3 樣態統整**:單筆輸入為一批整合報告;歸納問題
|
||||||
之三個發話脈絡統整——男聲(male)、女聲(female)、
|
**樣態**——問題呈現與運作的重複形態,非問題分類,
|
||||||
混合(mixed);基底看橫貫各
|
代表引句僅取自主清單;定義檔
|
||||||
脈絡之樣態,分組看各權力脈絡下之樣態。genderfluid
|
`prompts/05-03-synthesize.md`。四種輸入範圍各執行
|
||||||
與 non-binary 共 3 首不設群——樣本數不支持歸納——
|
一次:全 145 首之基底統整(歸檔
|
||||||
僅入基底統整。同一輸入不重複
|
`runs/05-03-synthesize/run1`),及依 `performer_gender`
|
||||||
執行:自由歸納之產出無機械合併可言,其變異由 5a
|
(演唱聲音之性別)切分之三個發話脈絡統整——男聲
|
||||||
三讀、5b 整合與草稿地位承接,四份草稿互為對照,由
|
(male,歸檔 `run2`)、女聲(female,歸檔 `run3`)、
|
||||||
研究者終審裁決。
|
混合(mixed,歸檔 `run4`);基底看橫貫各脈絡之樣態,
|
||||||
- **5d 樣態標註**:以(歌,樣態)對為可逐項機械比對之
|
分組看各權力脈絡下之樣態。genderfluid 與 non-binary
|
||||||
單位,回歸「三次執行+多數決」協定。對象為「有問題」
|
共 3 首不設群——樣本數不支持歸納——僅入基底統整,
|
||||||
的歌——三讀中至多一個「無」(多數決精神;恰兩「無」
|
由研究者以個案閱讀。同一輸入不重複執行:自由歸納之
|
||||||
者其整合報告主清單必為空,與 5b 主清單規則自洽),
|
產出無機械合併可言,其變異由 5-1 三讀、5-2 整合與
|
||||||
計 111 首(男聲 12、女聲 69、混合 29、genderfluid 1)。
|
草稿地位承接,四份草稿互為對照,由研究者終審裁決。
|
||||||
樣態表為三份分組統整草稿原文:男聲 13 條(M1–M13)、
|
- **5-4 樣態標註**:以 (歌, 樣態) 對為可逐項機械比對之
|
||||||
女聲 14 條(F1–F14)、混合 16 條(X1–X16);基底 15 條
|
單位,回歸「三次執行+多數決」協定。對象為「有問題」
|
||||||
|
的歌——三讀中至多一個「無」(多數決精神;恰兩「無」
|
||||||
|
者其整合報告主清單必為空,與 5-2 主清單規則自洽),
|
||||||
|
計 111 首(男聲 12、女聲 69、混合 29、genderfluid 1)。
|
||||||
|
樣態表為三份分組統整草稿原文:男聲 13 條(M1–M13)、
|
||||||
|
女聲 14 條(F1–F14)、混合 16 條(X1–X16);基底 15 條
|
||||||
不入矩陣——全體歸納之一條樣態可能疊合不同方向的
|
不入矩陣——全體歸納之一條樣態可能疊合不同方向的
|
||||||
權力關係(男對女、女對男),實為多個樣態共用一名。
|
權力關係(男對女、女對男),實為多個樣態共用一名。
|
||||||
檢驗範圍:男聲樣態不檢驗純女聲歌、女聲樣態不檢驗
|
檢驗範圍:男聲樣態不檢驗純女聲歌、女聲樣態不檢驗
|
||||||
純男聲歌(發話位置範疇錯置,檢查無意義);混合樣態
|
純男聲歌(發話位置範疇錯置,檢查無意義);混合樣態
|
||||||
檢驗全部(其男女聲部無系統化切分方式,無意義之標註
|
檢驗全部(其男女聲部無系統化切分方式,無意義之標註
|
||||||
容忍之);genderfluid 歌三套全查(無自身透鏡,發話
|
容忍之);genderfluid 歌三套全查(無自身透鏡,發話
|
||||||
位置無法先驗決定),其歸屬引用維持個案地位。每筆
|
位置無法先驗決定),其歸屬引用維持個案地位。每筆
|
||||||
輸入為一首歌之整合報告(「僅單獨提及」行於組裝時
|
輸入為一首歌之整合報告(「僅單獨提及」行於組裝時
|
||||||
剝除,標註僅依主清單)與該首適用之樣態表;定義檔
|
剝除,標註僅依主清單)與該首適用之樣態表;定義檔
|
||||||
獨立執行三次,(歌,樣態)對
|
`prompts/05-04-annotate.md`,獨立執行三次
|
||||||
|
(`runs/05-04-annotate/run1`~`run3`),(歌, 樣態) 對
|
||||||
得兩票以上者定案。
|
得兩票以上者定案。
|
||||||
- **模型**:`claude-fable-5`。
|
- **模型**:`claude-fable-5`(與先導深讀同儀器;
|
||||||
- **輸入組裝**:各步輸入檔由確定性程序自上游產物組裝。
|
`temperature` 與 `thinking` 參數不適用,均不送出)。
|
||||||
5d 之樣態條目自分組統整草稿機械切出,代表引句不
|
- **輸入組裝**:確定性行內腳本。5-1:145 首依歌曲 ID
|
||||||
隨附——引句出自特定歌曲,判該曲時形同預答。
|
升序,`content` 為歌詞逐字全文;5-2:每筆
|
||||||
|
`{"reports": [run1 輸出, run2 輸出, run3 輸出]}`;
|
||||||
|
5-3:單筆以 `song-<ID>` 為鍵、整合報告為值之 JSON
|
||||||
|
物件,鍵集合為該次統整之範圍(基底為全 145 首,
|
||||||
|
分組依工作庫 `performer_gender` 切分);5-4:每筆
|
||||||
|
`{"report": 主清單, "patterns": [{"id", "name",
|
||||||
|
"description"}]}`,樣態條目自分組統整草稿機械切出,
|
||||||
|
代表引句不隨附——引句出自特定歌曲,判該曲時形同
|
||||||
|
預答。各輸入檔之 SHA-256 記入該步 meta。
|
||||||
|
|
||||||
## 女性力量候選集
|
## 女性力量候選集
|
||||||
|
|
||||||
候選集為定案編碼含 `women-power` 或 `female-empowerment`
|
候選集為兩類歌曲的合集:定案編碼含 `women-power` 者,
|
||||||
(步驟 4 女性力量群的兩個編碼)之歌曲聯集:wp 66 首、
|
以及定案編碼含研究者指認之女性力量概念域分群組者。
|
||||||
fe 144 首,聯集 145 首。
|
指認於詞彙表定案後、黃金標準編碼開始前完成,指認清單
|
||||||
|
與理由記入決策日誌。
|
||||||
|
|
||||||
## 軌跡對映(診斷用)
|
## 軌跡對映(診斷用)
|
||||||
|
|
||||||
沿收斂軌跡的機械對映:原始關鍵字 →(兩份標註執行歸檔的
|
沿收斂軌跡的機械對映:原始關鍵字 →(兩份標註執行歸檔的
|
||||||
原始輸出)歌曲、原始關鍵字 →(分群)組,純機械查表,
|
`output.jsonl`)歌曲、原始關鍵字 →(分群)組,純程式查表,
|
||||||
決定性。以其結果與步驟 3 直接編碼的差異率作為「收斂軌跡
|
決定性。以其結果與步驟 3 直接編碼的差異率作為「收斂軌跡
|
||||||
扭曲」的診斷量,不作主結果。
|
扭曲」的診斷量,不作主結果。
|
||||||
|
|
||||||
@@ -171,36 +230,78 @@ fe 144 首,聯集 145 首。
|
|||||||
每一步的輸出如何變成下一步的輸入,皆為確定性程序,規則
|
每一步的輸出如何變成下一步的輸入,皆為確定性程序,規則
|
||||||
明定如下:
|
明定如下:
|
||||||
|
|
||||||
- **歌詞輸入檔(步驟 1)**:由確定性的匯出程序自工作
|
- **歌詞輸入檔(步驟 1)**:`export-llm-input` 自工作
|
||||||
儲存產出,一筆一首歌,依歌曲 ID 升序。步驟 3 的輸入
|
儲存產出,每筆 `{"id": "song-<ID>", "content": <歌詞>}`,
|
||||||
由同一匯出程序、同一工作儲存產出(見下),兩步的語料
|
依歌曲 ID 升序。步驟 3 的輸入由同一子命令、同一工作
|
||||||
同一性由此成立。
|
儲存產出(見下),兩步的語料同一性由此成立;各步
|
||||||
- **步驟 1 → 2**:確定性的分群程序讀兩份執行歸檔的
|
輸入檔的 SHA-256 記入該步 meta。
|
||||||
執行紀錄,進池後直接分群,產出詞彙表與交給
|
- **步驟 1 → 2**:`cluster-keywords` 讀兩份執行歸檔的
|
||||||
模型的碼表。
|
`output.jsonl`(一律以換行字元 `\n` 切行——歌詞含
|
||||||
- **步驟 2 → 3 輸入檔**:同一匯出程序自工作儲存產出
|
U+0085 等控制字元時,`str.splitlines()` 類的通用切行
|
||||||
步驟 3 的輸入,一筆一首歌,兼含歌詞與定案碼表,依
|
會截斷 JSON 字串,實測踩中),進池後直接分群,一次
|
||||||
歌曲 ID 升序。碼表以參數傳入而非填進定義檔——定義
|
產出上列五份檔案。
|
||||||
|
- **步驟 2 → 3 輸入檔**:`export-llm-input --extras
|
||||||
|
<定案碼表>` 自工作儲存產出步驟 3 的輸入,每筆
|
||||||
|
`{"id": "song-<ID>", "content": <字串>}`,`content` 為
|
||||||
|
固定鍵序序列化的 `{"lyrics": …, "keywords": [...]}`,
|
||||||
|
依歌曲 ID 升序。碼表以參數傳入而非填進定義檔——定義
|
||||||
檔只規定任務形狀,換詞彙表、換演算法都不必改它。
|
檔只規定任務形狀,換詞彙表、換演算法都不必改它。
|
||||||
- **步驟 3 定案**:確定性的計票程序讀三份執行歸檔的
|
- **步驟 3 定案**:`tally-codings <執行歸檔 1> <執行歸檔
|
||||||
執行紀錄,依序套用更正表、驗證所有標籤皆在合法碼
|
2> <執行歸檔 3> <輸出 CSV> --corrections <更正表>
|
||||||
|
--valid-keywords <合法碼清單>` 讀三份執行歸檔的
|
||||||
|
`output.jsonl`,依序套用更正表、驗證所有標籤皆在合法碼
|
||||||
清單之內、計票。
|
清單之內、計票。
|
||||||
- **更正表**:研究者逐列校定的人工著作,逐筆更正標籤
|
- **更正表**:`data/manual/coding-corrections.csv`,研究者
|
||||||
或引述——以替代字串取代,或刪去該筆標籤指派或該句
|
逐列校定的人工著作,欄位 `Song ID`、`Run`、`Type`、
|
||||||
引述。一筆引述更正套用於該首歌該次執行的所有出現處。
|
`To Be Replaced`、`Correct Term`。`Type` 為 `keyword`
|
||||||
表中任一列若在資料中找不到對應者,即中止;校定的
|
或 `evidence`,分別更正標籤與引述;`Correct Term` 為
|
||||||
判準記於 `decision-log.md`。
|
替代字串,或 `**REMOVE**` 表示刪去該筆標籤指派(`keyword`)
|
||||||
- **合法碼清單**:自詞彙表產出:分群組名加上
|
或該句引述(`evidence`)。一筆 `evidence` 更正套用於該
|
||||||
`women-power`。
|
首歌該次執行的所有出現處。兩個文字欄以歌詞慣例「 / 」
|
||||||
- **定案編碼表**:歌名與演出者名銜逐首查工作儲存取得。
|
表示換行(與載入後的執行紀錄同一表示法,逐字比對、不再
|
||||||
每個定案標籤隨附其在計票中各份執行所引的
|
轉換),故一列一行,純文字工具可逐列處理。表中任一列若
|
||||||
歌詞行,供逐碼查核(三份執行彼此無先後主從之別)。
|
在資料中找不到對應者,即中止;校定的判準記於
|
||||||
|
`decision-log.md`。
|
||||||
|
- **合法碼清單**:純文字、一行一個碼,自詞彙表產出:
|
||||||
|
`{ cat runs/02-cluster/result-keywords.txt; echo
|
||||||
|
women-power; } | sort`。
|
||||||
|
- **定案表**:`results/codings.csv`,欄位 `Song`、
|
||||||
|
`Artist Credit`、`Keyword`、`Quote`,一列一個標籤。歌名
|
||||||
|
與演出者名銜逐首查工作儲存取得,故本子命令須在
|
||||||
|
`build-db` 之後執行。`Quote` 為該標籤在計票中各份執行
|
||||||
|
所引的歌詞行:各份的引述串接後逐字去重,按 Unicode
|
||||||
|
碼位排序,以單一 `|` 相接(三份執行彼此無先後主從之
|
||||||
|
別,引述之序取決於引述本身);引述內的換行於執行紀錄
|
||||||
|
載入時一次換成歌詞慣例「 / 」,此後更正表、定案表與
|
||||||
|
工作儲存全鏈路同一表示法,不再還原。「 / 」的無歧義性
|
||||||
|
是語料事實而非結構保證:全 883 首歌詞經窮舉查核不含
|
||||||
|
「 / 」;換語料須重查。列序依印出的前三欄依序排:
|
||||||
|
歌名、演出者名銜、
|
||||||
|
標籤,一律以 Unicode 碼位比較,換行為 CRLF(同專案
|
||||||
|
其他 CSV)。
|
||||||
|
- **序列化通則**:所有中間檔為 UTF-8,欄序、鍵序與元素
|
||||||
|
序皆依上列規則明定,無時間戳、無隨機成分;JSON 解析
|
||||||
|
一律偵測重複鍵,違規即失敗。人讀為主的產物採純文字或
|
||||||
|
CSV(CSV 依 RFC 4180,標題列字首大寫),機器交接檔採
|
||||||
|
JSON。給定相同的 LLM 執行輸出,全部交接產物逐位元組
|
||||||
|
可再生。
|
||||||
|
|
||||||
## 執行與稽核
|
## 執行與稽核
|
||||||
|
|
||||||
- LLM 步驟以批次執行程序執行,一份定義檔配一份
|
- LLM 步驟以 `run-llm <定義檔> <輸入檔> <歸檔目錄>`
|
||||||
輸入檔;一步的 N 次執行為 N 次各自獨立的呼叫,各自
|
執行;一步的 N 次執行=重現命令清單上的 N 行命令,
|
||||||
歸檔自我完備。
|
各自歸檔(`runs/<步驟>/run1`、`run2`,三票制步驟另有
|
||||||
|
`run3`)。
|
||||||
|
- 確定性步驟(進池、分群、計票、對映)為子命令,其
|
||||||
|
輸入輸出檔同隨 `runs/` 歸檔;因無執行變異,歸檔目錄
|
||||||
|
下不分 `run<N>` 層。
|
||||||
- Batch API 的每筆請求自含全部脈絡且互不可見(平台
|
- Batch API 的每筆請求自含全部脈絡且互不可見(平台
|
||||||
契約),歌與歌之間的獨立性由此成立;各次執行的獨立
|
契約),歌與歌之間的獨立性由此成立;各次執行的獨立
|
||||||
性由「一次呼叫、一個批次、一份歸檔」的執行結構自明。
|
性由「一次呼叫、一個批次、一份歸檔」的執行結構自明。
|
||||||
|
- 每次 `run-llm` 執行的 token 用量與費用記入
|
||||||
|
`run-costs.md`,被取代的執行一併保留供總支出核算。
|
||||||
|
|
||||||
|
## 映射分析方法
|
||||||
|
|
||||||
|
(依 2026-07-30 決策,於看到結果前寫定;待黃金標準
|
||||||
|
編碼展開前補入。)
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
# LLM 輸出的契約查核
|
# LLM 輸出的契約查核
|
||||||
|
|
||||||
(2026-08-06 量測。對象為步驟 3 的三份執行歸檔
|
(2026-08-06 量測。對象為步驟 3 的三份執行歸檔
|
||||||
`data/runs/3a-code/run1`–`run3`,共 883 首歌、44,149 筆標籤
|
`runs/03-code/run1`–`run3`,共 883 首歌、44,149 筆標籤
|
||||||
指派、44,146 句引述。歌詞以模型實際看到的那一份為準,即
|
指派、44,146 句引述。歌詞以模型實際看到的那一份為準,即
|
||||||
`tools/instance/llm-input-code.jsonl`。)
|
`tools/instance/llm-input-code.jsonl`。)
|
||||||
|
|
||||||
定義檔 `prompts/3a-code.md` 對輸出下了四條明確要求:只用
|
定義檔 `prompts/03-code.md` 對輸出下了四條明確要求:只用
|
||||||
給定的碼且拼寫照給、每個標出的碼恰引一行歌詞、引述逐字、
|
給定的碼且拼寫照給、每個標出的碼恰引一行歌詞、引述逐字、
|
||||||
輸出為合法 JSON 且無其他文字。本檔記錄這四條各自被遵守到
|
輸出為合法 JSON 且無其他文字。本檔記錄這四條各自被遵守到
|
||||||
什麼程度。
|
什麼程度。
|
||||||
@@ -46,8 +46,9 @@ song-750 run3: -use=[] -abuse=["I got a thing for the hard
|
|||||||
liquor on ice"]
|
liquor on ice"]
|
||||||
```
|
```
|
||||||
|
|
||||||
song-750 的 run1 與 run2 則整筆使用錯拼的鍵並附上引述,
|
模型寫錯後綴、已輸出的 token 收不回,遂以空陣列收束該鍵,
|
||||||
故該首的三票分裂於兩種拼寫之間。
|
再於正確的鍵補上引述。song-750 的 run1 與 run2 則整筆使用
|
||||||
|
錯拼的鍵並附上引述,故該首的三票分裂於兩種拼寫之間。
|
||||||
|
|
||||||
## 引述的存在
|
## 引述的存在
|
||||||
|
|
||||||
|
|||||||
@@ -1,85 +0,0 @@
|
|||||||
# 先導研究的來歷與地位
|
|
||||||
|
|
||||||
(2026-08-17 認清並記錄。本檔記述正式研究之前的先導研究:
|
|
||||||
它做了什麼、研究者檢視到哪一層、哪些成果被沿用、哪些被
|
|
||||||
棄用,以及它如何促成正式研究的設計。)
|
|
||||||
|
|
||||||
## 一、先導研究是什麼
|
|
||||||
|
|
||||||
正式研究之前,研究者曾以 Claude Code 對 Billboard Year-End
|
|
||||||
Hot 100(2018–2025、684 首)的歌詞做過一輪探索性分析,檢視
|
|
||||||
「女性力量」語彙的使用狀況,並附帶分析 pussy 一詞作為女性
|
|
||||||
代稱的修辭。2026 年 4 月投出的研討會摘要即根據該輪分析撰寫。
|
|
||||||
|
|
||||||
## 二、它實際的執行方式
|
|
||||||
|
|
||||||
**研究問題與方向由研究者提出**:女性力量語彙被父權框架取用、
|
|
||||||
pussy 作為代稱的跨性別使用,都是研究者長期關注的問題。
|
|
||||||
|
|
||||||
**設計與執行由 Claude Code 進行**:分析管線、分類架構(genuine
|
|
||||||
/peripheral/fake 三分類)、五種「假女性力量」類型、以及
|
|
||||||
frame-aware 提示修正實驗,均由當時協作的 Claude Code 設計並
|
|
||||||
執行。
|
|
||||||
|
|
||||||
**研究者檢視的是問題與答案,不是中間過程**:研究者確認研究
|
|
||||||
問題的方向正確、答案值得追究,即據以投稿;分類類目如何產生、
|
|
||||||
提示詞如何下、數字如何算出,並未逐步檢視。投稿摘要中的具體
|
|
||||||
數字(如「44% 屬於假女性力量」、62 首的三分類、53 首的 pussy
|
|
||||||
修辭分類、假率由 44% 降至 5% 以下)研究者未曾稽核,其產生
|
|
||||||
過程亦無歸檔可回溯。
|
|
||||||
|
|
||||||
**當時的判斷**:投稿未上即罷;若上,正式研究由研究者自行
|
|
||||||
設計與執行。事實上正式研究即是如此重做的。
|
|
||||||
|
|
||||||
## 三、被沿用的成果
|
|
||||||
|
|
||||||
- **歌詞捕捉檔**:先導研究蒐集的 lyrics.json(684 首,
|
|
||||||
2018–2025)以私人腳本(不入版本庫)匯入歌詞快取,只取
|
|
||||||
識別欄位與歌詞本文,先導的分析欄位一概不匯入;出處記於
|
|
||||||
`data/captures/lyrics-provenance.csv`
|
|
||||||
(見 `decision-log.md` 2026-07-31、08-02 條)。
|
|
||||||
- **假說方向**:女性力量語彙的挪用與污染,成為正式研究的
|
|
||||||
研究問題。
|
|
||||||
- **粒度選擇**:正式研究鎖定 thematic keywords 這一粒度,
|
|
||||||
係繼承先導研究三種粒度的比較結果(keywords 過碎、themes
|
|
||||||
過早抽象)。
|
|
||||||
- **`women-power` 一詞的來歷**:考據先導研究的 local agent
|
|
||||||
存檔可知,其第一步指令含數十個範例 thematic keywords,
|
|
||||||
其中即有 women-power——為當時協作的 Claude Code 依研究者
|
|
||||||
長期表達的關注主動加入(研究者端播種,非明示指定);先導
|
|
||||||
的標籤 `women-power-and-empowerment` 則是第三步強制合併
|
|
||||||
兩個關鍵字的管線人工產物(見 `decision-log.md` 2026-08-04
|
|
||||||
條)。
|
|
||||||
- **簿記容量的教訓**:先導研究九百餘詞可以在單一回應內完成
|
|
||||||
分組;此法未沿用,詞彙表改由確定性程序產生(見
|
|
||||||
`decision-log.md` 2026-08-06 條)。
|
|
||||||
|
|
||||||
## 四、被棄用的成果
|
|
||||||
|
|
||||||
以下先導研究的產物未進入正式研究,論文亦未引用:
|
|
||||||
|
|
||||||
- **genuine/peripheral/fake 三分類與「44% 假女性力量」**:
|
|
||||||
構念與分母均未經稽核回溯,未沿用。
|
|
||||||
- **五種「假女性力量」類型(A–E)**:未沿用。
|
|
||||||
- **pussy 一詞的修辭分類**:未沿用。
|
|
||||||
- **frame-aware(框架感知)提示修正法**:先導研究以框架判準
|
|
||||||
寫進提示以提高準確率;未沿用。
|
|
||||||
- **三個獨立 LLM subagent+人工仲裁的流程**:未沿用,原因見
|
|
||||||
`decision-log.md` 2026-07-30 條。
|
|
||||||
|
|
||||||
## 五、它促成了可稽核的設計
|
|
||||||
|
|
||||||
先導研究的根本限制不在結論對錯,而在**不可稽核**:沒有定義檔
|
|
||||||
快照、沒有原始輸出歸檔、沒有參數紀錄,因此任何一個數字都無法
|
|
||||||
回溯到產生它的那一次執行。正式研究的可稽核設計,正是針對這一點
|
|
||||||
而立。
|
|
||||||
|
|
||||||
換言之,正式研究對 LLM 輸出所採取的「不信任、須查核」立場,
|
|
||||||
其第一個案例就是先導研究本身。
|
|
||||||
|
|
||||||
## 六、現存的紀錄
|
|
||||||
|
|
||||||
- 投稿時的先導研究摘要:已於 git 歷史中(`paper/abstract.md`
|
|
||||||
的前身),樹中不另存副本——摘要只是產物,不足以記述過程,
|
|
||||||
故另立本檔。
|
|
||||||
- 先導研究的歌詞捕捉檔沿用事實:`data/captures/lyrics-provenance.csv`。
|
|
||||||
@@ -0,0 +1,134 @@
|
|||||||
|
# 專案目錄結構
|
||||||
|
|
||||||
|
(2026-07-30 討論定案;2026-07-31 更新為 tools/ 子專案與
|
||||||
|
SQLite 工作儲存架構;2026-08-17 依完成後的現況更新)
|
||||||
|
|
||||||
|
```
|
||||||
|
pop-fem-audit/
|
||||||
|
├── README.md # 專案說明、重現步驟
|
||||||
|
├── CLAUDE.md # 極簡工作規範(subagent 會讀到,
|
||||||
|
│ # 絕不放理論、codebook、預期結果)
|
||||||
|
├── .gitignore # captures/lyrics/、.env、scratch
|
||||||
|
├── data/ # 依生命週期分層(文字格式)
|
||||||
|
│ ├── source/ # 源頭:手放後不動
|
||||||
|
│ │ └── yearend_hot100_2016_2025.csv # 原始榜單
|
||||||
|
│ ├── captures/ # 外部捕捉:只由 fetch 命令與
|
||||||
|
│ │ │ # 私人匯入腳本寫入
|
||||||
|
│ │ ├── artists-wikidata.csv # Wikidata 快照
|
||||||
|
│ │ ├── lyrics-provenance.csv # 歌詞出處
|
||||||
|
│ │ └── lyrics/ # 歌詞 .txt 快取
|
||||||
|
│ │ # (gitignored,版權)
|
||||||
|
│ ├── manual/ # 人工著作:只由研究者手寫
|
||||||
|
│ │ ├── coding-corrections.csv # 編碼與引述的校對表
|
||||||
|
│ │ └── performer-gender-corrections.csv # 演唱聲音性別的
|
||||||
|
│ │ # 手工修正
|
||||||
|
│ └── derived/ # 衍生:只由 build-db 寫入
|
||||||
|
│ ├── songs.csv # 歌曲報表(人讀;進 git)
|
||||||
|
│ └── artists.csv # 歌手報表(人讀;進 git)
|
||||||
|
├── prompts/ # LLM 定義檔(逐字作為 system prompt)
|
||||||
|
│ └── <步>-<次步>-<task>.md # 01-tag.md、03-code.md、
|
||||||
|
│ # 04-group.md、05-01-read.md、
|
||||||
|
│ # 05-02-consolidate.md、
|
||||||
|
│ # 05-03-synthesize.md、
|
||||||
|
│ # 05-04-annotate.md
|
||||||
|
│ # (步內僅一個執行時省略次步)
|
||||||
|
│ # 不帶版本號,版本即 git 歷史
|
||||||
|
│ # (編號的所指是工序:確定性
|
||||||
|
│ # 的步驟 2 無定義檔仍佔一號;
|
||||||
|
│ # 補零只為排序)
|
||||||
|
├── tools/ # 輔助工具子專案(src-layout)
|
||||||
|
│ ├── pyproject.toml # 發行名 pop-fem-audit-tools;
|
||||||
|
│ │ # pip install -e tools/ 安裝
|
||||||
|
│ ├── README.rst LICENSE MANIFEST.in .env.example .gitignore
|
||||||
|
│ ├── docs/ # Sphinx API 文件
|
||||||
|
│ ├── instance/ # SQLite 工作儲存(generated、
|
||||||
|
│ │ # gitignored;含歌詞全文)
|
||||||
|
│ ├── src/pop_fem_audit_tools/
|
||||||
|
│ │ ├── __main__.py # 套件 CLI 進入點(分派子命令)
|
||||||
|
│ │ ├── commands/ # CLI 子命令模組(登記於 __init__)
|
||||||
|
│ │ │ ├── build_db.py # build the SQLite working store
|
||||||
|
│ │ │ │ # from the inputs
|
||||||
|
│ │ │ ├── export_llm_input.py # export the LLM input JSONL
|
||||||
|
│ │ │ │ # (lyrics only) from the
|
||||||
|
│ │ │ │ # working store
|
||||||
|
│ │ │ ├── fetch_artists.py # fetch artist metadata from
|
||||||
|
│ │ │ │ # Wikidata into the snapshot CSV
|
||||||
|
│ │ │ ├── fetch_lyrics.py # fetch missing lyrics from the
|
||||||
|
│ │ │ │ # public APIs into the lyrics dir
|
||||||
|
│ │ │ ├── cluster_keywords.py # pool the tagging runs'
|
||||||
|
│ │ │ │ # keywords and cluster them
|
||||||
|
│ │ │ │ # into the codes (step 2)
|
||||||
|
│ │ │ ├── tally_codings.py # settle step 3 by majority
|
||||||
|
│ │ │ ├── tally_groups.py # settle step 4 by majority
|
||||||
|
│ │ │ ├── tally_annotations.py # settle step 5-4 by majority
|
||||||
|
│ │ │ └── run_llm.py # API 執行器:一份定義檔+一份輸入
|
||||||
|
│ │ │ # →歸檔至指定目錄(Batch API);
|
||||||
|
│ │ │ # 多次執行的計票由獨立子命令承擔
|
||||||
|
│ │ ├── config.py # pydantic-settings 設定(.env)
|
||||||
|
│ │ ├── database.py # SQLAlchemy engine / session / Base
|
||||||
|
│ │ ├── models.py # SQLAlchemy ORM 資料模型
|
||||||
|
│ │ └── utils.py # 共用工具(format_duration)
|
||||||
|
│ └── tests/ # 單元測試(unittest)
|
||||||
|
├── runs/ # 現行執行的完整稽核紀錄(進 git;
|
||||||
|
│ │ # 重跑同一 run 須明示 --replace)
|
||||||
|
│ ├── <步驟名>/ # 一步一個目錄(01-tag、03-code、
|
||||||
|
│ │ # 04-group、05-01-read、
|
||||||
|
│ │ # 05-02-consolidate、
|
||||||
|
│ │ # 05-03-synthesize、05-04-annotate)
|
||||||
|
│ │ └── run<N>/ # LLM 步驟:每個 run 一份自我
|
||||||
|
│ │ ├── prompt.md # 完備歸檔(定義檔快照)
|
||||||
|
│ │ ├── output.jsonl # 該次執行原始輸出
|
||||||
|
│ │ └── meta.json # model ID、temperature、時間戳、
|
||||||
|
│ │ # batch ID、token 用量
|
||||||
|
│ └── 02-cluster/ # 確定性步驟:無執行變異,
|
||||||
|
│ # 不分 run<N> 層
|
||||||
|
├── results/ # 論文引用的定案表 CSV(計票子命令
|
||||||
|
│ │ # 產出;「可再生仍 commit」的例外)
|
||||||
|
│ ├── codings.csv # 步驟 3 定案編碼
|
||||||
|
│ ├── groups.csv # 步驟 4 定案編碼群
|
||||||
|
│ ├── patterns.csv # 步驟 5-3 定案樣態表
|
||||||
|
│ ├── annotations.csv # 步驟 5-4 定案歌×樣態
|
||||||
|
│ └── pattern-matrix.csv # 前四者的人讀寬表
|
||||||
|
├── docs/
|
||||||
|
│ ├── research-plan.md # 研究步驟規劃(本檔之姊妹篇)
|
||||||
|
│ ├── project-structure.md # 本檔
|
||||||
|
│ ├── output-validation.md # LLM 輸出的契約查核紀錄
|
||||||
|
│ ├── decision-log.md # 決策日誌:每次改定義檔的原因
|
||||||
|
│ ├── run-costs.md # 每次執行的 token 用量與費用
|
||||||
|
│ └── methodology.md # 方法細節(全文方法節底稿;
|
||||||
|
│ # 映射分析方法須在看結果前寫定)
|
||||||
|
└── paper/
|
||||||
|
├── abstract.md # 摘要
|
||||||
|
└── 流行音樂中「女性力量」….odt # 全文
|
||||||
|
```
|
||||||
|
|
||||||
|
## 設計理由
|
||||||
|
|
||||||
|
- **`runs/` 自我完備**:每個執行目錄含定義檔快照 + 原始輸出 +
|
||||||
|
meta,讀者不需 git 考古即可稽核任一筆結果。
|
||||||
|
- **`runs/`(原始稽核資料)與 `results/`(最終表)分離**:
|
||||||
|
論文只引 `results/`,其來源可回溯至 `runs/`。
|
||||||
|
- **`prompts/` 檔名不帶版本號**:版本即 git 歷史,失敗的
|
||||||
|
版本不保留;論文引用的單位是 `runs/` 內隨執行保存的定義檔
|
||||||
|
快照(每個執行目錄自我完備),不需檔名可指的版本名。
|
||||||
|
- **工作儲存的資料表**:`songs`(含 `performer_gender`=演唱
|
||||||
|
聲音的性別)、`chart_entries`、`artists`、`song_artists`、
|
||||||
|
`codings`(定案編碼:一歌一標籤一列,`quotes` 存該標籤所據的
|
||||||
|
歌詞引述,多句以 `|` 相接)、`groups`(語意編碼群)、
|
||||||
|
`patterns`(深讀樣態)、`annotations`(歌×樣態定案矩陣)。
|
||||||
|
各定案表經 `build-db` 的 `--codings`、`--groups`、
|
||||||
|
`--patterns`、`--annotations` 匯入,性別修正經
|
||||||
|
`--gender-corrections` 套用,與其餘資料同一交易,儲存不會
|
||||||
|
半建;詳見 `research-plan.md`「資料儲存與模型」。
|
||||||
|
- **Commit 判準**:能由「committed 輸入+程式」決定性再生者不
|
||||||
|
commit(SQLite 工作儲存、LLM 輸入檔);源頭、捕捉、人工著作
|
||||||
|
一律以文字 commit。「可再生仍 commit」的例外有二:
|
||||||
|
`results/` 報表(引用穩定性、審稿人零門檻、撰稿期可 diff)
|
||||||
|
與 `data/derived/` 人讀報表(與工作儲存同一動作產出,稽核
|
||||||
|
鏈無中間空缺)。詳見 `research-plan.md`「資料儲存與模型」。
|
||||||
|
- **設定**經 pydantic-settings 統一:`.env`(gitignored,範本
|
||||||
|
`tools/.env.example`)供應 `SQLALCHEMY_DATABASE_URL` 與
|
||||||
|
`ANTHROPIC_API_KEY`,絕不寫入 repo。
|
||||||
|
- **CLAUDE.md 極簡**:實測證實 Claude Code subagent 會繼承專案
|
||||||
|
CLAUDE.md 全文,故其中只放工作流程規則,領域知識一律放
|
||||||
|
`docs/`(subagent 不會自動讀到)。
|
||||||
+20
-9
@@ -1,7 +1,8 @@
|
|||||||
# 信度:量測方式與結果
|
# 信度:量測方式與結果
|
||||||
|
|
||||||
(2026-08-16 量測。對象為步驟 3a 與步驟 5d 各三次執行的
|
(2026-08-16 量測。對象為步驟 3 的三份執行歸檔
|
||||||
原始輸出;數字由原始輸出直接計算。)
|
`runs/03-code/run1`–`run3`,與步驟 5-4 的三份執行歸檔
|
||||||
|
`runs/05-04-annotate/run1`–`run3`;數字由原始輸出直接計算。)
|
||||||
|
|
||||||
## 一、本研究的信度是什麼
|
## 一、本研究的信度是什麼
|
||||||
|
|
||||||
@@ -33,7 +34,11 @@ Krippendorff 依產生資料的設計,把信度分成三型
|
|||||||
**精密度(precision)**,而**準確度(accuracy)** 未經量測。
|
**精密度(precision)**,而**準確度(accuracy)** 未經量測。
|
||||||
若儀器是確定性的,重複性無須量測;但 LLM 不是——同樣的提示、
|
若儀器是確定性的,重複性無須量測;但 LLM 不是——同樣的提示、
|
||||||
同樣的輸入,三次會給出不同結果,因此重複性是必須報告的儀器
|
同樣的輸入,三次會給出不同結果,因此重複性是必須報告的儀器
|
||||||
規格。
|
規格,而非慣例儀式。
|
||||||
|
|
||||||
|
這一點與 LLM 標註的近期文獻一致:同一模型重複取樣量測的是
|
||||||
|
自我一致性(self-consistency),而重複執行後取多數決可提升
|
||||||
|
標註穩定度(如 Prompt Stability Scoring,arXiv:2407.02039)。
|
||||||
|
|
||||||
## 二、三個常用指標
|
## 二、三個常用指標
|
||||||
|
|
||||||
@@ -62,7 +67,7 @@ Krippendorff 依產生資料的設計,把信度分成三型
|
|||||||
|
|
||||||
## 三、實測結果
|
## 三、實測結果
|
||||||
|
|
||||||
### 步驟 3a(編碼:883 首 × 101 個碼 = 89,183 格,執行三次)
|
### 步驟 3(編碼:883 首 × 101 個碼 = 89,183 格,執行三次)
|
||||||
|
|
||||||
| 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard |
|
| 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard |
|
||||||
|---|---:|---:|---:|
|
|---|---:|---:|---:|
|
||||||
@@ -79,7 +84,7 @@ Krippendorff 依產生資料的設計,把信度分成三型
|
|||||||
(此處由原始輸出計算,未套用 936 筆人工校對表;論文引用的
|
(此處由原始輸出計算,未套用 936 筆人工校對表;論文引用的
|
||||||
定案數 14,664 為校對後的結果,差 1 筆。)
|
定案數 14,664 為校對後的結果,差 1 筆。)
|
||||||
|
|
||||||
### 步驟 5d(樣態標註:111 首 × 各自適用樣態 = 3,708 格,執行三次)
|
### 步驟 5-4(樣態標註:111 首 × 各自適用樣態 = 3,708 格,執行三次)
|
||||||
|
|
||||||
| 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard |
|
| 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard |
|
||||||
|---|---:|---:|---:|
|
|---|---:|---:|---:|
|
||||||
@@ -101,16 +106,22 @@ Jaccard(只看「有標到」的格子,忽略雙方都沒標的)約 0.89–0.91,
|
|||||||
比百分比一致率低而更誠實——因為 82% 的格子是雙方都判「無」,
|
比百分比一致率低而更誠實——因為 82% 的格子是雙方都判「無」,
|
||||||
那些一致並不費力。
|
那些一致並不費力。
|
||||||
|
|
||||||
## 四、隨機性的規模
|
## 四、隨機性的規模與三票制的作用
|
||||||
|
|
||||||
信度數字回答的實際問題是:**這台儀器的隨機性,大到會不會
|
信度數字回答的實際問題是:**這台儀器的隨機性,大到會不會
|
||||||
改變結論?**
|
改變結論?**
|
||||||
|
|
||||||
步驟 3a 的 89,183 格中,搖擺的格子共 2,473 格(兩票 1,163、
|
步驟 3 的 89,183 格中,搖擺的格子共 2,473 格(兩票 1,163、
|
||||||
一票 1,310),佔 2.8%。若只執行一次即定案,這批格子當中約
|
一票 1,310),佔 2.8%。若只執行一次即定案,這批格子當中約
|
||||||
一半會成為誤收、另一半會成為漏收。三票多數決把兩票以上者
|
一半會成為誤收、另一半會成為漏收。三票多數決把兩票以上者
|
||||||
收入、一票者剔除,處理的正是這一批。
|
收入、一票者剔除,處理的正是這一批。
|
||||||
|
|
||||||
|
三票制並非消除隨機性,而是**把隨機性往案例原本的傾向推**。
|
||||||
|
設某個邊緣案例的符合程度為 p,單次執行以機率 p 標出,三次
|
||||||
|
多數決則以 p³+3p²(1−p) 標出:p=0.9 者由 0.9 提高到 0.972,
|
||||||
|
p=0.1 者由 0.1 壓低到 0.028,而 p=0.5 者仍是 0.5——真正
|
||||||
|
模稜兩可的案例,任何票制都救不了。
|
||||||
|
|
||||||
以此規模判斷,結論層的三條帶狀結構(女性力量與陽剛群共現、
|
以此規模判斷,結論層的三條帶狀結構(女性力量與陽剛群共現、
|
||||||
與脆弱群互斥、與厭女群獨立)不可能由這個量級的雜訊翻轉。
|
與脆弱群互斥、與厭女群獨立)不可能由這個量級的雜訊翻轉。
|
||||||
反過來說,若一致率只有 0.6,同一組結論就不能採信——信度
|
反過來說,若一致率只有 0.6,同一組結論就不能採信——信度
|
||||||
@@ -135,8 +146,8 @@ Jaccard(只看「有標到」的格子,忽略雙方都沒標的)約 0.89–0.91,
|
|||||||
## 六、計算方式與依據
|
## 六、計算方式與依據
|
||||||
|
|
||||||
- **分析單位**:每一個「(歌曲, 編碼)」格為一個單位,是非題
|
- **分析單位**:每一個「(歌曲, 編碼)」格為一個單位,是非題
|
||||||
(有標/無標)。步驟 3a 為 883 首 × 101 碼 = 89,183 格;
|
(有標/無標)。步驟 3 為 883 首 × 101 碼 = 89,183 格;
|
||||||
步驟 5d 為各歌適用樣態數合計 3,708 格。
|
步驟 5-4 為各歌適用樣態數合計 3,708 格。
|
||||||
- **Cohen's κ**:兩次執行的 2×2 表,κ=(p_o−p_e)/(1−p_e),
|
- **Cohen's κ**:兩次執行的 2×2 表,κ=(p_o−p_e)/(1−p_e),
|
||||||
p_e 由兩次各自的邊際比例相乘求得。
|
p_e 由兩次各自的邊際比例相乘求得。
|
||||||
- **Krippendorff's α(名目、三位「編碼者」、無缺漏)**:
|
- **Krippendorff's α(名目、三位「編碼者」、無缺漏)**:
|
||||||
|
|||||||
+87
-23
@@ -11,33 +11,81 @@
|
|||||||
**不與先導研究做比較**;
|
**不與先導研究做比較**;
|
||||||
全文數字一律以正式研究結果為準。先導研究僅作為假說的
|
全文數字一律以正式研究結果為準。先導研究僅作為假說的
|
||||||
內部來源,記於決策日誌,不進入論文敘事。
|
內部來源,記於決策日誌,不進入論文敘事。
|
||||||
- **執行原則**:主會話只做討論,分析一律由確定性程序
|
- **執行原則**:主會話只做討論;所有分析由 deterministic
|
||||||
執行。可逐項機械比對的判斷(步驟 3 編碼、步驟 4 選群、
|
script 執行。LLM 步驟以 Python script 呼叫 Anthropic
|
||||||
步驟 5d 樣態標註)採「三次獨立執行+多數決」定案,
|
Messages API(個人 Console 帳號、Batch API 五折),定義
|
||||||
自由生成(步驟 1 自由標註)採兩次執行全數進池,步驟
|
檔逐字作為 system prompt。可逐項機械比對的判斷(步驟 3
|
||||||
5a 至 5c 定為質性閱讀協定;程序細節見 `methodology.md`,
|
編碼、步驟 4 選群、步驟 5-4 樣態標註)採「同一定義檔
|
||||||
工作規範見 `conventions.md`。
|
獨立執行三次+多數決」,計票由確定性子命令完成。自由
|
||||||
|
生成(步驟 1 自由標註)兩次執行全數進池。步驟 5-1 至
|
||||||
|
5-3 為質性閱讀協定:三次獨立閱讀為分析者三角檢核、
|
||||||
|
逐首整合、樣態統整產出草稿,不適用投票與仲裁。詞彙表
|
||||||
|
不經 LLM,由詞向量嵌入+確定性分群產生。驗證結果不符
|
||||||
|
預期則修訂定義檔重跑該循環,絕不手改結果。
|
||||||
- **提示詞只定格式、不定語意**:研究對象是通用 LLM 以其
|
- **提示詞只定格式、不定語意**:研究對象是通用 LLM 以其
|
||||||
網路語料知識背景所做的自然編碼與閱讀,其結果本身是
|
網路語料知識背景所做的自然編碼與閱讀,其結果本身是
|
||||||
批判對象。定義檔只規定任務形狀(輸入、數量範圍、輸出
|
批判對象。定義檔只規定任務形狀(輸入、數量範圍、輸出
|
||||||
格式),不給任何主題的定義、判準或範例。LLM 判斷一律
|
格式),不給任何主題的定義、判準或範例。LLM 判斷一律
|
||||||
要求逐項引述歌詞原句,作為檢視偏差的依據。
|
要求逐項引述歌詞原句,作為檢視偏差的依據。
|
||||||
- **模型**:步驟 1、3 用 `claude-sonnet-4-6`;步驟 4、5
|
- **模型**:步驟 1、3 用 `claude-sonnet-4-6`(temperature=0、
|
||||||
用 `claude-fable-5`——實測發現 sonnet 將「Women Power」
|
thinking 關閉);步驟 4、5 用 `claude-fable-5`(兩參數
|
||||||
拆讀為 women+power 的組合語意,fable-5 讀為詞彙化
|
不適用,均不送出;取樣變異由多數決或整合吸收)——實測
|
||||||
概念,語意層任務因此換用 fable-5(經過見決策日誌)。
|
發現 sonnet 將「Women Power」拆讀為 women+power 的組合
|
||||||
|
語意,fable-5 讀為詞彙化概念,語意層任務因此換用
|
||||||
|
fable-5(經過見決策日誌)。
|
||||||
- **信度與效度**:三次獨立執行量測穩定性(intra-rater
|
- **信度與效度**:三次獨立執行量測穩定性(intra-rater
|
||||||
reliability,可報告兩兩一致率);封閉母體全量檢查取代
|
reliability,可報告兩兩一致率);封閉母體全量檢查取代
|
||||||
抽樣防衛。可重現性定義為「程序透明+可稽核」:公開定義
|
抽樣防衛。可重現性定義為「程序透明+可稽核」:公開定義
|
||||||
檔、記錄 model ID 與執行時間、保存全部原始輸出。
|
檔、記錄 model ID 與執行時間、保存全部原始輸出。
|
||||||
|
|
||||||
## 範圍性定案
|
## 資料儲存與模型
|
||||||
|
|
||||||
|
- **Commit 判準**:凡能由「committed 的輸入+committed 的
|
||||||
|
程式」決定性再生者,不 commit;凡不能者——源頭資料、
|
||||||
|
外部世界的捕捉(Wikidata 快照、LLM 原始輸出)、人工
|
||||||
|
著作——一律以文字格式 commit。格式跟著層次走。
|
||||||
|
- **分層**:
|
||||||
|
- 源頭:原始榜單 CSV(進 git)。
|
||||||
|
- 捕捉:Wikidata 快照 CSV、`runs/` JSONL(皆進 git);
|
||||||
|
歌詞 `.txt` 快取(版權因素 gitignored,為已知的稽核
|
||||||
|
缺口)。
|
||||||
|
- 人工:`data/manual/`(僅研究者親手寫入:編碼修正、
|
||||||
|
演唱者性別修正)。
|
||||||
|
- 工作儲存:SQLite 單檔(`tools/instance/`,generated、
|
||||||
|
不進 git),SQLAlchemy 2.0 typed ORM 定義 schema,
|
||||||
|
設定經 pydantic-settings(`.env` 供應
|
||||||
|
`SQLALCHEMY_DATABASE_URL` 與 `ANTHROPIC_API_KEY`)。
|
||||||
|
歌詞全文入 DB(不進 git 故無版權疑慮)。
|
||||||
|
- 衍生:`build-db` 建置工作儲存的同一動作產出人讀報表
|
||||||
|
(`data/derived/`,進 git),與 SQLite 同交易語意。
|
||||||
|
- 報表:論文引用的定案表進 `results/`——
|
||||||
|
`codings.csv`(步驟 3 定案編碼)、`groups.csv`
|
||||||
|
(步驟 4 定案編碼群)、`patterns.csv`(步驟 5-3 定案
|
||||||
|
樣態表)、`annotations.csv`(步驟 5-4 定案歌×樣態
|
||||||
|
矩陣)。衍生與報表為「可再生仍 commit」的例外,理由:
|
||||||
|
引用穩定性、審稿人零門檻、撰稿期數字變動可 diff。
|
||||||
|
- **資料模型**:`songs`(含 lyrics、`performer_gender`——
|
||||||
|
演唱聲音之性別,由署名藝人之 Wikidata 性別推導後套用
|
||||||
|
`data/manual/performer-gender-corrections.csv` 手工修正)、
|
||||||
|
`chart_entries`、`artists`、`song_artists`、`codings`、
|
||||||
|
`groups`(語意編碼群)、`patterns`(深讀樣態)、
|
||||||
|
`annotations`(歌×樣態定案矩陣)。領域不變量(恰 1000
|
||||||
|
筆榜單、每歌至少一 primary 歌手等)檢查內建於
|
||||||
|
`build-db`,違規即建置失敗。
|
||||||
- **歌手背景防火牆**:歌手背景資料只進人工解讀階段,
|
- **歌手背景防火牆**:歌手背景資料只進人工解讀階段,
|
||||||
**絕不進 LLM 輸入**——LLM 任務的 user message 維持
|
**絕不進 LLM 輸入**——LLM 任務的 user message 維持
|
||||||
歌詞-only 或管線中間產物-only,避免光環偏誤。
|
歌詞-only 或管線中間產物-only,避免光環偏誤。
|
||||||
- **Pilot 歌詞沿用**:先導歌詞沿用之來歷與細節詳見該檔
|
- **Pilot 歌詞沿用(私人匯入,不進發布管線)**:先導研究
|
||||||
(`pilot-study.md`)。
|
捕捉檔(lyrics.json,684 首,2018–2025)以 `excludes/`
|
||||||
|
的私人腳本匯入歌詞快取;讀者的重現路徑純粹是
|
||||||
|
`fetch-lyrics`;沿用之事實記於
|
||||||
|
`data/captures/lyrics-provenance.csv`(進 git)。
|
||||||
|
- **子命令**(`pop-fem-audit-tools <cmd>`):`build-db`
|
||||||
|
(`--codings`、`--groups`、`--gender-corrections`、
|
||||||
|
`--patterns`、`--annotations`)、`cluster-keywords`、
|
||||||
|
`export-llm-input`、`fetch-artists`、`fetch-lyrics`、
|
||||||
|
`run-llm`(`--model` 於模型登錄表中擇一)、
|
||||||
|
`tally-codings`、`tally-groups`、`tally-annotations`。
|
||||||
|
|
||||||
## 分析管線(五步驟,全部完成)
|
## 分析管線(五步驟,全部完成)
|
||||||
|
|
||||||
@@ -48,19 +96,26 @@
|
|||||||
階層式聚合分群 k=100,組名取 medoid;再併入研究者先驗
|
階層式聚合分群 k=100,組名取 medoid;再併入研究者先驗
|
||||||
主題詞 `women-power`(據實揭露的儀器介入),共 101 碼。
|
主題詞 `women-power`(據實揭露的儀器介入),共 101 碼。
|
||||||
3. **編碼(步驟 3)**:以定稿詞彙表對全 883 首編碼,逐
|
3. **編碼(步驟 3)**:以定稿詞彙表對全 883 首編碼,逐
|
||||||
標籤附引述,×3+多數決,產出定案編碼表。「女性力量」
|
標籤附引述,×3+多數決(`tally-codings`),定案
|
||||||
候選集(wp 66 首、fe 144 首、wp∪fe 145 首)由此浮現。
|
`results/codings.csv`。「女性力量」候選集(wp 66 首、
|
||||||
|
fe 144 首、wp∪fe 145 首)由此浮現。
|
||||||
4. **語意編碼群(步驟 4)**:LLM 依編碼字面語意將 101 碼
|
4. **語意編碼群(步驟 4)**:LLM 依編碼字面語意將 101 碼
|
||||||
選入研究者指定的四個主題群(women-power/misogyny/
|
選入研究者指定的四個主題群(women-power/misogyny/
|
||||||
masculine/vulnerable),×3+多數決,產出定案分群表;
|
masculine/vulnerable),×3+多數決(`tally-groups`),
|
||||||
wp/fe 與各編碼、各編碼群之關聯統計(BH-FDR 校正)入
|
定案 `results/groups.csv`;wp/fe 與各編碼、各編碼群之
|
||||||
論文。
|
關聯統計(BH-FDR 校正)入論文。
|
||||||
5. **女性主義問題之質性深讀(步驟 5)**:對 wp∪fe 145 首
|
5. **女性主義問題之質性深讀(步驟 5)**:對 wp∪fe 145 首
|
||||||
——5a 逐首盲讀(僅歌詞全文)×3;5b 逐首整合(收斂
|
——5-1 逐首盲讀(僅歌詞全文)×3;5-2 逐首整合(收斂
|
||||||
註記、主清單限兩讀以上);5c 樣態統整(全體基底+
|
註記、主清單限兩讀以上);5-3 樣態統整(全體基底+
|
||||||
男聲/女聲/混合三個發話脈絡分組);5d 樣態標註——
|
男聲/女聲/混合三個發話脈絡分組);5-4 樣態標註——
|
||||||
以分組樣態表逐首標註「有問題」的 111 首,×3+多數決,
|
以分組樣態表逐首標註「有問題」的 111 首,×3+多數決
|
||||||
產出定案樣態表與歌×樣態定案矩陣。
|
(`tally-annotations`),定案 `results/patterns.csv` 與
|
||||||
|
`results/annotations.csv`。
|
||||||
|
|
||||||
|
定義檔命名 `prompts/<步>-<次步>-<task>.md`,編號的所指是
|
||||||
|
工序而非定義檔(確定性的第 2 步無定義檔仍佔編號);檔名
|
||||||
|
不帶版本號——版本即 git 歷史;每次執行的定義檔快照隨
|
||||||
|
`runs/` 自我完備,token 費用逐筆記於 `docs/run-costs.md`。
|
||||||
|
|
||||||
## 時程與剩餘工作
|
## 時程與剩餘工作
|
||||||
|
|
||||||
@@ -69,3 +124,12 @@
|
|||||||
污染」結論(步驟 5 素材已備)、信度說明、fe 與新自由
|
污染」結論(步驟 5 素材已備)、信度說明、fe 與新自由
|
||||||
主義敘事之詮釋標註。
|
主義敘事之詮釋標註。
|
||||||
|
|
||||||
|
## 其他已定案事項
|
||||||
|
|
||||||
|
- 歌詞受版權保護:完整歌詞不進 git
|
||||||
|
(`data/captures/lyrics/` gitignored),論文與 repo 只留
|
||||||
|
分析所引摘錄。
|
||||||
|
- 工程支援(寫 script、style check、審稿)用公司訂閱的
|
||||||
|
Claude Code;進論文的分析 token 由個人 API 帳號支付。
|
||||||
|
- Positionality statement 寫入論文(53 歲、長年婦運者、
|
||||||
|
資深工程師、離開學術圈多年、因整理榜單而起)。
|
||||||
|
|||||||
+51
-55
@@ -10,61 +10,57 @@ $3/$15、opus-4-6 $5/$25、opus-5 與 fable-5 $10/$50)
|
|||||||
被 `--replace` 取代的執行以「已取代」標記,數字保留供
|
被 `--replace` 取代的執行以「已取代」標記,數字保留供
|
||||||
總支出核算。
|
總支出核算。
|
||||||
|
|
||||||
「步驟」欄的名稱於 2026-08-18 一律換為現行編號(見
|
| 日期 | 步驟 | 執行 | 模型 | 批次 ID | 耗時 | input | output | 費用 (USD) | 狀態 |
|
||||||
`decision-log.md` 當日條目),執行當時的名稱記於末欄
|
|---|---|---|---|---|---|---:|---:|---:|---|
|
||||||
「原步驟名」,未曾改名者留空。已刪除的工序——步驟 2 的
|
| 2026-08-04 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01PTACDQMr8M6ahnshbedjtB | 6 分 27 秒 | 763,318 | 338,661 | $3.68 | 已取代(浮水印清洗與防圍欄修訂後重跑) |
|
||||||
LLM 合併嘗試(`01-02-01-merge`,詞彙表改採詞向量分群時
|
| 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01TikJNd2pZVxzQ8SaybVthu | 4 分 57 秒 | 774,604 | 323,651 | $3.59 | 已取代(合法 JSON 修訂後重跑) |
|
||||||
放棄)與步驟 3 的仲裁(`03-02-arbitration`,改採三票
|
| 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01JFBCNqnu1cwXyqmEKLHQYF | 4 分 30 秒 | 790,370 | 328,227 | $3.65 | 已取代(song-288 平台失敗,整批重跑驗證) |
|
||||||
多數決時刪除)——不另發新名,步驟欄標「已廢棄」,僅存
|
| 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01VSDneWuSbf8mShA32jiWrX | 6 分 15 秒 | 790,370 | 326,193 | $3.63 | 已取代(措辭修訂後全體重跑) |
|
||||||
本表供總支出核算。
|
| 2026-08-05 | 01-01-01-tag | run1-rescue-288 | claude-sonnet-4-6 | msgbatch_019Hq6bNVXVmp4DjRZ2cgVda | 2 分 5 秒 | 1,015 | 376 | $0.01 | 已取代(措辭修訂後全體重跑,該首原生通過)|
|
||||||
|
| 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01VgZ77KAPGWmuu3PnQFqZ7Q | 7 分 1 秒 | 794,913 | 326,435 | $3.64 | 現行 |
|
||||||
|
| 2026-08-05 | 01-01-01-tag | run2 | claude-sonnet-4-6 | msgbatch_01TLFey3L4fimKxcebTZQGYn | 4 分 21 秒 | 794,913 | 328,324 | $3.65 | 現行 |
|
||||||
|
| 2026-08-05 | 01-02-01-merge | run1 | claude-sonnet-4-6 | msgbatch_01BvYMFmH8zrUq9SNxWSNba7 | 9 分 14 秒 | 46,454 | 60,974 | $0.53 | 已取代(完整分割驗證不過:漏 366、重複分派 511、撞名 4) |
|
||||||
|
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-5 | msgbatch_018TkYpdT3DsZqq2q94FQCUN | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) |
|
||||||
|
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-4-8 | msgbatch_018Wuux4adz5mWSvzJjfgLxb | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) |
|
||||||
|
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-4-6 | msgbatch_01Lmt2j1Txyof9CctUK7CbHA | 12 分 19 秒 | 46,454 | 54,864 | $0.80 | 已取代(圍欄違規;漏 190、發明 151、重複 11) |
|
||||||
|
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-5 | msgbatch_012SEfebTnjU5uWN6hhfnhhh | 8 分 45 秒 | 63,368 | 64,000 | $1.92 | 已取代(64k 截斷) |
|
||||||
|
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-5 | msgbatch_01Wi9xEoQKCy1nZK6myVNg77 | 11 分 34 秒 | 63,368 | 70,175 | $2.07 | 已取代(驗證不過:漏 59、發明 140、重複 3) |
|
||||||
|
| 2026-08-05 | 01-02-01-merge | run1 | claude-fable-5 | msgbatch_01M15hUs8P9FDTqWvAHdc1cA | 32 分 45 秒 | 63,368 | 107,721 | $3.01 | 現行歸檔(JSON 語法毀損,驗證不過) |
|
||||||
|
| 2026-08-05 | 01-02-01-merge | run1 | claude-fable-5 | msgbatch_012stRP28uuQDQwobm41HMLh | — | 0 | 0 | $0.00 | 拒收(thinking.type.enabled 不支援) |
|
||||||
|
| 2026-08-05 | 01-02-01-merge | run1 | claude-fable-5 | msgbatch_01KpS7AMx1zAHTcNujgGsouJ | 26 分 28 秒 | 63,368 | 128,000 | $3.52 | 現行歸檔(effort max:128k 全耗於推理,正文空白) |
|
||||||
|
| 2026-08-05 | 03-01-code | run1 | claude-sonnet-4-6 | msgbatch_01L7VepTxSNz5vruvzzjuL2c | 6 分 36 秒 | 1,247,264 | 492,730 | $5.57 | 36 首輸出遭內容過濾攔阻,待修訂重跑 |
|
||||||
|
| 2026-08-05 | 03-01-code | 引述減量實驗(36 首)| claude-sonnet-4-6 | msgbatch_01Hskdhi2DkudYmgZhhgFts7 | 3 分 51 秒 | 56,458 | 11,074 | $0.17 | 實驗:每碼一行引述,36 首全數通過過濾;歸檔不入 repo |
|
||||||
|
| 2026-08-05 | 03-01-code | run1 | claude-sonnet-4-6 | msgbatch_01GG5Ez9KT1pPwtQaY4sW7tv | 4 分 35 秒 | 1,293,724 | 269,166 | $3.96 | 過濾零攔阻;song-168、song-590 因餘額用盡失敗,song-775 拒答 |
|
||||||
|
| 2026-08-05 | 03-01-code | 101 碼樹狀探測(148 首)| claude-sonnet-4-6 | msgbatch_017jo3E5gWVks9b39iWMTqz3 | 2 小時 11 分 | 385,270 | 79,690 | $0.87 | 實驗:k=100 葉碼+women-power;零違規碼; 歸檔不入 repo |
|
||||||
|
| 2026-08-06 | 命名實驗(100 組)| — | claude-sonnet-4-6 | msgbatch_01XSi1YtWzdVyYUzYh7DQRWg | 3 分 5 秒 | 55,090 | 1,168 | $0.09 | 實驗:LLM 命名對照 medoid,未採用;歸檔不入 repo |
|
||||||
|
| 2026-08-06 | 命名實驗(100 組)| — | claude-fable-5 | msgbatch_01Y8SJj1h1ZuSQRErhqkvgZE | 3 分 2 秒 | 76,211 | 2,049 | $0.43 | 實驗:同上,加禁用 themes;未採用;歸檔不入 repo |
|
||||||
|
| 2026-08-06 | 03-01-code | run1 | claude-sonnet-4-6 | msgbatch_01GS1opvurvsf62oknnQxhtx | 3 分 46 秒 | 1,625,458 | 362,759 | $5.16 | 現行(101 碼;883 首全數有效,零攔阻) |
|
||||||
|
| 2026-08-06 | 03-01-code | run2 | claude-sonnet-4-6 | msgbatch_01CxnwNLWzZbRpZK7UAdpb8i | 5 分 22 秒 | 1,625,458 | 364,098 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) |
|
||||||
|
| 2026-08-06 | 03-02-arbitration | — | claude-sonnet-4-6 | msgbatch_019xcQXwrbwDGFc9nE5M8AjE | 4 分 0 秒 | 763,193 | 42,389 | $1.46 | 已取代(2 首遭內容過濾攔阻、13 首輸出夾帶散文;定義檔修訂後重跑) |
|
||||||
|
| 2026-08-06 | 03-02-arbitration | — | claude-sonnet-4-6 | msgbatch_01N7bDbXRSfAVUzzaj2thKeR | 4 分 20 秒 | 781,037 | 39,315 | $1.47 | 現行(644 首全數有效,零攔阻;保留 1,481/送裁 1,699) |
|
||||||
|
| 2026-08-06 | 03-code | run3 | claude-sonnet-4-6 | msgbatch_01KnkCaGETnFJrPddrxZTYHA | 6 分 26 秒 | 1,625,458 | 363,840 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) |
|
||||||
|
| 2026-08-14 | 04-group | run1 | claude-sonnet-4-6 | msgbatch_01UPNedog6feQzJ9WVfSAxBD | 1 分 1 秒 | 3,129 | 470 | $0.01 | 已取代(僅 3 群;改納 women-power 群後重跑) |
|
||||||
|
| 2026-08-14 | 04-group | run1 | claude-sonnet-4-6 | msgbatch_01KntJgxdicStaMjNL12P3zi | 1 分 25 秒 | 4,170 | 558 | $0.01 | 已取代(改以 claude-fable-5 執行;vulnerable 輸出含詞彙表外碼 1 筆;歸檔備份於 excludes/experiments/) |
|
||||||
|
| 2026-08-14 | 04-group | run1 | claude-fable-5 | msgbatch_01Mr6goBb2Efa4YrqCprbP4U | 55 秒 | 5,553 | 2,049 | $0.08 | 現行(4 群;零違規碼;temperature 與 thinking 參數不適用於本模型,未送出) |
|
||||||
|
| 2026-08-14 | 04-group | run2 | claude-fable-5 | msgbatch_01C17xW3YBefThTYZ83g7KkL | 1 分 21 秒 | 5,553 | 1,891 | $0.08 | 現行(4 群;零違規碼) |
|
||||||
|
| 2026-08-14 | 04-group | run3 | claude-fable-5 | msgbatch_01DveEMYyjCAYe6wxpcCD87V | 2 分 9 秒 | 5,553 | 1,975 | $0.08 | 現行(4 群;零違規碼) |
|
||||||
|
| 2026-08-15 | 05-01-read | run1 | claude-fable-5 | msgbatch_018NJejke75o7kzKyjZeeKwi | 2 分 12 秒 | 189,607 | 181,299 | $5.48 | 現行(144/145 有效;song-444 平台錯誤,單筆補送) |
|
||||||
|
| 2026-08-15 | 05-01-read | run1-rescue-444 | claude-fable-5 | msgbatch_01DgygMm6iYn2qEnyqHkTMfa | 1 分 52 秒 | 1,852 | 1,557 | $0.05 | 現行(run1 之 song-444 單筆補送,成功) |
|
||||||
|
| 2026-08-15 | 05-01-read | run2 | claude-fable-5 | msgbatch_011txWpWDtdkNXNiPA74ZUfQ | 15 分 25 秒 | 191,459 | 185,630 | $5.60 | 現行(145/145 有效) |
|
||||||
|
| 2026-08-15 | 05-01-read | run3 | claude-fable-5 | msgbatch_01Unn7C7WCnuVoLraJRXksoG | 7 分 59 秒 | 191,459 | 190,433 | $5.72 | 現行(145/145 有效) |
|
||||||
|
| 2026-08-15 | 05-02-consolidate | run1 | claude-fable-5 | msgbatch_01WPyH4pCCiieHs4yU3xtHzN | 2 分 28 秒 | 326,287 | 242,785 | $7.70 | 現行(145/145 有效) |
|
||||||
|
| 2026-08-15 | 05-03-synthesize | run1 | claude-fable-5 | msgbatch_01XbNQb1SFrUxhk2ZWF7epPA | 3 分 0 秒 | 103,888 | 9,982 | $0.77 | 現行(15 個樣態;研究者審定用草稿) |
|
||||||
|
|
||||||
| 日期 | 步驟 | 執行 | 模型 | 批次 ID | 耗時 | input | output | 費用 (USD) | 狀態 | 原步驟名 |
|
| 2026-08-15 | 05-03-synthesize | run2 | claude-fable-5 | msgbatch_01FKqq8U6m5HLLsx7KpQES26 | 3 分 4 秒 | 13,573 | 6,883 | $0.24 | 現行(男聲群 14 首;13 個樣態) |
|
||||||
|---|---|---|---|---|---|---:|---:|---:|---|---|
|
| 2026-08-15 | 05-03-synthesize | run3 | claude-fable-5 | msgbatch_017tfhiRTnaxqCp6YFyoivsu | 4 分 4 秒 | 60,727 | 8,519 | $0.52 | 現行(女聲群 98 首;14 個樣態) |
|
||||||
| 2026-08-04 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01PTACDQMr8M6ahnshbedjtB | 6 分 27 秒 | 763,318 | 338,661 | $3.68 | 已取代(浮水印清洗與防圍欄修訂後重跑) | 01-01-01-tag |
|
| 2026-08-15 | 05-03-synthesize | run4 | claude-fable-5 | msgbatch_01SNjTLiNPEba4LW2tcUAcTJ | 4 分 3 秒 | 29,958 | 9,793 | $0.39 | 現行(混合群 30 首;16 個樣態) |
|
||||||
| 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01TikJNd2pZVxzQ8SaybVthu | 4 分 57 秒 | 774,604 | 323,651 | $3.59 | 已取代(合法 JSON 修訂後重跑) | 01-01-01-tag |
|
|
||||||
| 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01JFBCNqnu1cwXyqmEKLHQYF | 4 分 30 秒 | 790,370 | 328,227 | $3.65 | 已取代(song-288 平台失敗,整批重跑驗證) | 01-01-01-tag |
|
| 2026-08-15 | 05-04-annotate | run1 | claude-fable-5 | msgbatch_01H9gXXhNbhZG6gzZtvMHxhh | 3 分 9 秒 | 815,492 | 300,901 | $11.60 | 現行(109/111 有效;song-177、song-199 遭 max_tokens 截斷,單筆補送) |
|
||||||
| 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01VSDneWuSbf8mShA32jiWrX | 6 分 15 秒 | 790,370 | 326,193 | $3.63 | 已取代(措辭修訂後全體重跑) | 01-01-01-tag |
|
| 2026-08-15 | 05-04-annotate | run1-rescue-177-199 | claude-fable-5 | msgbatch_01YEdroZ51b8btPfG1aqPFCD | 3 分 5 秒 | 17,802 | 11,601 | $0.38 | 現行(run1 之 2 筆補送,max_tokens 提為 16000,成功) |
|
||||||
| 2026-08-05 | 1-tag | run1-rescue-288 | claude-sonnet-4-6 | msgbatch_019Hq6bNVXVmp4DjRZ2cgVda | 2 分 5 秒 | 1,015 | 376 | $0.01 | 已取代(措辭修訂後全體重跑,該首原生通過)| 01-01-01-tag |
|
| 2026-08-15 | 05-04-annotate | — | claude-fable-5 | msgbatch_01G67STqZsHpnhedat2HRU85 | 3 分 0 秒 | 17,802 | 14,162 | $0.44 | 誤送(補送批次輪詢中斷後誤判死亡而重送;原批次自行完成並歸檔為現行,本批次取消不及、兩筆皆完成,結果棄用) |
|
||||||
| 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01VgZ77KAPGWmuu3PnQFqZ7Q | 7 分 1 秒 | 794,913 | 326,435 | $3.64 | 現行 | 01-01-01-tag |
|
|
||||||
| 2026-08-05 | 1-tag | run2 | claude-sonnet-4-6 | msgbatch_01TLFey3L4fimKxcebTZQGYn | 4 分 21 秒 | 794,913 | 328,324 | $3.65 | 現行 | 01-01-01-tag |
|
| 2026-08-15 | 05-04-annotate | run2 | claude-fable-5 | msgbatch_01VVJpsfidNtF66GnHQdWnGR | 9 分 16 秒 | 815,492 | 306,582 | $11.74 | 現行(111/111 有效;max_tokens 16000,零截斷) |
|
||||||
| 2026-08-05 | 已廢棄 | run1 | claude-sonnet-4-6 | msgbatch_01BvYMFmH8zrUq9SNxWSNba7 | 9 分 14 秒 | 46,454 | 60,974 | $0.53 | 已取代(完整分割驗證不過:漏 366、重複分派 511、撞名 4) | 01-02-01-merge |
|
|
||||||
| 2026-08-05 | 已廢棄 | run1 | claude-opus-5 | msgbatch_018TkYpdT3DsZqq2q94FQCUN | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) | 01-02-01-merge |
|
| 2026-08-15 | 05-04-annotate | run3 | claude-fable-5 | msgbatch_01AsYvZd8rsTYVfUYLzWfeMK | 4 分 12 秒 | 815,492 | 298,512 | $11.54 | 現行(111/111 有效;max_tokens 16000,零截斷) |
|
||||||
| 2026-08-05 | 已廢棄 | run1 | claude-opus-4-8 | msgbatch_018Wuux4adz5mWSvzJjfgLxb | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) | 01-02-01-merge |
|
|
||||||
| 2026-08-05 | 已廢棄 | run1 | claude-opus-4-6 | msgbatch_01Lmt2j1Txyof9CctUK7CbHA | 12 分 19 秒 | 46,454 | 54,864 | $0.80 | 已取代(圍欄違規;漏 190、發明 151、重複 11) | 01-02-01-merge |
|
|
||||||
| 2026-08-05 | 已廢棄 | run1 | claude-opus-5 | msgbatch_012SEfebTnjU5uWN6hhfnhhh | 8 分 45 秒 | 63,368 | 64,000 | $1.92 | 已取代(64k 截斷) | 01-02-01-merge |
|
|
||||||
| 2026-08-05 | 已廢棄 | run1 | claude-opus-5 | msgbatch_01Wi9xEoQKCy1nZK6myVNg77 | 11 分 34 秒 | 63,368 | 70,175 | $2.07 | 已取代(驗證不過:漏 59、發明 140、重複 3) | 01-02-01-merge |
|
|
||||||
| 2026-08-05 | 已廢棄 | run1 | claude-fable-5 | msgbatch_01M15hUs8P9FDTqWvAHdc1cA | 32 分 45 秒 | 63,368 | 107,721 | $3.01 | 現行歸檔(JSON 語法毀損,驗證不過) | 01-02-01-merge |
|
|
||||||
| 2026-08-05 | 已廢棄 | run1 | claude-fable-5 | msgbatch_012stRP28uuQDQwobm41HMLh | — | 0 | 0 | $0.00 | 拒收(thinking.type.enabled 不支援) | 01-02-01-merge |
|
|
||||||
| 2026-08-05 | 已廢棄 | run1 | claude-fable-5 | msgbatch_01KpS7AMx1zAHTcNujgGsouJ | 26 分 28 秒 | 63,368 | 128,000 | $3.52 | 現行歸檔(effort max:128k 全耗於推理,正文空白) | 01-02-01-merge |
|
|
||||||
| 2026-08-05 | 3a-code | run1 | claude-sonnet-4-6 | msgbatch_01L7VepTxSNz5vruvzzjuL2c | 6 分 36 秒 | 1,247,264 | 492,730 | $5.57 | 36 首輸出遭內容過濾攔阻,待修訂重跑 | 03-01-code |
|
|
||||||
| 2026-08-05 | 3a-code | 引述減量實驗(36 首)| claude-sonnet-4-6 | msgbatch_01Hskdhi2DkudYmgZhhgFts7 | 3 分 51 秒 | 56,458 | 11,074 | $0.17 | 實驗:每碼一行引述,36 首全數通過過濾;歸檔不入 repo | 03-01-code |
|
|
||||||
| 2026-08-05 | 3a-code | run1 | claude-sonnet-4-6 | msgbatch_01GG5Ez9KT1pPwtQaY4sW7tv | 4 分 35 秒 | 1,293,724 | 269,166 | $3.96 | 過濾零攔阻;song-168、song-590 因餘額用盡失敗,song-775 拒答 | 03-01-code |
|
|
||||||
| 2026-08-05 | 3a-code | 101 碼樹狀探測(148 首)| claude-sonnet-4-6 | msgbatch_017jo3E5gWVks9b39iWMTqz3 | 2 小時 11 分 | 385,270 | 79,690 | $0.87 | 實驗:k=100 葉碼+women-power;零違規碼; 歸檔不入 repo | 03-01-code |
|
|
||||||
| 2026-08-06 | 命名實驗(100 組)| — | claude-sonnet-4-6 | msgbatch_01XSi1YtWzdVyYUzYh7DQRWg | 3 分 5 秒 | 55,090 | 1,168 | $0.09 | 實驗:LLM 命名對照 medoid,未採用;歸檔不入 repo | |
|
|
||||||
| 2026-08-06 | 命名實驗(100 組)| — | claude-fable-5 | msgbatch_01Y8SJj1h1ZuSQRErhqkvgZE | 3 分 2 秒 | 76,211 | 2,049 | $0.43 | 實驗:同上,加禁用 themes;未採用;歸檔不入 repo | |
|
|
||||||
| 2026-08-06 | 3a-code | run1 | claude-sonnet-4-6 | msgbatch_01GS1opvurvsf62oknnQxhtx | 3 分 46 秒 | 1,625,458 | 362,759 | $5.16 | 現行(101 碼;883 首全數有效,零攔阻) | 03-01-code |
|
|
||||||
| 2026-08-06 | 3a-code | run2 | claude-sonnet-4-6 | msgbatch_01CxnwNLWzZbRpZK7UAdpb8i | 5 分 22 秒 | 1,625,458 | 364,098 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) | 03-01-code |
|
|
||||||
| 2026-08-06 | 已廢棄 | — | claude-sonnet-4-6 | msgbatch_019xcQXwrbwDGFc9nE5M8AjE | 4 分 0 秒 | 763,193 | 42,389 | $1.46 | 已取代(2 首遭內容過濾攔阻、13 首輸出夾帶散文;定義檔修訂後重跑) | 03-02-arbitration |
|
|
||||||
| 2026-08-06 | 已廢棄 | — | claude-sonnet-4-6 | msgbatch_01N7bDbXRSfAVUzzaj2thKeR | 4 分 20 秒 | 781,037 | 39,315 | $1.47 | 現行(644 首全數有效,零攔阻;保留 1,481/送裁 1,699) | 03-02-arbitration |
|
|
||||||
| 2026-08-06 | 3a-code | run3 | claude-sonnet-4-6 | msgbatch_01KnkCaGETnFJrPddrxZTYHA | 6 分 26 秒 | 1,625,458 | 363,840 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) | 03-code |
|
|
||||||
| 2026-08-14 | 4-group | run1 | claude-sonnet-4-6 | msgbatch_01UPNedog6feQzJ9WVfSAxBD | 1 分 1 秒 | 3,129 | 470 | $0.01 | 已取代(僅 3 群;改納 women-power 群後重跑) | 04-group |
|
|
||||||
| 2026-08-14 | 4-group | run1 | claude-sonnet-4-6 | msgbatch_01KntJgxdicStaMjNL12P3zi | 1 分 25 秒 | 4,170 | 558 | $0.01 | 已取代(改以 claude-fable-5 執行;vulnerable 輸出含詞彙表外碼 1 筆;歸檔另行私人備份,不入版本庫) | 04-group |
|
|
||||||
| 2026-08-14 | 4-group | run1 | claude-fable-5 | msgbatch_01Mr6goBb2Efa4YrqCprbP4U | 55 秒 | 5,553 | 2,049 | $0.08 | 現行(4 群;零違規碼;temperature 與 thinking 參數不適用於本模型,未送出) | 04-group |
|
|
||||||
| 2026-08-14 | 4-group | run2 | claude-fable-5 | msgbatch_01C17xW3YBefThTYZ83g7KkL | 1 分 21 秒 | 5,553 | 1,891 | $0.08 | 現行(4 群;零違規碼) | 04-group |
|
|
||||||
| 2026-08-14 | 4-group | run3 | claude-fable-5 | msgbatch_01DveEMYyjCAYe6wxpcCD87V | 2 分 9 秒 | 5,553 | 1,975 | $0.08 | 現行(4 群;零違規碼) | 04-group |
|
|
||||||
| 2026-08-15 | 5a-read | run1 | claude-fable-5 | msgbatch_018NJejke75o7kzKyjZeeKwi | 2 分 12 秒 | 189,607 | 181,299 | $5.48 | 現行(144/145 有效;song-444 平台錯誤,單筆補送) | 05-01-read |
|
|
||||||
| 2026-08-15 | 5a-read | run1-rescue-444 | claude-fable-5 | msgbatch_01DgygMm6iYn2qEnyqHkTMfa | 1 分 52 秒 | 1,852 | 1,557 | $0.05 | 現行(run1 之 song-444 單筆補送,成功) | 05-01-read |
|
|
||||||
| 2026-08-15 | 5a-read | run2 | claude-fable-5 | msgbatch_011txWpWDtdkNXNiPA74ZUfQ | 15 分 25 秒 | 191,459 | 185,630 | $5.60 | 現行(145/145 有效) | 05-01-read |
|
|
||||||
| 2026-08-15 | 5a-read | run3 | claude-fable-5 | msgbatch_01Unn7C7WCnuVoLraJRXksoG | 7 分 59 秒 | 191,459 | 190,433 | $5.72 | 現行(145/145 有效) | 05-01-read |
|
|
||||||
| 2026-08-15 | 5b-consolidate | run1 | claude-fable-5 | msgbatch_01WPyH4pCCiieHs4yU3xtHzN | 2 分 28 秒 | 326,287 | 242,785 | $7.70 | 現行(145/145 有效) | 05-02-consolidate |
|
|
||||||
| 2026-08-15 | 5c-synthesize | run1 | claude-fable-5 | msgbatch_01XbNQb1SFrUxhk2ZWF7epPA | 3 分 0 秒 | 103,888 | 9,982 | $0.77 | 現行(15 個樣態;研究者審定用草稿) | 05-03-synthesize |
|
|
||||||
| 2026-08-15 | 5c-synthesize | run2 | claude-fable-5 | msgbatch_01FKqq8U6m5HLLsx7KpQES26 | 3 分 4 秒 | 13,573 | 6,883 | $0.24 | 現行(男聲群 14 首;13 個樣態) | 05-03-synthesize |
|
|
||||||
| 2026-08-15 | 5c-synthesize | run3 | claude-fable-5 | msgbatch_017tfhiRTnaxqCp6YFyoivsu | 4 分 4 秒 | 60,727 | 8,519 | $0.52 | 現行(女聲群 98 首;14 個樣態) | 05-03-synthesize |
|
|
||||||
| 2026-08-15 | 5c-synthesize | run4 | claude-fable-5 | msgbatch_01SNjTLiNPEba4LW2tcUAcTJ | 4 分 3 秒 | 29,958 | 9,793 | $0.39 | 現行(混合群 30 首;16 個樣態) | 05-03-synthesize |
|
|
||||||
| 2026-08-15 | 5d-annotate | run1 | claude-fable-5 | msgbatch_01H9gXXhNbhZG6gzZtvMHxhh | 3 分 9 秒 | 815,492 | 300,901 | $11.60 | 現行(109/111 有效;song-177、song-199 遭 max_tokens 截斷,單筆補送) | 05-04-annotate |
|
|
||||||
| 2026-08-15 | 5d-annotate | run1-rescue-177-199 | claude-fable-5 | msgbatch_01YEdroZ51b8btPfG1aqPFCD | 3 分 5 秒 | 17,802 | 11,601 | $0.38 | 現行(run1 之 2 筆補送,max_tokens 提為 16000,成功) | 05-04-annotate |
|
|
||||||
| 2026-08-15 | 5d-annotate | — | claude-fable-5 | msgbatch_01G67STqZsHpnhedat2HRU85 | 3 分 0 秒 | 17,802 | 14,162 | $0.44 | 誤送(補送批次輪詢中斷後誤判死亡而重送;原批次自行完成並歸檔為現行,本批次取消不及、兩筆皆完成,結果棄用) | 05-04-annotate |
|
|
||||||
| 2026-08-15 | 5d-annotate | run2 | claude-fable-5 | msgbatch_01VVJpsfidNtF66GnHQdWnGR | 9 分 16 秒 | 815,492 | 306,582 | $11.74 | 現行(111/111 有效;max_tokens 16000,零截斷) | 05-04-annotate |
|
|
||||||
| 2026-08-15 | 5d-annotate | run3 | claude-fable-5 | msgbatch_01AsYvZd8rsTYVfUYLzWfeMK | 4 分 12 秒 | 815,492 | 298,512 | $11.54 | 現行(111/111 有效;max_tokens 16000,零截斷) | 05-04-annotate |
|
|
||||||
|
|
||||||
累計支出:$125.65。
|
累計支出:$125.65。
|
||||||
|
|||||||
|
Can't render this file because it is too large.
|
@@ -1,10 +0,0 @@
|
|||||||
Change Log
|
|
||||||
==========
|
|
||||||
|
|
||||||
|
|
||||||
version 1.0.0
|
|
||||||
-------------
|
|
||||||
|
|
||||||
Released 2026/8/19
|
|
||||||
|
|
||||||
Initial release.
|
|
||||||
@@ -13,8 +13,6 @@ This is a collection of supporting tools for the conference paper "流行音樂
|
|||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
:caption: Contents:
|
:caption: Contents:
|
||||||
|
|
||||||
changelog
|
|
||||||
|
|
||||||
|
|
||||||
Indices and tables
|
Indices and tables
|
||||||
==================
|
==================
|
||||||
|
|||||||
@@ -6,5 +6,5 @@
|
|||||||
"""Tools for A Feminist Audit of Pop Music."""
|
"""Tools for A Feminist Audit of Pop Music."""
|
||||||
|
|
||||||
|
|
||||||
VERSION: str = "1.0.0"
|
VERSION: str = "0.0.0"
|
||||||
"""The package version."""
|
"""The package version."""
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -24,7 +24,8 @@ keyword set for ``export-llm-input --extras`` is written as a JSON
|
|||||||
file holding the group name keywords plus every extra a-priori
|
file holding the group name keywords plus every extra a-priori
|
||||||
keyword the caller gives with the repeatable ``--extra-keyword``
|
keyword the caller gives with the repeatable ``--extra-keyword``
|
||||||
command-line option, as
|
command-line option, as
|
||||||
:attr:`KeywordsToMerge.KEYWORDS_TO_MERGE_JSON`. No default
|
:attr:`KeywordsToMerge.KEYWORDS_TO_MERGE_JSON`; with no
|
||||||
|
``--extra-keyword``, it holds the group names alone. No default
|
||||||
extra keyword is ever injected; the caller supplies each one
|
extra keyword is ever injected; the caller supplies each one
|
||||||
consciously. Finally, the command-line choices and the
|
consciously. Finally, the command-line choices and the
|
||||||
environment that produced the numbers -- neither recoverable from
|
environment that produced the numbers -- neither recoverable from
|
||||||
@@ -159,8 +160,15 @@ class KeywordPooler:
|
|||||||
if line.strip() == "":
|
if line.strip() == "":
|
||||||
continue
|
continue
|
||||||
record: Any = json.loads(line)
|
record: Any = json.loads(line)
|
||||||
|
if not isinstance(record, dict) or "id" not in record:
|
||||||
|
raise ValueError(
|
||||||
|
f"{path}: record without \"id\": {line}")
|
||||||
if "error" in record:
|
if "error" in record:
|
||||||
continue
|
continue
|
||||||
|
if "text" not in record:
|
||||||
|
raise ValueError(
|
||||||
|
f"{path}: id {record['id']}: record without"
|
||||||
|
" \"text\" or \"error\"")
|
||||||
song_id: int = cls.__parse_song_id(record["id"], path)
|
song_id: int = cls.__parse_song_id(record["id"], path)
|
||||||
try:
|
try:
|
||||||
keywords: Any = json.loads(
|
keywords: Any = json.loads(
|
||||||
@@ -289,8 +297,7 @@ class KeywordGroups:
|
|||||||
class KeywordClusterer:
|
class KeywordClusterer:
|
||||||
"""The clusterer of the pooled keywords into coding groups."""
|
"""The clusterer of the pooled keywords into coding groups."""
|
||||||
|
|
||||||
DEFAULT_MODEL: ClassVar[str] \
|
DEFAULT_MODEL: str = "sentence-transformers/all-mpnet-base-v2"
|
||||||
= "sentence-transformers/all-mpnet-base-v2"
|
|
||||||
"""The sentence embedding model used when the caller names
|
"""The sentence embedding model used when the caller names
|
||||||
none."""
|
none."""
|
||||||
|
|
||||||
@@ -661,7 +668,11 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
|||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"output_dir", type=Path,
|
"output_dir", type=Path,
|
||||||
help="the output directory, created if missing, that"
|
help="the output directory, created if missing, that"
|
||||||
" receives the run's output artifacts")
|
f" receives {PooledKeywords.SOURCE_KEYWORDS_TXT},"
|
||||||
|
f" {KeywordGroups.RESULT_KEYWORDS_TXT},"
|
||||||
|
f" {KeywordGroups.RESULT_GROUPS_CSV},"
|
||||||
|
f" {KeywordsToMerge.KEYWORDS_TO_MERGE_JSON},"
|
||||||
|
f" and {RunMeta.META_JSON}")
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--model", default=model,
|
"--model", default=model,
|
||||||
help=f"the sentence embedding model (default \"{model}\")")
|
help=f"the sentence embedding model (default \"{model}\")")
|
||||||
@@ -687,9 +698,16 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
|||||||
def main(argv: list[str] | None = None) -> int:
|
def main(argv: list[str] | None = None) -> int:
|
||||||
"""Pool the two tagging runs' keywords and cluster them.
|
"""Pool the two tagging runs' keywords and cluster them.
|
||||||
|
|
||||||
Creates the output directory (with parents) if it does not
|
Writes the five fixed-named artifacts under the output
|
||||||
exist. Each output artifact is written as soon as its
|
directory, creating it (with parents) if it does not exist:
|
||||||
content is computed, so when the input is rejected, or an
|
the pooled keyword text file; then the group membership CSV
|
||||||
|
file, holding the clustering result alone; the group name
|
||||||
|
keyword text file, holding the same group names as a readable
|
||||||
|
list; the coding keyword set JSON file, holding the group
|
||||||
|
names plus every extra keyword given via ``--extra-keyword``;
|
||||||
|
and the run metadata JSON file, recording the command-line
|
||||||
|
choices and the environment. Each file is written as soon as
|
||||||
|
its content is computed, so when the input is rejected, or an
|
||||||
extra keyword duplicates a group name or another extra
|
extra keyword duplicates a group name or another extra
|
||||||
keyword, the output directory holds whatever the steps before
|
keyword, the output directory holds whatever the steps before
|
||||||
the failing one produced, and the error message names what
|
the failing one produced, and the error message names what
|
||||||
@@ -701,8 +719,8 @@ def main(argv: list[str] | None = None) -> int:
|
|||||||
"""
|
"""
|
||||||
started: float = time.monotonic()
|
started: float = time.monotonic()
|
||||||
args: argparse.Namespace = parse_args(argv)
|
args: argparse.Namespace = parse_args(argv)
|
||||||
|
args.output_dir.mkdir(parents=True, exist_ok=True)
|
||||||
try:
|
try:
|
||||||
args.output_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
source: PooledKeywords = KeywordPooler(
|
source: PooledKeywords = KeywordPooler(
|
||||||
args.run_dir_1, args.run_dir_2, args.output_dir).run()
|
args.run_dir_1, args.run_dir_2, args.output_dir).run()
|
||||||
clusters: KeywordGroups = KeywordClusterer(
|
clusters: KeywordGroups = KeywordClusterer(
|
||||||
@@ -717,7 +735,7 @@ def main(argv: list[str] | None = None) -> int:
|
|||||||
f"Done. Clustered {len(source.keywords)} keywords into"
|
f"Done. Clustered {len(source.keywords)} keywords into"
|
||||||
f" {len(clusters.names)}. {elapsed} elapsed.",
|
f" {len(clusters.names)}. {elapsed} elapsed.",
|
||||||
file=sys.stderr)
|
file=sys.stderr)
|
||||||
except (ClusterError, OSError) as error:
|
except ClusterError as error:
|
||||||
print(f"error: {error}", file=sys.stderr)
|
print(f"error: {error}", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
return 0
|
return 0
|
||||||
|
|||||||
@@ -11,17 +11,26 @@ project's lyrics-only firewall: the output carries only the
|
|||||||
lyrics text of each song, identified by an opaque song key; no
|
lyrics text of each song, identified by an opaque song key; no
|
||||||
title, artist, or chart data crosses into the LLM input.
|
title, artist, or chart data crosses into the LLM input.
|
||||||
|
|
||||||
With ``--extras`` and ``--extras-per-id``, a record's content may
|
With ``--extras``, each record's ``content`` becomes a JSON
|
||||||
carry extra parameters alongside the lyrics, so a step that needs
|
object serialized as a string, its ``lyrics`` key holding the
|
||||||
them can get them without this module knowing what they mean; see
|
song's lyrics followed by the keys of the given extras file in
|
||||||
the exporter's content-building step for how the two merge.
|
their file order, so a step that needs parameters alongside the
|
||||||
|
lyrics can carry them without this module knowing what they mean.
|
||||||
|
|
||||||
|
With ``--extras-per-id``, the same merge happens per song: the
|
||||||
|
given file maps a song ID to the extra keys of that one song, and
|
||||||
|
the export is restricted to the song IDs the file names, so a step
|
||||||
|
that revisits only some of the songs, each with its own parameters,
|
||||||
|
gets exactly those records. The two options may be given together,
|
||||||
|
in which case a record's keys are ``lyrics``, the shared extras'
|
||||||
|
keys, then that song's own keys, each group in its file order.
|
||||||
"""
|
"""
|
||||||
import argparse
|
import argparse
|
||||||
import json
|
import json
|
||||||
import sys
|
import sys
|
||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, ClassVar
|
from typing import Any
|
||||||
|
|
||||||
import sqlalchemy as sa
|
import sqlalchemy as sa
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
@@ -31,270 +40,6 @@ from ..models import Song
|
|||||||
from ..utils import format_duration
|
from ..utils import format_duration
|
||||||
|
|
||||||
|
|
||||||
class LlmInputExporter:
|
|
||||||
"""The exporter of the LLM input JSONL file."""
|
|
||||||
|
|
||||||
__LYRICS_KEY: ClassVar[str] = "lyrics"
|
|
||||||
"""The key holding the lyrics in a record's merged content,
|
|
||||||
and the key forbidden in an extras file."""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self, output_jsonl: Path, extras: Path | None = None,
|
|
||||||
extras_per_id: Path | None = None) -> None:
|
|
||||||
"""Set up the exporter of the LLM input JSONL file.
|
|
||||||
|
|
||||||
:param output_jsonl: The JSONL output file.
|
|
||||||
:param extras: The extras JSON file, or None for none.
|
|
||||||
:param extras_per_id: The per-ID extras JSON file, or
|
|
||||||
None for none.
|
|
||||||
"""
|
|
||||||
self.__output_jsonl: Path = output_jsonl
|
|
||||||
"""The JSONL output file."""
|
|
||||||
self.__extras_path: Path | None = extras
|
|
||||||
"""The extras JSON file, or None for none."""
|
|
||||||
self.__extras_per_id_path: Path | None = extras_per_id
|
|
||||||
"""The per-ID extras JSON file, or None for none."""
|
|
||||||
|
|
||||||
def run(self) -> int:
|
|
||||||
"""Export the songs' lyrics to the output JSONL file.
|
|
||||||
|
|
||||||
:return: The number of songs exported.
|
|
||||||
:raises OSError: When a file cannot be read or written.
|
|
||||||
:raises sqlalchemy.exc.SQLAlchemyError: When the working
|
|
||||||
store cannot be read.
|
|
||||||
:raises ValueError: When an extras file is malformed, an
|
|
||||||
exported song has no lyrics, or the per-ID extras
|
|
||||||
name a song the working store does not have.
|
|
||||||
"""
|
|
||||||
session: Session = ds.get_db()
|
|
||||||
try:
|
|
||||||
extras: dict[str, Any] | None = None
|
|
||||||
if self.__extras_path is not None:
|
|
||||||
extras = self.__load_extras(self.__extras_path)
|
|
||||||
extras_per_id: dict[str, dict[str, Any]] | None = None
|
|
||||||
if self.__extras_per_id_path is not None:
|
|
||||||
extras_per_id = self.__load_extras_per_id(
|
|
||||||
self.__extras_per_id_path)
|
|
||||||
lines: list[str] = self.__build_lines(
|
|
||||||
session, extras, extras_per_id)
|
|
||||||
finally:
|
|
||||||
session.close()
|
|
||||||
self.__write_output(lines)
|
|
||||||
return len(lines)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __no_duplicate_keys(
|
|
||||||
pairs: list[tuple[str, Any]]) -> dict[str, Any]:
|
|
||||||
"""Build a dict from JSON object pairs, rejecting
|
|
||||||
duplicates.
|
|
||||||
|
|
||||||
:param pairs: The key-value pairs of a JSON object, in
|
|
||||||
file order.
|
|
||||||
:return: The pairs as a dict, in file order.
|
|
||||||
:raises ValueError: When a key appears more than once.
|
|
||||||
"""
|
|
||||||
result: dict[str, Any] = {}
|
|
||||||
key: str
|
|
||||||
value: Any
|
|
||||||
for key, value in pairs:
|
|
||||||
if key in result:
|
|
||||||
raise ValueError(
|
|
||||||
f"duplicate key \"{key}\" in extras")
|
|
||||||
result[key] = value
|
|
||||||
return result
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def __load_json_object(
|
|
||||||
cls, path: Path, label: str) -> dict[str, Any]:
|
|
||||||
"""Load a single JSON object from a file, in file order.
|
|
||||||
|
|
||||||
:param path: The JSON file.
|
|
||||||
:param label: The kind of file, for the error messages.
|
|
||||||
:return: The object, in file order.
|
|
||||||
:raises OSError: When the file cannot be read.
|
|
||||||
:raises ValueError: When the file is not valid JSON, is
|
|
||||||
not a JSON object, or has duplicate keys.
|
|
||||||
"""
|
|
||||||
with open(path, encoding="utf-8") as file:
|
|
||||||
text: str = file.read()
|
|
||||||
try:
|
|
||||||
data: Any = json.loads(
|
|
||||||
text, object_pairs_hook=cls.__no_duplicate_keys)
|
|
||||||
except json.JSONDecodeError as error:
|
|
||||||
raise ValueError(
|
|
||||||
f"invalid JSON in {label} file {path}: {error}") \
|
|
||||||
from error
|
|
||||||
if not isinstance(data, dict):
|
|
||||||
raise ValueError(
|
|
||||||
f"{label} file {path} must contain a JSON object")
|
|
||||||
return data
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def __load_extras(cls, path: Path) -> dict[str, Any]:
|
|
||||||
"""Load the extras object from a JSON file.
|
|
||||||
|
|
||||||
:param path: The extras JSON file.
|
|
||||||
:return: The extras, in file order.
|
|
||||||
:raises OSError: When the file cannot be read.
|
|
||||||
:raises ValueError: When the file is not valid JSON, is
|
|
||||||
not a JSON object, has duplicate keys, or has a
|
|
||||||
"lyrics" key.
|
|
||||||
"""
|
|
||||||
data: dict[str, Any] = cls.__load_json_object(
|
|
||||||
path, "extras")
|
|
||||||
if cls.__LYRICS_KEY in data:
|
|
||||||
raise ValueError(
|
|
||||||
f"extras file {path} must not have a"
|
|
||||||
f" \"{cls.__LYRICS_KEY}\" key")
|
|
||||||
return data
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def __load_extras_per_id(
|
|
||||||
cls, path: Path) -> dict[str, dict[str, Any]]:
|
|
||||||
"""Load the per-ID extras object from a JSON file.
|
|
||||||
|
|
||||||
:param path: The per-ID extras JSON file, mapping a song
|
|
||||||
ID, as ``song-<N>``, to the extras of that one song.
|
|
||||||
:return: The extras of each song ID, in file order, every
|
|
||||||
song's own extras in their file order too.
|
|
||||||
:raises OSError: When the file cannot be read.
|
|
||||||
:raises ValueError: When the file is not valid JSON, is
|
|
||||||
not a JSON object, has duplicate keys, has a song
|
|
||||||
whose value is not a JSON object, or has a song with
|
|
||||||
a "lyrics" key.
|
|
||||||
"""
|
|
||||||
data: dict[str, Any] = cls.__load_json_object(
|
|
||||||
path, "per-ID extras")
|
|
||||||
song_id: str
|
|
||||||
extras: Any
|
|
||||||
for song_id, extras in data.items():
|
|
||||||
if not isinstance(extras, dict):
|
|
||||||
raise ValueError(
|
|
||||||
f"per-ID extras file {path}: id {song_id}"
|
|
||||||
" must have a JSON object")
|
|
||||||
if cls.__LYRICS_KEY in extras:
|
|
||||||
raise ValueError(
|
|
||||||
f"per-ID extras file {path}: id {song_id}"
|
|
||||||
f" must not have a \"{cls.__LYRICS_KEY}\""
|
|
||||||
" key")
|
|
||||||
return data
|
|
||||||
|
|
||||||
def __build_lines(
|
|
||||||
self, session: Session,
|
|
||||||
extras: dict[str, Any] | None = None,
|
|
||||||
extras_per_id: dict[str, dict[str, Any]] | None
|
|
||||||
= None) -> list[str]:
|
|
||||||
"""Build the JSONL lines of the exported songs' lyrics.
|
|
||||||
|
|
||||||
Every song is exported, unless per-ID extras are given,
|
|
||||||
in which case only the songs they name are; see
|
|
||||||
:meth:`__build_content` for how the extras merge into a
|
|
||||||
record's content.
|
|
||||||
|
|
||||||
:param session: The database session.
|
|
||||||
:param extras: The extra parameters merged into every
|
|
||||||
record's content alongside the lyrics, in the order
|
|
||||||
they are to appear, or None for none.
|
|
||||||
:param extras_per_id: The extra parameters merged into
|
|
||||||
the content of one record alone, keyed by that
|
|
||||||
record's song ID and in the order they are to appear,
|
|
||||||
restricting the export to the song IDs they name, or
|
|
||||||
None for no such extras and no such restriction.
|
|
||||||
:return: The JSON lines, one per exported song, ordered by
|
|
||||||
song ID.
|
|
||||||
:raises ValueError: When an exported song has no lyrics,
|
|
||||||
or the per-ID extras name a song the working store
|
|
||||||
does not have.
|
|
||||||
"""
|
|
||||||
lines: list[str] = []
|
|
||||||
exported: set[str] = set()
|
|
||||||
song: Song
|
|
||||||
for song in session.scalars(
|
|
||||||
sa.select(Song).order_by(Song.id)):
|
|
||||||
song_id: str = f"song-{song.id}"
|
|
||||||
if extras_per_id is not None \
|
|
||||||
and song_id not in extras_per_id:
|
|
||||||
continue
|
|
||||||
if song.lyrics is None:
|
|
||||||
raise ValueError(
|
|
||||||
f"song {song.id} \"{song.title}\": no lyrics")
|
|
||||||
song_extras: dict[str, Any] | None = None \
|
|
||||||
if extras_per_id is None \
|
|
||||||
else extras_per_id[song_id]
|
|
||||||
content: str = self.__build_content(
|
|
||||||
song.lyrics, extras, song_extras)
|
|
||||||
record: dict[str, str] = {
|
|
||||||
"id": song_id, "content": content}
|
|
||||||
lines.append(json.dumps(record, ensure_ascii=False))
|
|
||||||
exported.add(song_id)
|
|
||||||
if extras_per_id is not None:
|
|
||||||
missing: list[str] = sorted(
|
|
||||||
set(extras_per_id) - exported)
|
|
||||||
if len(missing) > 0:
|
|
||||||
raise ValueError(
|
|
||||||
"the per-ID extras name songs the working"
|
|
||||||
f" store does not have: {', '.join(missing)}")
|
|
||||||
return lines
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def __build_content(
|
|
||||||
cls, lyrics: str, extras: dict[str, Any] | None,
|
|
||||||
song_extras: dict[str, Any] | None) -> str:
|
|
||||||
"""Build the content of one exported record.
|
|
||||||
|
|
||||||
Without extras of either kind, a record's content is the
|
|
||||||
bare lyrics string. With ``--extras``, the content
|
|
||||||
becomes a JSON object serialized as a string, its
|
|
||||||
"lyrics" key holding the song's lyrics followed by the
|
|
||||||
keys of the given extras file, in their file order. With
|
|
||||||
``--extras-per-id``, the same merge happens per song: the
|
|
||||||
song's own extra keys follow the lyrics instead. When
|
|
||||||
both are given, a record's keys are "lyrics", the shared
|
|
||||||
extras' keys, then that song's own keys, each group in
|
|
||||||
its file order.
|
|
||||||
|
|
||||||
:param lyrics: The lyrics of the song.
|
|
||||||
:param extras: The extra parameters shared by every
|
|
||||||
record, in the order they are to appear, or None for
|
|
||||||
none.
|
|
||||||
:param song_extras: The extra parameters of this record
|
|
||||||
alone, in the order they are to appear, or None for
|
|
||||||
none.
|
|
||||||
:return: The bare lyrics when there are no extras of
|
|
||||||
either kind, or otherwise a JSON object serialized as
|
|
||||||
a string, whose first key is "lyrics" holding the
|
|
||||||
lyrics, followed by the shared extras' keys and then
|
|
||||||
this record's own keys, each group in its given
|
|
||||||
order.
|
|
||||||
"""
|
|
||||||
if extras is None and song_extras is None:
|
|
||||||
return lyrics
|
|
||||||
payload: dict[str, Any] = {cls.__LYRICS_KEY: lyrics}
|
|
||||||
if extras is not None:
|
|
||||||
payload.update(extras)
|
|
||||||
if song_extras is not None:
|
|
||||||
payload.update(song_extras)
|
|
||||||
return json.dumps(payload, ensure_ascii=False)
|
|
||||||
|
|
||||||
def __write_output(self, lines: list[str]) -> None:
|
|
||||||
"""Write the exported lines to the output JSONL file.
|
|
||||||
|
|
||||||
Creates the parent directory when it does not exist.
|
|
||||||
|
|
||||||
:param lines: The JSONL lines, in the output order.
|
|
||||||
:return: None.
|
|
||||||
:raises OSError: When the file cannot be written.
|
|
||||||
"""
|
|
||||||
self.__output_jsonl.parent.mkdir(
|
|
||||||
parents=True, exist_ok=True)
|
|
||||||
with open(
|
|
||||||
self.__output_jsonl, "w",
|
|
||||||
encoding="utf-8") as file:
|
|
||||||
line: str
|
|
||||||
for line in lines:
|
|
||||||
file.write(line + "\n")
|
|
||||||
|
|
||||||
|
|
||||||
def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
||||||
"""Parse the command-line arguments.
|
"""Parse the command-line arguments.
|
||||||
|
|
||||||
@@ -311,33 +56,227 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
|||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--extras", type=Path, default=None,
|
"--extras", type=Path, default=None,
|
||||||
help="a JSON file holding a single JSON object of extra"
|
help="a JSON file holding a single JSON object of extra"
|
||||||
" parameters merged into every record's content")
|
" parameters; when given, each record's \"content\""
|
||||||
|
" becomes a JSON object string with a \"lyrics\" key"
|
||||||
|
" followed by the extras' keys, instead of the bare"
|
||||||
|
" lyrics string")
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--extras-per-id", type=Path, default=None,
|
"--extras-per-id", type=Path, default=None,
|
||||||
help="a JSON file holding a single JSON object that maps a"
|
help="a JSON file holding a single JSON object that maps a"
|
||||||
" song ID, as \"song-<N>\", to a JSON object of extra"
|
" song ID, as \"song-<N>\", to a JSON object of extra"
|
||||||
" parameters for that one song, restricting the"
|
" parameters for that one song; the song's object is"
|
||||||
" export to the song IDs the file names")
|
" merged into its \"content\" the same way as with"
|
||||||
|
" --extras, and the export is restricted to the song"
|
||||||
|
" IDs the file names")
|
||||||
return parser.parse_args(argv)
|
return parser.parse_args(argv)
|
||||||
|
|
||||||
|
|
||||||
|
def __no_duplicate_keys(
|
||||||
|
pairs: list[tuple[str, Any]]) -> dict[str, Any]:
|
||||||
|
"""Build a dict from JSON object pairs, rejecting duplicates.
|
||||||
|
|
||||||
|
:param pairs: The key-value pairs of a JSON object, in file
|
||||||
|
order.
|
||||||
|
:return: The pairs as a dict, in file order.
|
||||||
|
:raises ValueError: When a key appears more than once.
|
||||||
|
"""
|
||||||
|
result: dict[str, Any] = {}
|
||||||
|
key: str
|
||||||
|
value: Any
|
||||||
|
for key, value in pairs:
|
||||||
|
if key in result:
|
||||||
|
raise ValueError(f"duplicate key \"{key}\" in extras")
|
||||||
|
result[key] = value
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def __load_json_object(path: Path, label: str) -> dict[str, Any]:
|
||||||
|
"""Load a single JSON object from a file, in file order.
|
||||||
|
|
||||||
|
:param path: The JSON file.
|
||||||
|
:param label: The kind of file, for the error messages.
|
||||||
|
:return: The object, in file order.
|
||||||
|
:raises OSError: When the file cannot be read.
|
||||||
|
:raises ValueError: When the file is not valid JSON, is not
|
||||||
|
a JSON object, or has duplicate keys.
|
||||||
|
"""
|
||||||
|
with open(path, encoding="utf-8") as file:
|
||||||
|
text: str = file.read()
|
||||||
|
try:
|
||||||
|
data: Any = json.loads(
|
||||||
|
text, object_pairs_hook=__no_duplicate_keys)
|
||||||
|
except json.JSONDecodeError as error:
|
||||||
|
raise ValueError(
|
||||||
|
f"invalid JSON in {label} file {path}: {error}") \
|
||||||
|
from error
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
raise ValueError(
|
||||||
|
f"{label} file {path} must contain a JSON object")
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def load_extras(path: Path) -> dict[str, Any]:
|
||||||
|
"""Load the extras object from a JSON file.
|
||||||
|
|
||||||
|
:param path: The extras JSON file.
|
||||||
|
:return: The extras, in file order.
|
||||||
|
:raises OSError: When the file cannot be read.
|
||||||
|
:raises ValueError: When the file is not valid JSON, is not
|
||||||
|
a JSON object, has duplicate keys, or has a "lyrics" key.
|
||||||
|
"""
|
||||||
|
data: dict[str, Any] = __load_json_object(path, "extras")
|
||||||
|
if "lyrics" in data:
|
||||||
|
raise ValueError(
|
||||||
|
f"extras file {path} must not have a \"lyrics\" key")
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def load_extras_per_id(path: Path) -> dict[str, dict[str, Any]]:
|
||||||
|
"""Load the per-ID extras object from a JSON file.
|
||||||
|
|
||||||
|
:param path: The per-ID extras JSON file, mapping a song ID,
|
||||||
|
as ``song-<N>``, to the extras of that one song.
|
||||||
|
:return: The extras of each song ID, in file order, every
|
||||||
|
song's own extras in their file order too.
|
||||||
|
:raises OSError: When the file cannot be read.
|
||||||
|
:raises ValueError: When the file is not valid JSON, is not
|
||||||
|
a JSON object, has duplicate keys, has a song whose value
|
||||||
|
is not a JSON object, or has a song with a "lyrics" key.
|
||||||
|
"""
|
||||||
|
data: dict[str, Any] = __load_json_object(path, "per-ID extras")
|
||||||
|
song_id: str
|
||||||
|
extras: Any
|
||||||
|
for song_id, extras in data.items():
|
||||||
|
if not isinstance(extras, dict):
|
||||||
|
raise ValueError(
|
||||||
|
f"per-ID extras file {path}: id {song_id} must"
|
||||||
|
" have a JSON object")
|
||||||
|
if "lyrics" in extras:
|
||||||
|
raise ValueError(
|
||||||
|
f"per-ID extras file {path}: id {song_id} must not"
|
||||||
|
" have a \"lyrics\" key")
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def __build_content(
|
||||||
|
lyrics: str,
|
||||||
|
extras: dict[str, Any] | None,
|
||||||
|
song_extras: dict[str, Any] | None) -> str:
|
||||||
|
"""Build the content of one exported record.
|
||||||
|
|
||||||
|
:param lyrics: The lyrics of the song.
|
||||||
|
:param extras: The extra parameters shared by every record,
|
||||||
|
in the order they are to appear, or None for none.
|
||||||
|
:param song_extras: The extra parameters of this record
|
||||||
|
alone, in the order they are to appear, or None for none.
|
||||||
|
:return: The bare lyrics when there are no extras of either
|
||||||
|
kind, or otherwise a JSON object serialized as a string,
|
||||||
|
whose first key is ``"lyrics"`` holding the lyrics,
|
||||||
|
followed by the shared extras' keys and then this
|
||||||
|
record's own keys, each group in its given order.
|
||||||
|
"""
|
||||||
|
if extras is None and song_extras is None:
|
||||||
|
return lyrics
|
||||||
|
payload: dict[str, Any] = {"lyrics": lyrics}
|
||||||
|
if extras is not None:
|
||||||
|
payload.update(extras)
|
||||||
|
if song_extras is not None:
|
||||||
|
payload.update(song_extras)
|
||||||
|
return json.dumps(payload, ensure_ascii=False)
|
||||||
|
|
||||||
|
|
||||||
|
def build_lines(
|
||||||
|
session: Session,
|
||||||
|
extras: dict[str, Any] | None = None,
|
||||||
|
extras_per_id: dict[str, dict[str, Any]] | None = None) \
|
||||||
|
-> list[str]:
|
||||||
|
"""Build the JSONL lines of the exported songs' lyrics.
|
||||||
|
|
||||||
|
Without extras of either kind, each record's ``content`` is
|
||||||
|
the bare lyrics string. With extras, ``content`` is a JSON
|
||||||
|
object serialized as a string, whose first key is ``"lyrics"``
|
||||||
|
holding the lyrics string, followed by the shared extras' keys
|
||||||
|
and then the song's own per-ID extras' keys, each group in its
|
||||||
|
given order.
|
||||||
|
|
||||||
|
Every song is exported, unless per-ID extras are given, in
|
||||||
|
which case only the songs they name are.
|
||||||
|
|
||||||
|
:param session: The database session.
|
||||||
|
:param extras: The extra parameters merged into every
|
||||||
|
record's content alongside the lyrics, in the order they
|
||||||
|
are to appear, or None for none.
|
||||||
|
:param extras_per_id: The extra parameters merged into the
|
||||||
|
content of one record alone, keyed by that record's song
|
||||||
|
ID and in the order they are to appear, restricting the
|
||||||
|
export to the song IDs they name, or None for no such
|
||||||
|
extras and no such restriction.
|
||||||
|
:return: The JSON lines, one per exported song, ordered by
|
||||||
|
song ID.
|
||||||
|
:raises ValueError: When an exported song has no lyrics, or
|
||||||
|
the per-ID extras name a song the working store does not
|
||||||
|
have.
|
||||||
|
"""
|
||||||
|
lines: list[str] = []
|
||||||
|
exported: set[str] = set()
|
||||||
|
song: Song
|
||||||
|
for song in session.scalars(sa.select(Song).order_by(Song.id)):
|
||||||
|
song_id: str = f"song-{song.id}"
|
||||||
|
if extras_per_id is not None and song_id not in extras_per_id:
|
||||||
|
continue
|
||||||
|
if song.lyrics is None:
|
||||||
|
raise ValueError(
|
||||||
|
f"song {song.id} \"{song.title}\": no lyrics")
|
||||||
|
song_extras: dict[str, Any] | None = None \
|
||||||
|
if extras_per_id is None else extras_per_id[song_id]
|
||||||
|
content: str = __build_content(
|
||||||
|
song.lyrics, extras, song_extras)
|
||||||
|
record: dict[str, str] = {
|
||||||
|
"id": song_id, "content": content}
|
||||||
|
lines.append(json.dumps(record, ensure_ascii=False))
|
||||||
|
exported.add(song_id)
|
||||||
|
if extras_per_id is not None:
|
||||||
|
missing: list[str] = sorted(set(extras_per_id) - exported)
|
||||||
|
if len(missing) > 0:
|
||||||
|
raise ValueError(
|
||||||
|
"the per-ID extras name songs the working store"
|
||||||
|
f" does not have: {', '.join(missing)}")
|
||||||
|
return lines
|
||||||
|
|
||||||
|
|
||||||
def main(argv: list[str] | None = None) -> int:
|
def main(argv: list[str] | None = None) -> int:
|
||||||
"""Export the LLM input JSONL file from the working store.
|
"""Export the LLM input JSONL file from the working store.
|
||||||
|
|
||||||
|
Every song is exported, unless ``--extras-per-id`` is given,
|
||||||
|
in which case only the songs its file names are.
|
||||||
|
|
||||||
:param argv: The command-line arguments, or None for
|
:param argv: The command-line arguments, or None for
|
||||||
``sys.argv``.
|
``sys.argv``.
|
||||||
:return: The exit status: 0 on success, non-zero on failure.
|
:return: The exit status: 0 on success, non-zero on failure.
|
||||||
"""
|
"""
|
||||||
started: float = time.monotonic()
|
started: float = time.monotonic()
|
||||||
args: argparse.Namespace = parse_args(argv)
|
args: argparse.Namespace = parse_args(argv)
|
||||||
|
session: Session = ds.get_db()
|
||||||
|
lines: list[str]
|
||||||
try:
|
try:
|
||||||
count: int = LlmInputExporter(
|
extras: dict[str, Any] | None = None
|
||||||
args.output_jsonl, args.extras,
|
if args.extras is not None:
|
||||||
args.extras_per_id).run()
|
extras = load_extras(args.extras)
|
||||||
|
extras_per_id: dict[str, dict[str, Any]] | None = None
|
||||||
|
if args.extras_per_id is not None:
|
||||||
|
extras_per_id = load_extras_per_id(args.extras_per_id)
|
||||||
|
lines = build_lines(session, extras, extras_per_id)
|
||||||
except (OSError, sa.exc.SQLAlchemyError, ValueError) as error:
|
except (OSError, sa.exc.SQLAlchemyError, ValueError) as error:
|
||||||
print(f"error: {error}", file=sys.stderr)
|
print(f"error: {error}", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
|
finally:
|
||||||
|
session.close()
|
||||||
|
args.output_jsonl.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with open(args.output_jsonl, "w", encoding="utf-8") as file:
|
||||||
|
line: str
|
||||||
|
for line in lines:
|
||||||
|
file.write(line + "\n")
|
||||||
elapsed: str = format_duration(time.monotonic() - started)
|
elapsed: str = format_duration(time.monotonic() - started)
|
||||||
print(f"Done. {count} songs exported."
|
print(f"Done. {len(lines)} songs exported."
|
||||||
f" {elapsed} elapsed.", file=sys.stderr)
|
f" {elapsed} elapsed.", file=sys.stderr)
|
||||||
return 0
|
return 0
|
||||||
|
|||||||
@@ -12,10 +12,12 @@ layer. The working store is only read, never written; the
|
|||||||
``build-db`` subcommand assembles the captured files into the
|
``build-db`` subcommand assembles the captured files into the
|
||||||
store on the next rebuild.
|
store on the next rebuild.
|
||||||
|
|
||||||
An unresolved artist or an error on one artist is noted on its
|
Every fetched row is meant for later human verification: the
|
||||||
row and does not fail the run. A row whose name is no longer an
|
description of the resolved item is recorded in the note column
|
||||||
artist of the store is dropped from the snapshot and reported on
|
so that a bad match can be spotted. An unresolved artist or an
|
||||||
the standard error.
|
error on one artist is noted on its row and does not fail the
|
||||||
|
run. A row whose name is no longer an artist of the store is
|
||||||
|
dropped from the snapshot and reported on the standard error.
|
||||||
"""
|
"""
|
||||||
import argparse
|
import argparse
|
||||||
import csv
|
import csv
|
||||||
@@ -31,7 +33,7 @@ import urllib.request
|
|||||||
from collections.abc import Container, Sequence
|
from collections.abc import Container, Sequence
|
||||||
from dataclasses import asdict, dataclass, field, fields
|
from dataclasses import asdict, dataclass, field, fields
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, ClassVar, Literal, TextIO
|
from typing import Any, Literal, TextIO
|
||||||
|
|
||||||
import sqlalchemy as sa
|
import sqlalchemy as sa
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
@@ -41,6 +43,71 @@ from ..database import ds
|
|||||||
from ..models import Artist, Song, SongArtist
|
from ..models import Artist, Song, SongArtist
|
||||||
from ..utils import format_duration
|
from ..utils import format_duration
|
||||||
|
|
||||||
|
API_URL: str = "https://www.wikidata.org/w/api.php"
|
||||||
|
"""The URL of the Wikidata API endpoint."""
|
||||||
|
SPARQL_URL: str = "https://query.wikidata.org/sparql"
|
||||||
|
"""The URL of the Wikidata Query Service SPARQL endpoint."""
|
||||||
|
USER_AGENT: str = (
|
||||||
|
f"pop-fem-audit-tools/{VERSION}"
|
||||||
|
" (https://github.com/imacat/pop-fem-audit;"
|
||||||
|
" mailto:imacat@mail.imacat.idv.tw)")
|
||||||
|
"""The User-Agent header sent on every HTTP request."""
|
||||||
|
TIMEOUT: float = 30.0
|
||||||
|
"""The timeout of an API HTTP request, in seconds."""
|
||||||
|
SPARQL_TIMEOUT: float = 90.0
|
||||||
|
"""The timeout of a SPARQL HTTP request, in seconds.
|
||||||
|
|
||||||
|
Higher than the API timeout: the WDQS server aborts a slow
|
||||||
|
query at 60 seconds, and a lower client timeout would race
|
||||||
|
that server-side abort and misclassify a slow-but-answerable
|
||||||
|
query as a client-side timeout instead of letting the server's
|
||||||
|
own HTTP error response arrive and enter the retry path."""
|
||||||
|
SLEEP_SECONDS: float = 1.0
|
||||||
|
"""The delay between consecutive HTTP requests, in seconds."""
|
||||||
|
MAX_ATTEMPTS: int = 5
|
||||||
|
"""The maximum number of attempts on a transient error."""
|
||||||
|
RETRY_SECONDS: float = 15.0
|
||||||
|
"""The back-off unit on a transient error, in seconds;
|
||||||
|
multiplied by the attempt number already made."""
|
||||||
|
RETRY_STATUSES: frozenset[int] = frozenset({429, 500, 502, 503})
|
||||||
|
"""The HTTP statuses that are retried with a back-off."""
|
||||||
|
MAX_STAGE1_TITLES: int = 3
|
||||||
|
"""The maximum number of charted titles used for the stage-1 song
|
||||||
|
corroboration."""
|
||||||
|
HUMAN_QID: str = "Q5"
|
||||||
|
"""The Wikidata item ID of "human"."""
|
||||||
|
ENSEMBLE_QID: str = "Q2088357"
|
||||||
|
"""The Wikidata item ID of "musical ensemble"."""
|
||||||
|
ORIGINAL_CAST_QID: str = "Q106497009"
|
||||||
|
"""The Wikidata item ID of "original cast"."""
|
||||||
|
GROUP_KEYWORDS: Sequence[str] = ("band", "group", "duo", "trio")
|
||||||
|
"""The label keywords that suggest a musical ensemble, covering
|
||||||
|
labels like "boy band" and "girl group"."""
|
||||||
|
NOTE_NOT_FOUND: str = "not found"
|
||||||
|
"""The note sentinel of an artist without a resolved Wikidata
|
||||||
|
item, written to the snapshot and read back for the
|
||||||
|
classification."""
|
||||||
|
CORPUS_START_YEAR: int = 2016
|
||||||
|
"""The first year of the corpus window: a member who left a group
|
||||||
|
before it never performed a corpus song."""
|
||||||
|
MIXED_GENDER: str = "mixed"
|
||||||
|
"""The gender recorded for a group whose members do not share one
|
||||||
|
gender."""
|
||||||
|
TIME_YEAR_PATTERN: re.Pattern[str] = re.compile(r"^[+-]?\d+")
|
||||||
|
"""The leading year of a Wikidata time value."""
|
||||||
|
PINNED_QIDS: dict[str, str] = {}
|
||||||
|
"""The last-resort pinned item IDs, keyed by the artist name.
|
||||||
|
|
||||||
|
An entry is for an artist the algorithm documented on
|
||||||
|
``ArtistFetcher`` is structurally unable to resolve, with its
|
||||||
|
justification recorded here. Currently empty: the only pin ever
|
||||||
|
needed, "Pinkfong" (typed as a brand, which the type gate
|
||||||
|
excludes by design), became moot when the store's artist entity
|
||||||
|
behind that credit was identified as Hope Segoine.
|
||||||
|
|
||||||
|
A pinned name skips the candidate retrieval and corroboration
|
||||||
|
steps; its item ID is used directly."""
|
||||||
|
|
||||||
|
|
||||||
class ArtistType(enum.StrEnum):
|
class ArtistType(enum.StrEnum):
|
||||||
"""The decided artist type of a snapshot row."""
|
"""The decided artist type of a snapshot row."""
|
||||||
@@ -80,14 +147,15 @@ class ArtistSnapshot:
|
|||||||
return asdict(self)
|
return asdict(self)
|
||||||
|
|
||||||
|
|
||||||
|
SNAPSHOT_FIELDS: Sequence[str] = tuple(
|
||||||
|
x.name for x in fields(ArtistSnapshot))
|
||||||
|
"""The header columns of the Wikidata artist snapshot CSV file."""
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class GroupMember:
|
class GroupMember:
|
||||||
"""One has-part member of a Wikidata group item."""
|
"""One has-part member of a Wikidata group item."""
|
||||||
|
|
||||||
__CORPUS_START_YEAR: ClassVar[int] = 2016
|
|
||||||
"""The first year of the corpus window: a member who left a
|
|
||||||
group before it never performed a corpus song."""
|
|
||||||
|
|
||||||
qid: str
|
qid: str
|
||||||
"""The item ID of the member."""
|
"""The item ID of the member."""
|
||||||
start_years: list[int] = field(default_factory=list)
|
start_years: list[int] = field(default_factory=list)
|
||||||
@@ -113,7 +181,7 @@ class GroupMember:
|
|||||||
if len(self.end_years) == 0:
|
if len(self.end_years) == 0:
|
||||||
return True
|
return True
|
||||||
last_end: int = max(self.end_years)
|
last_end: int = max(self.end_years)
|
||||||
if last_end >= self.__CORPUS_START_YEAR:
|
if last_end >= CORPUS_START_YEAR:
|
||||||
return True
|
return True
|
||||||
if len(self.start_years) == 0:
|
if len(self.start_years) == 0:
|
||||||
return False
|
return False
|
||||||
@@ -161,8 +229,20 @@ class RetryExhausted(Exception):
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
NOTE_NOT_FOUND: str = "not found"
|
def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
||||||
"""The note marking an artist that could not be resolved."""
|
"""Parse the command-line arguments.
|
||||||
|
|
||||||
|
:param argv: The command-line arguments, or None for
|
||||||
|
``sys.argv``.
|
||||||
|
:return: The parsed arguments.
|
||||||
|
"""
|
||||||
|
parser: argparse.ArgumentParser = argparse.ArgumentParser(
|
||||||
|
description="Fetch the artist metadata from Wikidata"
|
||||||
|
" into the capture layer.")
|
||||||
|
parser.add_argument(
|
||||||
|
"wikidata_csv", type=Path,
|
||||||
|
help="the Wikidata artist snapshot CSV file")
|
||||||
|
return parser.parse_args(argv)
|
||||||
|
|
||||||
|
|
||||||
class ArtistFetcher:
|
class ArtistFetcher:
|
||||||
@@ -211,71 +291,6 @@ class ArtistFetcher:
|
|||||||
in the note.
|
in the note.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
__API_URL: ClassVar[str] = "https://www.wikidata.org/w/api.php"
|
|
||||||
"""The URL of the Wikidata API endpoint."""
|
|
||||||
__SPARQL_URL: ClassVar[str] \
|
|
||||||
= "https://query.wikidata.org/sparql"
|
|
||||||
"""The URL of the Wikidata Query Service SPARQL endpoint."""
|
|
||||||
__USER_AGENT: ClassVar[str] = (
|
|
||||||
f"pop-fem-audit-tools/{VERSION}"
|
|
||||||
" (https://github.com/imacat/pop-fem-audit;"
|
|
||||||
" mailto:imacat@mail.imacat.idv.tw)")
|
|
||||||
"""The User-Agent header sent on every HTTP request."""
|
|
||||||
__TIMEOUT: ClassVar[float] = 30.0
|
|
||||||
"""The timeout of an API HTTP request, in seconds."""
|
|
||||||
__SPARQL_TIMEOUT: ClassVar[float] = 90.0
|
|
||||||
"""The timeout of a SPARQL HTTP request, in seconds.
|
|
||||||
|
|
||||||
Higher than the API timeout: the WDQS server aborts a slow
|
|
||||||
query at 60 seconds, and a lower client timeout would race
|
|
||||||
that server-side abort and misclassify a slow-but-answerable
|
|
||||||
query as a client-side timeout instead of letting the
|
|
||||||
server's own HTTP error response arrive and enter the retry
|
|
||||||
path."""
|
|
||||||
__SLEEP_SECONDS: ClassVar[float] = 1.0
|
|
||||||
"""The delay between consecutive HTTP requests, in
|
|
||||||
seconds."""
|
|
||||||
__MAX_ATTEMPTS: ClassVar[int] = 5
|
|
||||||
"""The maximum number of attempts on a transient error."""
|
|
||||||
__RETRY_SECONDS: ClassVar[float] = 15.0
|
|
||||||
"""The back-off unit on a transient error, in seconds;
|
|
||||||
multiplied by the attempt number already made."""
|
|
||||||
__RETRY_STATUSES: ClassVar[frozenset[int]] \
|
|
||||||
= frozenset({429, 500, 502, 503})
|
|
||||||
"""The HTTP statuses that are retried with a back-off."""
|
|
||||||
__MAX_STAGE1_TITLES: ClassVar[int] = 3
|
|
||||||
"""The maximum number of charted titles used for the
|
|
||||||
stage-1 song corroboration."""
|
|
||||||
__HUMAN_QID: ClassVar[str] = "Q5"
|
|
||||||
"""The Wikidata item ID of "human"."""
|
|
||||||
__ENSEMBLE_QID: ClassVar[str] = "Q2088357"
|
|
||||||
"""The Wikidata item ID of "musical ensemble"."""
|
|
||||||
__ORIGINAL_CAST_QID: ClassVar[str] = "Q106497009"
|
|
||||||
"""The Wikidata item ID of "original cast"."""
|
|
||||||
__GROUP_KEYWORDS: ClassVar[Sequence[str]] \
|
|
||||||
= ("band", "group", "duo", "trio")
|
|
||||||
"""The label keywords that suggest a musical ensemble,
|
|
||||||
covering labels like "boy band" and "girl group"."""
|
|
||||||
__MIXED_GENDER: ClassVar[str] = "mixed"
|
|
||||||
"""The gender recorded for a group whose members do not
|
|
||||||
share one gender."""
|
|
||||||
__TIME_YEAR_PATTERN: ClassVar[re.Pattern[str]] \
|
|
||||||
= re.compile(r"^[+-]?\d+")
|
|
||||||
"""The leading year of a Wikidata time value."""
|
|
||||||
__PINNED_QIDS: ClassVar[dict[str, str]] = {}
|
|
||||||
"""The last-resort pinned item IDs, keyed by the artist name.
|
|
||||||
|
|
||||||
An entry is for an artist the algorithm documented on
|
|
||||||
``ArtistFetcher`` is structurally unable to resolve, with its
|
|
||||||
justification recorded here. Currently empty: the only pin
|
|
||||||
ever needed, "Pinkfong" (typed as a brand, which the type
|
|
||||||
gate excludes by design), became moot when the store's
|
|
||||||
artist entity behind that credit was identified as Hope
|
|
||||||
Segoine.
|
|
||||||
|
|
||||||
A pinned name skips the candidate retrieval and corroboration
|
|
||||||
steps; its item ID is used directly."""
|
|
||||||
|
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
"""Construct the fetcher."""
|
"""Construct the fetcher."""
|
||||||
self.__sent: int = 0
|
self.__sent: int = 0
|
||||||
@@ -327,8 +342,8 @@ class ArtistFetcher:
|
|||||||
transient error are exhausted.
|
transient error are exhausted.
|
||||||
:raises ValueError: On a JSON decoding error.
|
:raises ValueError: On a JSON decoding error.
|
||||||
"""
|
"""
|
||||||
if name in self.__PINNED_QIDS:
|
if name in PINNED_QIDS:
|
||||||
return self.__PINNED_QIDS[name]
|
return PINNED_QIDS[name]
|
||||||
candidates: list[str] = self.__candidates(name)
|
candidates: list[str] = self.__candidates(name)
|
||||||
if len(candidates) == 0:
|
if len(candidates) == 0:
|
||||||
return None
|
return None
|
||||||
@@ -358,11 +373,11 @@ class ArtistFetcher:
|
|||||||
{{ ?item rdfs:label ?name }}
|
{{ ?item rdfs:label ?name }}
|
||||||
UNION {{ ?item skos:altLabel ?name }}
|
UNION {{ ?item skos:altLabel ?name }}
|
||||||
{{
|
{{
|
||||||
?item wdt:P31 wd:{self.__HUMAN_QID}
|
?item wdt:P31 wd:{HUMAN_QID}
|
||||||
}} UNION {{
|
}} UNION {{
|
||||||
?item wdt:P31/wdt:P279* wd:{self.__ENSEMBLE_QID}
|
?item wdt:P31/wdt:P279* wd:{ENSEMBLE_QID}
|
||||||
}} UNION {{
|
}} UNION {{
|
||||||
?item wdt:P31 wd:{self.__ORIGINAL_CAST_QID}
|
?item wdt:P31 wd:{ORIGINAL_CAST_QID}
|
||||||
}}
|
}}
|
||||||
}}
|
}}
|
||||||
"""
|
"""
|
||||||
@@ -388,7 +403,7 @@ class ArtistFetcher:
|
|||||||
transient error are exhausted.
|
transient error are exhausted.
|
||||||
:raises ValueError: On a JSON decoding error.
|
:raises ValueError: On a JSON decoding error.
|
||||||
"""
|
"""
|
||||||
subset: Sequence[str] = titles[:self.__MAX_STAGE1_TITLES]
|
subset: Sequence[str] = titles[:MAX_STAGE1_TITLES]
|
||||||
if len(subset) == 0:
|
if len(subset) == 0:
|
||||||
return None
|
return None
|
||||||
query: str = f"""
|
query: str = f"""
|
||||||
@@ -517,7 +532,7 @@ class ArtistFetcher:
|
|||||||
qid: str
|
qid: str
|
||||||
for qid in qids:
|
for qid in qids:
|
||||||
member: MemberClaims = claims.get(qid, MemberClaims())
|
member: MemberClaims = claims.get(qid, MemberClaims())
|
||||||
if self.__HUMAN_QID not in member.instance_of_ids:
|
if HUMAN_QID not in member.instance_of_ids:
|
||||||
continue
|
continue
|
||||||
if len(member.gender_ids) == 0:
|
if len(member.gender_ids) == 0:
|
||||||
return
|
return
|
||||||
@@ -528,7 +543,7 @@ class ArtistFetcher:
|
|||||||
[x[1] for x in genders], any_language=True)
|
[x[1] for x in genders], any_language=True)
|
||||||
unique: set[str] = {x[1] for x in genders}
|
unique: set[str] = {x[1] for x in genders}
|
||||||
snapshot.gender = labels[genders[0][1]] \
|
snapshot.gender = labels[genders[0][1]] \
|
||||||
if len(unique) == 1 else self.__MIXED_GENDER
|
if len(unique) == 1 else MIXED_GENDER
|
||||||
basis: str = "gender derived from members: " + "; ".join(
|
basis: str = "gender derived from members: " + "; ".join(
|
||||||
f"{x} {labels[y]}" for x, y in genders)
|
f"{x} {labels[y]}" for x, y in genders)
|
||||||
snapshot.note = f"{snapshot.note}; {basis}" \
|
snapshot.note = f"{snapshot.note}; {basis}" \
|
||||||
@@ -662,8 +677,7 @@ class ArtistFetcher:
|
|||||||
or not isinstance(value.get("time"), str):
|
or not isinstance(value.get("time"), str):
|
||||||
continue
|
continue
|
||||||
match: re.Match[str] | None \
|
match: re.Match[str] | None \
|
||||||
= ArtistFetcher.__TIME_YEAR_PATTERN.match(
|
= TIME_YEAR_PATTERN.match(value["time"])
|
||||||
value["time"])
|
|
||||||
if match is not None:
|
if match is not None:
|
||||||
years.append(int(match.group()))
|
years.append(int(match.group()))
|
||||||
return years
|
return years
|
||||||
@@ -792,13 +806,12 @@ class ArtistFetcher:
|
|||||||
``ArtistType.GROUP`` for a musical ensemble, or the
|
``ArtistType.GROUP`` for a musical ensemble, or the
|
||||||
empty string for the human to decide.
|
empty string for the human to decide.
|
||||||
"""
|
"""
|
||||||
if ArtistFetcher.__HUMAN_QID in type_ids:
|
if HUMAN_QID in type_ids:
|
||||||
return ArtistType.SOLO
|
return ArtistType.SOLO
|
||||||
qid: str
|
qid: str
|
||||||
for qid in type_ids:
|
for qid in type_ids:
|
||||||
label: str = labels.get(qid, "").lower()
|
label: str = labels.get(qid, "").lower()
|
||||||
if any(x in label
|
if any(x in label for x in GROUP_KEYWORDS):
|
||||||
for x in ArtistFetcher.__GROUP_KEYWORDS):
|
|
||||||
return ArtistType.GROUP
|
return ArtistType.GROUP
|
||||||
return ""
|
return ""
|
||||||
|
|
||||||
@@ -814,15 +827,14 @@ class ArtistFetcher:
|
|||||||
transient error are exhausted.
|
transient error are exhausted.
|
||||||
:raises ValueError: On a JSON decoding error.
|
:raises ValueError: On a JSON decoding error.
|
||||||
"""
|
"""
|
||||||
url: str = (
|
url: str = (f"{SPARQL_URL}?"
|
||||||
f"{self.__SPARQL_URL}?"
|
f"{urllib.parse.urlencode({'query': query})}")
|
||||||
f"{urllib.parse.urlencode({'query': query})}")
|
|
||||||
request: urllib.request.Request = urllib.request.Request(
|
request: urllib.request.Request = urllib.request.Request(
|
||||||
url, headers={
|
url, headers={
|
||||||
"User-Agent": self.__USER_AGENT,
|
"User-Agent": USER_AGENT,
|
||||||
"Accept": "application/sparql-results+json"})
|
"Accept": "application/sparql-results+json"})
|
||||||
body: bytes = self.__send(
|
body: bytes = self.__send(
|
||||||
request, timeout=self.__SPARQL_TIMEOUT)
|
request, timeout=SPARQL_TIMEOUT)
|
||||||
data: Any = json.loads(body)
|
data: Any = json.loads(body)
|
||||||
bindings: Any = None
|
bindings: Any = None
|
||||||
if isinstance(data, dict) \
|
if isinstance(data, dict) \
|
||||||
@@ -856,14 +868,13 @@ class ArtistFetcher:
|
|||||||
transient error are exhausted.
|
transient error are exhausted.
|
||||||
:raises ValueError: On a JSON decoding error.
|
:raises ValueError: On a JSON decoding error.
|
||||||
"""
|
"""
|
||||||
url: str \
|
url: str = f"{API_URL}?{urllib.parse.urlencode(params)}"
|
||||||
= f"{self.__API_URL}?{urllib.parse.urlencode(params)}"
|
|
||||||
request: urllib.request.Request = urllib.request.Request(
|
request: urllib.request.Request = urllib.request.Request(
|
||||||
url, headers={"User-Agent": self.__USER_AGENT})
|
url, headers={"User-Agent": USER_AGENT})
|
||||||
return json.loads(self.__send(request))
|
return json.loads(self.__send(request))
|
||||||
|
|
||||||
def __send(self, request: urllib.request.Request,
|
def __send(self, request: urllib.request.Request,
|
||||||
timeout: float = __TIMEOUT) -> bytes:
|
timeout: float = TIMEOUT) -> bytes:
|
||||||
"""Send an HTTP request, retrying on a transient error.
|
"""Send an HTTP request, retrying on a transient error.
|
||||||
|
|
||||||
Consecutive requests are separated by a fixed delay. A
|
Consecutive requests are separated by a fixed delay. A
|
||||||
@@ -881,7 +892,7 @@ class ArtistFetcher:
|
|||||||
transient error are exhausted.
|
transient error are exhausted.
|
||||||
"""
|
"""
|
||||||
if self.__sent > 0:
|
if self.__sent > 0:
|
||||||
time.sleep(self.__SLEEP_SECONDS)
|
time.sleep(SLEEP_SECONDS)
|
||||||
self.__sent += 1
|
self.__sent += 1
|
||||||
attempt: int = 1
|
attempt: int = 1
|
||||||
reason: str | None
|
reason: str | None
|
||||||
@@ -894,10 +905,10 @@ class ArtistFetcher:
|
|||||||
reason = self.__retry_reason(error)
|
reason = self.__retry_reason(error)
|
||||||
if reason is None:
|
if reason is None:
|
||||||
raise
|
raise
|
||||||
if attempt >= self.__MAX_ATTEMPTS:
|
if attempt >= MAX_ATTEMPTS:
|
||||||
raise RetryExhausted(
|
raise RetryExhausted(
|
||||||
f"retries exhausted ({reason})") from error
|
f"retries exhausted ({reason})") from error
|
||||||
time.sleep(self.__RETRY_SECONDS * attempt)
|
time.sleep(RETRY_SECONDS * attempt)
|
||||||
attempt += 1
|
attempt += 1
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
@@ -912,7 +923,7 @@ class ArtistFetcher:
|
|||||||
the error is not transient and must not be retried.
|
the error is not transient and must not be retried.
|
||||||
"""
|
"""
|
||||||
if isinstance(error, urllib.error.HTTPError):
|
if isinstance(error, urllib.error.HTTPError):
|
||||||
if error.code not in ArtistFetcher.__RETRY_STATUSES:
|
if error.code not in RETRY_STATUSES:
|
||||||
return None
|
return None
|
||||||
return str(error)
|
return str(error)
|
||||||
if isinstance(error, TimeoutError):
|
if isinstance(error, TimeoutError):
|
||||||
@@ -957,216 +968,92 @@ class ArtistFetcher:
|
|||||||
return uri.rsplit("/", 1)[-1]
|
return uri.rsplit("/", 1)[-1]
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
def read_snapshot_rows(file: TextIO) -> list[dict[str, str]]:
|
||||||
class FetchCounts:
|
"""Read the current rows of a snapshot CSV file handle.
|
||||||
"""The outcome counts of one snapshot update run."""
|
|
||||||
|
|
||||||
fetched: int
|
:param file: The open, seekable snapshot CSV file.
|
||||||
"""The number of artists newly resolved."""
|
:return: The rows, keyed by the column name.
|
||||||
not_found: int
|
:raises OSError: When the file cannot be read.
|
||||||
"""The number of artists left unresolved."""
|
|
||||||
errors: int
|
|
||||||
"""The number of artists that ended in an error."""
|
|
||||||
|
|
||||||
|
|
||||||
class ArtistSnapshotUpdater:
|
|
||||||
"""The updater of the Wikidata artist snapshot CSV file.
|
|
||||||
|
|
||||||
Fetches the metadata of every artist of the working store
|
|
||||||
that the snapshot does not resolve yet, appends a row for
|
|
||||||
each to the snapshot as it is fetched, and rewrites the
|
|
||||||
snapshot sorted by artist name with its stale rows dropped.
|
|
||||||
"""
|
"""
|
||||||
|
file.seek(0)
|
||||||
|
reader: csv.DictReader[str] = csv.DictReader(file)
|
||||||
|
return list(reader)
|
||||||
|
|
||||||
__SNAPSHOT_FIELDS: ClassVar[Sequence[str]] = tuple(
|
|
||||||
x.name for x in fields(ArtistSnapshot))
|
|
||||||
"""The header columns of the Wikidata artist snapshot CSV file."""
|
|
||||||
|
|
||||||
def __init__(self, wikidata_csv: Path) -> None:
|
def read_artist_titles(session: Session,
|
||||||
"""Set up the updater.
|
artist_id: int) -> list[str]:
|
||||||
|
"""Read the charted song titles credited to an artist.
|
||||||
|
|
||||||
:param wikidata_csv: The Wikidata artist snapshot CSV
|
:param session: The database session.
|
||||||
file.
|
:param artist_id: The artist ID.
|
||||||
"""
|
:return: The song titles credited to the artist, ordered by
|
||||||
self.__wikidata_csv: Path = wikidata_csv
|
the song ID, with the duplicate titles removed.
|
||||||
"""The Wikidata artist snapshot CSV file."""
|
"""
|
||||||
|
titles: Sequence[str] = session.scalars(
|
||||||
|
sa.select(Song.title)
|
||||||
|
.join(SongArtist, SongArtist.song_id == Song.id)
|
||||||
|
.where(SongArtist.artist_id == artist_id)
|
||||||
|
.order_by(Song.id)).all()
|
||||||
|
return list(dict.fromkeys(titles))
|
||||||
|
|
||||||
def run(self) -> FetchCounts:
|
|
||||||
"""Fetch every unresolved artist and update the snapshot.
|
|
||||||
|
|
||||||
:return: The counts of the run.
|
def ensure_snapshot_header(file: TextIO) -> None:
|
||||||
:raises OSError: When the snapshot file, or its parent
|
"""Write the snapshot CSV header row if the file is empty.
|
||||||
directory, cannot be read or written.
|
|
||||||
:raises sqlalchemy.exc.SQLAlchemyError: When the working
|
|
||||||
store cannot be read.
|
|
||||||
"""
|
|
||||||
session: Session = ds.get_db()
|
|
||||||
try:
|
|
||||||
return self.__run(session)
|
|
||||||
finally:
|
|
||||||
session.close()
|
|
||||||
|
|
||||||
def __run(self, session: Session) -> FetchCounts:
|
:param file: The open, seekable snapshot CSV file.
|
||||||
"""Run the fetch loop with an open database session.
|
:return: None.
|
||||||
|
:raises OSError: When the file cannot be written.
|
||||||
:param session: The database session.
|
"""
|
||||||
:return: The counts of the run.
|
file.seek(0, os.SEEK_END)
|
||||||
:raises OSError: When the snapshot file, or its parent
|
if file.tell() == 0:
|
||||||
directory, cannot be read or written.
|
csv.writer(file).writerow(SNAPSHOT_FIELDS)
|
||||||
"""
|
|
||||||
fetcher: ArtistFetcher = ArtistFetcher()
|
|
||||||
fetched: int = 0
|
|
||||||
not_found: int = 0
|
|
||||||
errors: int = 0
|
|
||||||
self.__wikidata_csv.parent.mkdir(
|
|
||||||
parents=True, exist_ok=True)
|
|
||||||
with open(self.__wikidata_csv, "a+", encoding="utf-8",
|
|
||||||
newline="") as csv_file:
|
|
||||||
done: set[str] = {
|
|
||||||
x["name"] for x in
|
|
||||||
self.__read_snapshot_rows(csv_file)
|
|
||||||
if x["gender"] != ""}
|
|
||||||
self.__ensure_snapshot_header(csv_file)
|
|
||||||
names: set[str] = set()
|
|
||||||
artist: Artist
|
|
||||||
for artist in session.scalars(
|
|
||||||
sa.select(Artist).order_by(Artist.id)):
|
|
||||||
names.add(artist.name)
|
|
||||||
if artist.name in done:
|
|
||||||
continue
|
|
||||||
titles: list[str] = self.__read_artist_titles(
|
|
||||||
session, artist.id)
|
|
||||||
snapshot: ArtistSnapshot = fetcher.fetch(
|
|
||||||
artist.name, titles)
|
|
||||||
self.__append_row(csv_file, snapshot)
|
|
||||||
status: str = snapshot.qid
|
|
||||||
if snapshot.note == NOTE_NOT_FOUND:
|
|
||||||
not_found += 1
|
|
||||||
status = NOTE_NOT_FOUND
|
|
||||||
elif snapshot.note.startswith("error: "):
|
|
||||||
errors += 1
|
|
||||||
status = snapshot.note
|
|
||||||
else:
|
|
||||||
fetched += 1
|
|
||||||
print(f"artist \"{artist.name}\": {status}",
|
|
||||||
file=sys.stderr)
|
|
||||||
self.__write_snapshot(csv_file, names)
|
|
||||||
return FetchCounts(
|
|
||||||
fetched=fetched, not_found=not_found, errors=errors)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __read_snapshot_rows(file: TextIO) \
|
|
||||||
-> list[dict[str, str]]:
|
|
||||||
"""Read the current rows of a snapshot CSV file handle.
|
|
||||||
|
|
||||||
:param file: The open, seekable snapshot CSV file.
|
|
||||||
:return: The rows, keyed by the column name.
|
|
||||||
:raises OSError: When the file cannot be read.
|
|
||||||
"""
|
|
||||||
file.seek(0)
|
|
||||||
reader: csv.DictReader[str] = csv.DictReader(file)
|
|
||||||
return list(reader)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __read_artist_titles(session: Session,
|
|
||||||
artist_id: int) -> list[str]:
|
|
||||||
"""Read the charted song titles credited to an artist.
|
|
||||||
|
|
||||||
:param session: The database session.
|
|
||||||
:param artist_id: The artist ID.
|
|
||||||
:return: The song titles credited to the artist, ordered
|
|
||||||
by the song ID, with the duplicate titles removed.
|
|
||||||
"""
|
|
||||||
titles: Sequence[str] = session.scalars(
|
|
||||||
sa.select(Song.title)
|
|
||||||
.join(SongArtist, SongArtist.song_id == Song.id)
|
|
||||||
.where(SongArtist.artist_id == artist_id)
|
|
||||||
.order_by(Song.id)).all()
|
|
||||||
return list(dict.fromkeys(titles))
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __ensure_snapshot_header(file: TextIO) -> None:
|
|
||||||
"""Write the snapshot CSV header row if the file is
|
|
||||||
empty.
|
|
||||||
|
|
||||||
:param file: The open, seekable snapshot CSV file.
|
|
||||||
:return: None.
|
|
||||||
:raises OSError: When the file cannot be written.
|
|
||||||
"""
|
|
||||||
file.seek(0, os.SEEK_END)
|
|
||||||
if file.tell() == 0:
|
|
||||||
csv.writer(file).writerow(
|
|
||||||
ArtistSnapshotUpdater.__SNAPSHOT_FIELDS)
|
|
||||||
file.flush()
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __append_row(file: TextIO,
|
|
||||||
snapshot: ArtistSnapshot) -> None:
|
|
||||||
"""Append a snapshot row to a snapshot CSV file handle.
|
|
||||||
|
|
||||||
:param file: The open snapshot CSV file, opened for
|
|
||||||
append.
|
|
||||||
:param snapshot: The snapshot of an artist.
|
|
||||||
:return: None.
|
|
||||||
:raises OSError: When the file cannot be written.
|
|
||||||
"""
|
|
||||||
csv.DictWriter(
|
|
||||||
file, ArtistSnapshotUpdater.__SNAPSHOT_FIELDS).writerow(
|
|
||||||
snapshot.to_row())
|
|
||||||
file.flush()
|
file.flush()
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __write_snapshot(file: TextIO, names: Container[str]) \
|
|
||||||
-> None:
|
|
||||||
"""Rewrite a snapshot CSV file handle sorted by artist
|
|
||||||
name.
|
|
||||||
|
|
||||||
The rows are ordered by the case-folded artist name,
|
def append_row(file: TextIO, snapshot: ArtistSnapshot) -> None:
|
||||||
matching the convention of the derived ``artists.csv``.
|
"""Append a snapshot row to a snapshot CSV file handle.
|
||||||
An artist keeps one row only, the last one of the file,
|
|
||||||
so that a re-fetched artist replaces its earlier row. A
|
|
||||||
row whose name is not an artist of the store is dropped
|
|
||||||
and reported on the standard error.
|
|
||||||
|
|
||||||
:param file: The open, seekable snapshot CSV file.
|
:param file: The open snapshot CSV file, opened for append.
|
||||||
:param names: The artist names of the working store.
|
:param snapshot: The snapshot of an artist.
|
||||||
:return: None.
|
:return: None.
|
||||||
:raises OSError: When the file cannot be read or written.
|
:raises OSError: When the file cannot be written.
|
||||||
"""
|
|
||||||
kept: dict[str, dict[str, str]] = {}
|
|
||||||
row: dict[str, str]
|
|
||||||
for row in ArtistSnapshotUpdater.__read_snapshot_rows(
|
|
||||||
file):
|
|
||||||
if row["name"] not in names:
|
|
||||||
print(f"dropped stale row \"{row['name']}\":"
|
|
||||||
" no such artist in the store",
|
|
||||||
file=sys.stderr)
|
|
||||||
continue
|
|
||||||
kept[row["name"]] = row
|
|
||||||
ordered: list[dict[str, str]] = sorted(
|
|
||||||
kept.values(), key=lambda x: x["name"].casefold())
|
|
||||||
file.seek(0)
|
|
||||||
file.truncate()
|
|
||||||
writer: csv.DictWriter[str] = csv.DictWriter(
|
|
||||||
file, ArtistSnapshotUpdater.__SNAPSHOT_FIELDS)
|
|
||||||
writer.writeheader()
|
|
||||||
writer.writerows(ordered)
|
|
||||||
|
|
||||||
|
|
||||||
def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
|
||||||
"""Parse the command-line arguments.
|
|
||||||
|
|
||||||
:param argv: The command-line arguments, or None for
|
|
||||||
``sys.argv``.
|
|
||||||
:return: The parsed arguments.
|
|
||||||
"""
|
"""
|
||||||
parser: argparse.ArgumentParser = argparse.ArgumentParser(
|
csv.DictWriter(file, SNAPSHOT_FIELDS).writerow(
|
||||||
description="Fetch the artist metadata from Wikidata"
|
snapshot.to_row())
|
||||||
" into the capture layer.")
|
file.flush()
|
||||||
parser.add_argument(
|
|
||||||
"wikidata_csv", type=Path,
|
|
||||||
help="the Wikidata artist snapshot CSV file")
|
def write_snapshot(file: TextIO, names: Container[str]) -> None:
|
||||||
return parser.parse_args(argv)
|
"""Rewrite a snapshot CSV file handle sorted by artist name.
|
||||||
|
|
||||||
|
The rows are ordered by the case-folded artist name, matching
|
||||||
|
the convention of the derived ``artists.csv``. An artist
|
||||||
|
keeps one row only, the last one of the file, so that a
|
||||||
|
re-fetched artist replaces its earlier row. A row whose name
|
||||||
|
is not an artist of the store is dropped and reported on the
|
||||||
|
standard error.
|
||||||
|
|
||||||
|
:param file: The open, seekable snapshot CSV file.
|
||||||
|
:param names: The artist names of the working store.
|
||||||
|
:return: None.
|
||||||
|
:raises OSError: When the file cannot be read or written.
|
||||||
|
"""
|
||||||
|
kept: dict[str, dict[str, str]] = {}
|
||||||
|
row: dict[str, str]
|
||||||
|
for row in read_snapshot_rows(file):
|
||||||
|
if row["name"] not in names:
|
||||||
|
print(f"dropped stale row \"{row['name']}\":"
|
||||||
|
" no such artist in the store", file=sys.stderr)
|
||||||
|
continue
|
||||||
|
kept[row["name"]] = row
|
||||||
|
ordered: list[dict[str, str]] = sorted(
|
||||||
|
kept.values(), key=lambda x: x["name"].casefold())
|
||||||
|
file.seek(0)
|
||||||
|
file.truncate()
|
||||||
|
writer: csv.DictWriter[str] = csv.DictWriter(
|
||||||
|
file, SNAPSHOT_FIELDS)
|
||||||
|
writer.writeheader()
|
||||||
|
writer.writerows(ordered)
|
||||||
|
|
||||||
|
|
||||||
def main(argv: list[str] | None = None) -> int:
|
def main(argv: list[str] | None = None) -> int:
|
||||||
@@ -1179,16 +1066,52 @@ def main(argv: list[str] | None = None) -> int:
|
|||||||
"""
|
"""
|
||||||
started: float = time.monotonic()
|
started: float = time.monotonic()
|
||||||
args: argparse.Namespace = parse_args(argv)
|
args: argparse.Namespace = parse_args(argv)
|
||||||
|
fetcher: ArtistFetcher = ArtistFetcher()
|
||||||
|
fetched: int = 0
|
||||||
|
not_found: int = 0
|
||||||
|
errors: int = 0
|
||||||
|
session: Session = ds.get_db()
|
||||||
try:
|
try:
|
||||||
counts: FetchCounts \
|
args.wikidata_csv.parent.mkdir(
|
||||||
= ArtistSnapshotUpdater(args.wikidata_csv).run()
|
parents=True, exist_ok=True)
|
||||||
|
with open(args.wikidata_csv, "a+", encoding="utf-8",
|
||||||
|
newline="") as csv_file:
|
||||||
|
done: set[str] = {x["name"] for x in
|
||||||
|
read_snapshot_rows(csv_file)
|
||||||
|
if x["gender"] != ""}
|
||||||
|
ensure_snapshot_header(csv_file)
|
||||||
|
names: set[str] = set()
|
||||||
|
artist: Artist
|
||||||
|
for artist in session.scalars(
|
||||||
|
sa.select(Artist).order_by(Artist.id)):
|
||||||
|
names.add(artist.name)
|
||||||
|
if artist.name in done:
|
||||||
|
continue
|
||||||
|
titles: list[str] = read_artist_titles(
|
||||||
|
session, artist.id)
|
||||||
|
snapshot: ArtistSnapshot = fetcher.fetch(
|
||||||
|
artist.name, titles)
|
||||||
|
append_row(csv_file, snapshot)
|
||||||
|
status: str = snapshot.qid
|
||||||
|
if snapshot.note == NOTE_NOT_FOUND:
|
||||||
|
not_found += 1
|
||||||
|
status = "not found"
|
||||||
|
elif snapshot.note.startswith("error: "):
|
||||||
|
errors += 1
|
||||||
|
status = snapshot.note
|
||||||
|
else:
|
||||||
|
fetched += 1
|
||||||
|
print(f"artist \"{artist.name}\": {status}",
|
||||||
|
file=sys.stderr)
|
||||||
|
write_snapshot(csv_file, names)
|
||||||
except (OSError, sa.exc.SQLAlchemyError) as error:
|
except (OSError, sa.exc.SQLAlchemyError) as error:
|
||||||
print(f"error: {error}", file=sys.stderr)
|
print(f"error: {error}", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
attempted: int = counts.fetched + counts.not_found \
|
finally:
|
||||||
+ counts.errors
|
session.close()
|
||||||
|
attempted: int = fetched + not_found + errors
|
||||||
elapsed: str = format_duration(time.monotonic() - started)
|
elapsed: str = format_duration(time.monotonic() - started)
|
||||||
print(f"Done. Resolved {counts.fetched}/{attempted}"
|
print(f"Done. Resolved {fetched}/{attempted} artists."
|
||||||
f" artists. {elapsed} elapsed.",
|
f" {elapsed} elapsed.",
|
||||||
file=sys.stderr)
|
file=sys.stderr)
|
||||||
return 0
|
return 0
|
||||||
|
|||||||
@@ -30,9 +30,8 @@ import time
|
|||||||
import urllib.parse
|
import urllib.parse
|
||||||
import urllib.request
|
import urllib.request
|
||||||
from collections.abc import Sequence
|
from collections.abc import Sequence
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, ClassVar
|
from typing import Any
|
||||||
|
|
||||||
import sqlalchemy as sa
|
import sqlalchemy as sa
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
@@ -46,6 +45,60 @@ from ..models import (
|
|||||||
)
|
)
|
||||||
from ..utils import format_duration
|
from ..utils import format_duration
|
||||||
|
|
||||||
|
PROVENANCE_FIELDS: Sequence[str] = (
|
||||||
|
"song_id", "source", "method", "acquired_at", "note")
|
||||||
|
"""The header columns of the lyrics provenance CSV file."""
|
||||||
|
USER_AGENT: str = ("pop-fem-audit-tools"
|
||||||
|
" (https://github.com/imacat/pop-fem-audit)")
|
||||||
|
"""The User-Agent header sent on every HTTP request."""
|
||||||
|
TIMEOUT: float = 30.0
|
||||||
|
"""The timeout of an HTTP request, in seconds."""
|
||||||
|
SLEEP_SECONDS: float = 1.0
|
||||||
|
"""The delay between consecutive HTTP requests, in seconds."""
|
||||||
|
|
||||||
|
|
||||||
|
def __build_normalization() -> dict[int, str | None]:
|
||||||
|
"""Build the lyrics normalization translation table.
|
||||||
|
|
||||||
|
:return: The codepoint-to-replacement mapping, a replacement
|
||||||
|
of None meaning removal.
|
||||||
|
"""
|
||||||
|
table: dict[int, str | None] = {}
|
||||||
|
codepoint: int
|
||||||
|
for codepoint in range(0x80, 0xa0):
|
||||||
|
try:
|
||||||
|
table[codepoint] = bytes([codepoint]).decode("cp1252")
|
||||||
|
except UnicodeDecodeError:
|
||||||
|
table[codepoint] = None
|
||||||
|
table[0x0435] = "e"
|
||||||
|
table[0x03cc] = "ó"
|
||||||
|
for codepoint in (0x2005, 0x205f, 0x200a):
|
||||||
|
table[codepoint] = " "
|
||||||
|
for codepoint in (0x200b, 0x200c, 0x200d, 0xfeff):
|
||||||
|
table[codepoint] = None
|
||||||
|
return table
|
||||||
|
|
||||||
|
|
||||||
|
NORMALIZATION: dict[int, str | None] = __build_normalization()
|
||||||
|
"""The codepoint-to-replacement mapping applied to fetched
|
||||||
|
lyrics: cp1252-mojibake restoration for U+0080-U+009F (with the
|
||||||
|
five byte values undefined in cp1252 removed), homoglyph
|
||||||
|
restoration for the Cyrillic "e" and the Greek "o" with tonos,
|
||||||
|
ASCII-space restoration for exotic space variants, and removal
|
||||||
|
of zero-width characters. A replacement of None removes the
|
||||||
|
codepoint."""
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_lyrics(text: str) -> str:
|
||||||
|
"""Restore or remove watermark and mojibake characters.
|
||||||
|
|
||||||
|
:param text: The lyrics text as fetched from an API.
|
||||||
|
:return: The text with the codepoints in
|
||||||
|
:data:`NORMALIZATION` replaced or removed; every other
|
||||||
|
character is unchanged.
|
||||||
|
"""
|
||||||
|
return text.translate(NORMALIZATION)
|
||||||
|
|
||||||
|
|
||||||
def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
||||||
"""Parse the command-line arguments.
|
"""Parse the command-line arguments.
|
||||||
@@ -69,16 +122,6 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
|||||||
class LyricsFetcher:
|
class LyricsFetcher:
|
||||||
"""A fetcher of song lyrics from the public lyrics APIs."""
|
"""A fetcher of song lyrics from the public lyrics APIs."""
|
||||||
|
|
||||||
__USER_AGENT: ClassVar[str] = (
|
|
||||||
"pop-fem-audit-tools"
|
|
||||||
" (https://github.com/imacat/pop-fem-audit)")
|
|
||||||
"""The User-Agent header sent on every HTTP request."""
|
|
||||||
__TIMEOUT: ClassVar[float] = 30.0
|
|
||||||
"""The timeout of an HTTP request, in seconds."""
|
|
||||||
__SLEEP_SECONDS: ClassVar[float] = 1.0
|
|
||||||
"""The delay between consecutive HTTP requests, in
|
|
||||||
seconds."""
|
|
||||||
|
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
"""Construct the fetcher."""
|
"""Construct the fetcher."""
|
||||||
self.__sent: int = 0
|
self.__sent: int = 0
|
||||||
@@ -150,216 +193,78 @@ class LyricsFetcher:
|
|||||||
network, or decoding error.
|
network, or decoding error.
|
||||||
"""
|
"""
|
||||||
if self.__sent > 0:
|
if self.__sent > 0:
|
||||||
time.sleep(self.__SLEEP_SECONDS)
|
time.sleep(SLEEP_SECONDS)
|
||||||
self.__sent += 1
|
self.__sent += 1
|
||||||
request: urllib.request.Request = urllib.request.Request(
|
request: urllib.request.Request = urllib.request.Request(
|
||||||
url, headers={"User-Agent": self.__USER_AGENT})
|
url, headers={"User-Agent": USER_AGENT})
|
||||||
try:
|
try:
|
||||||
with urllib.request.urlopen(
|
with urllib.request.urlopen(
|
||||||
request, timeout=self.__TIMEOUT) as response:
|
request, timeout=TIMEOUT) as response:
|
||||||
return json.load(response)
|
return json.load(response)
|
||||||
except (OSError, ValueError):
|
except (OSError, ValueError):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
def query_artist(session: Session, song_id: int) -> str:
|
||||||
class LyricsFetchCounts:
|
"""Find the artist name to query the APIs with.
|
||||||
"""The outcome of one run of fetching the missing lyrics."""
|
|
||||||
|
|
||||||
fetched: int
|
:param session: The database session.
|
||||||
"""The number of songs newly fetched."""
|
:param song_id: The song ID.
|
||||||
missed: int
|
:return: The name of the primary-role artist with the lowest
|
||||||
"""The number of songs every API missed."""
|
position.
|
||||||
|
"""
|
||||||
|
name: str | None = session.scalar(
|
||||||
|
sa.select(Artist.name)
|
||||||
|
.join(SongArtist, SongArtist.artist_id == Artist.id)
|
||||||
|
.where(SongArtist.song_id == song_id,
|
||||||
|
SongArtist.role == Role.PRIMARY)
|
||||||
|
.order_by(SongArtist.position)
|
||||||
|
.limit(1))
|
||||||
|
assert name is not None
|
||||||
|
return name
|
||||||
|
|
||||||
|
|
||||||
class LyricsFetchRunner:
|
def save_lyrics(lyrics_dir: Path, song_id: int,
|
||||||
"""The orchestrator of one run of fetching missing lyrics."""
|
lyrics: str) -> None:
|
||||||
|
"""Write the lyrics of a song into the cache directory.
|
||||||
|
|
||||||
__PROVENANCE_FIELDS: ClassVar[Sequence[str]] = (
|
The cache directory is created when missing.
|
||||||
"song_id", "source", "method", "acquired_at", "note")
|
|
||||||
"""The header columns of the lyrics provenance CSV file."""
|
|
||||||
|
|
||||||
@staticmethod
|
The lyrics text is normalized with :func:`normalize_lyrics`
|
||||||
def __build_normalization() -> dict[int, str | None]:
|
before being written.
|
||||||
"""Build the lyrics normalization translation table.
|
|
||||||
|
|
||||||
:return: The codepoint-to-replacement mapping, a
|
:param lyrics_dir: The lyrics cache directory.
|
||||||
replacement of None meaning removal.
|
:param song_id: The song ID.
|
||||||
"""
|
:param lyrics: The lyrics text.
|
||||||
table: dict[int, str | None] = {}
|
:return: None.
|
||||||
codepoint: int
|
:raises OSError: When the file cannot be written.
|
||||||
for codepoint in range(0x80, 0xa0):
|
"""
|
||||||
try:
|
lyrics_dir.mkdir(parents=True, exist_ok=True)
|
||||||
table[codepoint] = bytes(
|
(lyrics_dir / f"{song_id}.txt").write_text(
|
||||||
[codepoint]).decode("cp1252")
|
normalize_lyrics(lyrics), encoding="utf-8")
|
||||||
except UnicodeDecodeError:
|
|
||||||
table[codepoint] = None
|
|
||||||
table[0x0435] = "e"
|
|
||||||
table[0x03cc] = "ó"
|
|
||||||
for codepoint in (0x2005, 0x205f, 0x200a):
|
|
||||||
table[codepoint] = " "
|
|
||||||
for codepoint in (0x200b, 0x200c, 0x200d, 0xfeff):
|
|
||||||
table[codepoint] = None
|
|
||||||
return table
|
|
||||||
|
|
||||||
__NORMALIZATION: ClassVar[dict[int, str | None]] \
|
|
||||||
= __build_normalization()
|
|
||||||
"""The codepoint-to-replacement mapping applied to fetched
|
|
||||||
lyrics: cp1252-mojibake restoration for U+0080-U+009F (with
|
|
||||||
the five byte values undefined in cp1252 removed), homoglyph
|
|
||||||
restoration for the Cyrillic "e" and the Greek "o" with
|
|
||||||
tonos, ASCII-space restoration for exotic space variants, and
|
|
||||||
removal of zero-width characters. A replacement of None
|
|
||||||
removes the codepoint."""
|
|
||||||
|
|
||||||
def __init__(self, lyrics_dir: Path,
|
def append_provenance(path: Path, song_id: int,
|
||||||
provenance_csv: Path) -> None:
|
source: str) -> None:
|
||||||
"""Set up the fetch run.
|
"""Append a provenance row for a fetched lyrics file.
|
||||||
|
|
||||||
:param lyrics_dir: The lyrics cache directory.
|
The CSV file is created with the header row when missing.
|
||||||
:param provenance_csv: The lyrics provenance CSV file.
|
|
||||||
"""
|
|
||||||
self.__lyrics_dir: Path = lyrics_dir
|
|
||||||
"""The lyrics cache directory."""
|
|
||||||
self.__provenance_csv: Path = provenance_csv
|
|
||||||
"""The lyrics provenance CSV file."""
|
|
||||||
self.__fetcher: LyricsFetcher = LyricsFetcher()
|
|
||||||
"""The fetcher of the public lyrics APIs."""
|
|
||||||
|
|
||||||
def run(self) -> LyricsFetchCounts:
|
:param path: The lyrics provenance CSV file.
|
||||||
"""Fetch the missing lyrics of every song in the store.
|
:param song_id: The song ID.
|
||||||
|
:param source: The source name of the fetched lyrics.
|
||||||
Every song fetched or missed is reported on the standard
|
:return: None.
|
||||||
error as an observable side effect.
|
:raises OSError: When the file cannot be written.
|
||||||
|
"""
|
||||||
:return: The number of songs fetched and missed.
|
is_new: bool = not path.exists()
|
||||||
:raises OSError: When a cache file or the provenance CSV
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
cannot be written.
|
with open(path, "a", encoding="utf-8",
|
||||||
:raises sqlalchemy.exc.SQLAlchemyError: On a database
|
newline="") as file:
|
||||||
error.
|
writer: Any = csv.writer(file)
|
||||||
"""
|
if is_new:
|
||||||
fetched: int = 0
|
writer.writerow(PROVENANCE_FIELDS)
|
||||||
missed: int = 0
|
writer.writerow([song_id, source, "api-fetch",
|
||||||
session: Session = ds.get_db()
|
datetime.date.today().isoformat(), ""])
|
||||||
try:
|
|
||||||
song: Song
|
|
||||||
for song in session.scalars(
|
|
||||||
sa.select(Song).order_by(Song.id)):
|
|
||||||
if (self.__lyrics_dir
|
|
||||||
/ f"{song.id}.txt").exists():
|
|
||||||
continue
|
|
||||||
if self.__fetch_one(session, song):
|
|
||||||
fetched += 1
|
|
||||||
else:
|
|
||||||
missed += 1
|
|
||||||
finally:
|
|
||||||
session.close()
|
|
||||||
return LyricsFetchCounts(fetched=fetched, missed=missed)
|
|
||||||
|
|
||||||
def __fetch_one(self, session: Session, song: Song) -> bool:
|
|
||||||
"""Fetch and save the lyrics of one song.
|
|
||||||
|
|
||||||
The song is queried by its primary-role artist name; when
|
|
||||||
every API misses and the song's full artist credit
|
|
||||||
differs from that name, the same APIs are queried again
|
|
||||||
with the artist credit.
|
|
||||||
|
|
||||||
:param session: The database session.
|
|
||||||
:param song: The song to fetch.
|
|
||||||
:return: True when a lyrics text was fetched and saved,
|
|
||||||
False when every API missed on both queries.
|
|
||||||
:raises OSError: When the cache file or the provenance
|
|
||||||
CSV cannot be written.
|
|
||||||
"""
|
|
||||||
artist: str = self.__query_artist(session, song.id)
|
|
||||||
result: tuple[str, str] | None = self.__fetcher.fetch(
|
|
||||||
artist, song.title)
|
|
||||||
if result is None and song.artist_credit != artist:
|
|
||||||
result = self.__fetcher.fetch(
|
|
||||||
song.artist_credit, song.title)
|
|
||||||
if result is None:
|
|
||||||
print(f"song {song.id} \"{song.title}\": miss",
|
|
||||||
file=sys.stderr)
|
|
||||||
return False
|
|
||||||
lyrics: str
|
|
||||||
source: str
|
|
||||||
lyrics, source = result
|
|
||||||
self.__save_lyrics(song.id, lyrics)
|
|
||||||
self.__append_provenance(song.id, source)
|
|
||||||
print(f"song {song.id} \"{song.title}\": {source}",
|
|
||||||
file=sys.stderr)
|
|
||||||
return True
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __query_artist(session: Session, song_id: int) -> str:
|
|
||||||
"""Find the artist name to query the APIs with.
|
|
||||||
|
|
||||||
:param session: The database session.
|
|
||||||
:param song_id: The song ID.
|
|
||||||
:return: The name of the primary-role artist with the
|
|
||||||
lowest position.
|
|
||||||
"""
|
|
||||||
name: str | None = session.scalar(
|
|
||||||
sa.select(Artist.name)
|
|
||||||
.join(SongArtist, SongArtist.artist_id == Artist.id)
|
|
||||||
.where(SongArtist.song_id == song_id,
|
|
||||||
SongArtist.role == Role.PRIMARY)
|
|
||||||
.order_by(SongArtist.position)
|
|
||||||
.limit(1))
|
|
||||||
assert name is not None
|
|
||||||
return name
|
|
||||||
|
|
||||||
def __save_lyrics(self, song_id: int, lyrics: str) -> None:
|
|
||||||
"""Write the lyrics of a song into the cache directory.
|
|
||||||
|
|
||||||
The cache directory is created when missing.
|
|
||||||
|
|
||||||
The lyrics text is normalized with
|
|
||||||
:meth:`normalize_lyrics` before being written.
|
|
||||||
|
|
||||||
:param song_id: The song ID.
|
|
||||||
:param lyrics: The lyrics text.
|
|
||||||
:return: None.
|
|
||||||
:raises OSError: When the file cannot be written.
|
|
||||||
"""
|
|
||||||
self.__lyrics_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
(self.__lyrics_dir / f"{song_id}.txt").write_text(
|
|
||||||
self.normalize_lyrics(lyrics), encoding="utf-8")
|
|
||||||
|
|
||||||
def __append_provenance(self, song_id: int,
|
|
||||||
source: str) -> None:
|
|
||||||
"""Append a provenance row for a fetched lyrics file.
|
|
||||||
|
|
||||||
The CSV file is created with the header row when
|
|
||||||
missing.
|
|
||||||
|
|
||||||
:param song_id: The song ID.
|
|
||||||
:param source: The source name of the fetched lyrics.
|
|
||||||
:return: None.
|
|
||||||
:raises OSError: When the file cannot be written.
|
|
||||||
"""
|
|
||||||
is_new: bool = not self.__provenance_csv.exists()
|
|
||||||
self.__provenance_csv.parent.mkdir(
|
|
||||||
parents=True, exist_ok=True)
|
|
||||||
with open(self.__provenance_csv, "a", encoding="utf-8",
|
|
||||||
newline="") as file:
|
|
||||||
writer: Any = csv.writer(file)
|
|
||||||
if is_new:
|
|
||||||
writer.writerow(self.__PROVENANCE_FIELDS)
|
|
||||||
writer.writerow(
|
|
||||||
[song_id, source, "api-fetch",
|
|
||||||
datetime.date.today().isoformat(), ""])
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def normalize_lyrics(cls, text: str) -> str:
|
|
||||||
"""Restore or remove watermark and mojibake characters.
|
|
||||||
|
|
||||||
:param text: The lyrics text as fetched from an API.
|
|
||||||
:return: The text with the codepoints of the
|
|
||||||
normalization table replaced or removed; every other
|
|
||||||
character is unchanged.
|
|
||||||
"""
|
|
||||||
return text.translate(cls.__NORMALIZATION)
|
|
||||||
|
|
||||||
|
|
||||||
def main(argv: list[str] | None = None) -> int:
|
def main(argv: list[str] | None = None) -> int:
|
||||||
@@ -372,15 +277,44 @@ def main(argv: list[str] | None = None) -> int:
|
|||||||
"""
|
"""
|
||||||
started: float = time.monotonic()
|
started: float = time.monotonic()
|
||||||
args: argparse.Namespace = parse_args(argv)
|
args: argparse.Namespace = parse_args(argv)
|
||||||
|
fetcher: LyricsFetcher = LyricsFetcher()
|
||||||
|
fetched: int = 0
|
||||||
|
missed: int = 0
|
||||||
|
session: Session = ds.get_db()
|
||||||
try:
|
try:
|
||||||
counts: LyricsFetchCounts = LyricsFetchRunner(
|
song: Song
|
||||||
args.lyrics_dir, args.provenance_csv).run()
|
for song in session.scalars(
|
||||||
|
sa.select(Song).order_by(Song.id)):
|
||||||
|
if (args.lyrics_dir / f"{song.id}.txt").exists():
|
||||||
|
continue
|
||||||
|
artist: str = query_artist(session, song.id)
|
||||||
|
result: tuple[str, str] | None = fetcher.fetch(
|
||||||
|
artist, song.title)
|
||||||
|
if result is None and song.artist_credit != artist:
|
||||||
|
result = fetcher.fetch(
|
||||||
|
song.artist_credit, song.title)
|
||||||
|
if result is None:
|
||||||
|
missed += 1
|
||||||
|
print(f"song {song.id} \"{song.title}\": miss",
|
||||||
|
file=sys.stderr)
|
||||||
|
continue
|
||||||
|
lyrics: str
|
||||||
|
source: str
|
||||||
|
lyrics, source = result
|
||||||
|
save_lyrics(args.lyrics_dir, song.id, lyrics)
|
||||||
|
append_provenance(args.provenance_csv, song.id,
|
||||||
|
source)
|
||||||
|
fetched += 1
|
||||||
|
print(f"song {song.id} \"{song.title}\": {source}",
|
||||||
|
file=sys.stderr)
|
||||||
except (OSError, sa.exc.SQLAlchemyError) as error:
|
except (OSError, sa.exc.SQLAlchemyError) as error:
|
||||||
print(f"error: {error}", file=sys.stderr)
|
print(f"error: {error}", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
attempted: int = counts.fetched + counts.missed
|
finally:
|
||||||
|
session.close()
|
||||||
|
attempted: int = fetched + missed
|
||||||
elapsed: str = format_duration(time.monotonic() - started)
|
elapsed: str = format_duration(time.monotonic() - started)
|
||||||
print(f"Done. Fetched lyrics for {counts.fetched}/"
|
print(f"Done. Fetched lyrics for {fetched}/{attempted}"
|
||||||
f"{attempted} songs. {elapsed} elapsed.",
|
f" songs. {elapsed} elapsed.",
|
||||||
file=sys.stderr)
|
file=sys.stderr)
|
||||||
return 0
|
return 0
|
||||||
|
|||||||
@@ -28,13 +28,27 @@ import time
|
|||||||
from dataclasses import asdict, dataclass
|
from dataclasses import asdict, dataclass
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, ClassVar, Self
|
from typing import Any, Self
|
||||||
|
|
||||||
import anthropic
|
import anthropic
|
||||||
|
|
||||||
from ..config import get_settings
|
from ..config import get_settings
|
||||||
from ..utils import format_duration
|
from ..utils import format_duration
|
||||||
|
|
||||||
|
# claude-fable-5 accepts neither "temperature" nor "thinking";
|
||||||
|
# a model's entry holds exactly the extra request parameters it
|
||||||
|
# accepts.
|
||||||
|
MODELS: dict[str, dict[str, Any]] = {
|
||||||
|
"claude-sonnet-4-6": {
|
||||||
|
"temperature": 0.0,
|
||||||
|
"thinking": {"type": "disabled"},
|
||||||
|
},
|
||||||
|
"claude-fable-5": {},
|
||||||
|
}
|
||||||
|
DEFAULT_MODEL: str = "claude-sonnet-4-6"
|
||||||
|
SCRIPT_VERSION: str = "run_llm.py 3.1.0"
|
||||||
|
POLL_INTERVAL_SECONDS: float = 60.0
|
||||||
|
|
||||||
|
|
||||||
class InputFormatError(Exception):
|
class InputFormatError(Exception):
|
||||||
"""An error in the JSONL input file."""
|
"""An error in the JSONL input file."""
|
||||||
@@ -122,23 +136,13 @@ class BatchResult:
|
|||||||
if x.type == "text")
|
if x.type == "text")
|
||||||
return cls(id=entry.custom_id, text=text,
|
return cls(id=entry.custom_id, text=text,
|
||||||
stop_reason=message.stop_reason,
|
stop_reason=message.stop_reason,
|
||||||
usage=cls.__usage_to_dict(message.usage))
|
usage=usage_to_dict(message.usage))
|
||||||
case "errored":
|
case "errored":
|
||||||
return cls(id=entry.custom_id,
|
return cls(id=entry.custom_id,
|
||||||
error=result.error.error.type)
|
error=result.error.error.type)
|
||||||
case other:
|
case other:
|
||||||
return cls(id=entry.custom_id, error=str(other))
|
return cls(id=entry.custom_id, error=str(other))
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __usage_to_dict(usage: Any) -> dict[str, Any]:
|
|
||||||
"""Convert a usage object to a plain dictionary.
|
|
||||||
|
|
||||||
:param usage: The usage object of a message.
|
|
||||||
:return: The usage as a dictionary, without null entries.
|
|
||||||
"""
|
|
||||||
return {k: v for k, v in usage.model_dump().items()
|
|
||||||
if v is not None}
|
|
||||||
|
|
||||||
def to_record(self) -> dict[str, Any]:
|
def to_record(self) -> dict[str, Any]:
|
||||||
"""Return this result as an archive JSONL record.
|
"""Return this result as an archive JSONL record.
|
||||||
|
|
||||||
@@ -165,409 +169,10 @@ class BatchInfo:
|
|||||||
is still processing."""
|
is still processing."""
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class ExecutionOutcome:
|
|
||||||
"""The outcome of submitting and awaiting one batch."""
|
|
||||||
|
|
||||||
batch: BatchInfo
|
|
||||||
"""The submitted batch's bookkeeping."""
|
|
||||||
results: Results
|
|
||||||
"""The batch's results, keyed by item ID."""
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class RunOutcome:
|
|
||||||
"""The outcome of one LLM definition file run."""
|
|
||||||
|
|
||||||
item_count: int
|
|
||||||
"""The number of loaded input items."""
|
|
||||||
dry_run: bool
|
|
||||||
"""Whether this was a dry run."""
|
|
||||||
dry_run_request: dict[str, Any] | None
|
|
||||||
"""The first item's preview request, for a dry run; None for
|
|
||||||
an actual run."""
|
|
||||||
failed: list[str]
|
|
||||||
"""The failed item IDs, in item order; always empty for a dry
|
|
||||||
run."""
|
|
||||||
|
|
||||||
|
|
||||||
class LLMRunner:
|
|
||||||
"""The orchestrator of one LLM definition file run."""
|
|
||||||
|
|
||||||
# claude-fable-5 accepts neither "temperature" nor "thinking";
|
|
||||||
# a model's entry holds exactly the extra request parameters
|
|
||||||
# it accepts.
|
|
||||||
MODELS: ClassVar[dict[str, dict[str, Any]]] = {
|
|
||||||
"claude-sonnet-4-6": {
|
|
||||||
"temperature": 0.0,
|
|
||||||
"thinking": {"type": "disabled"},
|
|
||||||
},
|
|
||||||
"claude-fable-5": {},
|
|
||||||
}
|
|
||||||
"""The supported model IDs and their extra request
|
|
||||||
parameters."""
|
|
||||||
DEFAULT_MODEL: ClassVar[str] = "claude-sonnet-4-6"
|
|
||||||
"""The default model ID."""
|
|
||||||
__SCRIPT_VERSION: ClassVar[str] = "run_llm.py 3.1.0"
|
|
||||||
"""The script version recorded into the archive metadata."""
|
|
||||||
__POLL_INTERVAL_SECONDS: ClassVar[float] = 60.0
|
|
||||||
"""The interval between batch status polls."""
|
|
||||||
|
|
||||||
def __init__(self, prompt: Path, input_path: Path,
|
|
||||||
archive_dir: Path, model: str, max_tokens: int,
|
|
||||||
dry_run: bool, replace: bool) -> None:
|
|
||||||
"""Set up the run of one LLM definition file.
|
|
||||||
|
|
||||||
:param prompt: The prompt definition file, used as the
|
|
||||||
system prompt.
|
|
||||||
:param input_path: The JSONL input file with "id" and
|
|
||||||
"content".
|
|
||||||
:param archive_dir: The destination archive directory.
|
|
||||||
:param model: The model ID, a key of :attr:`MODELS`.
|
|
||||||
:param max_tokens: The maximum output tokens per request.
|
|
||||||
:param dry_run: Whether to validate and archive without
|
|
||||||
calling the API.
|
|
||||||
:param replace: Whether to replace an already existing
|
|
||||||
archive directory.
|
|
||||||
"""
|
|
||||||
self.__prompt: Path = prompt
|
|
||||||
"""The prompt definition file."""
|
|
||||||
self.__input: Path = input_path
|
|
||||||
"""The JSONL input file."""
|
|
||||||
self.__archive_dir: Path = archive_dir
|
|
||||||
"""The destination archive directory."""
|
|
||||||
self.__model: str = model
|
|
||||||
"""The model ID."""
|
|
||||||
self.__max_tokens: int = max_tokens
|
|
||||||
"""The maximum output tokens per request."""
|
|
||||||
self.__dry_run: bool = dry_run
|
|
||||||
"""Whether to validate and archive without calling the
|
|
||||||
API."""
|
|
||||||
self.__replace: bool = replace
|
|
||||||
"""Whether to replace an already existing archive
|
|
||||||
directory."""
|
|
||||||
|
|
||||||
def run(self) -> RunOutcome:
|
|
||||||
"""Load the input, archive the prompt, and run the batch.
|
|
||||||
|
|
||||||
Always writes ``prompt.md`` and ``meta.json`` into the
|
|
||||||
archive directory. A dry run stops there, previewing the
|
|
||||||
first item's request; an actual run also submits the
|
|
||||||
batch, awaits it, and writes ``output.jsonl``.
|
|
||||||
|
|
||||||
:return: The outcome of the run.
|
|
||||||
:raises InputFormatError: When the input file is
|
|
||||||
malformed.
|
|
||||||
:raises OSError: When the input or prompt file cannot be
|
|
||||||
read, the archive directory already exists without
|
|
||||||
``replace``, or an output file cannot be written.
|
|
||||||
"""
|
|
||||||
items: list[InputItem] = self.__load_items()
|
|
||||||
prompt_text: str = self.__prompt.read_text(encoding="utf-8")
|
|
||||||
archive_dir: Path = self.__create_archive_dir()
|
|
||||||
(archive_dir / "prompt.md").write_bytes(
|
|
||||||
self.__prompt.read_bytes())
|
|
||||||
meta: dict[str, Any] = self.__build_meta(items)
|
|
||||||
meta_path: Path = archive_dir / "meta.json"
|
|
||||||
if self.__dry_run:
|
|
||||||
self.__write_json(meta_path, meta)
|
|
||||||
request: dict[str, Any] = self.__build_request(
|
|
||||||
items[0], prompt_text)
|
|
||||||
return RunOutcome(
|
|
||||||
item_count=len(items), dry_run=True,
|
|
||||||
dry_run_request=request, failed=[])
|
|
||||||
client: anthropic.Anthropic = anthropic.Anthropic(
|
|
||||||
api_key=get_settings().ANTHROPIC_API_KEY)
|
|
||||||
outcome: ExecutionOutcome = self.__execute_run(
|
|
||||||
client, items, prompt_text)
|
|
||||||
item_ids: list[str] = [x.id for x in items]
|
|
||||||
self.__write_jsonl(
|
|
||||||
archive_dir / "output.jsonl",
|
|
||||||
[outcome.results[x].to_record() for x in item_ids
|
|
||||||
if x in outcome.results])
|
|
||||||
meta["batch"] = outcome.batch
|
|
||||||
meta["usage"] = self.__sum_usage(outcome.results)
|
|
||||||
self.__write_meta(meta_path, meta)
|
|
||||||
failed: list[str] = self.__find_failures(
|
|
||||||
item_ids, outcome.results)
|
|
||||||
return RunOutcome(
|
|
||||||
item_count=len(items), dry_run=False,
|
|
||||||
dry_run_request=None, failed=failed)
|
|
||||||
|
|
||||||
def __load_items(self) -> list[InputItem]:
|
|
||||||
"""Load and validate the JSONL input items.
|
|
||||||
|
|
||||||
:return: The input items, in file order.
|
|
||||||
:raises InputFormatError: When a line is malformed, an ID
|
|
||||||
is duplicated, or the file contains no item.
|
|
||||||
:raises OSError: When the file cannot be read.
|
|
||||||
"""
|
|
||||||
items: list[InputItem] = []
|
|
||||||
seen: set[str] = set()
|
|
||||||
with open(self.__input, encoding="utf-8") as file:
|
|
||||||
for number, line in enumerate(file, start=1):
|
|
||||||
if line.strip() == "":
|
|
||||||
continue
|
|
||||||
data: Any
|
|
||||||
try:
|
|
||||||
data = json.loads(line)
|
|
||||||
except json.JSONDecodeError as error:
|
|
||||||
raise InputFormatError(
|
|
||||||
f"{self.__input}: line {number}: malformed"
|
|
||||||
f" JSON: {error}")
|
|
||||||
item: InputItem = InputItem.get_instance(
|
|
||||||
data, self.__input, number)
|
|
||||||
if item.id in seen:
|
|
||||||
raise InputFormatError(
|
|
||||||
f"{self.__input}: line {number}:"
|
|
||||||
f" duplicated ID \"{item.id}\"")
|
|
||||||
seen.add(item.id)
|
|
||||||
items.append(item)
|
|
||||||
if len(items) == 0:
|
|
||||||
raise InputFormatError(f"{self.__input}: no input items")
|
|
||||||
return items
|
|
||||||
|
|
||||||
def __create_archive_dir(self) -> Path:
|
|
||||||
"""Create the archive directory.
|
|
||||||
|
|
||||||
Only this directory is ever created or removed; no other
|
|
||||||
directory is ever touched.
|
|
||||||
|
|
||||||
:return: The created archive directory.
|
|
||||||
:raises FileExistsError: When the archive directory
|
|
||||||
already exists and ``replace`` is False.
|
|
||||||
"""
|
|
||||||
if self.__archive_dir.exists():
|
|
||||||
if not self.__replace:
|
|
||||||
raise FileExistsError(
|
|
||||||
f"{self.__archive_dir} already exists; pass"
|
|
||||||
" --replace to replace it")
|
|
||||||
shutil.rmtree(self.__archive_dir)
|
|
||||||
self.__archive_dir.mkdir(parents=True)
|
|
||||||
return self.__archive_dir
|
|
||||||
|
|
||||||
def __build_meta(self, items: list[InputItem]) -> dict[str, Any]:
|
|
||||||
"""Build the initial archive metadata.
|
|
||||||
|
|
||||||
:param items: The loaded input items.
|
|
||||||
:return: The metadata, "batch" and "usage" not yet filled
|
|
||||||
in for an actual run.
|
|
||||||
"""
|
|
||||||
return {
|
|
||||||
"script_version": self.__SCRIPT_VERSION,
|
|
||||||
"model": self.__model,
|
|
||||||
"temperature": self.MODELS[self.__model].get(
|
|
||||||
"temperature"),
|
|
||||||
"thinking": self.MODELS[self.__model].get("thinking"),
|
|
||||||
"max_tokens": self.__max_tokens,
|
|
||||||
"prompt_path": str(self.__prompt),
|
|
||||||
"prompt_sha256": self.__sha256_of(self.__prompt),
|
|
||||||
"input_path": str(self.__input),
|
|
||||||
"input_sha256": self.__sha256_of(self.__input),
|
|
||||||
"item_count": len(items),
|
|
||||||
"dry_run": self.__dry_run,
|
|
||||||
"started_at": self.__now_iso(),
|
|
||||||
"batch": None,
|
|
||||||
"usage": {},
|
|
||||||
}
|
|
||||||
|
|
||||||
def __build_request(self, item: InputItem, system_prompt: str) \
|
|
||||||
-> dict[str, Any]:
|
|
||||||
"""Build one Message Batches request for an input item.
|
|
||||||
|
|
||||||
:param item: The input item.
|
|
||||||
:param system_prompt: The system prompt text.
|
|
||||||
:return: The batch request with "custom_id" and "params".
|
|
||||||
"""
|
|
||||||
return {
|
|
||||||
"custom_id": item.id,
|
|
||||||
"params": {
|
|
||||||
"model": self.__model,
|
|
||||||
"max_tokens": self.__max_tokens,
|
|
||||||
**self.MODELS[self.__model],
|
|
||||||
"system": system_prompt,
|
|
||||||
"messages": [
|
|
||||||
{"role": "user", "content": item.content},
|
|
||||||
],
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
def __execute_run(
|
|
||||||
self, client: anthropic.Anthropic,
|
|
||||||
items: list[InputItem], system_prompt: str) \
|
|
||||||
-> ExecutionOutcome:
|
|
||||||
"""Submit the batch of this run and await its results.
|
|
||||||
|
|
||||||
:param client: The Anthropic client.
|
|
||||||
:param items: The input items.
|
|
||||||
:param system_prompt: The system prompt text.
|
|
||||||
:return: The submitted batch's bookkeeping and its
|
|
||||||
results.
|
|
||||||
"""
|
|
||||||
requests: list[dict[str, Any]] = [
|
|
||||||
self.__build_request(x, system_prompt) for x in items]
|
|
||||||
info: BatchInfo = BatchInfo(
|
|
||||||
batch_id=self.__submit_batch(client, requests),
|
|
||||||
submitted_at=self.__now_iso())
|
|
||||||
print(f"submitted batch {info.batch_id}", file=sys.stderr)
|
|
||||||
batches: dict[str, Any] = self.__poll_batches(
|
|
||||||
client, [info.batch_id])
|
|
||||||
info.ended_at = batches[info.batch_id].ended_at.isoformat()
|
|
||||||
results: Results = self.__collect_results(
|
|
||||||
client, info.batch_id)
|
|
||||||
return ExecutionOutcome(batch=info, results=results)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __submit_batch(client: anthropic.Anthropic,
|
|
||||||
requests: list[dict[str, Any]]) -> str:
|
|
||||||
"""Submit one message batch.
|
|
||||||
|
|
||||||
:param client: The Anthropic client.
|
|
||||||
:param requests: The batch requests.
|
|
||||||
:return: The batch ID.
|
|
||||||
"""
|
|
||||||
return client.messages.batches.create(requests=requests).id
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def __poll_batches(cls, client: anthropic.Anthropic,
|
|
||||||
batch_ids: list[str]) -> dict[str, Any]:
|
|
||||||
"""Poll the batches until every one of them has ended.
|
|
||||||
|
|
||||||
Progress is printed to the standard error every poll.
|
|
||||||
|
|
||||||
:param client: The Anthropic client.
|
|
||||||
:param batch_ids: The batch IDs to poll.
|
|
||||||
:return: The final batch object of each batch, keyed by
|
|
||||||
batch ID.
|
|
||||||
"""
|
|
||||||
while True:
|
|
||||||
batches: dict[str, Any] = {
|
|
||||||
x: client.messages.batches.retrieve(x)
|
|
||||||
for x in batch_ids}
|
|
||||||
pending: list[str] = [
|
|
||||||
x for x in batch_ids
|
|
||||||
if batches[x].processing_status != "ended"]
|
|
||||||
for batch_id in batch_ids:
|
|
||||||
status: str = batches[batch_id].processing_status
|
|
||||||
print(f"batch {batch_id}: {status}", file=sys.stderr)
|
|
||||||
if len(pending) == 0:
|
|
||||||
return batches
|
|
||||||
time.sleep(cls.__POLL_INTERVAL_SECONDS)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __collect_results(client: anthropic.Anthropic,
|
|
||||||
batch_id: str) -> Results:
|
|
||||||
"""Collect the results of an ended batch.
|
|
||||||
|
|
||||||
:param client: The Anthropic client.
|
|
||||||
:param batch_id: The batch ID.
|
|
||||||
:return: The result records, keyed by custom ID.
|
|
||||||
"""
|
|
||||||
results: Results = {}
|
|
||||||
for entry in client.messages.batches.results(batch_id):
|
|
||||||
results[entry.custom_id] = BatchResult.get_instance(
|
|
||||||
entry)
|
|
||||||
return results
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __find_failures(item_ids: list[str],
|
|
||||||
results: Results) -> list[str]:
|
|
||||||
"""Find the item IDs that failed in a result set.
|
|
||||||
|
|
||||||
An item failed when it is missing from the results or when
|
|
||||||
its record is a failure.
|
|
||||||
|
|
||||||
:param item_ids: The item IDs to check, in order.
|
|
||||||
:param results: The result records, keyed by item ID.
|
|
||||||
:return: The failed item IDs, in the given order.
|
|
||||||
"""
|
|
||||||
return [x for x in item_ids
|
|
||||||
if x not in results or results[x].is_failure]
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __sum_usage(results: Results) -> dict[str, int]:
|
|
||||||
"""Sum the token usage of every succeeded result.
|
|
||||||
|
|
||||||
:param results: The result records, keyed by item ID.
|
|
||||||
:return: The summed integer usage fields.
|
|
||||||
"""
|
|
||||||
totals: dict[str, int] = {}
|
|
||||||
for result in results.values():
|
|
||||||
if result.usage is None:
|
|
||||||
continue
|
|
||||||
for key, value in result.usage.items():
|
|
||||||
if isinstance(value, int):
|
|
||||||
totals[key] = totals.get(key, 0) + value
|
|
||||||
return totals
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __write_jsonl(path: Path,
|
|
||||||
records: list[dict[str, Any]]) -> None:
|
|
||||||
"""Write records to a file as JSON Lines.
|
|
||||||
|
|
||||||
:param path: The path of the file to write.
|
|
||||||
:param records: The records, one per line.
|
|
||||||
:return: None.
|
|
||||||
:raises OSError: When the file cannot be written.
|
|
||||||
"""
|
|
||||||
with open(path, "w", encoding="utf-8") as file:
|
|
||||||
for record in records:
|
|
||||||
file.write(
|
|
||||||
json.dumps(record, ensure_ascii=False) + "\n")
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __write_json(path: Path, data: dict[str, Any]) -> None:
|
|
||||||
"""Write data to a file as pretty-printed JSON.
|
|
||||||
|
|
||||||
:param path: The path of the file to write.
|
|
||||||
:param data: The data to write.
|
|
||||||
:return: None.
|
|
||||||
:raises OSError: When the file cannot be written.
|
|
||||||
"""
|
|
||||||
path.write_text(
|
|
||||||
json.dumps(data, ensure_ascii=False, indent=2) + "\n",
|
|
||||||
encoding="utf-8")
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def __write_meta(cls, path: Path, meta: dict[str, Any]) -> None:
|
|
||||||
"""Write the metadata to the ``meta.json`` file.
|
|
||||||
|
|
||||||
The ``BatchInfo`` value under "batch" is written as a
|
|
||||||
plain JSON object.
|
|
||||||
|
|
||||||
:param path: The path of the ``meta.json`` file.
|
|
||||||
:param meta: The metadata to write.
|
|
||||||
:return: None.
|
|
||||||
:raises OSError: When the file cannot be written.
|
|
||||||
"""
|
|
||||||
cls.__write_json(
|
|
||||||
path, {**meta, "batch": asdict(meta["batch"])})
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __sha256_of(path: Path) -> str:
|
|
||||||
"""Calculate the SHA-256 digest of a file.
|
|
||||||
|
|
||||||
:param path: The path of the file.
|
|
||||||
:return: The hexadecimal SHA-256 digest.
|
|
||||||
"""
|
|
||||||
with open(path, "rb") as file:
|
|
||||||
return hashlib.file_digest(file, "sha256").hexdigest()
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def __now_iso() -> str:
|
|
||||||
"""Return the current local time in ISO 8601 format.
|
|
||||||
|
|
||||||
:return: The current local time with the timezone offset.
|
|
||||||
"""
|
|
||||||
return datetime.now().astimezone().isoformat(
|
|
||||||
timespec="seconds")
|
|
||||||
|
|
||||||
|
|
||||||
def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
||||||
"""Parse the command-line arguments.
|
"""Parse the command-line arguments.
|
||||||
|
|
||||||
:param argv: The command-line arguments, or None for
|
:param argv: The command-line arguments, or None for ``sys.argv``.
|
||||||
``sys.argv``.
|
|
||||||
:return: The parsed arguments.
|
:return: The parsed arguments.
|
||||||
"""
|
"""
|
||||||
parser: argparse.ArgumentParser = argparse.ArgumentParser(
|
parser: argparse.ArgumentParser = argparse.ArgumentParser(
|
||||||
@@ -575,8 +180,7 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
|||||||
" and archive the result.")
|
" and archive the result.")
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"prompt", type=Path,
|
"prompt", type=Path,
|
||||||
help="the prompt definition file, used as the system"
|
help="the prompt definition file, used as the system prompt")
|
||||||
" prompt")
|
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"input", type=Path,
|
"input", type=Path,
|
||||||
help="the JSONL input file with \"id\" and \"content\"")
|
help="the JSONL input file with \"id\" and \"content\"")
|
||||||
@@ -584,13 +188,11 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
|||||||
"archive_dir", type=Path,
|
"archive_dir", type=Path,
|
||||||
help="the destination archive directory")
|
help="the destination archive directory")
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--model", choices=sorted(LLMRunner.MODELS),
|
"--model", choices=sorted(MODELS), default=DEFAULT_MODEL,
|
||||||
default=LLMRunner.DEFAULT_MODEL,
|
help=f"the model ID (default {DEFAULT_MODEL})")
|
||||||
help=f"the model ID (default {LLMRunner.DEFAULT_MODEL})")
|
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--max-tokens", type=int, default=2048,
|
"--max-tokens", type=int, default=2048,
|
||||||
help="the maximum output tokens per request (default"
|
help="the maximum output tokens per request (default 2048)")
|
||||||
" 2048)")
|
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--dry-run", action="store_true",
|
"--dry-run", action="store_true",
|
||||||
help="validate and archive without calling the API")
|
help="validate and archive without calling the API")
|
||||||
@@ -600,30 +202,327 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
|||||||
return parser.parse_args(argv)
|
return parser.parse_args(argv)
|
||||||
|
|
||||||
|
|
||||||
|
def load_items(path: Path) -> list[InputItem]:
|
||||||
|
"""Load and validate the JSONL input items.
|
||||||
|
|
||||||
|
:param path: The path of the JSONL input file.
|
||||||
|
:return: The input items, in file order.
|
||||||
|
:raises InputFormatError: When a line is malformed, an ID is
|
||||||
|
duplicated, or the file contains no item.
|
||||||
|
:raises OSError: When the file cannot be read.
|
||||||
|
"""
|
||||||
|
items: list[InputItem] = []
|
||||||
|
seen: set[str] = set()
|
||||||
|
with open(path, encoding="utf-8") as file:
|
||||||
|
for number, line in enumerate(file, start=1):
|
||||||
|
if line.strip() == "":
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
data: Any = json.loads(line)
|
||||||
|
except json.JSONDecodeError as error:
|
||||||
|
raise InputFormatError(
|
||||||
|
f"{path}: line {number}: malformed JSON: {error}")
|
||||||
|
item: InputItem = InputItem.get_instance(
|
||||||
|
data, path, number)
|
||||||
|
if item.id in seen:
|
||||||
|
raise InputFormatError(
|
||||||
|
f"{path}: line {number}: duplicated ID"
|
||||||
|
f" \"{item.id}\"")
|
||||||
|
seen.add(item.id)
|
||||||
|
items.append(item)
|
||||||
|
if len(items) == 0:
|
||||||
|
raise InputFormatError(f"{path}: no input items")
|
||||||
|
return items
|
||||||
|
|
||||||
|
|
||||||
|
def build_request(item: InputItem, system_prompt: str,
|
||||||
|
max_tokens: int, model: str) -> dict[str, Any]:
|
||||||
|
"""Build one Message Batches request for an input item.
|
||||||
|
|
||||||
|
:param item: The input item.
|
||||||
|
:param system_prompt: The system prompt text.
|
||||||
|
:param max_tokens: The maximum output tokens.
|
||||||
|
:param model: The model ID, a key of ``MODELS``.
|
||||||
|
:return: The batch request with "custom_id" and "params".
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
"custom_id": item.id,
|
||||||
|
"params": {
|
||||||
|
"model": model,
|
||||||
|
"max_tokens": max_tokens,
|
||||||
|
**MODELS[model],
|
||||||
|
"system": system_prompt,
|
||||||
|
"messages": [
|
||||||
|
{"role": "user", "content": item.content},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def submit_batch(client: anthropic.Anthropic,
|
||||||
|
requests: list[dict[str, Any]]) -> str:
|
||||||
|
"""Submit one message batch.
|
||||||
|
|
||||||
|
:param client: The Anthropic client.
|
||||||
|
:param requests: The batch requests.
|
||||||
|
:return: The batch ID.
|
||||||
|
"""
|
||||||
|
return client.messages.batches.create(requests=requests).id
|
||||||
|
|
||||||
|
|
||||||
|
def poll_batches(client: anthropic.Anthropic,
|
||||||
|
batch_ids: list[str]) -> dict[str, Any]:
|
||||||
|
"""Poll the batches until every one of them has ended.
|
||||||
|
|
||||||
|
Progress is printed to the standard error every poll.
|
||||||
|
|
||||||
|
:param client: The Anthropic client.
|
||||||
|
:param batch_ids: The batch IDs to poll.
|
||||||
|
:return: The final batch object of each batch, keyed by batch ID.
|
||||||
|
"""
|
||||||
|
while True:
|
||||||
|
batches: dict[str, Any] = {
|
||||||
|
x: client.messages.batches.retrieve(x) for x in batch_ids}
|
||||||
|
pending: list[str] = [
|
||||||
|
x for x in batch_ids
|
||||||
|
if batches[x].processing_status != "ended"]
|
||||||
|
for batch_id in batch_ids:
|
||||||
|
status: str = batches[batch_id].processing_status
|
||||||
|
print(f"batch {batch_id}: {status}", file=sys.stderr)
|
||||||
|
if len(pending) == 0:
|
||||||
|
return batches
|
||||||
|
time.sleep(POLL_INTERVAL_SECONDS)
|
||||||
|
|
||||||
|
|
||||||
|
def usage_to_dict(usage: Any) -> dict[str, Any]:
|
||||||
|
"""Convert a usage object to a plain dictionary.
|
||||||
|
|
||||||
|
:param usage: The usage object of a message.
|
||||||
|
:return: The usage as a dictionary, without null entries.
|
||||||
|
"""
|
||||||
|
return {k: v for k, v in usage.model_dump().items()
|
||||||
|
if v is not None}
|
||||||
|
|
||||||
|
|
||||||
|
def sum_usage(results: Results) -> dict[str, int]:
|
||||||
|
"""Sum the token usage of every succeeded result.
|
||||||
|
|
||||||
|
:param results: The result records, keyed by item ID.
|
||||||
|
:return: The summed integer usage fields.
|
||||||
|
"""
|
||||||
|
totals: dict[str, int] = {}
|
||||||
|
for result in results.values():
|
||||||
|
if result.usage is None:
|
||||||
|
continue
|
||||||
|
for key, value in result.usage.items():
|
||||||
|
if isinstance(value, int):
|
||||||
|
totals[key] = totals.get(key, 0) + value
|
||||||
|
return totals
|
||||||
|
|
||||||
|
|
||||||
|
def collect_results(client: anthropic.Anthropic,
|
||||||
|
batch_id: str) -> Results:
|
||||||
|
"""Collect the results of an ended batch.
|
||||||
|
|
||||||
|
:param client: The Anthropic client.
|
||||||
|
:param batch_id: The batch ID.
|
||||||
|
:return: The result records, keyed by custom ID.
|
||||||
|
"""
|
||||||
|
results: Results = {}
|
||||||
|
for entry in client.messages.batches.results(batch_id):
|
||||||
|
results[entry.custom_id] = BatchResult.get_instance(entry)
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def find_failures(item_ids: list[str],
|
||||||
|
results: Results) -> list[str]:
|
||||||
|
"""Find the item IDs that failed in a result set.
|
||||||
|
|
||||||
|
An item failed when it is missing from the results or when its
|
||||||
|
record is a failure.
|
||||||
|
|
||||||
|
:param item_ids: The item IDs to check, in order.
|
||||||
|
:param results: The result records, keyed by item ID.
|
||||||
|
:return: The failed item IDs, in the given order.
|
||||||
|
"""
|
||||||
|
return [x for x in item_ids
|
||||||
|
if x not in results or results[x].is_failure]
|
||||||
|
|
||||||
|
|
||||||
|
def create_archive_dir(directory: Path, replace: bool) -> Path:
|
||||||
|
"""Create the archive directory.
|
||||||
|
|
||||||
|
Only this directory is ever created or removed; no other
|
||||||
|
directory is ever touched.
|
||||||
|
|
||||||
|
:param directory: The destination archive directory.
|
||||||
|
:param replace: Whether to remove an already existing archive
|
||||||
|
directory before creating it.
|
||||||
|
:return: The created archive directory.
|
||||||
|
:raises FileExistsError: When the archive directory already
|
||||||
|
exists and ``replace`` is False.
|
||||||
|
"""
|
||||||
|
if directory.exists():
|
||||||
|
if not replace:
|
||||||
|
raise FileExistsError(
|
||||||
|
f"{directory} already exists; pass --replace to"
|
||||||
|
" replace it")
|
||||||
|
shutil.rmtree(directory)
|
||||||
|
directory.mkdir(parents=True)
|
||||||
|
return directory
|
||||||
|
|
||||||
|
|
||||||
|
def write_jsonl(path: Path, records: list[dict[str, Any]]) -> None:
|
||||||
|
"""Write records to a file as JSON Lines.
|
||||||
|
|
||||||
|
:param path: The path of the file to write.
|
||||||
|
:param records: The records, one per line.
|
||||||
|
:return: None.
|
||||||
|
"""
|
||||||
|
with open(path, "w", encoding="utf-8") as file:
|
||||||
|
for record in records:
|
||||||
|
file.write(json.dumps(record, ensure_ascii=False) + "\n")
|
||||||
|
|
||||||
|
|
||||||
|
def write_json(path: Path, data: dict[str, Any]) -> None:
|
||||||
|
"""Write data to a file as pretty-printed JSON.
|
||||||
|
|
||||||
|
:param path: The path of the file to write.
|
||||||
|
:param data: The data to write.
|
||||||
|
:return: None.
|
||||||
|
"""
|
||||||
|
path.write_text(
|
||||||
|
json.dumps(data, ensure_ascii=False, indent=2) + "\n",
|
||||||
|
encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def write_meta(path: Path, meta: dict[str, Any]) -> None:
|
||||||
|
"""Write the metadata to the ``meta.json`` file.
|
||||||
|
|
||||||
|
The ``BatchInfo`` value under ``batch`` is written as a plain
|
||||||
|
JSON object.
|
||||||
|
|
||||||
|
:param path: The path of the ``meta.json`` file.
|
||||||
|
:param meta: The metadata to write.
|
||||||
|
:return: None.
|
||||||
|
"""
|
||||||
|
write_json(path, {**meta, "batch": asdict(meta["batch"])})
|
||||||
|
|
||||||
|
|
||||||
|
def sha256_of(path: Path) -> str:
|
||||||
|
"""Calculate the SHA-256 digest of a file.
|
||||||
|
|
||||||
|
:param path: The path of the file.
|
||||||
|
:return: The hexadecimal SHA-256 digest.
|
||||||
|
"""
|
||||||
|
with open(path, "rb") as file:
|
||||||
|
return hashlib.file_digest(file, "sha256").hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def now_iso() -> str:
|
||||||
|
"""Return the current local time in ISO 8601 format.
|
||||||
|
|
||||||
|
:return: The current local time with the timezone offset.
|
||||||
|
"""
|
||||||
|
return datetime.now().astimezone().isoformat(timespec="seconds")
|
||||||
|
|
||||||
|
|
||||||
|
def execute_run(
|
||||||
|
client: anthropic.Anthropic, items: list[InputItem],
|
||||||
|
system_prompt: str, max_tokens: int, model: str,
|
||||||
|
meta: dict[str, Any],
|
||||||
|
) -> Results:
|
||||||
|
"""Submit the batch of this run and await its results.
|
||||||
|
|
||||||
|
The batch ID and timestamps are recorded into the metadata as an
|
||||||
|
observable side effect.
|
||||||
|
|
||||||
|
:param client: The Anthropic client.
|
||||||
|
:param items: The input items.
|
||||||
|
:param system_prompt: The system prompt text.
|
||||||
|
:param max_tokens: The maximum output tokens per request.
|
||||||
|
:param model: The model ID, a key of ``MODELS``.
|
||||||
|
:param meta: The metadata to record the batch bookkeeping into.
|
||||||
|
:return: The results of this run, keyed by item ID.
|
||||||
|
"""
|
||||||
|
requests: list[dict[str, Any]] = [
|
||||||
|
build_request(x, system_prompt, max_tokens, model)
|
||||||
|
for x in items]
|
||||||
|
info: BatchInfo = BatchInfo(
|
||||||
|
batch_id=submit_batch(client, requests),
|
||||||
|
submitted_at=now_iso())
|
||||||
|
meta["batch"] = info
|
||||||
|
print(f"submitted batch {info.batch_id}", file=sys.stderr)
|
||||||
|
batches: dict[str, Any] = poll_batches(client, [info.batch_id])
|
||||||
|
info.ended_at = batches[info.batch_id].ended_at.isoformat()
|
||||||
|
return collect_results(client, info.batch_id)
|
||||||
|
|
||||||
|
|
||||||
def main(argv: list[str] | None = None) -> int:
|
def main(argv: list[str] | None = None) -> int:
|
||||||
"""Run one LLM definition file against one input and archive it.
|
"""Run one LLM definition file against one input and archive it.
|
||||||
|
|
||||||
:param argv: The command-line arguments, or None for
|
:param argv: The command-line arguments, or None for ``sys.argv``.
|
||||||
``sys.argv``.
|
|
||||||
:return: The exit status: 0 on success, non-zero on failure.
|
:return: The exit status: 0 on success, non-zero on failure.
|
||||||
"""
|
"""
|
||||||
started: float = time.monotonic()
|
started: float = time.monotonic()
|
||||||
args: argparse.Namespace = parse_args(argv)
|
args: argparse.Namespace = parse_args(argv)
|
||||||
try:
|
try:
|
||||||
outcome: RunOutcome = LLMRunner(
|
items: list[InputItem] = load_items(args.input)
|
||||||
args.prompt, args.input, args.archive_dir, args.model,
|
prompt_text: str = args.prompt.read_text(encoding="utf-8")
|
||||||
args.max_tokens, args.dry_run, args.replace).run()
|
except (OSError, InputFormatError) as error:
|
||||||
except (InputFormatError, OSError) as error:
|
|
||||||
print(f"error: {error}", file=sys.stderr)
|
print(f"error: {error}", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
if not outcome.dry_run and len(outcome.failed) > 0:
|
try:
|
||||||
print(f"error: failed items: {', '.join(outcome.failed)}",
|
archive_dir: Path = create_archive_dir(
|
||||||
|
args.archive_dir, args.replace)
|
||||||
|
except FileExistsError as error:
|
||||||
|
print(f"error: {error}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
meta_path: Path = archive_dir / "meta.json"
|
||||||
|
(archive_dir / "prompt.md").write_bytes(args.prompt.read_bytes())
|
||||||
|
meta: dict[str, Any] = {
|
||||||
|
"script_version": SCRIPT_VERSION,
|
||||||
|
"model": args.model,
|
||||||
|
"temperature": MODELS[args.model].get("temperature"),
|
||||||
|
"thinking": MODELS[args.model].get("thinking"),
|
||||||
|
"max_tokens": args.max_tokens,
|
||||||
|
"prompt_path": str(args.prompt),
|
||||||
|
"prompt_sha256": sha256_of(args.prompt),
|
||||||
|
"input_path": str(args.input),
|
||||||
|
"input_sha256": sha256_of(args.input),
|
||||||
|
"item_count": len(items),
|
||||||
|
"dry_run": args.dry_run,
|
||||||
|
"started_at": now_iso(),
|
||||||
|
"batch": None,
|
||||||
|
"usage": {},
|
||||||
|
}
|
||||||
|
if args.dry_run:
|
||||||
|
write_json(meta_path, meta)
|
||||||
|
print(json.dumps(
|
||||||
|
build_request(items[0], prompt_text, args.max_tokens,
|
||||||
|
args.model),
|
||||||
|
ensure_ascii=False, indent=2))
|
||||||
|
elapsed: str = format_duration(time.monotonic() - started)
|
||||||
|
print(f"Done. {len(items)} jobs finished."
|
||||||
|
f" {elapsed} elapsed.", file=sys.stderr)
|
||||||
|
return 0
|
||||||
|
client: anthropic.Anthropic = anthropic.Anthropic(
|
||||||
|
api_key=get_settings().ANTHROPIC_API_KEY)
|
||||||
|
results: Results = execute_run(
|
||||||
|
client, items, prompt_text, args.max_tokens, args.model,
|
||||||
|
meta)
|
||||||
|
item_ids: list[str] = [x.id for x in items]
|
||||||
|
write_jsonl(
|
||||||
|
archive_dir / "output.jsonl",
|
||||||
|
[results[x].to_record() for x in item_ids if x in results])
|
||||||
|
meta["usage"] = sum_usage(results)
|
||||||
|
write_meta(meta_path, meta)
|
||||||
|
failed: list[str] = find_failures(item_ids, results)
|
||||||
|
if len(failed) > 0:
|
||||||
|
print(f"error: failed items: {', '.join(failed)}",
|
||||||
file=sys.stderr)
|
file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
if outcome.dry_run:
|
|
||||||
print(json.dumps(
|
|
||||||
outcome.dry_run_request, ensure_ascii=False, indent=2))
|
|
||||||
elapsed: str = format_duration(time.monotonic() - started)
|
elapsed: str = format_duration(time.monotonic() - started)
|
||||||
print(f"Done. {outcome.item_count} jobs finished."
|
print(f"Done. {len(items)} jobs finished."
|
||||||
f" {elapsed} elapsed.", file=sys.stderr)
|
f" {elapsed} elapsed.", file=sys.stderr)
|
||||||
return 0
|
return 0
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user