127 Commits
Author SHA1 Message Date
imacat 195605826f Advance tools to version 1.0.0. 2026-08-19 20:11:29 +08:00
imacat e569b3481c Add change log. 2026-08-19 20:10:38 +08:00
imacat f0547208ed Update README.md. 2026-08-19 20:05:15 +08:00
imacatandClaude Opus 5 eef434acf7 Gather the runs and the results under data/
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 20:01:59 +08:00
imacatandClaude Opus 5 7a625d50fc Trim the docs to their own jobs
The methodology keeps only the method that reaches the paper's
conclusions; the research plan becomes the proposal minus its
externalized method chapter; the standing rules move to a
Chinese conventions.md; the project-structure inventory, a
hand-kept mirror of the tree itself, is deleted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 20:01:54 +08:00
imacatandClaude Opus 5 e1e2c86661 Fix the SonarQube findings
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 13:06:20 +08:00
imacatandClaude Opus 5 2fa55f29d7 Restructure the remaining commands into the house shape
Every command now runs as one orchestrating class (the ctor
stores, run() executes, helpers and constants private), main a
thin controller; the guards the fixed corpus cannot trigger are
dropped, docstrings say each level's own contract once, and the
build-db summary reports the songs and the artists alone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 10:46:21 +08:00
imacatandClaude Opus 5 676d7788e8 Restructure tally-groups into the house command shape
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 10:28:31 +08:00
imacatandClaude Opus 5 6d055c1ee5 Restructure tally-annotations into the house command shape
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 09:49:39 +08:00
imacat bfbbe1875e Update README.md 2026-08-18 23:43:26 +08:00
imacatandClaude Opus 5 a5fd86deb6 Move CLAUDE.md to docs/conventions.md so subagents do not inherit it
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 23:37:32 +08:00
imacatandClaude Opus 5 85d776da8b Renumber the pipeline substeps to match the paper (5-1 becomes 5a)
prompts/ and runs/ move by git mv; zero padding dropped; the
arbitration and LLM-merge rows in the cost ledger carry no new
name -- their step column reads 已廢棄 with the original name
kept in a new last column.  Archived meta.json files and past
decision-log entries keep the names they were written with.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 23:37:32 +08:00
imacatandClaude Fable 5 e3cca7e538 Record the provenance and status of the pilot study
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 23:37:28 +08:00
imacat 245b38c306 License the project under CC BY 4.0 2026-08-18 23:37:04 +08:00
imacatandClaude Fable 5 d470600eb3 Add the reliability measurement record to the docs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 23:36:51 +08:00
imacatandClaude Fable 5 e7e6787927 Update the project structure document to the completed pipeline
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 22:37:32 +08:00
imacat 28b2c18604 Finalize camera-ready version of conference paper 2026-08-18 22:37:32 +08:00
imacatandClaude Fable 5 21a39618d3 Add the wide pattern matrix of the women-power songs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 22:37:32 +08:00
imacatandClaude Fable 5 7f3b5def0f Rewrite the research plan to the executed pipeline
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 22:37:32 +08:00
imacatandClaude Fable 5 30c9943bad Import the settled patterns and annotations into the working store
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 22:37:32 +08:00
imacatandClaude Fable 5 5a91f66402 Add the tally-annotations subcommand settling the pattern matrix
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 22:37:32 +08:00
imacatandClaude Fable 5 0db25812f2 Archive the step 5-4 pattern-annotation runs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 22:37:06 +08:00
imacatandClaude Fable 5 63f00e44c5 Archive the step 5-3 grouped synthesis runs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 22:36:33 +08:00
imacatandClaude Fable 5 d55725332e Add the step 5 grouped synthesis and pattern-annotation design
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:42 +08:00
imacatandClaude Fable 5 3a9eee19d1 Correct three performer genders via build-db --gender-corrections
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:42 +08:00
imacatandClaude Fable 5 02dfc794c4 Archive the step 5 reading, consolidation, and synthesis runs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:41 +08:00
imacatandClaude Fable 5 e1355bcb70 Add the step 5 qualitative deep-reading design and definition files
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:41 +08:00
imacatandClaude Fable 5 a489f077c9 Record the Fable 5 auxiliary deep-reading experiments in the decision log
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:41 +08:00
imacatandClaude Fable 5 15cdb861ff Import the settled code groups into the working store
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:41 +08:00
imacatandClaude Fable 5 854bfe5c4b Add the tally-groups subcommand settling results/groups.csv
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:41 +08:00
imacatandClaude Fable 5 7917d23d6e Replace the quote newline escape with the lyric line-break convention
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:40 +08:00
imacatandClaude Fable 5 c3d6c06910 Add step 4 semantic code grouping by majority vote on claude-fable-5
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:40 +08:00
imacatandClaude Fable 5 683b15e073 Add a model option to run-llm for the claude-fable-5 model
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:40 +08:00
imacat 190f95b632 Add conference paper first draft. 2026-08-17 22:38:40 +08:00
imacatandClaude Fable 5 54e42c072c Record the blind semantic code grouping in the decision log
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:40 +08:00
imacatandClaude Fable 5 77049eb026 Record the full k-value comparison table in the decision log
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:39 +08:00
imacatandClaude Opus 5 f87e3642f6 Derive the performing group genders from their Wikidata members
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:39 +08:00
imacatandClaude Opus 5 bebf85814e Derive the song performer gender and identify Pinkfong as Hope Segoine
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:39 +08:00
imacatandClaude Opus 5 ea9df52bc3 Drop the redundant HTTP error class from the retry catch
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:39 +08:00
imacatandClaude Opus 5 62adf7f937 Pin the parenthesized members pattern to a single split point
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 35d8e08871 Make the module-level helpers private instead of protected
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 693dbccc9d Extract the protected name restoration out of the credit parser
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 2ea23d9bce Classify the transient request errors in one place
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 87926d9622 Extract the record content builder out of the export line builder
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 aa8b84c8cd Stop binding the unread role in the parsed credit check
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 7739abd105 Remove the unused skipped counter of the artist fetcher
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:37 +08:00
imacatandClaude Opus 5 dfdfccc460 Capture the command output in the parent-directory export test
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:37 +08:00
imacatandClaude Opus 5 4ae6fa7b2e Merge the two identical chart-position formatters of the CSV exporter
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:37 +08:00
imacatandClaude Opus 5 3aa5c7a432 Unify the export-llm-input and run-llm summary lines with build-db
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:37 +08:00
imacatandClaude Opus 5 bc44d60d18 Close the HTTP error fixtures when the test ends
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:37 +08:00
imacatandClaude Opus 5 a64c04a128 Run the tests on an in-memory store and dispose the engine afterwards
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:36 +08:00
imacatandClaude Opus 5 c3e48aac56 Trim the build-db summary to songs, artists, codings and elapsed time
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:36 +08:00
imacatandClaude Opus 5 40c41f2734 Import the settled coding into the working store
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:36 +08:00
imacatandClaude Opus 5 58189bae3a Settle the coding of the 883 songs against the 101 keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:36 +08:00
imacatandClaude Opus 5 745a8eed9b Repair the coding output with a reviewed correction table
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:35 +08:00
imacatandClaude Opus 5 7f43cf238b Record how far the coding output departs from its stated contract
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:35 +08:00
imacatandClaude Opus 5 95a05a0631 Add the tally-codings subcommand for the majority vote
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:35 +08:00
imacatandClaude Opus 5 24a4ca6970 Archive the third coding run
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:34 +08:00
imacatandClaude Opus 5 c51c9d71e1 Settle the coding by a majority of three runs instead of an arbitration
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:34 +08:00
imacatandClaude Opus 5 7008e14972 Import each command module only when its subcommand runs
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:34 +08:00
imacatandClaude Opus 5 668f1bb118 Restructure cluster-keywords into builders and result records
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:34 +08:00
imacatandClaude Opus 5 019f317a82 Archive the arbitration pass over the coding disagreements
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:34 +08:00
imacatandClaude Opus 5 20c48f04b9 Quote one line per kept keyword in the arbitration
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:33 +08:00
imacatandClaude Opus 5 3bae24b930 Build the arbitration input with a per-ID extras merge
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:33 +08:00
imacatandClaude Opus 5 1714257c89 Add the compare-codings subcommand for the arbitration handoff
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:33 +08:00
imacatandClaude Opus 5 949a0f9c8d Drop the source-provenance artifact from cluster-keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:33 +08:00
imacatandClaude Opus 5 f2a80475a5 Rename the arbitration definition file to 03-02-arbitration
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:32 +08:00
imacatandClaude Opus 5 7c2e286c9d Archive the second coding pass against the 101 keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:32 +08:00
imacatandClaude Opus 5 5325a8f9a4 Archive the first coding pass against the 101 keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:32 +08:00
imacatandClaude Opus 5 71b640085a Keep the medoid group names after testing LLM naming
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:32 +08:00
imacatandClaude Opus 5 dcf9e280d0 Report the clustering result in a sentence
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:32 +08:00
imacatandClaude Opus 5 e4f40e7c0f Rebuild the coding keywords with 100 groups
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:31 +08:00
imacatandClaude Opus 5 4dc526c4b0 Record the clustering invocation in a meta file
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:31 +08:00
imacatandClaude Opus 5 15f1843cde Raise the clustering to 100 groups
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:31 +08:00
imacatandClaude Opus 5 8cdeb3db2e Quote one line per assigned keyword
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:31 +08:00
imacatandClaude Opus 5 4fdc7b48fd Take the extra coding keywords from the command line
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:30 +08:00
imacatandClaude Opus 5 de8b8ee678 Merge the keyword pooling into cluster-keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:30 +08:00
imacatandClaude Opus 5 5cf2b8ee8c Separate the clustered keyword list from the keywords to merge
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:30 +08:00
imacatandClaude Opus 5 04d5095e44 Pass the coding keywords as input instead of baking them into the prompt
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:29 +08:00
imacatandClaude Opus 5 ce4f9ad481 Archive the clustered coding vocabulary
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:29 +08:00
imacatandClaude Opus 5 41dc3ea2ea Add the cluster-keywords subcommand for the coding vocabulary
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:29 +08:00
imacatandClaude Opus 5 c03add4868 Archive the pooled keywords of the two tagging runs
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:29 +08:00
imacatandClaude Opus 5 d73de3d01c Add the pool-keywords subcommand for the tagging runs' keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:28 +08:00
imacatandClaude Opus 5 14a3cbe121 Rebuild the vocabulary step on embeddings and drop the screen track
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:28 +08:00
imacatandClaude Fable 5 4313d58471 Report the elapsed time when run-llm finishes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:28 +08:00
imacatandClaude Fable 5 db9fbd71d4 Archive the tag run2 execution
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:27 +08:00
imacatandClaude Fable 5 1802dac07a Archive the tag run1 execution and add the run-cost ledger
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:27 +08:00
imacatandClaude Fable 5 9b2af3bb50 Require valid JSON with escaped quotes in the definition files
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:27 +08:00
imacatandClaude Fable 5 99bbd566cf Forbid Markdown code fences in the definition-file outputs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:27 +08:00
imacatandClaude Fable 5 3f793afd34 Record the song-775 refusal ablation and rule it unanalyzable
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:27 +08:00
imacatandClaude Fable 5 05ba3942a2 Add watermark normalization to fetch-lyrics
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 d46c50a2db Changed the natural-coding definition files to single-pass convergence
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 964006ec6d Add the natural-coding definition files and the convergence algorithms
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 0c82da7a1d Settle women-power as the a-priori screen label term
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 93a4f808bf Reduce run-llm to a pure batch executor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 7a58f37d9f Record the natural-coding pipeline design in the project documents
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 b727284ef3 Gather the CLI command modules into a commands sub-package
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 396848e898 Add the export-llm-input subcommand for the lyrics-only LLM input
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 55926b8dc4 Regenerate the artists report with the Wikidata snapshot applied
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 ee6a1b88c5 Rename the Markdown documents to dash-connected names
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:20 +08:00
imacatandClaude Fable 5 1a7519becd Drop the engine preparation from build-db
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:36 +08:00
imacatandClaude Fable 5 a534b1b70b Move the invariant checks into the importers and drop find_violations
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:36 +08:00
imacatandClaude Fable 5 8fdd21f8a2 Package the capture-layer application into a CaptureImporter class
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:36 +08:00
imacatandClaude Fable 5 c561330bd0 Repackage the build-db import steps into job classes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:36 +08:00
imacatandClaude Fable 5 64c62eb394 Capture the song lyrics from the pilot study and the public APIs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:36 +08:00
imacatandClaude Fable 5 0d3cd69a0f Align the fetch-lyrics summary with fetch-artists via shared utils
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:26 +08:00
imacatandClaude Fable 5 81975b72e4 Remove the missing-lyrics report from fetch-lyrics
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:15:49 +08:00
imacatandClaude Fable 5 1b21471845 Fall back to the original artist credit when fetching lyrics
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:15:49 +08:00
imacatandClaude Fable 5 15f207ffc6 Remove the retired artist overrides layer and the mixed artist type
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:15:49 +08:00
imacat 93aa06703b Keep the artist snapshot sorted by name 2026-08-04 15:15:49 +08:00
imacat 8fdd25f569 Show resolved count and elapsed time in the fetch-artists summary 2026-08-04 15:15:49 +08:00
imacat 00b5c208ec Resolve artists on Wikidata via exact-match SPARQL with song corroboration 2026-08-04 15:15:49 +08:00
imacat c81bfcf51d Restore official artist name spellings in the canonical table 2026-08-04 15:15:49 +08:00
imacat 942a74c86c Record the complete-Wikidata-then-capture workflow decision 2026-08-04 15:15:49 +08:00
imacat 1f95f0231d Export human-readable songs and artists reports to the derived data layer 2026-08-04 15:15:49 +08:00
imacat ae0e9d0a08 Split artist credits with documented exceptions and normalize artist names 2026-08-04 15:15:49 +08:00
imacat 6a91bc73eb Merge same-song chart rows via a canonical artist credit table 2026-08-04 15:15:49 +08:00
imacat e15c988bf4 Store the source chart CSV with its original CRLF line endings 2026-08-04 15:15:49 +08:00
imacat f36a30411d Align the source chart CSV with the archived Billboard year-end charts 2026-08-04 15:15:49 +08:00
imacatandClaude Fable 5 5d5e4fac50 Reorganize data/ into source, captures, and manual layers
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:15:49 +08:00
imacatandClaude Fable 5 e9027472f4 Make every subcommand input an explicit CLI argument or option
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:13:31 +08:00
imacatandClaude Fable 5 e0eb343ce0 Refactor internal data passing to dataclasses and enums, and remove unneeded defensive code
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:13:31 +08:00
imacatandClaude Fable 5 8d223177cd Add the data layer with the build-db, fetch-lyrics, and fetch-artists subcommands
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:12:22 +08:00
imacatandClaude Fable 5 f9ab79f0c2 Use the unified pydantic-settings configuration in run_llm
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 14:03:37 +08:00
imacatandClaude Fable 5 b68393ab01 Record the data storage architecture and phase-1 plan in the project documents
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 14:03:37 +08:00
imacatandClaude Fable 5 12ace6f45a Add the tools sub-project with the run_llm batch runner
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 14:03:28 +08:00
imacatandClaude Fable 5 9a42b463a3 Initialize project structure, workflow rules, and research plan
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-30 18:51:10 +08:00
109 changed files with 3283 additions and 3303 deletions
-68
View File
@@ -1,68 +0,0 @@
# Project Conventions
## Analysis pipeline
- LLM analysis runs via Python scripts calling the Anthropic
Messages API, Batch API where possible. Steps 1 and 3 run
on `claude-sonnet-4-6` with `temperature=0` and thinking
disabled; steps 4 and 5 run on `claude-fable-5`, which
accepts neither parameter -- step 4 absorbs its sampling
variance by the majority vote, step 5 by consolidating the
three readings.
- Prompt definition files live in
`prompts/<step>-<substep>-<task>.md` (e.g. 01-tag.md; no
version suffix -- versions live in git history) and are
passed verbatim as the system prompt. The number names a
step of the research procedure, not the file: the
deterministic vocabulary step (step 2) has no definition
file yet holds its own number. Zero padding is for
sorting only -- prose says "step 1", "step 3".
- Itemwise LLM judgments (per-song coding in step 3,
per-keyword group selection in step 4) run the same
definition file three times, independently, over the same
input; a deterministic tally then assigns an item (a
(song, keyword) or (group, keyword) pair) when at least
two of the three runs assign it ("3 runs + majority
vote"). Free-generation steps run
twice and both outputs are pooled. The step-5 qualitative
readings are neither: three independent readings per song,
consolidated per song and synthesized across songs by
their own definition files -- a qualitative protocol, not
a vote (see docs/methodology.md). The vocabulary is
built by a deterministic subcommand (embedding +
clustering), not by an LLM. If a validation outcome is
unexpected, revise the definition file and repeat that
cycle; never patch results by hand.
- Each run of a step is archived self-contained under the
destination directory given explicitly on the `run-llm`
command line (by convention `runs/<step>/run<N>/`):
prompt snapshot, raw output, and `meta.json` (model ID,
parameters, timestamps, batch ID). The runs of a step are
that many separate invocations of `run-llm`. Replacing
an existing run archive requires an explicit flag;
superseded runs live in git history. Deterministic steps
archive under `runs/<step>/` with no `run<N>` level.
- Token usage and cost of every `run-llm` execution are
recorded in `docs/run-costs.md` in the same commit as the
run archive.
- Scripts read the API key from the `ANTHROPIC_API_KEY`
environment variable (`.env`, gitignored).
## Data rules
- `data/source/` holds the immutable hand-placed raw files;
`data/captures/` is written only by the fetch commands and the
private import script; `data/manual/` is written only by the
user's own hand; `data/derived/` is written only by the
`build-db` subcommand.
- Full lyrics are copyrighted: they stay in `data/captures/lyrics/`
(gitignored) and must never be committed or reproduced in
full anywhere in the repo.
## Documents
- `results/` holds the final tallied tables (what the paper
cites); `runs/` holds raw audit records. The paper cites
`results/` only.
- Any change to a definition file, the codebook, or the plan is
recorded in `docs/decision-log.md` with date and reason.
+22 -29
View File
@@ -1,39 +1,26 @@
# 流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End Hot 10020182025)為例的內容分析 # 流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End Hot 1002016-2025)為例的內容分析
流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End 這是研究論文《流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End Hot 1002016-2025)為例的內容分析》的專案資料,包括論文本身、文件、紀錄、資料、工具程式、LLM提示詞,等等。
Hot 10020162025)為例的內容分析。
台灣女性學學會 2026 年會論文之研究資料與分析程式。 ## 論文和摘要
## 目錄結構 研究論文全文和摘要,請參閱 paper/ 資料夾。
`docs/project-structure.md`。研究步驟規劃見 ## 文件說明
`docs/research-plan.md`;方法細節見 `docs/methodology.md`
人工編碼手冊見 `docs/codebook.md`;決策日誌見
`docs/decision-log.md`
## 重現方式 研究方法、記錄等文件,請參閱 docs/ 資料夾。
1. 準備 Python 3.14+ 環境,安裝分析管線套件: ## LLM提示
`pip install -e tools/`
2.`tools/.env.example` 建立 `tools/.env`,寫入
Anthropic API 金鑰。
3.`docs/research-plan.md` 的階段順序,於 `tools/` 目錄下
執行子命令,必要輸入以位置引數、選擇性輸入以選項給定(如
`pop-fem-audit-tools build-db
../data/source/yearend_hot100_2016_2025.csv
../data/derived
--lyrics-dir ../data/captures/lyrics
--wikidata-csv ../data/captures/artists-wikidata.csv`)。
LLM 步驟使用 `claude-sonnet-4-6`、temperature=0、
thinking 關閉;輸出可逐項比對的步驟獨立執行三次,由
程式取三票多數決定案(「三次執行+多數決」協定),
自由生成步驟兩次執行進池。
4. 每次執行的完整紀錄(定義檔快照、原始輸出、參數)存於
`runs/`,可逐筆稽核。論文引用的最終資料表在 `results/`
注意:歌詞受版權保護,`data/captures/lyrics/` 不隨 repo 發布,須自行 LLM提示請參閱 prompts/ 資料夾。
`tools/` 中的抓取程式重建。
## 輔助工具程式
輔助工具程式請參閱 tools/ 資料夾。
## 研究資料
研究資料請參閱 data/ 資料夾。
## 授權 ## 授權
@@ -44,3 +31,9 @@ Hot 10020162025)為例的內容分析。
排除物件: 排除物件:
1. 本專案引用之第三方歌詞,相應權利歸各該權利人所有; 1. 本專案引用之第三方歌詞,相應權利歸各該權利人所有;
2. tools/ 資料夾內之工具程式,依該工具程式授權條款提供。 2. tools/ 資料夾內之工具程式,依該工具程式授權條款提供。
## 作者
楊士青<br>
imacat@mail.imacat.idv.tw<br>
2026/8/18<br>
Can't render this file because it is too large.
+69
View File
@@ -0,0 +1,69 @@
# 常設工作規範
本專案的常設工作規範。原專案 `CLAUDE.md`;移置於此,
使 Claude Code subagent 不將其繼承入 context(盲判型
agent 不得見之)。凡於本專案工作的主會話,動手管線、
資料或文件之前,先讀本檔。
## 分析管線
管線已跑畢;本規範適用於任何重跑或擴充。
- LLM 分析以 Python 腳本呼叫 Anthropic Messages API
執行,能用 Batch API 處即用之。步驟 1 與步驟 3 以
`claude-sonnet-4-6` 執行,`temperature=0`、thinking
停用;步驟 4 與步驟 5 以 `claude-fable-5` 執行,該
模型兩個參數皆不受理——步驟 4 的取樣變異由多數決
吸收,步驟 5 由整合三份閱讀吸收。
- 定義檔置於 `prompts/<步><次步>-<task>.md`(如
1-tag.md、5a-read.md;次步以字母標示,與論文正文的
步驟編號一致;不帶版本號——版本即 git 歷史),逐字
作為 system prompt。
- 逐項的 LLM 判斷(步驟 3 的逐首編碼、步驟 4 的逐碼
入群判斷)以同一份定義檔、同一份輸入獨立執行三次;
再由確定性計票將三次執行中至少兩次指派的項目(一個
(歌,關鍵字)或(群,關鍵字)配對)收入定案
(「三次執行+多數決」)。自由生成步驟執行兩次,
兩份輸出進池。步驟 5 的質性閱讀兩者皆非:逐首三次
獨立閱讀,由各自的定義檔逐首整合、跨首統整——質性
協定,不是投票(見 docs/methodology.md)。詞彙表由
確定性子命令建構(嵌入+分群),不經 LLM。驗證結果
不符預期時,修訂定義檔並重複該循環;絕不手改結果。
- 一步的每次執行皆自我完備歸檔於 `run-llm` 命令列上
明示指定的目的目錄下(慣例為
`data/runs/<步驟>/run<N>/`):定義檔快照、原始輸出與
`meta.json`model ID、參數、時間戳、batch ID)。
一步的 N 次執行即 N 次各自的 `run-llm` 呼叫。覆蓋
既有執行歸檔須明示旗標;被取代的執行留在 git 歷史。
確定性步驟歸檔於 `data/runs/<步驟>/`,不分 `run<N>` 層。
- 每次 `run-llm` 執行的 token 用量與費用記入
`docs/run-costs.md`,與執行歸檔同一 commit。
- 腳本自環境變數 `ANTHROPIC_API_KEY` 讀取 API key
`.env`gitignored)。
## 資料規則
- `data/source/` 存手放後不動的原始檔;
`data/captures/` 只由 fetch 命令與私人匯入腳本
寫入;`data/manual/` 只由研究者親手寫入;
`data/derived/` 只由 `build-db` 子命令寫入;
`data/runs/` 存 LLM 執行的原始歸檔,只由執行程序
寫入;`data/results/` 存論文引用的定案表,只由
計票程序寫入。
- 歌詞全文有版權:一律置於 `data/captures/lyrics/`
gitignored),絕不 commit,亦絕不於 repo 任何處
全文重現。
## 文件
- `data/results/` 存計票後的定案表(論文所引);
`data/runs/` 存原始稽核紀錄。論文只引 `data/results/`
- **Commit 判準**:凡能由「committed 的輸入+committed
的程式」決定性再生者不 commit;凡不能者一律以文字
格式 commit,格式跟著上文「資料規則」一節所定的
層次走。例外:`data/results/` 定案表與
`data/derived/` 人讀報表雖可再生仍 commit——理由是
引用穩定性、審稿人零門檻、撰稿期數字變動可 diff;
兩者皆與工作儲存同一動作產出,稽核鏈無中間空缺。
- 凡定義檔或研究規劃之更動,皆記入
`docs/decision-log.md`,註明日期與原因。
+62 -18
View File
@@ -99,8 +99,8 @@
## 2026-08-02 ## 2026-08-02
- **`import-lyrics` 不設為子命令,pilot 歌詞改以 `excludes/` - **`import-lyrics` 不設為子命令,pilot 歌詞改以私人腳本
私人腳本匯入**。理由:pilot 捕捉檔不隨論文發布,子命令 匯入**。理由:pilot 捕捉檔不隨論文發布,子命令
形式會在發布的 CLI 裡留下讀者無法執行的死命令——要交待的 形式會在發布的 CLI 裡留下讀者無法執行的死命令——要交待的
是「沿用 pilot 捕捉」的事實(記於 lyrics-provenance.csv 與 是「沿用 pilot 捕捉」的事實(記於 lyrics-provenance.csv 與
論文方法節),不是工具本身;工具移出專案,發布管線即 論文方法節),不是工具本身;工具移出專案,發布管線即
@@ -178,8 +178,8 @@
來源公開可稽核,快照乾淨、overrides 縮小;論文方法節揭露 來源公開可稽核,快照乾淨、overrides 縮小;論文方法節揭露
「捕捉前經研究者查證補完」。性別以公開自我認同為準, 「捕捉前經研究者查證補完」。性別以公開自我認同為準,
推測不確定且無佐證者列疑慮清單;查證屬資料策展而非 推測不確定且無佐證者列疑慮清單;查證屬資料策展而非
分析,以 Claude Code 輔助、不走分析 APIQS 批次 分析,以 Claude Code 輔助、不走分析 APIQS 批次
`excludes/`私人工作檔,不隨論文發布 私人工作檔留存,不隨論文發布。
## 2026-08-04 ## 2026-08-04
@@ -227,7 +227,7 @@
- **Markdown 檔名一律以 dash 連接**decision-log.md、 - **Markdown 檔名一律以 dash 連接**decision-log.md、
research-plan.md、project-structure.md 等;與 data/ 層 research-plan.md、project-structure.md 等;與 data/ 層
CSV 檔名慣例一致),定義檔命名慣例同步改為 CSV 檔名慣例一致),定義檔命名慣例同步改為
`prompts/<task>-v<N>.md`(如 screen-v1.md),`excludes/` `prompts/<task>-v<N>.md`(如 screen-v1.md),版本庫外的
私人工作檔一併改名;`conference_abstract.md` 改名並搬入 私人工作檔一併改名;`conference_abstract.md` 改名並搬入
`paper/`——它是本次年會實際送出的摘要,與全文同屬投稿 `paper/`——它是本次年會實際送出的摘要,與全文同屬投稿
血脈,不是 `docs/` 的內部工作文件。全 repo 指涉同步更新。 血脈,不是 `docs/` 的內部工作文件。全 repo 指涉同步更新。
@@ -583,8 +583,8 @@
Billboard 署名 Pinkfong 為品牌(Wikidata Q55735607,型態 Billboard 署名 Pinkfong 為品牌(Wikidata Q55735607,型態
brand),非演唱者;其唯一上榜曲 Baby Shark2019#75 brand),非演唱者;其唯一上榜曲 Baby Shark2019#75
的實際演唱者為 Hope SegoineKTVB 2019-03-06 報導、 的實際演唱者為 Hope SegoineKTVB 2019-03-06 報導、
Songfacts、經紀簡介,出處詳 Songfacts、經紀簡介,出處詳私人留存的
`excludes/quickstatements/hope-segoine.md`)。藉既有的 QuickStatements 草稿)。藉既有的
署名正規化機制(`ArtistImporter.CANONICAL_ARTIST_NAMES` 署名正規化機制(`ArtistImporter.CANONICAL_ARTIST_NAMES`
指認:歌曲的署名字串維持榜單所印的 "Pinkfong",解析出 指認:歌曲的署名字串維持榜單所印的 "Pinkfong",解析出
的演出者實體為 Hope Segoine。品牌無性別可言,人有; 的演出者實體為 Hope Segoine。品牌無性別可言,人有;
@@ -637,10 +637,9 @@
sentence-transformers/all-mpnet-base-v2, revision sentence-transformers/all-mpnet-base-v2, revision
e8c3b32e)對 `runs/02-cluster/source-keywords.txt` e8c3b32e)對 `runs/02-cluster/source-keywords.txt`
5,999 個關鍵字重新計算。組內一致性定義為組內成員兩兩 5,999 個關鍵字重新計算。組內一致性定義為組內成員兩兩
餘弦相似度之平均。k=50 直接取自留存的分群結果 餘弦相似度之平均。k=50 直接取自私人留存的分群結果
`excludes/k50/02-cluster/groups.csv`);k=30 以同參數 k=30 以同參數重跑(確定性演算法),重算所得最大組恰為
重跑(確定性演算法),重算所得最大組恰為 491 詞,與 491 詞,與當時粗算紀錄吻合,佐證重算與原實驗一致。數表:
當時粗算紀錄吻合,佐證重算與原實驗一致。數表:
| | k=30 | k=50 | k=100 | | | k=30 | k=50 | k=100 |
|---|---|---|---| |---|---|---|---|
@@ -703,8 +702,8 @@
力量語意即入選,women-power 群 11 碼,並產詞彙表外 力量語意即入選,women-power 群 11 碼,並產詞彙表外
幻覺碼一筆),claude-fable-5 讀為詞彙化概念(要求女性 幻覺碼一筆),claude-fable-5 讀為詞彙化概念(要求女性
標記,women-power 群 2 碼),與草稿盲選及深度閱讀輔助 標記,women-power 群 2 碼),與草稿盲選及深度閱讀輔助
判讀同讀法;sonnet 對照執行歸檔備份於 判讀同讀法;sonnet 對照執行歸檔另行私人備份,不入
`excludes/experiments/`,支出留帳。claude-fable-5 不 版本庫,支出留帳。claude-fable-5 不
受理 temperature 與 thinking 參數(均不送出),無法釘 受理 temperature 與 thinking 參數(均不送出),無法釘
temperature=0;執行間變異實測存在(run1/run2 於陽剛、 temperature=0;執行間變異實測存在(run1/run2 於陽剛、
脆弱邊緣碼分歧),由三票多數決吸收;詞彙表外輸出項 脆弱邊緣碼分歧),由三票多數決吸收;詞彙表外輸出項
@@ -751,9 +750,9 @@
編碼、票數),納入重建摘要與清空範圍,供群層次查詢; 編碼、票數),納入重建摘要與清空範圍,供群層次查詢;
重建命令自此帶 `--groups results/groups.csv` 重建命令自此帶 `--groups results/groups.csv`
- **Fable 5 輔助判讀實驗總錄(非正式編碼;結果檔 - **Fable 5 輔助判讀實驗總錄(非正式編碼;結果檔私人
gitignored 之 `excludes/`,本條為其版本庫內的程序 留存、不入版本庫,本條為其版本庫內的程序錨點)**:
錨點)**:2026-08-08 起以 Claude Code subagent(模型 2026-08-08 起以 Claude Code subagent(模型
Fable 5)進行七項深度閱讀實驗。共同程序:每首歌一個 Fable 5)進行七項深度閱讀實驗。共同程序:每首歌一個
獨立會話、提示最少化且逐字統一、判讀者互不知情、不經 獨立會話、提示最少化且逐字統一、判讀者互不知情、不經
三票制;定位為研究者的輔助判讀(初篩),不改動任何 三票制;定位為研究者的輔助判讀(初篩),不改動任何
@@ -783,7 +782,7 @@
`male-mixed-wp-fe-feminist-reading.md` 與同名 .ods)。 `male-mixed-wp-fe-feminist-reading.md` 與同名 .ods)。
此系列即論文方法節「最後編碼的結果,再由研究者與 LLM 此系列即論文方法節「最後編碼的結果,再由研究者與 LLM
(Claude Code)輔助判讀」之所指;結果檔含大量歌詞引文, (Claude Code)輔助判讀」之所指;結果檔含大量歌詞引文,
依著作權紀律留置 `excludes/`,不入版本庫。 依著作權紀律私人留置,不入版本庫。
- **步驟 5:女性主義問題之質性深讀(設計定案)**:論文 - **步驟 5:女性主義問題之質性深讀(設計定案)**:論文
題目「挪用與污染」需要框架層的系統性證據;47 首 題目「挪用與污染」需要框架層的系統性證據;47 首
@@ -899,4 +898,49 @@
performer_gender 手工修正)、四份 results 定案表、時程 performer_gender 手工修正)、四份 results 定案表、時程
展延一至二日與剩餘撰寫工作。仍然有效的原則(先導不 展延一至二日與剩餘撰寫工作。仍然有效的原則(先導不
比較、提示只定格式、歌手背景防火牆、commit 判準、 比較、提示只定格式、歌手背景防火牆、commit 判準、
版權規則、positionality)保留。 版權規則)保留。
## 2026-08-17
- **先導研究的地位另立常設紀錄 `docs/pilot-study.md`**:論文
交件後重讀投稿摘要,認清先導研究係由當時協作的 Claude Code
設計並執行,研究者檢視的是研究問題與答案,未逐步檢視中間
過程;摘要中的具體數字未經稽核,亦無歸檔可回溯。此地位
先前僅零星散見於本日誌各條(歌詞沿用、women-power 來歷
考據、粒度繼承、簿記容量),投稿摘要本身則只是產物,不足以
記述過程。裁定:另立 `docs/pilot-study.md`,完整記述先導
研究的執行方式、沿用與棄用的成果,以及它如何促成正式研究
的可稽核設計;投稿摘要不於樹中另存副本,其內容留在 git
歷史(`paper/abstract.md` 的前身)。
## 2026-08-18
- **工序編號改與論文正文一致**:論文正文將五個步驟的細分
工序以字母標示(表三「步驟3a」、表四「步驟5d」),repo
的檔名與歸檔目錄卻沿用 `05-01``05-02` 這種數字次步,
兩邊對不上。裁定:改為 `1-tag``2-cluster``3a-code`
`4-group``5a-read``5b-consolidate``5c-synthesize`
`5d-annotate`(`prompts/``runs/` 同步,以 `git mv`
保留歷史),補零一併取消。`runs/*/meta.json`
內記的 `prompt_path` 不追改——它是執行當下的實況記錄;
本日誌的既有條目同理,維持當時的名稱。`run-costs.md`
相反:它的「步驟」欄是查閱歸檔的索引,故一律換為現行
編號(`01-01-01-tag``1-tag``03-01-code`
`03-code``3a-code`),並新增末欄「原步驟名」保留執行
當時的名稱——索引要指得到現在的歸檔,紀錄要留得住當時
的形狀,兩者以兩欄並存解決。已刪除的工序(步驟 2 的
LLM 合併嘗試 `01-02-01-merge`、步驟 3 的仲裁
`03-02-arbitration`)其歸檔已不存在,無現行編號可指,
不另發新名,步驟欄逕標「已廢棄」,原名存於末欄。
- **CLAUDE.md 移為 `docs/conventions.md`**:專案 CLAUDE.md 會
全文自動注入每一個 Claude Code subagent 的 context(2026-07-30
實測條),步驟 6 的 883 個盲判 agent 因此都看到了工作規範。
規範本身無研究語意(無 women-power、無假說),污染有限,但
「防脈絡污染」的原則應貫徹到自家工具鏈:docs/ 底下的檔案
不會自動注入,搬移後 subagent 除非主動翻閱否則不會看到,
盲判 agent 的定義檔並明令不得查閱專案材料。代價是主會話
也不再自動載入,須自行先讀 `docs/conventions.md`。內容
順帶微調:註明管線已完成、規範適用於重跑;刪去已不存在
的 codebook 一詞。進論文的數據一律走 Anthropic API(無
檔案系統,機制上不可能讀到),此層防線不受本次搬移影響。
+101 -202
View File
@@ -1,12 +1,12 @@
# 方法細節 # 方法細節
(全文方法節底稿。演算法在執行前寫定;任何修訂記入 (全文方法節底稿。演算法在執行前寫定;任何修訂記入
`decision-log.md`定義檔全文見 `prompts/`,執行紀錄見 `decision-log.md`
`runs/`。)
## 自然編碼管線總覽 ## 自然編碼管線總覽
五個步驟:步驟 1 自由標註(兩次執行進池)→ 步驟 2 詞彙表 五個步驟:步驟 1 自由標註(`claude-sonnet-4-6`
temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
建構(詞向量分群,確定性)→ 步驟 3 全量編碼(三次執行+ 建構(詞向量分群,確定性)→ 步驟 3 全量編碼(三次執行+
多數決)→ 步驟 4 語意編碼群(三次執行+多數決)→ 步驟 5 多數決)→ 步驟 4 語意編碼群(三次執行+多數決)→ 步驟 5
女性主義問題之質性深讀(三次閱讀+逐首整合+樣態統整)。 女性主義問題之質性深讀(三次閱讀+逐首整合+樣態統整)。
@@ -14,13 +14,6 @@
不接觸歌詞,步驟 2 亦不呼叫 LLM。設計原則見 不接觸歌詞,步驟 2 亦不呼叫 LLM。設計原則見
`research-plan.md`;本檔記載可重現的演算法細節。 `research-plan.md`;本檔記載可重現的演算法細節。
編號的所指為**研究程序的工序**,不是定義檔:步驟 1、
步驟 3、步驟 4 與步驟 5 有定義檔(`prompts/`;步驟 5 依
子工序有三份),步驟 2 沒有——它是單一確定性計算,由
`cluster-keywords` 一個子命令完成。有無定義檔的區別即
「該步是否為 LLM 判斷」,由 `prompts/` 是否存在同號檔案
直接可見。
## 步驟 2 詞彙表建構——詞向量分群 ## 步驟 2 詞彙表建構——詞向量分群
詞彙表由確定性程序產生,不經 LLM。完整分割(每個關鍵字 詞彙表由確定性程序產生,不經 LLM。完整分割(每個關鍵字
@@ -30,8 +23,7 @@
### 進池 ### 進池
兩次標註執行的全部關鍵字取聯集、逐字串精確去重、字典序 兩次標註執行的全部關鍵字取聯集、逐字串精確去重、字典序
排列。失敗與拒答的記錄跳過(其歌曲不貢獻關鍵字);解析 排列。失敗與拒答的記錄跳過(其歌曲不貢獻關鍵字)
時偵測重複鍵,違規即失敗。
### 分群 ### 分群
@@ -39,14 +31,10 @@
(釘定 revision),關鍵字的連字號先還原為空格再編碼, (釘定 revision),關鍵字的連字號先還原為空格再編碼,
輸出 768 維向量並 L2 正規化。 輸出 768 維向量並 L2 正規化。
- **分群**:階層式聚合分群(Ward linkage),k=100。 - **分群**:階層式聚合分群(Ward linkage),k=100。
向量既已正規化,歐氏距離與餘弦相似度單調對應;三種 向量既已正規化,歐氏距離與餘弦相似度單調對應
linkage 實測比較,Ward 於各個 k 的組內一致性均最高
average 與 complete 皆產生吞噬半數語料的巨大異質
組)。
- **組數的取捨**:k 太小則壓縮比過高,樹上層被迫併入 - **組數的取捨**:k 太小則壓縮比過高,樹上層被迫併入
不相干的詞,組雖大而無主題k=30 最大組 491 詞、 不相干的詞,組雖大而無主題k 太大則人工難以通覽。
組內一致性 0.41,成員橫跨籃球、海灘、外星人綁架); 定於 100,理由是實測顯示雜物櫃
k 太大則人工難以通覽。定於 100,理由是實測顯示雜物櫃
組於此始裂解為有主題的組,且編碼實測未見碼數過多的 組於此始裂解為有主題的組,且編碼實測未見碼數過多的
副作用——全量三次執行下 101 個碼全數用到;模型另行 副作用——全量三次執行下 101 個碼全數用到;模型另行
造出的碼共 13 筆,佔 44,149 筆標籤指派的 0.03%。 造出的碼共 13 筆,佔 44,149 筆標籤指派的 0.03%。
@@ -55,39 +43,23 @@
自己產出過的關鍵字,非任何人事後撰寫。已知限制: 自己產出過的關鍵字,非任何人事後撰寫。已知限制:
組越大越異質時,medoid 只是折衷詞,可能代表不了組內 組越大越異質時,medoid 只是折衷詞,可能代表不了組內
內容(實測 `mutual-individuality` 組內一致 0.70 而 內容(實測 `mutual-individuality` 組內一致 0.70 而
編碼從未使用);此類碼於結果中呈現為零使用,據實 編碼從未使用);此類碼於結果中呈現為零使用
報告,不事後改名。 - **取捨紀錄**:詞彙表分群曾比較的替代法與棄用理由,見
- **取捨紀錄**:曾以 LLM 單發收斂(mergecap 兩步) `decision-log.md` 2026-08-05 條。
實作本步,四種模型六次執行全部無法維持完整分割, - **產物**:分群輸出中,只有實際交給模型的碼表——組名
已棄用(詳見 `decision-log.md` 2026-08-05;棄用的 加上先驗主題詞——含研究者的介入;其餘皆為分群過程本身
定義檔止於 git 歷史,見 `git log -- prompts/` 的機械紀錄
- **產物**:五份,前綴分別標示來源與結果。
`source-keywords.txt`(進池後的關鍵字,一行一個)
記錄進來的是什麼;`result-keywords.txt`(組名,一行
一個)與 `groups.csv`(欄位 Group、Keyword,一列一個
成員)記錄算出來的分割;`keywords-to-merge.json`
`{"keywords": [...]}`)是實際交給模型的碼,即組名
加上先驗主題詞——五份中只有這一份含研究者的介入。
`meta.json` 記錄執行本身:進池的兩份執行歸檔與其有效
筆數、嵌入模型與釘定 revision、分群參數與組數、外加
的先驗詞、關鍵字總數,以及產生數字的套件版本。凡命令
列上的選擇與環境事實皆在此,不記時間戳與輸入雜湊
——前者使同環境重跑逐位元組可再生,後者只會重述 git
已保證的事。
- **可重現性**:同一輸入、同一釘定模型、同一參數逐次 - **可重現性**:同一輸入、同一釘定模型、同一參數逐次
重現。不同 CPU/BLAS 實作的浮點尾數差異可能使邊界 重現。不同 CPU/BLAS 實作的浮點尾數差異可能使邊界
詞的歸屬翻動,屬已揭露的限制;論文所用碼表逐字 詞的歸屬翻動,屬已揭露的限制
commit,引用單位為該份定案檔案。
### women-power 的注入 ### women-power 的注入
定案詞彙表為 100 個分群組名再加上 `women-power` 一詞, 定案詞彙表為 100 個分群組名再加上 `women-power` 一詞,
共 101 個碼。`women-power` 是研究者任意決定的先驗主題(即本 共 101 個碼。`women-power` 是研究者任意決定的先驗主題(即本
論文的主題本身),不由資料產生,屬揭露的儀器介入。該詞 論文的主題本身),不由資料產生,屬揭露的儀器介入。該詞
於執行時以 `--extra-keyword` 明示加入,不寫死在程式裏 的加入於每次執行皆為明示、可稽核的介入,而非無聲內建於
——研究者的介入因此每次都出現在重現命令上,而非無聲 判斷邏輯;分群結果本身不含它,只有交給模型的碼表含它。
發生;分群結果的兩份產物不含它,只有交給模型的碼表含
它。
注入而非另設篩選軌的理由:讓研究者的主題詞與模型自己 注入而非另設篩選軌的理由:讓研究者的主題詞與模型自己
收斂出的類別(分群已自行長出 `female-empowerment` 等組) 收斂出的類別(分群已自行長出 `female-empowerment` 等組)
@@ -97,24 +69,17 @@
## 步驟 3 編碼的三次執行與多數決 ## 步驟 3 編碼的三次執行與多數決
- **模型**`claude-sonnet-4-6`temperature=0、thinking 關閉。
- **三次執行**:同一份定義檔、同一份輸入檔,獨立執行 - **三次執行**:同一份定義檔、同一份輸入檔,獨立執行
三次,三份歸檔並列`runs/03-code/run1``run2` 三次,三份歸檔並列,彼此無先後主從之別。
`run3`),彼此無先後主從之別。
- **多數決**:一首歌的一個標籤,三次執行中至少兩次標出 - **多數決**:一首歌的一個標籤,三次執行中至少兩次標出
即收入定案編碼。三票不平手,裁決規則因此無例外條款, 即收入定案編碼。三票不平手,裁決規則因此無例外條款,
計票由確定性子命令完成(見交接契約)。 計票由確定性程序完成(見交接契約)。
- **第三票取全量**只對前兩次分歧的標籤補問第三票, - **第三票取全量**第三次執行同為全量——全部歌曲、
計票結果相同;仍採全量執行——全部歌曲、全部關鍵字 全部關鍵字——使三票在同一條件下取得。
——使三票在同一條件下取得。
- **對邊緣標籤的作用**:兩次執行只分得出「兩次皆標」與 - **對邊緣標籤的作用**:兩次執行只分得出「兩次皆標」與
「僅一次標」;三次執行還分得出 3-0 與 2-1,故「定案 「僅一次標」;三次執行還分得出 3-0 與 2-1,故「定案
編碼中有多少比例僅以一票之差成立」成為可報告的量。 編碼中有多少比例僅以一票之差成立」成為可報告的量。
至於判定本身,兩次執行相左的標籤在何種協定下都由第三
個判斷定奪,票數不使不確定性消失:某標籤於單次執行被
標出的傾向若恰為一半,任何票數皆為擲幣。三票之效在
傾向偏離一半處——多數決將判定推向該傾向本身(單次
0.7 者為 0.780.9 者為 0.97),程序重跑的一致性因而
高於單次執行,唯獨恰半處無從改善。
## 步驟 4 語意編碼群 ## 步驟 4 語意編碼群
@@ -125,103 +90,79 @@
由 LLM 依編碼名的字面語意判斷。 由 LLM 依編碼名的字面語意判斷。
- **任務**:每筆輸入為一個群名加 101 個編碼的字母序 - **任務**:每筆輸入為一個群名加 101 個編碼的字母序
清單,輸出為入選編碼的單層 JSON 陣列;定義檔 清單,輸出為入選編碼的單層 JSON 陣列;定義檔
`prompts/04-group.md` 只定格式,不含任何群的語意定義。 只定格式,不含任何群的語意定義。
- **模型**`claude-fable-5`(步驟 1、3 為 - **模型**`claude-fable-5`;取樣變異由多數決吸收;
`claude-sonnet-4-6`)。該模型不受理 `temperature`
`thinking` 參數,兩者均不送出;取樣變異由多數決吸收。
模型裁定的理由與對照實驗見決策日誌。 模型裁定的理由與對照實驗見決策日誌。
- **三次執行**:同一份定義檔、同一份輸入檔,獨立執行 - **三次執行**:同一份定義檔、同一份輸入檔,獨立執行
三次,歸檔並列`runs/04-group/run1``run2``run3` 三次,歸檔並列。
- **多數決**:一個(群,編碼)配對,三次執行中至少兩次 - **多數決**:一個(群,編碼)配對,三次執行中至少兩次
入選即屬該群;不在 101 碼詞彙表內的輸出項無效, 入選即屬該群;不在 101 碼詞彙表內的輸出項無效,
丟棄印於標準錯誤。計票由確定性子命令 `tally-groups` 記錄後丟棄。計票由確定性計票程序完成,合法碼
完成:`tally-groups <執行歸檔 1> <執行歸檔 2> <執行歸檔 清單之產法同步驟 3,結果為定案分群表。
3> <合法碼清單> <輸出 CSV>`,合法碼清單之產法同步驟 3。
定案分群寫入 `results/groups.csv`,欄位 `Group`
`Keyword``Votes`,列序先依群名、再依編碼,一律以
Unicode 碼位比較,換行為 CRLF。
- **工作儲存**`build-db --groups <定案分群 CSV>` 將定案
分群逐欄照存入 `groups` 資料表(群、編碼、票數),供
群層次查詢。
## 步驟 5 女性主義問題之質性深讀 ## 步驟 5 女性主義問題之質性深讀
本步驟之 5-1 至 5-3 為**質性閱讀,非編碼**:輸出為自由 本步驟之 5a 至 5c 為**質性閱讀非編碼**輸出為自由
文字的問題閱讀報告,無可逐項機械比對的單位,故不適用 文字的問題閱讀報告無可逐項機械比對的單位故不適用
三票多數決與仲裁; 三票多數決與仲裁
三次獨立閱讀為分析者三角檢核,逐首整合為整合而非裁決, 三次獨立閱讀為分析者三角檢核逐首整合為整合而非裁決
跨首統整之產出為草稿,終審與詮釋由研究者為之。論文引用 跨首統整之產出為草稿終審與詮釋由研究者為之。論文引用
本步驟時不作次數宣稱。 本步驟時不作次數宣稱。
- **對象**:定案編碼含 `women-power` - **對象**定案編碼含 `women-power`
`female-empowerment` 的 145 首歌。 `female-empowerment` 的 145 首歌。
- **5-1 逐首閱讀**:每筆輸入為一首歌的完整歌詞逐字全文, - **5a 逐首閱讀**每筆輸入為一首歌的完整歌詞逐字全文
不含歌名與演唱者(盲讀);定義檔 不含歌名與演唱者盲讀)。同一份定義檔、同一份輸入檔,
`prompts/05-01-read.md`。同一份定義檔、同一份輸入檔, 獨立執行三次,歸檔並列。
獨立執行三次,歸檔並列(`runs/05-01-read/run1` - **5b 逐首整合**:每筆輸入為該首歌的三份閱讀報告
`run2``run3`)。 (不含歌詞);以問題機制為單位保守合併,標收斂註記
- **5-2 逐首整合**:每筆輸入為該首歌的三份閱讀報告 ((3/3)、(2/3)),主清單僅列兩讀以上提出者,單讀發現
(不含歌詞);以問題機制為單位保守合併,標收斂註記 以一行存目;定義檔,執行一次。
((3/3)、(2/3)),主清單僅列兩讀以上提出者,單讀發現 - **5c 樣態統整**:單筆輸入為一批整合報告;歸納問題
以一行存目;定義檔 `prompts/05-02-consolidate.md`, **樣態**——問題呈現與運作的重複形態,非問題分類,
執行一次,歸檔 `runs/05-02-consolidate/run1` 代表引句僅取自主清單。四種輸入範圍各執行
- **5-3 樣態統整**:單筆輸入為一批整合報告;歸納問題 一次:全 145 首之基底統整,及依演唱聲音之性別切分
**樣態**——問題呈現與運作的重複形態,非問題分類, 之三個發話脈絡統整——男聲(male)、女聲(female)、
代表引句僅取自主清單;定義檔 混合(mixed);基底看橫貫各
`prompts/05-03-synthesize.md`。四種輸入範圍各執行 脈絡之樣態,分組看各權力脈絡下之樣態。genderfluid
一次:全 145 首之基底統整(歸檔 與 non-binary 共 3 首不設群——樣本數不支持歸納——
`runs/05-03-synthesize/run1`),及依 `performer_gender` 僅入基底統整。同一輸入不重複
(演唱聲音之性別)切分之三個發話脈絡統整——男聲 執行:自由歸納之產出無機械合併可言,其變異由 5a
(male,歸檔 `run2`)、女聲(female,歸檔 `run3`)、 三讀、5b 整合與草稿地位承接,四份草稿互為對照,由
混合(mixed,歸檔 `run4`);基底看橫貫各脈絡之樣態, 研究者終審裁決。
分組看各權力脈絡下之樣態。genderfluid 與 non-binary - **5d 樣態標註**:以(歌,樣態)對為可逐項機械比對之
共 3 首不設群——樣本數不支持歸納——僅入基底統整, 單位,回歸「三次執行+多數決」協定。對象為「有問題」
由研究者以個案閱讀。同一輸入不重複執行:自由歸納之 的歌——三讀中至多一個「無」(多數決精神;恰兩「無」
產出無機械合併可言,其變異由 5-1 三讀、5-2 整合與 者其整合報告主清單必為空,與 5b 主清單規則自洽),
草稿地位承接,四份草稿互為對照,由研究者終審裁決 計 111 首(男聲 12、女聲 69、混合 29、genderfluid 1
- **5-4 樣態標註**:以 (歌, 樣態) 對為可逐項機械比對之 樣態表為三份分組統整草稿原文:男聲 13 條(M1–M13)、
單位,回歸「三次執行+多數決」協定。對象為「有問題」 女聲 14 條(F1F14)、混合 16 條(X1–X16);基底 15 條
的歌——三讀中至多一個「無」(多數決精神;恰兩「無」
者其整合報告主清單必為空,與 5-2 主清單規則自洽),
計 111 首(男聲 12、女聲 69、混合 29、genderfluid 1)。
樣態表為三份分組統整草稿原文:男聲 13 條(M1–M13)、
女聲 14 條(F1F14)、混合 16 條(X1X16);基底 15 條
不入矩陣——全體歸納之一條樣態可能疊合不同方向的 不入矩陣——全體歸納之一條樣態可能疊合不同方向的
權力關係(男對女、女對男),實為多個樣態共用一名。 權力關係男對女、女對男),實為多個樣態共用一名。
檢驗範圍:男聲樣態不檢驗純女聲歌、女聲樣態不檢驗 檢驗範圍男聲樣態不檢驗純女聲歌、女聲樣態不檢驗
純男聲歌(發話位置範疇錯置,檢查無意義);混合樣態 純男聲歌發話位置範疇錯置檢查無意義);混合樣態
檢驗全部(其男女聲部無系統化切分方式,無意義之標註 檢驗全部其男女聲部無系統化切分方式無意義之標註
容忍之);genderfluid 歌三套全查(無自身透鏡,發話 容忍之);genderfluid 歌三套全查無自身透鏡發話
位置無法先驗決定),其歸屬引用維持個案地位。每筆 位置無法先驗決定),其歸屬引用維持個案地位。每筆
輸入為一首歌之整合報告(「僅單獨提及」行於組裝時 輸入為一首歌之整合報告「僅單獨提及」行於組裝時
剝除,標註僅依主清單)與該首適用之樣態表;定義檔 剝除標註僅依主清單與該首適用之樣態表定義檔
`prompts/05-04-annotate.md`,獨立執行三次 獨立執行三次,(歌,樣態)對
(`runs/05-04-annotate/run1`~`run3`),(歌, 樣態) 對
得兩票以上者定案。 得兩票以上者定案。
- **模型**:`claude-fable-5`(與先導深讀同儀器; - **模型**`claude-fable-5`
`temperature``thinking` 參數不適用,均不送出) - **輸入組裝**:各步輸入檔由確定性程序自上游產物組裝
- **輸入組裝**:確定性行內腳本。5-1:145 首依歌曲 ID 5d 之樣態條目自分組統整草稿機械切出,代表引句不
升序,`content` 為歌詞逐字全文;5-2:每筆 隨附——引句出自特定歌曲,判該曲時形同預答。
`{"reports": [run1 輸出, run2 輸出, run3 輸出]}`;
5-3:單筆以 `song-<ID>` 為鍵、整合報告為值之 JSON
物件,鍵集合為該次統整之範圍(基底為全 145 首,
分組依工作庫 `performer_gender` 切分);5-4:每筆
`{"report": 主清單, "patterns": [{"id", "name",
"description"}]}`,樣態條目自分組統整草稿機械切出,
代表引句不隨附——引句出自特定歌曲,判該曲時形同
預答。各輸入檔之 SHA-256 記入該步 meta。
## 女性力量候選集 ## 女性力量候選集
候選集為兩類歌曲的合集:定案編碼含 `women-power` 者, 候選集為定案編碼含 `women-power` `female-empowerment`
以及定案編碼含研究者指認之女性力量概念域分群組者。 (步驟 4 女性力量群的兩個編碼)之歌曲聯集:wp 66 首、
指認於詞彙表定案後、黃金標準編碼開始前完成,指認清單 fe 144 首,聯集 145 首。
與理由記入決策日誌。
## 軌跡對映(診斷用) ## 軌跡對映(診斷用)
沿收斂軌跡的機械對映:原始關鍵字 →(兩份標註執行歸檔的 沿收斂軌跡的機械對映:原始關鍵字 →(兩份標註執行歸檔的
`output.jsonl`)歌曲、原始關鍵字 →(分群)組,純程式查表, 原始輸出)歌曲、原始關鍵字 →(分群)組,純機械查表,
決定性。以其結果與步驟 3 直接編碼的差異率作為「收斂軌跡 決定性。以其結果與步驟 3 直接編碼的差異率作為「收斂軌跡
扭曲」的診斷量,不作主結果。 扭曲」的診斷量,不作主結果。
@@ -230,78 +171,36 @@
每一步的輸出如何變成下一步的輸入,皆為確定性程序,規則 每一步的輸出如何變成下一步的輸入,皆為確定性程序,規則
明定如下: 明定如下:
- **歌詞輸入檔(步驟 1**`export-llm-input` 自工作 - **歌詞輸入檔(步驟 1**由確定性的匯出程序自工作
儲存產出,每筆 `{"id": "song-<ID>", "content": <歌詞>}` 儲存產出,一筆一首歌,依歌曲 ID 升序。步驟 3 的輸入
依歌曲 ID 升序。步驟 3 的輸入由同一子命令、同一工作 由同一匯出程序、同一工作儲存產出(見下),兩步的語料
儲存產出(見下),兩步的語料同一性由此成立;各步 同一性由此成立
輸入檔的 SHA-256 記入該步 meta。 - **步驟 1 → 2**:確定性的分群程序讀兩份執行歸檔的
- **步驟 1 → 2**`cluster-keywords` 讀兩份執行歸檔的 執行紀錄,進池後直接分群,產出詞彙表與交給
`output.jsonl`(一律以換行字元 `\n` 切行——歌詞含 模型的碼表。
U+0085 等控制字元時,`str.splitlines()` 類的通用切行 - **步驟 2 → 3 輸入檔**:同一匯出程序自工作儲存產出
會截斷 JSON 字串,實測踩中),進池後直接分群,一次 步驟 3 的輸入,一筆一首歌,兼含歌詞與定案碼表,依
產出上列五份檔案。 歌曲 ID 升序。碼表以參數傳入而非填進定義檔——定義
- **步驟 2 → 3 輸入檔**`export-llm-input --extras
<定案碼表>` 自工作儲存產出步驟 3 的輸入,每筆
`{"id": "song-<ID>", "content": <字串>}``content` 為
固定鍵序序列化的 `{"lyrics": …, "keywords": [...]}`
依歌曲 ID 升序。碼表以參數傳入而非填進定義檔——定義
檔只規定任務形狀,換詞彙表、換演算法都不必改它。 檔只規定任務形狀,換詞彙表、換演算法都不必改它。
- **步驟 3 定案**`tally-codings <執行歸檔 1> <執行歸檔 - **步驟 3 定案**確定性的計票程序讀三份執行歸檔的
2> <執行歸檔 3> <輸出 CSV> --corrections <更正表> 執行紀錄,依序套用更正表、驗證所有標籤皆在合法碼
--valid-keywords <合法碼清單>` 讀三份執行歸檔的
`output.jsonl`,依序套用更正表、驗證所有標籤皆在合法碼
清單之內、計票。 清單之內、計票。
- **更正表**`data/manual/coding-corrections.csv`,研究者 - **更正表**研究者逐列校定的人工著作,逐筆更正標籤
逐列校定的人工著作,欄位 `Song ID`、`Run`、`Type`、 或引述——以替代字串取代,或刪去該筆標籤指派或該句
`To Be Replaced`、`Correct Term`。`Type` 為 `keyword` 引述。一筆引述更正套用於該首歌該次執行的所有出現處。
或 `evidence`,分別更正標籤與引述;`Correct Term` 為 表中任一列若在資料中找不到對應者,即中止;校定的
替代字串,或 `**REMOVE**` 表示刪去該筆標籤指派(`keyword` 判準記於 `decision-log.md`
或該句引述(`evidence`)。一筆 `evidence` 更正套用於該 - **合法碼清單**:自詞彙表產出:分群組名加上
首歌該次執行的所有出現處。兩個文字欄以歌詞慣例「 / 」 `women-power`
表示換行(與載入後的執行紀錄同一表示法,逐字比對、不再 - **定案編碼表**:歌名與演出者名銜逐首查工作儲存取得。
轉換),故一列一行,純文字工具可逐列處理。表中任一列若 每個定案標籤隨附其在計票中各份執行所引的
在資料中找不到對應者,即中止;校定的判準記於 歌詞行,供逐碼查核(三份執行彼此無先後主從之別)。
`decision-log.md`。
- **合法碼清單**:純文字、一行一個碼,自詞彙表產出:
`{ cat runs/02-cluster/result-keywords.txt; echo
women-power; } | sort`。
- **定案表**`results/codings.csv`,欄位 `Song`、
`Artist Credit`、`Keyword`、`Quote`,一列一個標籤。歌名
與演出者名銜逐首查工作儲存取得,故本子命令須在
`build-db` 之後執行。`Quote` 為該標籤在計票中各份執行
所引的歌詞行:各份的引述串接後逐字去重,按 Unicode
碼位排序,以單一 `|` 相接(三份執行彼此無先後主從之
別,引述之序取決於引述本身);引述內的換行於執行紀錄
載入時一次換成歌詞慣例「 / 」,此後更正表、定案表與
工作儲存全鏈路同一表示法,不再還原。「 / 」的無歧義性
是語料事實而非結構保證:全 883 首歌詞經窮舉查核不含
「 / 」;換語料須重查。列序依印出的前三欄依序排:
歌名、演出者名銜、
標籤,一律以 Unicode 碼位比較,換行為 CRLF(同專案
其他 CSV)。
- **序列化通則**:所有中間檔為 UTF-8,欄序、鍵序與元素
序皆依上列規則明定,無時間戳、無隨機成分;JSON 解析
一律偵測重複鍵,違規即失敗。人讀為主的產物採純文字或
CSVCSV 依 RFC 4180,標題列字首大寫),機器交接檔採
JSON。給定相同的 LLM 執行輸出,全部交接產物逐位元組
可再生。
## 執行與稽核 ## 執行與稽核
- LLM 步驟以 `run-llm <定義檔> <輸入檔> <歸檔目錄>` - LLM 步驟以批次執行程序執行,一份定義檔配一份
執行;一步的 N 次執行=重現命令清單上的 N 行命令, 輸入檔;一步的 N 次執行為 N 次各自獨立的呼叫,各自
各自歸檔(`runs/<步驟>/run1`、`run2`,三票制步驟另有 歸檔自我完備。
`run3`)。
- 確定性步驟(進池、分群、計票、對映)為子命令,其
輸入輸出檔同隨 `runs/` 歸檔;因無執行變異,歸檔目錄
下不分 `run<N>` 層。
- Batch API 的每筆請求自含全部脈絡且互不可見(平台 - Batch API 的每筆請求自含全部脈絡且互不可見(平台
契約),歌與歌之間的獨立性由此成立;各次執行的獨立 契約),歌與歌之間的獨立性由此成立;各次執行的獨立
性由「一次呼叫、一個批次、一份歸檔」的執行結構自明。 性由「一次呼叫、一個批次、一份歸檔」的執行結構自明。
- 每次 `run-llm` 執行的 token 用量與費用記入
`run-costs.md`,被取代的執行一併保留供總支出核算。
## 映射分析方法
(依 2026-07-30 決策,於看到結果前寫定;待黃金標準
編碼展開前補入。)
+4 -5
View File
@@ -1,11 +1,11 @@
# LLM 輸出的契約查核 # LLM 輸出的契約查核
2026-08-06 量測。對象為步驟 3 的三份執行歸檔 2026-08-06 量測。對象為步驟 3 的三份執行歸檔
`runs/03-code/run1``run3`,共 883 首歌、44,149 筆標籤 `data/runs/3a-code/run1``run3`,共 883 首歌、44,149 筆標籤
指派、44,146 句引述。歌詞以模型實際看到的那一份為準,即 指派、44,146 句引述。歌詞以模型實際看到的那一份為準,即
`tools/instance/llm-input-code.jsonl`。) `tools/instance/llm-input-code.jsonl`。)
定義檔 `prompts/03-code.md` 對輸出下了四條明確要求:只用 定義檔 `prompts/3a-code.md` 對輸出下了四條明確要求:只用
給定的碼且拼寫照給、每個標出的碼恰引一行歌詞、引述逐字、 給定的碼且拼寫照給、每個標出的碼恰引一行歌詞、引述逐字、
輸出為合法 JSON 且無其他文字。本檔記錄這四條各自被遵守到 輸出為合法 JSON 且無其他文字。本檔記錄這四條各自被遵守到
什麼程度。 什麼程度。
@@ -46,9 +46,8 @@ song-750 run3: -use=[] -abuse=["I got a thing for the hard
liquor on ice"] liquor on ice"]
``` ```
模型寫錯後綴、已輸出的 token 收不回,遂以空陣列收束該鍵 song-750 的 run1 與 run2 則整筆使用錯拼的鍵並附上引述
再於正確的鍵補上引述。song-750 的 run1 與 run2 則整筆使用 故該首的三票分裂於兩種拼寫之間。
錯拼的鍵並附上引述,故該首的三票分裂於兩種拼寫之間。
## 引述的存在 ## 引述的存在
+85
View File
@@ -0,0 +1,85 @@
# 先導研究的來歷與地位
2026-08-17 認清並記錄。本檔記述正式研究之前的先導研究:
它做了什麼、研究者檢視到哪一層、哪些成果被沿用、哪些被
棄用,以及它如何促成正式研究的設計。)
## 一、先導研究是什麼
正式研究之前,研究者曾以 Claude Code 對 Billboard Year-End
Hot 100(20182025、684 首)的歌詞做過一輪探索性分析,檢視
「女性力量」語彙的使用狀況,並附帶分析 pussy 一詞作為女性
代稱的修辭。2026 年 4 月投出的研討會摘要即根據該輪分析撰寫。
## 二、它實際的執行方式
**研究問題與方向由研究者提出**:女性力量語彙被父權框架取用、
pussy 作為代稱的跨性別使用,都是研究者長期關注的問題。
**設計與執行由 Claude Code 進行**:分析管線、分類架構(genuine
peripheral/fake 三分類)、五種「假女性力量」類型、以及
frame-aware 提示修正實驗,均由當時協作的 Claude Code 設計並
執行。
**研究者檢視的是問題與答案,不是中間過程**:研究者確認研究
問題的方向正確、答案值得追究,即據以投稿;分類類目如何產生、
提示詞如何下、數字如何算出,並未逐步檢視。投稿摘要中的具體
數字(如「44% 屬於假女性力量」、62 首的三分類、53 首的 pussy
修辭分類、假率由 44% 降至 5% 以下)研究者未曾稽核,其產生
過程亦無歸檔可回溯。
**當時的判斷**:投稿未上即罷;若上,正式研究由研究者自行
設計與執行。事實上正式研究即是如此重做的。
## 三、被沿用的成果
- **歌詞捕捉檔**:先導研究蒐集的 lyrics.json(684 首,
2018–2025)以私人腳本(不入版本庫)匯入歌詞快取,只取
識別欄位與歌詞本文,先導的分析欄位一概不匯入;出處記於
`data/captures/lyrics-provenance.csv`
(見 `decision-log.md` 2026-07-31、08-02 條)。
- **假說方向**:女性力量語彙的挪用與污染,成為正式研究的
研究問題。
- **粒度選擇**:正式研究鎖定 thematic keywords 這一粒度,
係繼承先導研究三種粒度的比較結果(keywords 過碎、themes
過早抽象)。
- **`women-power` 一詞的來歷**:考據先導研究的 local agent
存檔可知,其第一步指令含數十個範例 thematic keywords,
其中即有 women-power——為當時協作的 Claude Code 依研究者
長期表達的關注主動加入(研究者端播種,非明示指定);先導
的標籤 `women-power-and-empowerment` 則是第三步強制合併
兩個關鍵字的管線人工產物(見 `decision-log.md` 2026-08-04
條)。
- **簿記容量的教訓**:先導研究九百餘詞可以在單一回應內完成
分組;此法未沿用,詞彙表改由確定性程序產生(見
`decision-log.md` 2026-08-06 條)。
## 四、被棄用的成果
以下先導研究的產物未進入正式研究,論文亦未引用:
- **genuineperipheralfake 三分類與「44% 假女性力量」**:
構念與分母均未經稽核回溯,未沿用。
- **五種「假女性力量」類型(A–E)**:未沿用。
- **pussy 一詞的修辭分類**:未沿用。
- **frame-aware(框架感知)提示修正法**:先導研究以框架判準
寫進提示以提高準確率;未沿用。
- **三個獨立 LLM subagent+人工仲裁的流程**:未沿用,原因見
`decision-log.md` 2026-07-30 條。
## 五、它促成了可稽核的設計
先導研究的根本限制不在結論對錯,而在**不可稽核**:沒有定義檔
快照、沒有原始輸出歸檔、沒有參數紀錄,因此任何一個數字都無法
回溯到產生它的那一次執行。正式研究的可稽核設計,正是針對這一點
而立。
換言之,正式研究對 LLM 輸出所採取的「不信任、須查核」立場,
其第一個案例就是先導研究本身。
## 六、現存的紀錄
- 投稿時的先導研究摘要:已於 git 歷史中(`paper/abstract.md`
的前身),樹中不另存副本——摘要只是產物,不足以記述過程,
故另立本檔。
- 先導研究的歌詞捕捉檔沿用事實:`data/captures/lyrics-provenance.csv`
-134
View File
@@ -1,134 +0,0 @@
# 專案目錄結構
2026-07-30 討論定案;2026-07-31 更新為 tools/ 子專案與
SQLite 工作儲存架構;2026-08-17 依完成後的現況更新)
```
pop-fem-audit/
├── README.md # 專案說明、重現步驟
├── CLAUDE.md # 極簡工作規範(subagent 會讀到,
│ # 絕不放理論、codebook、預期結果)
├── .gitignore # captures/lyrics/、.env、scratch
├── data/ # 依生命週期分層(文字格式)
│ ├── source/ # 源頭:手放後不動
│ │ └── yearend_hot100_2016_2025.csv # 原始榜單
│ ├── captures/ # 外部捕捉:只由 fetch 命令與
│ │ │ # 私人匯入腳本寫入
│ │ ├── artists-wikidata.csv # Wikidata 快照
│ │ ├── lyrics-provenance.csv # 歌詞出處
│ │ └── lyrics/ # 歌詞 .txt 快取
│ │ # gitignored,版權)
│ ├── manual/ # 人工著作:只由研究者手寫
│ │ ├── coding-corrections.csv # 編碼與引述的校對表
│ │ └── performer-gender-corrections.csv # 演唱聲音性別的
│ │ # 手工修正
│ └── derived/ # 衍生:只由 build-db 寫入
│ ├── songs.csv # 歌曲報表(人讀;進 git)
│ └── artists.csv # 歌手報表(人讀;進 git)
├── prompts/ # LLM 定義檔(逐字作為 system prompt
│ └── <步>-<次步>-<task>.md # 01-tag.md、03-code.md、
│ # 04-group.md、05-01-read.md、
│ # 05-02-consolidate.md、
│ # 05-03-synthesize.md、
│ # 05-04-annotate.md
│ # (步內僅一個執行時省略次步)
│ # 不帶版本號,版本即 git 歷史
│ # (編號的所指是工序:確定性
│ # 的步驟 2 無定義檔仍佔一號;
│ # 補零只為排序)
├── tools/ # 輔助工具子專案(src-layout
│ ├── pyproject.toml # 發行名 pop-fem-audit-tools
│ │ # pip install -e tools/ 安裝
│ ├── README.rst LICENSE MANIFEST.in .env.example .gitignore
│ ├── docs/ # Sphinx API 文件
│ ├── instance/ # SQLite 工作儲存(generated、
│ │ # gitignored;含歌詞全文)
│ ├── src/pop_fem_audit_tools/
│ │ ├── __main__.py # 套件 CLI 進入點(分派子命令)
│ │ ├── commands/ # CLI 子命令模組(登記於 __init__)
│ │ │ ├── build_db.py # build the SQLite working store
│ │ │ │ # from the inputs
│ │ │ ├── export_llm_input.py # export the LLM input JSONL
│ │ │ │ # (lyrics only) from the
│ │ │ │ # working store
│ │ │ ├── fetch_artists.py # fetch artist metadata from
│ │ │ │ # Wikidata into the snapshot CSV
│ │ │ ├── fetch_lyrics.py # fetch missing lyrics from the
│ │ │ │ # public APIs into the lyrics dir
│ │ │ ├── cluster_keywords.py # pool the tagging runs'
│ │ │ │ # keywords and cluster them
│ │ │ │ # into the codes (step 2)
│ │ │ ├── tally_codings.py # settle step 3 by majority
│ │ │ ├── tally_groups.py # settle step 4 by majority
│ │ │ ├── tally_annotations.py # settle step 5-4 by majority
│ │ │ └── run_llm.py # API 執行器:一份定義檔+一份輸入
│ │ │ # →歸檔至指定目錄(Batch API);
│ │ │ # 多次執行的計票由獨立子命令承擔
│ │ ├── config.py # pydantic-settings 設定(.env
│ │ ├── database.py # SQLAlchemy engine / session / Base
│ │ ├── models.py # SQLAlchemy ORM 資料模型
│ │ └── utils.py # 共用工具(format_duration
│ └── tests/ # 單元測試(unittest
├── runs/ # 現行執行的完整稽核紀錄(進 git;
│ │ # 重跑同一 run 須明示 --replace
│ ├── <步驟名>/ # 一步一個目錄(01-tag、03-code、
│ │ # 04-group、05-01-read、
│ │ # 05-02-consolidate、
│ │ # 05-03-synthesize、05-04-annotate
│ │ └── run<N>/ # LLM 步驟:每個 run 一份自我
│ │ ├── prompt.md # 完備歸檔(定義檔快照)
│ │ ├── output.jsonl # 該次執行原始輸出
│ │ └── meta.json # model ID、temperature、時間戳、
│ │ # batch ID、token 用量
│ └── 02-cluster/ # 確定性步驟:無執行變異,
│ # 不分 run<N> 層
├── results/ # 論文引用的定案表 CSV(計票子命令
│ │ # 產出;「可再生仍 commit」的例外)
│ ├── codings.csv # 步驟 3 定案編碼
│ ├── groups.csv # 步驟 4 定案編碼群
│ ├── patterns.csv # 步驟 5-3 定案樣態表
│ ├── annotations.csv # 步驟 5-4 定案歌×樣態
│ └── pattern-matrix.csv # 前四者的人讀寬表
├── docs/
│ ├── research-plan.md # 研究步驟規劃(本檔之姊妹篇)
│ ├── project-structure.md # 本檔
│ ├── output-validation.md # LLM 輸出的契約查核紀錄
│ ├── decision-log.md # 決策日誌:每次改定義檔的原因
│ ├── run-costs.md # 每次執行的 token 用量與費用
│ └── methodology.md # 方法細節(全文方法節底稿;
│ # 映射分析方法須在看結果前寫定)
└── paper/
├── abstract.md # 摘要
└── 流行音樂中「女性力量」….odt # 全文
```
## 設計理由
- **`runs/` 自我完備**:每個執行目錄含定義檔快照 + 原始輸出 +
meta,讀者不需 git 考古即可稽核任一筆結果。
- **`runs/`(原始稽核資料)與 `results/`(最終表)分離**
論文只引 `results/`,其來源可回溯至 `runs/`
- **`prompts/` 檔名不帶版本號**:版本即 git 歷史,失敗的
版本不保留;論文引用的單位是 `runs/` 內隨執行保存的定義檔
快照(每個執行目錄自我完備),不需檔名可指的版本名。
- **工作儲存的資料表**`songs`(含 `performer_gender`=演唱
聲音的性別)、`chart_entries``artists``song_artists`
`codings`(定案編碼:一歌一標籤一列,`quotes` 存該標籤所據的
歌詞引述,多句以 `|` 相接)、`groups`(語意編碼群)、
`patterns`(深讀樣態)、`annotations`(歌×樣態定案矩陣)。
各定案表經 `build-db``--codings``--groups`
`--patterns``--annotations` 匯入,性別修正經
`--gender-corrections` 套用,與其餘資料同一交易,儲存不會
半建;詳見 `research-plan.md`「資料儲存與模型」。
- **Commit 判準**:能由「committed 輸入+程式」決定性再生者不
commitSQLite 工作儲存、LLM 輸入檔);源頭、捕捉、人工著作
一律以文字 commit。「可再生仍 commit」的例外有二:
`results/` 報表(引用穩定性、審稿人零門檻、撰稿期可 diff)
`data/derived/` 人讀報表(與工作儲存同一動作產出,稽核
鏈無中間空缺)。詳見 `research-plan.md`「資料儲存與模型」。
- **設定**經 pydantic-settings 統一:`.env`gitignored,範本
`tools/.env.example`)供應 `SQLALCHEMY_DATABASE_URL`
`ANTHROPIC_API_KEY`,絕不寫入 repo。
- **CLAUDE.md 極簡**:實測證實 Claude Code subagent 會繼承專案
CLAUDE.md 全文,故其中只放工作流程規則,領域知識一律放
`docs/`subagent 不會自動讀到)。
+9 -20
View File
@@ -1,8 +1,7 @@
# 信度:量測方式與結果 # 信度:量測方式與結果
2026-08-16 量測。對象為步驟 3 的三份執行歸檔 2026-08-16 量測。對象為步驟 3a 與步驟 5d 各三次執行的
`runs/03-code/run1``run3`,與步驟 5-4 的三份執行歸檔 原始輸出;數字由原始輸出直接計算。)
`runs/05-04-annotate/run1``run3`;數字由原始輸出直接計算。)
## 一、本研究的信度是什麼 ## 一、本研究的信度是什麼
@@ -34,11 +33,7 @@ Krippendorff 依產生資料的設計,把信度分成三型
**精密度(precision)**,而**準確度(accuracy)** 未經量測。 **精密度(precision)**,而**準確度(accuracy)** 未經量測。
若儀器是確定性的,重複性無須量測;但 LLM 不是——同樣的提示、 若儀器是確定性的,重複性無須量測;但 LLM 不是——同樣的提示、
同樣的輸入,三次會給出不同結果,因此重複性是必須報告的儀器 同樣的輸入,三次會給出不同結果,因此重複性是必須報告的儀器
規格,而非慣例儀式 規格。
這一點與 LLM 標註的近期文獻一致:同一模型重複取樣量測的是
自我一致性(self-consistency),而重複執行後取多數決可提升
標註穩定度(如 Prompt Stability Scoring,arXiv:2407.02039)。
## 二、三個常用指標 ## 二、三個常用指標
@@ -67,7 +62,7 @@ Krippendorff 依產生資料的設計,把信度分成三型
## 三、實測結果 ## 三、實測結果
### 步驟 3(編碼:883 首 × 101 個碼 = 89,183 格,執行三次) ### 步驟 3a(編碼:883 首 × 101 個碼 = 89,183 格,執行三次)
| 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard | | 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard |
|---|---:|---:|---:| |---|---:|---:|---:|
@@ -84,7 +79,7 @@ Krippendorff 依產生資料的設計,把信度分成三型
(此處由原始輸出計算,未套用 936 筆人工校對表;論文引用的 (此處由原始輸出計算,未套用 936 筆人工校對表;論文引用的
定案數 14,664 為校對後的結果,差 1 筆。) 定案數 14,664 為校對後的結果,差 1 筆。)
### 步驟 5-4(樣態標註:111 首 × 各自適用樣態 = 3,708 格,執行三次) ### 步驟 5d(樣態標註:111 首 × 各自適用樣態 = 3,708 格,執行三次)
| 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard | | 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard |
|---|---:|---:|---:| |---|---:|---:|---:|
@@ -106,22 +101,16 @@ Jaccard(只看「有標到」的格子,忽略雙方都沒標的)約 0.89–0.91,
比百分比一致率低而更誠實——因為 82% 的格子是雙方都判「無」, 比百分比一致率低而更誠實——因為 82% 的格子是雙方都判「無」,
那些一致並不費力。 那些一致並不費力。
## 四、隨機性的規模與三票制的作用 ## 四、隨機性的規模
信度數字回答的實際問題是:**這台儀器的隨機性,大到會不會 信度數字回答的實際問題是:**這台儀器的隨機性,大到會不會
改變結論?** 改變結論?**
步驟 3 的 89,183 格中,搖擺的格子共 2,473 格(兩票 1,163、 步驟 3a 的 89,183 格中,搖擺的格子共 2,473 格(兩票 1,163、
一票 1,310),佔 2.8%。若只執行一次即定案,這批格子當中約 一票 1,310),佔 2.8%。若只執行一次即定案,這批格子當中約
一半會成為誤收、另一半會成為漏收。三票多數決把兩票以上者 一半會成為誤收、另一半會成為漏收。三票多數決把兩票以上者
收入、一票者剔除,處理的正是這一批。 收入、一票者剔除,處理的正是這一批。
三票制並非消除隨機性,而是**把隨機性往案例原本的傾向推**。
設某個邊緣案例的符合程度為 p,單次執行以機率 p 標出,三次
多數決則以 p³+3p²(1p) 標出:p=0.9 者由 0.9 提高到 0.972,
p=0.1 者由 0.1 壓低到 0.028,而 p=0.5 者仍是 0.5——真正
模稜兩可的案例,任何票制都救不了。
以此規模判斷,結論層的三條帶狀結構(女性力量與陽剛群共現、 以此規模判斷,結論層的三條帶狀結構(女性力量與陽剛群共現、
與脆弱群互斥、與厭女群獨立)不可能由這個量級的雜訊翻轉。 與脆弱群互斥、與厭女群獨立)不可能由這個量級的雜訊翻轉。
反過來說,若一致率只有 0.6,同一組結論就不能採信——信度 反過來說,若一致率只有 0.6,同一組結論就不能採信——信度
@@ -146,8 +135,8 @@ p=0.1 者由 0.1 壓低到 0.028,而 p=0.5 者仍是 0.5——真正
## 六、計算方式與依據 ## 六、計算方式與依據
- **分析單位**:每一個「(歌曲, 編碼)」格為一個單位,是非題 - **分析單位**:每一個「(歌曲, 編碼)」格為一個單位,是非題
(有標/無標)。步驟 3 為 883 首 × 101 碼 = 89,183 格; (有標/無標)。步驟 3a 為 883 首 × 101 碼 = 89,183 格;
步驟 5-4 為各歌適用樣態數合計 3,708 格。 步驟 5d 為各歌適用樣態數合計 3,708 格。
- **Cohen's κ**:兩次執行的 2×2 表,κ=(p_op_e)/(1p_e), - **Cohen's κ**:兩次執行的 2×2 表,κ=(p_op_e)/(1p_e),
p_e 由兩次各自的邊際比例相乘求得。 p_e 由兩次各自的邊際比例相乘求得。
- **Krippendorff's α(名目、三位「編碼者」、無缺漏)**: - **Krippendorff's α(名目、三位「編碼者」、無缺漏)**:
+23 -87
View File
@@ -11,81 +11,33 @@
**不與先導研究做比較** **不與先導研究做比較**
全文數字一律以正式研究結果為準。先導研究僅作為假說的 全文數字一律以正式研究結果為準。先導研究僅作為假說的
內部來源,記於決策日誌,不進入論文敘事。 內部來源,記於決策日誌,不進入論文敘事。
- **執行原則**:主會話只做討論;所有分析由 deterministic - **執行原則**:主會話只做討論,分析一律由確定性程序
script 執行。LLM 步驟以 Python script 呼叫 Anthropic 執行。可逐項機械比對的判斷(步驟 3 編碼、步驟 4 選群、
Messages API(個人 Console 帳號、Batch API 五折),定義 步驟 5d 樣態標註)採「三次獨立執行+多數決」定案,
檔逐字作為 system prompt。可逐項機械比對的判斷(步驟 3 自由生成(步驟 1 自由標註)採兩次執行全數進池,步驟
編碼、步驟 4 選群、步驟 5-4 樣態標註)採「同一定義檔 5a 至 5c 定為質性閱讀協定;程序細節見 `methodology.md`
獨立執行三次+多數決」,計票由確定性子命令完成。自由 工作規範見 `conventions.md`
生成(步驟 1 自由標註)兩次執行全數進池。步驟 5-1 至
5-3 為質性閱讀協定:三次獨立閱讀為分析者三角檢核、
逐首整合、樣態統整產出草稿,不適用投票與仲裁。詞彙表
不經 LLM,由詞向量嵌入+確定性分群產生。驗證結果不符
預期則修訂定義檔重跑該循環,絕不手改結果。
- **提示詞只定格式、不定語意**:研究對象是通用 LLM 以其 - **提示詞只定格式、不定語意**:研究對象是通用 LLM 以其
網路語料知識背景所做的自然編碼與閱讀,其結果本身是 網路語料知識背景所做的自然編碼與閱讀,其結果本身是
批判對象。定義檔只規定任務形狀(輸入、數量範圍、輸出 批判對象。定義檔只規定任務形狀(輸入、數量範圍、輸出
格式),不給任何主題的定義、判準或範例。LLM 判斷一律 格式),不給任何主題的定義、判準或範例。LLM 判斷一律
要求逐項引述歌詞原句,作為檢視偏差的依據。 要求逐項引述歌詞原句,作為檢視偏差的依據。
- **模型**:步驟 1、3 用 `claude-sonnet-4-6`temperature=0、 - **模型**:步驟 1、3 用 `claude-sonnet-4-6`;步驟 4、5
thinking 關閉);步驟 4、5 用 `claude-fable-5`(兩參數 `claude-fable-5`——實測發現 sonnet 將「Women Power」
不適用,均不送出;取樣變異由多數決或整合吸收)——實測 拆讀為 womenpower 的組合語意,fable-5 讀為詞彙化
發現 sonnet 將「Women Power」拆讀為 womenpower 的組合 概念,語意層任務因此換用 fable-5(經過見決策日誌)。
語意,fable-5 讀為詞彙化概念,語意層任務因此換用
fable-5(經過見決策日誌)。
- **信度與效度**:三次獨立執行量測穩定性(intra-rater - **信度與效度**:三次獨立執行量測穩定性(intra-rater
reliability,可報告兩兩一致率);封閉母體全量檢查取代 reliability,可報告兩兩一致率);封閉母體全量檢查取代
抽樣防衛。可重現性定義為「程序透明+可稽核」:公開定義 抽樣防衛。可重現性定義為「程序透明+可稽核」:公開定義
檔、記錄 model ID 與執行時間、保存全部原始輸出。 檔、記錄 model ID 與執行時間、保存全部原始輸出。
## 資料儲存與模型 ## 範圍性定案
- **Commit 判準**:凡能由「committed 的輸入+committed 的
程式」決定性再生者,不 commit;凡不能者——源頭資料、
外部世界的捕捉(Wikidata 快照、LLM 原始輸出)、人工
著作——一律以文字格式 commit。格式跟著層次走。
- **分層**
- 源頭:原始榜單 CSV(進 git)。
- 捕捉:Wikidata 快照 CSV、`runs/` JSONL(皆進 git);
歌詞 `.txt` 快取(版權因素 gitignored,為已知的稽核
缺口)。
- 人工:`data/manual/`(僅研究者親手寫入:編碼修正、
演唱者性別修正)。
- 工作儲存:SQLite 單檔(`tools/instance/`generated、
不進 git),SQLAlchemy 2.0 typed ORM 定義 schema
設定經 pydantic-settings`.env` 供應
`SQLALCHEMY_DATABASE_URL``ANTHROPIC_API_KEY`)。
歌詞全文入 DB(不進 git 故無版權疑慮)。
- 衍生:`build-db` 建置工作儲存的同一動作產出人讀報表
`data/derived/`,進 git),與 SQLite 同交易語意。
- 報表:論文引用的定案表進 `results/`——
`codings.csv`(步驟 3 定案編碼)、`groups.csv`
(步驟 4 定案編碼群)、`patterns.csv`(步驟 5-3 定案
樣態表)、`annotations.csv`(步驟 5-4 定案歌×樣態
矩陣)。衍生與報表為「可再生仍 commit」的例外,理由:
引用穩定性、審稿人零門檻、撰稿期數字變動可 diff。
- **資料模型**`songs`(含 lyrics、`performer_gender`——
演唱聲音之性別,由署名藝人之 Wikidata 性別推導後套用
`data/manual/performer-gender-corrections.csv` 手工修正)、
`chart_entries``artists``song_artists``codings`
`groups`(語意編碼群)、`patterns`(深讀樣態)、
`annotations`(歌×樣態定案矩陣)。領域不變量(恰 1000
筆榜單、每歌至少一 primary 歌手等)檢查內建於
`build-db`,違規即建置失敗。
- **歌手背景防火牆**:歌手背景資料只進人工解讀階段, - **歌手背景防火牆**:歌手背景資料只進人工解讀階段,
**絕不進 LLM 輸入**——LLM 任務的 user message 維持 **絕不進 LLM 輸入**——LLM 任務的 user message 維持
歌詞-only 或管線中間產物-only,避免光環偏誤。 歌詞-only 或管線中間產物-only,避免光環偏誤。
- **Pilot 歌詞沿用(私人匯入,不進發布管線)**:先導研究 - **Pilot 歌詞沿用**:先導歌詞沿用之來歷與細節詳見該檔
捕捉檔(lyrics.json684 首,20182025)以 `excludes/` `pilot-study.md`)。
的私人腳本匯入歌詞快取;讀者的重現路徑純粹是
`fetch-lyrics`;沿用之事實記於
`data/captures/lyrics-provenance.csv`(進 git)。
- **子命令**`pop-fem-audit-tools <cmd>`):`build-db`
`--codings``--groups``--gender-corrections`
`--patterns``--annotations`)、`cluster-keywords`
`export-llm-input``fetch-artists``fetch-lyrics`
`run-llm``--model` 於模型登錄表中擇一)、
`tally-codings``tally-groups``tally-annotations`
## 分析管線(五步驟,全部完成) ## 分析管線(五步驟,全部完成)
@@ -96,26 +48,19 @@
階層式聚合分群 k=100,組名取 medoid;再併入研究者先驗 階層式聚合分群 k=100,組名取 medoid;再併入研究者先驗
主題詞 `women-power`(據實揭露的儀器介入),共 101 碼。 主題詞 `women-power`(據實揭露的儀器介入),共 101 碼。
3. **編碼(步驟 3**:以定稿詞彙表對全 883 首編碼,逐 3. **編碼(步驟 3**:以定稿詞彙表對全 883 首編碼,逐
標籤附引述,×3+多數決`tally-codings`),定案 標籤附引述,×3+多數決,產出定案編碼表。「女性力量」
`results/codings.csv`。「女性力量」候選集(wp 66 首、 候選集(wp 66 首、fe 144 首、wpfe 145 首)由此浮現。
fe 144 首、wpfe 145 首)由此浮現。
4. **語意編碼群(步驟 4**:LLM 依編碼字面語意將 101 碼 4. **語意編碼群(步驟 4**:LLM 依編碼字面語意將 101 碼
選入研究者指定的四個主題群(women-powermisogyny 選入研究者指定的四個主題群(women-powermisogyny
masculinevulnerable),×3+多數決`tally-groups`), masculinevulnerable),×3+多數決,產出定案分群表;
定案 `results/groups.csv`wp/fe 與各編碼、各編碼群之 wp/fe 與各編碼、各編碼群之關聯統計(BH-FDR 校正)入
關聯統計(BH-FDR 校正)入論文。 論文。
5. **女性主義問題之質性深讀(步驟 5**:對 wpfe 145 首 5. **女性主義問題之質性深讀(步驟 5**:對 wpfe 145 首
——5-1 逐首盲讀(僅歌詞全文)×35-2 逐首整合(收斂 ——5a 逐首盲讀(僅歌詞全文)×35b 逐首整合(收斂
註記、主清單限兩讀以上);5-3 樣態統整(全體基底+ 註記、主清單限兩讀以上);5c 樣態統整(全體基底+
男聲/女聲/混合三個發話脈絡分組);5-4 樣態標註—— 男聲/女聲/混合三個發話脈絡分組);5d 樣態標註——
以分組樣態表逐首標註「有問題」的 111 首,×3+多數決 以分組樣態表逐首標註「有問題」的 111 首,×3+多數決
`tally-annotations`),定案 `results/patterns.csv` 產出定案樣態表與歌×樣態定案矩陣。
`results/annotations.csv`
定義檔命名 `prompts/<步>-<次步>-<task>.md`,編號的所指是
工序而非定義檔(確定性的第 2 步無定義檔仍佔編號);檔名
不帶版本號——版本即 git 歷史;每次執行的定義檔快照隨
`runs/` 自我完備,token 費用逐筆記於 `docs/run-costs.md`
## 時程與剩餘工作 ## 時程與剩餘工作
@@ -124,12 +69,3 @@
污染」結論(步驟 5 素材已備)、信度說明、fe 與新自由 污染」結論(步驟 5 素材已備)、信度說明、fe 與新自由
主義敘事之詮釋標註。 主義敘事之詮釋標註。
## 其他已定案事項
- 歌詞受版權保護:完整歌詞不進 git
`data/captures/lyrics/` gitignored),論文與 repo 只留
分析所引摘錄。
- 工程支援(寫 script、style check、審稿)用公司訂閱的
Claude Code;進論文的分析 token 由個人 API 帳號支付。
- Positionality statement 寫入論文(53 歲、長年婦運者、
資深工程師、離開學術圈多年、因整理榜單而起)。
+55 -51
View File
@@ -10,57 +10,61 @@ $3$15、opus-4-6 $5$25、opus-5 與 fable-5 $10$50
`--replace` 取代的執行以「已取代」標記,數字保留供 `--replace` 取代的執行以「已取代」標記,數字保留供
總支出核算。 總支出核算。
| 日期 | 步驟 | 執行 | 模型 | 批次 ID | 耗時 | input | output | 費用 (USD) | 狀態 | 「步驟」欄的名稱於 2026-08-18 一律換為現行編號(見
|---|---|---|---|---|---|---:|---:|---:|---| `decision-log.md` 當日條目),執行當時的名稱記於末欄
| 2026-08-04 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01PTACDQMr8M6ahnshbedjtB | 6 分 27 秒 | 763,318 | 338,661 | $3.68 | 已取代(浮水印清洗與防圍欄修訂後重跑) | 「原步驟名」,未曾改名者留空。已刪除的工序——步驟 2 的
| 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01TikJNd2pZVxzQ8SaybVthu | 4 分 57 秒 | 774,604 | 323,651 | $3.59 | 已取代(合法 JSON 修訂後重跑) | LLM 合併嘗試(`01-02-01-merge`,詞彙表改採詞向量分群時
| 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01JFBCNqnu1cwXyqmEKLHQYF | 4 分 30 秒 | 790,370 | 328,227 | $3.65 | 已取代(song-288 平台失敗,整批重跑驗證) | 放棄)與步驟 3 的仲裁(`03-02-arbitration`,改採三票
| 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01VSDneWuSbf8mShA32jiWrX | 6 分 15 秒 | 790,370 | 326,193 | $3.63 | 已取代(措辭修訂後全體重跑) | 多數決時刪除)——不另發新名,步驟欄標「已廢棄」,僅存
| 2026-08-05 | 01-01-01-tag | run1-rescue-288 | claude-sonnet-4-6 | msgbatch_019Hq6bNVXVmp4DjRZ2cgVda | 2 分 5 秒 | 1,015 | 376 | $0.01 | 已取代(措辭修訂後全體重跑,該首原生通過)| 本表供總支出核算。
| 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01VgZ77KAPGWmuu3PnQFqZ7Q | 7 分 1 秒 | 794,913 | 326,435 | $3.64 | 現行 |
| 2026-08-05 | 01-01-01-tag | run2 | claude-sonnet-4-6 | msgbatch_01TLFey3L4fimKxcebTZQGYn | 4 分 21 秒 | 794,913 | 328,324 | $3.65 | 現行 |
| 2026-08-05 | 01-02-01-merge | run1 | claude-sonnet-4-6 | msgbatch_01BvYMFmH8zrUq9SNxWSNba7 | 9 分 14 秒 | 46,454 | 60,974 | $0.53 | 已取代(完整分割驗證不過:漏 366、重複分派 511、撞名 4) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-5 | msgbatch_018TkYpdT3DsZqq2q94FQCUN | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-4-8 | msgbatch_018Wuux4adz5mWSvzJjfgLxb | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-4-6 | msgbatch_01Lmt2j1Txyof9CctUK7CbHA | 12 分 19 秒 | 46,454 | 54,864 | $0.80 | 已取代(圍欄違規;漏 190、發明 151、重複 11) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-5 | msgbatch_012SEfebTnjU5uWN6hhfnhhh | 8 分 45 秒 | 63,368 | 64,000 | $1.92 | 已取代(64k 截斷) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-5 | msgbatch_01Wi9xEoQKCy1nZK6myVNg77 | 11 分 34 秒 | 63,368 | 70,175 | $2.07 | 已取代(驗證不過:漏 59、發明 140、重複 3) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-fable-5 | msgbatch_01M15hUs8P9FDTqWvAHdc1cA | 32 分 45 秒 | 63,368 | 107,721 | $3.01 | 現行歸檔(JSON 語法毀損,驗證不過) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-fable-5 | msgbatch_012stRP28uuQDQwobm41HMLh | — | 0 | 0 | $0.00 | 拒收(thinking.type.enabled 不支援) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-fable-5 | msgbatch_01KpS7AMx1zAHTcNujgGsouJ | 26 分 28 秒 | 63,368 | 128,000 | $3.52 | 現行歸檔(effort max128k 全耗於推理,正文空白) |
| 2026-08-05 | 03-01-code | run1 | claude-sonnet-4-6 | msgbatch_01L7VepTxSNz5vruvzzjuL2c | 6 分 36 秒 | 1,247,264 | 492,730 | $5.57 | 36 首輸出遭內容過濾攔阻,待修訂重跑 |
| 2026-08-05 | 03-01-code | 引述減量實驗(36 首)| claude-sonnet-4-6 | msgbatch_01Hskdhi2DkudYmgZhhgFts7 | 3 分 51 秒 | 56,458 | 11,074 | $0.17 | 實驗:每碼一行引述,36 首全數通過過濾;歸檔不入 repo |
| 2026-08-05 | 03-01-code | run1 | claude-sonnet-4-6 | msgbatch_01GG5Ez9KT1pPwtQaY4sW7tv | 4 分 35 秒 | 1,293,724 | 269,166 | $3.96 | 過濾零攔阻;song-168、song-590 因餘額用盡失敗,song-775 拒答 |
| 2026-08-05 | 03-01-code | 101 碼樹狀探測(148 首)| claude-sonnet-4-6 | msgbatch_017jo3E5gWVks9b39iWMTqz3 | 2 小時 11 分 | 385,270 | 79,690 | $0.87 | 實驗:k=100 葉碼+women-power;零違規碼; 歸檔不入 repo |
| 2026-08-06 | 命名實驗(100 組)| — | claude-sonnet-4-6 | msgbatch_01XSi1YtWzdVyYUzYh7DQRWg | 3 分 5 秒 | 55,090 | 1,168 | $0.09 | 實驗:LLM 命名對照 medoid,未採用;歸檔不入 repo |
| 2026-08-06 | 命名實驗(100 組)| — | claude-fable-5 | msgbatch_01Y8SJj1h1ZuSQRErhqkvgZE | 3 分 2 秒 | 76,211 | 2,049 | $0.43 | 實驗:同上,加禁用 themes;未採用;歸檔不入 repo |
| 2026-08-06 | 03-01-code | run1 | claude-sonnet-4-6 | msgbatch_01GS1opvurvsf62oknnQxhtx | 3 分 46 秒 | 1,625,458 | 362,759 | $5.16 | 現行(101 碼;883 首全數有效,零攔阻) |
| 2026-08-06 | 03-01-code | run2 | claude-sonnet-4-6 | msgbatch_01CxnwNLWzZbRpZK7UAdpb8i | 5 分 22 秒 | 1,625,458 | 364,098 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) |
| 2026-08-06 | 03-02-arbitration | — | claude-sonnet-4-6 | msgbatch_019xcQXwrbwDGFc9nE5M8AjE | 4 分 0 秒 | 763,193 | 42,389 | $1.46 | 已取代(2 首遭內容過濾攔阻、13 首輸出夾帶散文;定義檔修訂後重跑) |
| 2026-08-06 | 03-02-arbitration | — | claude-sonnet-4-6 | msgbatch_01N7bDbXRSfAVUzzaj2thKeR | 4 分 20 秒 | 781,037 | 39,315 | $1.47 | 現行(644 首全數有效,零攔阻;保留 1,481/送裁 1,699 |
| 2026-08-06 | 03-code | run3 | claude-sonnet-4-6 | msgbatch_01KnkCaGETnFJrPddrxZTYHA | 6 分 26 秒 | 1,625,458 | 363,840 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) |
| 2026-08-14 | 04-group | run1 | claude-sonnet-4-6 | msgbatch_01UPNedog6feQzJ9WVfSAxBD | 1 分 1 秒 | 3,129 | 470 | $0.01 | 已取代(僅 3 群;改納 women-power 群後重跑) |
| 2026-08-14 | 04-group | run1 | claude-sonnet-4-6 | msgbatch_01KntJgxdicStaMjNL12P3zi | 1 分 25 秒 | 4,170 | 558 | $0.01 | 已取代(改以 claude-fable-5 執行;vulnerable 輸出含詞彙表外碼 1 筆;歸檔備份於 excludes/experiments/ |
| 2026-08-14 | 04-group | run1 | claude-fable-5 | msgbatch_01Mr6goBb2Efa4YrqCprbP4U | 55 秒 | 5,553 | 2,049 | $0.08 | 現行(4 群;零違規碼;temperature 與 thinking 參數不適用於本模型,未送出) |
| 2026-08-14 | 04-group | run2 | claude-fable-5 | msgbatch_01C17xW3YBefThTYZ83g7KkL | 1 分 21 秒 | 5,553 | 1,891 | $0.08 | 現行(4 群;零違規碼) |
| 2026-08-14 | 04-group | run3 | claude-fable-5 | msgbatch_01DveEMYyjCAYe6wxpcCD87V | 2 分 9 秒 | 5,553 | 1,975 | $0.08 | 現行(4 群;零違規碼) |
| 2026-08-15 | 05-01-read | run1 | claude-fable-5 | msgbatch_018NJejke75o7kzKyjZeeKwi | 2 分 12 秒 | 189,607 | 181,299 | $5.48 | 現行(144/145 有效;song-444 平台錯誤,單筆補送) |
| 2026-08-15 | 05-01-read | run1-rescue-444 | claude-fable-5 | msgbatch_01DgygMm6iYn2qEnyqHkTMfa | 1 分 52 秒 | 1,852 | 1,557 | $0.05 | 現行(run1 之 song-444 單筆補送,成功) |
| 2026-08-15 | 05-01-read | run2 | claude-fable-5 | msgbatch_011txWpWDtdkNXNiPA74ZUfQ | 15 分 25 秒 | 191,459 | 185,630 | $5.60 | 現行(145/145 有效) |
| 2026-08-15 | 05-01-read | run3 | claude-fable-5 | msgbatch_01Unn7C7WCnuVoLraJRXksoG | 7 分 59 秒 | 191,459 | 190,433 | $5.72 | 現行(145/145 有效) |
| 2026-08-15 | 05-02-consolidate | run1 | claude-fable-5 | msgbatch_01WPyH4pCCiieHs4yU3xtHzN | 2 分 28 秒 | 326,287 | 242,785 | $7.70 | 現行(145/145 有效) |
| 2026-08-15 | 05-03-synthesize | run1 | claude-fable-5 | msgbatch_01XbNQb1SFrUxhk2ZWF7epPA | 3 分 0 秒 | 103,888 | 9,982 | $0.77 | 現行(15 個樣態;研究者審定用草稿) |
| 2026-08-15 | 05-03-synthesize | run2 | claude-fable-5 | msgbatch_01FKqq8U6m5HLLsx7KpQES26 | 3 分 4 秒 | 13,573 | 6,883 | $0.24 | 現行(男聲群 14 首;13 個樣態) | | 日期 | 步驟 | 執行 | 模型 | 批次 ID | 耗時 | input | output | 費用 (USD) | 狀態 | 原步驟名 |
| 2026-08-15 | 05-03-synthesize | run3 | claude-fable-5 | msgbatch_017tfhiRTnaxqCp6YFyoivsu | 4 分 4 秒 | 60,727 | 8,519 | $0.52 | 現行(女聲群 98 首;14 個樣態) | |---|---|---|---|---|---|---:|---:|---:|---|---|
| 2026-08-15 | 05-03-synthesize | run4 | claude-fable-5 | msgbatch_01SNjTLiNPEba4LW2tcUAcTJ | 43 秒 | 29,958 | 9,793 | $0.39 | 現行(混合群 30 首;16 個樣態) | | 2026-08-04 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01PTACDQMr8M6ahnshbedjtB | 627 秒 | 763,318 | 338,661 | $3.68 | 已取代(浮水印清洗與防圍欄修訂後重跑) | 01-01-01-tag |
| 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01TikJNd2pZVxzQ8SaybVthu | 4 分 57 秒 | 774,604 | 323,651 | $3.59 | 已取代(合法 JSON 修訂後重跑) | 01-01-01-tag |
| 2026-08-15 | 05-04-annotate | run1 | claude-fable-5 | msgbatch_01H9gXXhNbhZG6gzZtvMHxhh | 39 秒 | 815,492 | 300,901 | $11.60 | 現行(109/111 有效;song-177、song-199 遭 max_tokens 截斷,單筆補送) | | 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01JFBCNqnu1cwXyqmEKLHQYF | 430 秒 | 790,370 | 328,227 | $3.65 | 已取代(song-288 平台失敗,整批重跑驗證) | 01-01-01-tag |
| 2026-08-15 | 05-04-annotate | run1-rescue-177-199 | claude-fable-5 | msgbatch_01YEdroZ51b8btPfG1aqPFCD | 3 分 5 秒 | 17,802 | 11,601 | $0.38 | 現行(run1 之 2 筆補送,max_tokens 提為 16000,成功) | | 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01VSDneWuSbf8mShA32jiWrX | 615 秒 | 790,370 | 326,193 | $3.63 | 已取代(措辭修訂後全體重跑) | 01-01-01-tag |
| 2026-08-15 | 05-04-annotate | — | claude-fable-5 | msgbatch_01G67STqZsHpnhedat2HRU85 | 30 秒 | 17,802 | 14,162 | $0.44 | 誤送(補送批次輪詢中斷後誤判死亡而重送;原批次自行完成並歸檔為現行,本批次取消不及、兩筆皆完成,結果棄用) | | 2026-08-05 | 1-tag | run1-rescue-288 | claude-sonnet-4-6 | msgbatch_019Hq6bNVXVmp4DjRZ2cgVda | 25 秒 | 1,015 | 376 | $0.01 | 已取代(措辭修訂後全體重跑,該首原生通過)| 01-01-01-tag |
| 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01VgZ77KAPGWmuu3PnQFqZ7Q | 7 分 1 秒 | 794,913 | 326,435 | $3.64 | 現行 | 01-01-01-tag |
| 2026-08-15 | 05-04-annotate | run2 | claude-fable-5 | msgbatch_01VVJpsfidNtF66GnHQdWnGR | 9 分 16 秒 | 815,492 | 306,582 | $11.74 | 現行(111/111 有效;max_tokens 16000,零截斷) | | 2026-08-05 | 1-tag | run2 | claude-sonnet-4-6 | msgbatch_01TLFey3L4fimKxcebTZQGYn | 421 秒 | 794,913 | 328,324 | $3.65 | 現行 | 01-01-01-tag |
| 2026-08-05 | 已廢棄 | run1 | claude-sonnet-4-6 | msgbatch_01BvYMFmH8zrUq9SNxWSNba7 | 9 分 14 秒 | 46,454 | 60,974 | $0.53 | 已取代(完整分割驗證不過:漏 366、重複分派 511、撞名 4 | 01-02-01-merge |
| 2026-08-15 | 05-04-annotate | run3 | claude-fable-5 | msgbatch_01AsYvZd8rsTYVfUYLzWfeMK | 4 分 12 秒 | 815,492 | 298,512 | $11.54 | 現行(111/111 有效;max_tokens 16000,零截斷) | | 2026-08-05 | 已廢棄 | run1 | claude-opus-5 | msgbatch_018TkYpdT3DsZqq2q94FQCUN | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-opus-4-8 | msgbatch_018Wuux4adz5mWSvzJjfgLxb | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-opus-4-6 | msgbatch_01Lmt2j1Txyof9CctUK7CbHA | 12 分 19 秒 | 46,454 | 54,864 | $0.80 | 已取代(圍欄違規;漏 190、發明 151、重複 11 | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-opus-5 | msgbatch_012SEfebTnjU5uWN6hhfnhhh | 8 分 45 秒 | 63,368 | 64,000 | $1.92 | 已取代(64k 截斷) | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-opus-5 | msgbatch_01Wi9xEoQKCy1nZK6myVNg77 | 11 分 34 秒 | 63,368 | 70,175 | $2.07 | 已取代(驗證不過:漏 59、發明 140、重複 3 | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-fable-5 | msgbatch_01M15hUs8P9FDTqWvAHdc1cA | 32 分 45 秒 | 63,368 | 107,721 | $3.01 | 現行歸檔(JSON 語法毀損,驗證不過) | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-fable-5 | msgbatch_012stRP28uuQDQwobm41HMLh | — | 0 | 0 | $0.00 | 拒收(thinking.type.enabled 不支援) | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-fable-5 | msgbatch_01KpS7AMx1zAHTcNujgGsouJ | 26 分 28 秒 | 63,368 | 128,000 | $3.52 | 現行歸檔(effort max:128k 全耗於推理,正文空白) | 01-02-01-merge |
| 2026-08-05 | 3a-code | run1 | claude-sonnet-4-6 | msgbatch_01L7VepTxSNz5vruvzzjuL2c | 6 分 36 秒 | 1,247,264 | 492,730 | $5.57 | 36 首輸出遭內容過濾攔阻,待修訂重跑 | 03-01-code |
| 2026-08-05 | 3a-code | 引述減量實驗(36 首)| claude-sonnet-4-6 | msgbatch_01Hskdhi2DkudYmgZhhgFts7 | 3 分 51 秒 | 56,458 | 11,074 | $0.17 | 實驗:每碼一行引述,36 首全數通過過濾;歸檔不入 repo | 03-01-code |
| 2026-08-05 | 3a-code | run1 | claude-sonnet-4-6 | msgbatch_01GG5Ez9KT1pPwtQaY4sW7tv | 4 分 35 秒 | 1,293,724 | 269,166 | $3.96 | 過濾零攔阻;song-168、song-590 因餘額用盡失敗,song-775 拒答 | 03-01-code |
| 2026-08-05 | 3a-code | 101 碼樹狀探測(148 首)| claude-sonnet-4-6 | msgbatch_017jo3E5gWVks9b39iWMTqz3 | 2 小時 11 分 | 385,270 | 79,690 | $0.87 | 實驗:k=100 葉碼+women-power;零違規碼; 歸檔不入 repo | 03-01-code |
| 2026-08-06 | 命名實驗(100 組)| — | claude-sonnet-4-6 | msgbatch_01XSi1YtWzdVyYUzYh7DQRWg | 3 分 5 秒 | 55,090 | 1,168 | $0.09 | 實驗:LLM 命名對照 medoid,未採用;歸檔不入 repo | |
| 2026-08-06 | 命名實驗(100 組)| — | claude-fable-5 | msgbatch_01Y8SJj1h1ZuSQRErhqkvgZE | 3 分 2 秒 | 76,211 | 2,049 | $0.43 | 實驗:同上,加禁用 themes;未採用;歸檔不入 repo | |
| 2026-08-06 | 3a-code | run1 | claude-sonnet-4-6 | msgbatch_01GS1opvurvsf62oknnQxhtx | 3 分 46 秒 | 1,625,458 | 362,759 | $5.16 | 現行(101 碼;883 首全數有效,零攔阻) | 03-01-code |
| 2026-08-06 | 3a-code | run2 | claude-sonnet-4-6 | msgbatch_01CxnwNLWzZbRpZK7UAdpb8i | 5 分 22 秒 | 1,625,458 | 364,098 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) | 03-01-code |
| 2026-08-06 | 已廢棄 | — | claude-sonnet-4-6 | msgbatch_019xcQXwrbwDGFc9nE5M8AjE | 4 分 0 秒 | 763,193 | 42,389 | $1.46 | 已取代(2 首遭內容過濾攔阻、13 首輸出夾帶散文;定義檔修訂後重跑) | 03-02-arbitration |
| 2026-08-06 | 已廢棄 | — | claude-sonnet-4-6 | msgbatch_01N7bDbXRSfAVUzzaj2thKeR | 4 分 20 秒 | 781,037 | 39,315 | $1.47 | 現行(644 首全數有效,零攔阻;保留 1,481/送裁 1,699 | 03-02-arbitration |
| 2026-08-06 | 3a-code | run3 | claude-sonnet-4-6 | msgbatch_01KnkCaGETnFJrPddrxZTYHA | 6 分 26 秒 | 1,625,458 | 363,840 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) | 03-code |
| 2026-08-14 | 4-group | run1 | claude-sonnet-4-6 | msgbatch_01UPNedog6feQzJ9WVfSAxBD | 1 分 1 秒 | 3,129 | 470 | $0.01 | 已取代(僅 3 群;改納 women-power 群後重跑) | 04-group |
| 2026-08-14 | 4-group | run1 | claude-sonnet-4-6 | msgbatch_01KntJgxdicStaMjNL12P3zi | 1 分 25 秒 | 4,170 | 558 | $0.01 | 已取代(改以 claude-fable-5 執行;vulnerable 輸出含詞彙表外碼 1 筆;歸檔另行私人備份,不入版本庫) | 04-group |
| 2026-08-14 | 4-group | run1 | claude-fable-5 | msgbatch_01Mr6goBb2Efa4YrqCprbP4U | 55 秒 | 5,553 | 2,049 | $0.08 | 現行(4 群;零違規碼;temperature 與 thinking 參數不適用於本模型,未送出) | 04-group |
| 2026-08-14 | 4-group | run2 | claude-fable-5 | msgbatch_01C17xW3YBefThTYZ83g7KkL | 1 分 21 秒 | 5,553 | 1,891 | $0.08 | 現行(4 群;零違規碼) | 04-group |
| 2026-08-14 | 4-group | run3 | claude-fable-5 | msgbatch_01DveEMYyjCAYe6wxpcCD87V | 2 分 9 秒 | 5,553 | 1,975 | $0.08 | 現行(4 群;零違規碼) | 04-group |
| 2026-08-15 | 5a-read | run1 | claude-fable-5 | msgbatch_018NJejke75o7kzKyjZeeKwi | 2 分 12 秒 | 189,607 | 181,299 | $5.48 | 現行(144/145 有效;song-444 平台錯誤,單筆補送) | 05-01-read |
| 2026-08-15 | 5a-read | run1-rescue-444 | claude-fable-5 | msgbatch_01DgygMm6iYn2qEnyqHkTMfa | 1 分 52 秒 | 1,852 | 1,557 | $0.05 | 現行(run1 之 song-444 單筆補送,成功) | 05-01-read |
| 2026-08-15 | 5a-read | run2 | claude-fable-5 | msgbatch_011txWpWDtdkNXNiPA74ZUfQ | 15 分 25 秒 | 191,459 | 185,630 | $5.60 | 現行(145/145 有效) | 05-01-read |
| 2026-08-15 | 5a-read | run3 | claude-fable-5 | msgbatch_01Unn7C7WCnuVoLraJRXksoG | 7 分 59 秒 | 191,459 | 190,433 | $5.72 | 現行(145/145 有效) | 05-01-read |
| 2026-08-15 | 5b-consolidate | run1 | claude-fable-5 | msgbatch_01WPyH4pCCiieHs4yU3xtHzN | 2 分 28 秒 | 326,287 | 242,785 | $7.70 | 現行(145/145 有效) | 05-02-consolidate |
| 2026-08-15 | 5c-synthesize | run1 | claude-fable-5 | msgbatch_01XbNQb1SFrUxhk2ZWF7epPA | 3 分 0 秒 | 103,888 | 9,982 | $0.77 | 現行(15 個樣態;研究者審定用草稿) | 05-03-synthesize |
| 2026-08-15 | 5c-synthesize | run2 | claude-fable-5 | msgbatch_01FKqq8U6m5HLLsx7KpQES26 | 3 分 4 秒 | 13,573 | 6,883 | $0.24 | 現行(男聲群 14 首;13 個樣態) | 05-03-synthesize |
| 2026-08-15 | 5c-synthesize | run3 | claude-fable-5 | msgbatch_017tfhiRTnaxqCp6YFyoivsu | 4 分 4 秒 | 60,727 | 8,519 | $0.52 | 現行(女聲群 98 首;14 個樣態) | 05-03-synthesize |
| 2026-08-15 | 5c-synthesize | run4 | claude-fable-5 | msgbatch_01SNjTLiNPEba4LW2tcUAcTJ | 4 分 3 秒 | 29,958 | 9,793 | $0.39 | 現行(混合群 30 首;16 個樣態) | 05-03-synthesize |
| 2026-08-15 | 5d-annotate | run1 | claude-fable-5 | msgbatch_01H9gXXhNbhZG6gzZtvMHxhh | 3 分 9 秒 | 815,492 | 300,901 | $11.60 | 現行(109/111 有效;song-177、song-199 遭 max_tokens 截斷,單筆補送) | 05-04-annotate |
| 2026-08-15 | 5d-annotate | run1-rescue-177-199 | claude-fable-5 | msgbatch_01YEdroZ51b8btPfG1aqPFCD | 3 分 5 秒 | 17,802 | 11,601 | $0.38 | 現行(run1 之 2 筆補送,max_tokens 提為 16000,成功) | 05-04-annotate |
| 2026-08-15 | 5d-annotate | — | claude-fable-5 | msgbatch_01G67STqZsHpnhedat2HRU85 | 3 分 0 秒 | 17,802 | 14,162 | $0.44 | 誤送(補送批次輪詢中斷後誤判死亡而重送;原批次自行完成並歸檔為現行,本批次取消不及、兩筆皆完成,結果棄用) | 05-04-annotate |
| 2026-08-15 | 5d-annotate | run2 | claude-fable-5 | msgbatch_01VVJpsfidNtF66GnHQdWnGR | 9 分 16 秒 | 815,492 | 306,582 | $11.74 | 現行(111/111 有效;max_tokens 16000,零截斷) | 05-04-annotate |
| 2026-08-15 | 5d-annotate | run3 | claude-fable-5 | msgbatch_01AsYvZd8rsTYVfUYLzWfeMK | 4 分 12 秒 | 815,492 | 298,512 | $11.54 | 現行(111/111 有效;max_tokens 16000,零截斷) | 05-04-annotate |
累計支出:$125.65。 累計支出:$125.65。
+10
View File
@@ -0,0 +1,10 @@
Change Log
==========
version 1.0.0
-------------
Released 2026/8/19
Initial release.
+2
View File
@@ -13,6 +13,8 @@ This is a collection of supporting tools for the conference paper "流行音樂
:maxdepth: 2 :maxdepth: 2
:caption: Contents: :caption: Contents:
changelog
Indices and tables Indices and tables
================== ==================
+1 -1
View File
@@ -6,5 +6,5 @@
"""Tools for A Feminist Audit of Pop Music.""" """Tools for A Feminist Audit of Pop Music."""
VERSION: str = "0.0.0" VERSION: str = "1.0.0"
"""The package version.""" """The package version."""
File diff suppressed because it is too large Load Diff
@@ -24,8 +24,7 @@ keyword set for ``export-llm-input --extras`` is written as a JSON
file holding the group name keywords plus every extra a-priori file holding the group name keywords plus every extra a-priori
keyword the caller gives with the repeatable ``--extra-keyword`` keyword the caller gives with the repeatable ``--extra-keyword``
command-line option, as command-line option, as
:attr:`KeywordsToMerge.KEYWORDS_TO_MERGE_JSON`; with no :attr:`KeywordsToMerge.KEYWORDS_TO_MERGE_JSON`. No default
``--extra-keyword``, it holds the group names alone. No default
extra keyword is ever injected; the caller supplies each one extra keyword is ever injected; the caller supplies each one
consciously. Finally, the command-line choices and the consciously. Finally, the command-line choices and the
environment that produced the numbers -- neither recoverable from environment that produced the numbers -- neither recoverable from
@@ -160,15 +159,8 @@ class KeywordPooler:
if line.strip() == "": if line.strip() == "":
continue continue
record: Any = json.loads(line) record: Any = json.loads(line)
if not isinstance(record, dict) or "id" not in record:
raise ValueError(
f"{path}: record without \"id\": {line}")
if "error" in record: if "error" in record:
continue continue
if "text" not in record:
raise ValueError(
f"{path}: id {record['id']}: record without"
" \"text\" or \"error\"")
song_id: int = cls.__parse_song_id(record["id"], path) song_id: int = cls.__parse_song_id(record["id"], path)
try: try:
keywords: Any = json.loads( keywords: Any = json.loads(
@@ -297,7 +289,8 @@ class KeywordGroups:
class KeywordClusterer: class KeywordClusterer:
"""The clusterer of the pooled keywords into coding groups.""" """The clusterer of the pooled keywords into coding groups."""
DEFAULT_MODEL: str = "sentence-transformers/all-mpnet-base-v2" DEFAULT_MODEL: ClassVar[str] \
= "sentence-transformers/all-mpnet-base-v2"
"""The sentence embedding model used when the caller names """The sentence embedding model used when the caller names
none.""" none."""
@@ -668,11 +661,7 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser.add_argument( parser.add_argument(
"output_dir", type=Path, "output_dir", type=Path,
help="the output directory, created if missing, that" help="the output directory, created if missing, that"
f" receives {PooledKeywords.SOURCE_KEYWORDS_TXT}," " receives the run's output artifacts")
f" {KeywordGroups.RESULT_KEYWORDS_TXT},"
f" {KeywordGroups.RESULT_GROUPS_CSV},"
f" {KeywordsToMerge.KEYWORDS_TO_MERGE_JSON},"
f" and {RunMeta.META_JSON}")
parser.add_argument( parser.add_argument(
"--model", default=model, "--model", default=model,
help=f"the sentence embedding model (default \"{model}\")") help=f"the sentence embedding model (default \"{model}\")")
@@ -698,16 +687,9 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
"""Pool the two tagging runs' keywords and cluster them. """Pool the two tagging runs' keywords and cluster them.
Writes the five fixed-named artifacts under the output Creates the output directory (with parents) if it does not
directory, creating it (with parents) if it does not exist: exist. Each output artifact is written as soon as its
the pooled keyword text file; then the group membership CSV content is computed, so when the input is rejected, or an
file, holding the clustering result alone; the group name
keyword text file, holding the same group names as a readable
list; the coding keyword set JSON file, holding the group
names plus every extra keyword given via ``--extra-keyword``;
and the run metadata JSON file, recording the command-line
choices and the environment. Each file is written as soon as
its content is computed, so when the input is rejected, or an
extra keyword duplicates a group name or another extra extra keyword duplicates a group name or another extra
keyword, the output directory holds whatever the steps before keyword, the output directory holds whatever the steps before
the failing one produced, and the error message names what the failing one produced, and the error message names what
@@ -719,8 +701,8 @@ def main(argv: list[str] | None = None) -> int:
""" """
started: float = time.monotonic() started: float = time.monotonic()
args: argparse.Namespace = parse_args(argv) args: argparse.Namespace = parse_args(argv)
args.output_dir.mkdir(parents=True, exist_ok=True)
try: try:
args.output_dir.mkdir(parents=True, exist_ok=True)
source: PooledKeywords = KeywordPooler( source: PooledKeywords = KeywordPooler(
args.run_dir_1, args.run_dir_2, args.output_dir).run() args.run_dir_1, args.run_dir_2, args.output_dir).run()
clusters: KeywordGroups = KeywordClusterer( clusters: KeywordGroups = KeywordClusterer(
@@ -735,7 +717,7 @@ def main(argv: list[str] | None = None) -> int:
f"Done. Clustered {len(source.keywords)} keywords into" f"Done. Clustered {len(source.keywords)} keywords into"
f" {len(clusters.names)}. {elapsed} elapsed.", f" {len(clusters.names)}. {elapsed} elapsed.",
file=sys.stderr) file=sys.stderr)
except ClusterError as error: except (ClusterError, OSError) as error:
print(f"error: {error}", file=sys.stderr) print(f"error: {error}", file=sys.stderr)
return 1 return 1
return 0 return 0
@@ -11,26 +11,17 @@ project's lyrics-only firewall: the output carries only the
lyrics text of each song, identified by an opaque song key; no lyrics text of each song, identified by an opaque song key; no
title, artist, or chart data crosses into the LLM input. title, artist, or chart data crosses into the LLM input.
With ``--extras``, each record's ``content`` becomes a JSON With ``--extras`` and ``--extras-per-id``, a record's content may
object serialized as a string, its ``lyrics`` key holding the carry extra parameters alongside the lyrics, so a step that needs
song's lyrics followed by the keys of the given extras file in them can get them without this module knowing what they mean; see
their file order, so a step that needs parameters alongside the the exporter's content-building step for how the two merge.
lyrics can carry them without this module knowing what they mean.
With ``--extras-per-id``, the same merge happens per song: the
given file maps a song ID to the extra keys of that one song, and
the export is restricted to the song IDs the file names, so a step
that revisits only some of the songs, each with its own parameters,
gets exactly those records. The two options may be given together,
in which case a record's keys are ``lyrics``, the shared extras'
keys, then that song's own keys, each group in its file order.
""" """
import argparse import argparse
import json import json
import sys import sys
import time import time
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any, ClassVar
import sqlalchemy as sa import sqlalchemy as sa
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -40,6 +31,270 @@ from ..models import Song
from ..utils import format_duration from ..utils import format_duration
class LlmInputExporter:
"""The exporter of the LLM input JSONL file."""
__LYRICS_KEY: ClassVar[str] = "lyrics"
"""The key holding the lyrics in a record's merged content,
and the key forbidden in an extras file."""
def __init__(
self, output_jsonl: Path, extras: Path | None = None,
extras_per_id: Path | None = None) -> None:
"""Set up the exporter of the LLM input JSONL file.
:param output_jsonl: The JSONL output file.
:param extras: The extras JSON file, or None for none.
:param extras_per_id: The per-ID extras JSON file, or
None for none.
"""
self.__output_jsonl: Path = output_jsonl
"""The JSONL output file."""
self.__extras_path: Path | None = extras
"""The extras JSON file, or None for none."""
self.__extras_per_id_path: Path | None = extras_per_id
"""The per-ID extras JSON file, or None for none."""
def run(self) -> int:
"""Export the songs' lyrics to the output JSONL file.
:return: The number of songs exported.
:raises OSError: When a file cannot be read or written.
:raises sqlalchemy.exc.SQLAlchemyError: When the working
store cannot be read.
:raises ValueError: When an extras file is malformed, an
exported song has no lyrics, or the per-ID extras
name a song the working store does not have.
"""
session: Session = ds.get_db()
try:
extras: dict[str, Any] | None = None
if self.__extras_path is not None:
extras = self.__load_extras(self.__extras_path)
extras_per_id: dict[str, dict[str, Any]] | None = None
if self.__extras_per_id_path is not None:
extras_per_id = self.__load_extras_per_id(
self.__extras_per_id_path)
lines: list[str] = self.__build_lines(
session, extras, extras_per_id)
finally:
session.close()
self.__write_output(lines)
return len(lines)
@staticmethod
def __no_duplicate_keys(
pairs: list[tuple[str, Any]]) -> dict[str, Any]:
"""Build a dict from JSON object pairs, rejecting
duplicates.
:param pairs: The key-value pairs of a JSON object, in
file order.
:return: The pairs as a dict, in file order.
:raises ValueError: When a key appears more than once.
"""
result: dict[str, Any] = {}
key: str
value: Any
for key, value in pairs:
if key in result:
raise ValueError(
f"duplicate key \"{key}\" in extras")
result[key] = value
return result
@classmethod
def __load_json_object(
cls, path: Path, label: str) -> dict[str, Any]:
"""Load a single JSON object from a file, in file order.
:param path: The JSON file.
:param label: The kind of file, for the error messages.
:return: The object, in file order.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is
not a JSON object, or has duplicate keys.
"""
with open(path, encoding="utf-8") as file:
text: str = file.read()
try:
data: Any = json.loads(
text, object_pairs_hook=cls.__no_duplicate_keys)
except json.JSONDecodeError as error:
raise ValueError(
f"invalid JSON in {label} file {path}: {error}") \
from error
if not isinstance(data, dict):
raise ValueError(
f"{label} file {path} must contain a JSON object")
return data
@classmethod
def __load_extras(cls, path: Path) -> dict[str, Any]:
"""Load the extras object from a JSON file.
:param path: The extras JSON file.
:return: The extras, in file order.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is
not a JSON object, has duplicate keys, or has a
"lyrics" key.
"""
data: dict[str, Any] = cls.__load_json_object(
path, "extras")
if cls.__LYRICS_KEY in data:
raise ValueError(
f"extras file {path} must not have a"
f" \"{cls.__LYRICS_KEY}\" key")
return data
@classmethod
def __load_extras_per_id(
cls, path: Path) -> dict[str, dict[str, Any]]:
"""Load the per-ID extras object from a JSON file.
:param path: The per-ID extras JSON file, mapping a song
ID, as ``song-<N>``, to the extras of that one song.
:return: The extras of each song ID, in file order, every
song's own extras in their file order too.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is
not a JSON object, has duplicate keys, has a song
whose value is not a JSON object, or has a song with
a "lyrics" key.
"""
data: dict[str, Any] = cls.__load_json_object(
path, "per-ID extras")
song_id: str
extras: Any
for song_id, extras in data.items():
if not isinstance(extras, dict):
raise ValueError(
f"per-ID extras file {path}: id {song_id}"
" must have a JSON object")
if cls.__LYRICS_KEY in extras:
raise ValueError(
f"per-ID extras file {path}: id {song_id}"
f" must not have a \"{cls.__LYRICS_KEY}\""
" key")
return data
def __build_lines(
self, session: Session,
extras: dict[str, Any] | None = None,
extras_per_id: dict[str, dict[str, Any]] | None
= None) -> list[str]:
"""Build the JSONL lines of the exported songs' lyrics.
Every song is exported, unless per-ID extras are given,
in which case only the songs they name are; see
:meth:`__build_content` for how the extras merge into a
record's content.
:param session: The database session.
:param extras: The extra parameters merged into every
record's content alongside the lyrics, in the order
they are to appear, or None for none.
:param extras_per_id: The extra parameters merged into
the content of one record alone, keyed by that
record's song ID and in the order they are to appear,
restricting the export to the song IDs they name, or
None for no such extras and no such restriction.
:return: The JSON lines, one per exported song, ordered by
song ID.
:raises ValueError: When an exported song has no lyrics,
or the per-ID extras name a song the working store
does not have.
"""
lines: list[str] = []
exported: set[str] = set()
song: Song
for song in session.scalars(
sa.select(Song).order_by(Song.id)):
song_id: str = f"song-{song.id}"
if extras_per_id is not None \
and song_id not in extras_per_id:
continue
if song.lyrics is None:
raise ValueError(
f"song {song.id} \"{song.title}\": no lyrics")
song_extras: dict[str, Any] | None = None \
if extras_per_id is None \
else extras_per_id[song_id]
content: str = self.__build_content(
song.lyrics, extras, song_extras)
record: dict[str, str] = {
"id": song_id, "content": content}
lines.append(json.dumps(record, ensure_ascii=False))
exported.add(song_id)
if extras_per_id is not None:
missing: list[str] = sorted(
set(extras_per_id) - exported)
if len(missing) > 0:
raise ValueError(
"the per-ID extras name songs the working"
f" store does not have: {', '.join(missing)}")
return lines
@classmethod
def __build_content(
cls, lyrics: str, extras: dict[str, Any] | None,
song_extras: dict[str, Any] | None) -> str:
"""Build the content of one exported record.
Without extras of either kind, a record's content is the
bare lyrics string. With ``--extras``, the content
becomes a JSON object serialized as a string, its
"lyrics" key holding the song's lyrics followed by the
keys of the given extras file, in their file order. With
``--extras-per-id``, the same merge happens per song: the
song's own extra keys follow the lyrics instead. When
both are given, a record's keys are "lyrics", the shared
extras' keys, then that song's own keys, each group in
its file order.
:param lyrics: The lyrics of the song.
:param extras: The extra parameters shared by every
record, in the order they are to appear, or None for
none.
:param song_extras: The extra parameters of this record
alone, in the order they are to appear, or None for
none.
:return: The bare lyrics when there are no extras of
either kind, or otherwise a JSON object serialized as
a string, whose first key is "lyrics" holding the
lyrics, followed by the shared extras' keys and then
this record's own keys, each group in its given
order.
"""
if extras is None and song_extras is None:
return lyrics
payload: dict[str, Any] = {cls.__LYRICS_KEY: lyrics}
if extras is not None:
payload.update(extras)
if song_extras is not None:
payload.update(song_extras)
return json.dumps(payload, ensure_ascii=False)
def __write_output(self, lines: list[str]) -> None:
"""Write the exported lines to the output JSONL file.
Creates the parent directory when it does not exist.
:param lines: The JSONL lines, in the output order.
:return: None.
:raises OSError: When the file cannot be written.
"""
self.__output_jsonl.parent.mkdir(
parents=True, exist_ok=True)
with open(
self.__output_jsonl, "w",
encoding="utf-8") as file:
line: str
for line in lines:
file.write(line + "\n")
def parse_args(argv: list[str] | None) -> argparse.Namespace: def parse_args(argv: list[str] | None) -> argparse.Namespace:
"""Parse the command-line arguments. """Parse the command-line arguments.
@@ -56,227 +311,33 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser.add_argument( parser.add_argument(
"--extras", type=Path, default=None, "--extras", type=Path, default=None,
help="a JSON file holding a single JSON object of extra" help="a JSON file holding a single JSON object of extra"
" parameters; when given, each record's \"content\"" " parameters merged into every record's content")
" becomes a JSON object string with a \"lyrics\" key"
" followed by the extras' keys, instead of the bare"
" lyrics string")
parser.add_argument( parser.add_argument(
"--extras-per-id", type=Path, default=None, "--extras-per-id", type=Path, default=None,
help="a JSON file holding a single JSON object that maps a" help="a JSON file holding a single JSON object that maps a"
" song ID, as \"song-<N>\", to a JSON object of extra" " song ID, as \"song-<N>\", to a JSON object of extra"
" parameters for that one song; the song's object is" " parameters for that one song, restricting the"
" merged into its \"content\" the same way as with" " export to the song IDs the file names")
" --extras, and the export is restricted to the song"
" IDs the file names")
return parser.parse_args(argv) return parser.parse_args(argv)
def __no_duplicate_keys(
pairs: list[tuple[str, Any]]) -> dict[str, Any]:
"""Build a dict from JSON object pairs, rejecting duplicates.
:param pairs: The key-value pairs of a JSON object, in file
order.
:return: The pairs as a dict, in file order.
:raises ValueError: When a key appears more than once.
"""
result: dict[str, Any] = {}
key: str
value: Any
for key, value in pairs:
if key in result:
raise ValueError(f"duplicate key \"{key}\" in extras")
result[key] = value
return result
def __load_json_object(path: Path, label: str) -> dict[str, Any]:
"""Load a single JSON object from a file, in file order.
:param path: The JSON file.
:param label: The kind of file, for the error messages.
:return: The object, in file order.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is not
a JSON object, or has duplicate keys.
"""
with open(path, encoding="utf-8") as file:
text: str = file.read()
try:
data: Any = json.loads(
text, object_pairs_hook=__no_duplicate_keys)
except json.JSONDecodeError as error:
raise ValueError(
f"invalid JSON in {label} file {path}: {error}") \
from error
if not isinstance(data, dict):
raise ValueError(
f"{label} file {path} must contain a JSON object")
return data
def load_extras(path: Path) -> dict[str, Any]:
"""Load the extras object from a JSON file.
:param path: The extras JSON file.
:return: The extras, in file order.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is not
a JSON object, has duplicate keys, or has a "lyrics" key.
"""
data: dict[str, Any] = __load_json_object(path, "extras")
if "lyrics" in data:
raise ValueError(
f"extras file {path} must not have a \"lyrics\" key")
return data
def load_extras_per_id(path: Path) -> dict[str, dict[str, Any]]:
"""Load the per-ID extras object from a JSON file.
:param path: The per-ID extras JSON file, mapping a song ID,
as ``song-<N>``, to the extras of that one song.
:return: The extras of each song ID, in file order, every
song's own extras in their file order too.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is not
a JSON object, has duplicate keys, has a song whose value
is not a JSON object, or has a song with a "lyrics" key.
"""
data: dict[str, Any] = __load_json_object(path, "per-ID extras")
song_id: str
extras: Any
for song_id, extras in data.items():
if not isinstance(extras, dict):
raise ValueError(
f"per-ID extras file {path}: id {song_id} must"
" have a JSON object")
if "lyrics" in extras:
raise ValueError(
f"per-ID extras file {path}: id {song_id} must not"
" have a \"lyrics\" key")
return data
def __build_content(
lyrics: str,
extras: dict[str, Any] | None,
song_extras: dict[str, Any] | None) -> str:
"""Build the content of one exported record.
:param lyrics: The lyrics of the song.
:param extras: The extra parameters shared by every record,
in the order they are to appear, or None for none.
:param song_extras: The extra parameters of this record
alone, in the order they are to appear, or None for none.
:return: The bare lyrics when there are no extras of either
kind, or otherwise a JSON object serialized as a string,
whose first key is ``"lyrics"`` holding the lyrics,
followed by the shared extras' keys and then this
record's own keys, each group in its given order.
"""
if extras is None and song_extras is None:
return lyrics
payload: dict[str, Any] = {"lyrics": lyrics}
if extras is not None:
payload.update(extras)
if song_extras is not None:
payload.update(song_extras)
return json.dumps(payload, ensure_ascii=False)
def build_lines(
session: Session,
extras: dict[str, Any] | None = None,
extras_per_id: dict[str, dict[str, Any]] | None = None) \
-> list[str]:
"""Build the JSONL lines of the exported songs' lyrics.
Without extras of either kind, each record's ``content`` is
the bare lyrics string. With extras, ``content`` is a JSON
object serialized as a string, whose first key is ``"lyrics"``
holding the lyrics string, followed by the shared extras' keys
and then the song's own per-ID extras' keys, each group in its
given order.
Every song is exported, unless per-ID extras are given, in
which case only the songs they name are.
:param session: The database session.
:param extras: The extra parameters merged into every
record's content alongside the lyrics, in the order they
are to appear, or None for none.
:param extras_per_id: The extra parameters merged into the
content of one record alone, keyed by that record's song
ID and in the order they are to appear, restricting the
export to the song IDs they name, or None for no such
extras and no such restriction.
:return: The JSON lines, one per exported song, ordered by
song ID.
:raises ValueError: When an exported song has no lyrics, or
the per-ID extras name a song the working store does not
have.
"""
lines: list[str] = []
exported: set[str] = set()
song: Song
for song in session.scalars(sa.select(Song).order_by(Song.id)):
song_id: str = f"song-{song.id}"
if extras_per_id is not None and song_id not in extras_per_id:
continue
if song.lyrics is None:
raise ValueError(
f"song {song.id} \"{song.title}\": no lyrics")
song_extras: dict[str, Any] | None = None \
if extras_per_id is None else extras_per_id[song_id]
content: str = __build_content(
song.lyrics, extras, song_extras)
record: dict[str, str] = {
"id": song_id, "content": content}
lines.append(json.dumps(record, ensure_ascii=False))
exported.add(song_id)
if extras_per_id is not None:
missing: list[str] = sorted(set(extras_per_id) - exported)
if len(missing) > 0:
raise ValueError(
"the per-ID extras name songs the working store"
f" does not have: {', '.join(missing)}")
return lines
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
"""Export the LLM input JSONL file from the working store. """Export the LLM input JSONL file from the working store.
Every song is exported, unless ``--extras-per-id`` is given,
in which case only the songs its file names are.
:param argv: The command-line arguments, or None for :param argv: The command-line arguments, or None for
``sys.argv``. ``sys.argv``.
:return: The exit status: 0 on success, non-zero on failure. :return: The exit status: 0 on success, non-zero on failure.
""" """
started: float = time.monotonic() started: float = time.monotonic()
args: argparse.Namespace = parse_args(argv) args: argparse.Namespace = parse_args(argv)
session: Session = ds.get_db()
lines: list[str]
try: try:
extras: dict[str, Any] | None = None count: int = LlmInputExporter(
if args.extras is not None: args.output_jsonl, args.extras,
extras = load_extras(args.extras) args.extras_per_id).run()
extras_per_id: dict[str, dict[str, Any]] | None = None
if args.extras_per_id is not None:
extras_per_id = load_extras_per_id(args.extras_per_id)
lines = build_lines(session, extras, extras_per_id)
except (OSError, sa.exc.SQLAlchemyError, ValueError) as error: except (OSError, sa.exc.SQLAlchemyError, ValueError) as error:
print(f"error: {error}", file=sys.stderr) print(f"error: {error}", file=sys.stderr)
return 1 return 1
finally:
session.close()
args.output_jsonl.parent.mkdir(parents=True, exist_ok=True)
with open(args.output_jsonl, "w", encoding="utf-8") as file:
line: str
for line in lines:
file.write(line + "\n")
elapsed: str = format_duration(time.monotonic() - started) elapsed: str = format_duration(time.monotonic() - started)
print(f"Done. {len(lines)} songs exported." print(f"Done. {count} songs exported."
f" {elapsed} elapsed.", file=sys.stderr) f" {elapsed} elapsed.", file=sys.stderr)
return 0 return 0
@@ -12,12 +12,10 @@ layer. The working store is only read, never written; the
``build-db`` subcommand assembles the captured files into the ``build-db`` subcommand assembles the captured files into the
store on the next rebuild. store on the next rebuild.
Every fetched row is meant for later human verification: the An unresolved artist or an error on one artist is noted on its
description of the resolved item is recorded in the note column row and does not fail the run. A row whose name is no longer an
so that a bad match can be spotted. An unresolved artist or an artist of the store is dropped from the snapshot and reported on
error on one artist is noted on its row and does not fail the the standard error.
run. A row whose name is no longer an artist of the store is
dropped from the snapshot and reported on the standard error.
""" """
import argparse import argparse
import csv import csv
@@ -33,7 +31,7 @@ import urllib.request
from collections.abc import Container, Sequence from collections.abc import Container, Sequence
from dataclasses import asdict, dataclass, field, fields from dataclasses import asdict, dataclass, field, fields
from pathlib import Path from pathlib import Path
from typing import Any, Literal, TextIO from typing import Any, ClassVar, Literal, TextIO
import sqlalchemy as sa import sqlalchemy as sa
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -43,71 +41,6 @@ from ..database import ds
from ..models import Artist, Song, SongArtist from ..models import Artist, Song, SongArtist
from ..utils import format_duration from ..utils import format_duration
API_URL: str = "https://www.wikidata.org/w/api.php"
"""The URL of the Wikidata API endpoint."""
SPARQL_URL: str = "https://query.wikidata.org/sparql"
"""The URL of the Wikidata Query Service SPARQL endpoint."""
USER_AGENT: str = (
f"pop-fem-audit-tools/{VERSION}"
" (https://github.com/imacat/pop-fem-audit;"
" mailto:imacat@mail.imacat.idv.tw)")
"""The User-Agent header sent on every HTTP request."""
TIMEOUT: float = 30.0
"""The timeout of an API HTTP request, in seconds."""
SPARQL_TIMEOUT: float = 90.0
"""The timeout of a SPARQL HTTP request, in seconds.
Higher than the API timeout: the WDQS server aborts a slow
query at 60 seconds, and a lower client timeout would race
that server-side abort and misclassify a slow-but-answerable
query as a client-side timeout instead of letting the server's
own HTTP error response arrive and enter the retry path."""
SLEEP_SECONDS: float = 1.0
"""The delay between consecutive HTTP requests, in seconds."""
MAX_ATTEMPTS: int = 5
"""The maximum number of attempts on a transient error."""
RETRY_SECONDS: float = 15.0
"""The back-off unit on a transient error, in seconds;
multiplied by the attempt number already made."""
RETRY_STATUSES: frozenset[int] = frozenset({429, 500, 502, 503})
"""The HTTP statuses that are retried with a back-off."""
MAX_STAGE1_TITLES: int = 3
"""The maximum number of charted titles used for the stage-1 song
corroboration."""
HUMAN_QID: str = "Q5"
"""The Wikidata item ID of "human"."""
ENSEMBLE_QID: str = "Q2088357"
"""The Wikidata item ID of "musical ensemble"."""
ORIGINAL_CAST_QID: str = "Q106497009"
"""The Wikidata item ID of "original cast"."""
GROUP_KEYWORDS: Sequence[str] = ("band", "group", "duo", "trio")
"""The label keywords that suggest a musical ensemble, covering
labels like "boy band" and "girl group"."""
NOTE_NOT_FOUND: str = "not found"
"""The note sentinel of an artist without a resolved Wikidata
item, written to the snapshot and read back for the
classification."""
CORPUS_START_YEAR: int = 2016
"""The first year of the corpus window: a member who left a group
before it never performed a corpus song."""
MIXED_GENDER: str = "mixed"
"""The gender recorded for a group whose members do not share one
gender."""
TIME_YEAR_PATTERN: re.Pattern[str] = re.compile(r"^[+-]?\d+")
"""The leading year of a Wikidata time value."""
PINNED_QIDS: dict[str, str] = {}
"""The last-resort pinned item IDs, keyed by the artist name.
An entry is for an artist the algorithm documented on
``ArtistFetcher`` is structurally unable to resolve, with its
justification recorded here. Currently empty: the only pin ever
needed, "Pinkfong" (typed as a brand, which the type gate
excludes by design), became moot when the store's artist entity
behind that credit was identified as Hope Segoine.
A pinned name skips the candidate retrieval and corroboration
steps; its item ID is used directly."""
class ArtistType(enum.StrEnum): class ArtistType(enum.StrEnum):
"""The decided artist type of a snapshot row.""" """The decided artist type of a snapshot row."""
@@ -147,15 +80,14 @@ class ArtistSnapshot:
return asdict(self) return asdict(self)
SNAPSHOT_FIELDS: Sequence[str] = tuple(
x.name for x in fields(ArtistSnapshot))
"""The header columns of the Wikidata artist snapshot CSV file."""
@dataclass @dataclass
class GroupMember: class GroupMember:
"""One has-part member of a Wikidata group item.""" """One has-part member of a Wikidata group item."""
__CORPUS_START_YEAR: ClassVar[int] = 2016
"""The first year of the corpus window: a member who left a
group before it never performed a corpus song."""
qid: str qid: str
"""The item ID of the member.""" """The item ID of the member."""
start_years: list[int] = field(default_factory=list) start_years: list[int] = field(default_factory=list)
@@ -181,7 +113,7 @@ class GroupMember:
if len(self.end_years) == 0: if len(self.end_years) == 0:
return True return True
last_end: int = max(self.end_years) last_end: int = max(self.end_years)
if last_end >= CORPUS_START_YEAR: if last_end >= self.__CORPUS_START_YEAR:
return True return True
if len(self.start_years) == 0: if len(self.start_years) == 0:
return False return False
@@ -229,20 +161,8 @@ class RetryExhausted(Exception):
""" """
def parse_args(argv: list[str] | None) -> argparse.Namespace: NOTE_NOT_FOUND: str = "not found"
"""Parse the command-line arguments. """The note marking an artist that could not be resolved."""
:param argv: The command-line arguments, or None for
``sys.argv``.
:return: The parsed arguments.
"""
parser: argparse.ArgumentParser = argparse.ArgumentParser(
description="Fetch the artist metadata from Wikidata"
" into the capture layer.")
parser.add_argument(
"wikidata_csv", type=Path,
help="the Wikidata artist snapshot CSV file")
return parser.parse_args(argv)
class ArtistFetcher: class ArtistFetcher:
@@ -291,6 +211,71 @@ class ArtistFetcher:
in the note. in the note.
""" """
__API_URL: ClassVar[str] = "https://www.wikidata.org/w/api.php"
"""The URL of the Wikidata API endpoint."""
__SPARQL_URL: ClassVar[str] \
= "https://query.wikidata.org/sparql"
"""The URL of the Wikidata Query Service SPARQL endpoint."""
__USER_AGENT: ClassVar[str] = (
f"pop-fem-audit-tools/{VERSION}"
" (https://github.com/imacat/pop-fem-audit;"
" mailto:imacat@mail.imacat.idv.tw)")
"""The User-Agent header sent on every HTTP request."""
__TIMEOUT: ClassVar[float] = 30.0
"""The timeout of an API HTTP request, in seconds."""
__SPARQL_TIMEOUT: ClassVar[float] = 90.0
"""The timeout of a SPARQL HTTP request, in seconds.
Higher than the API timeout: the WDQS server aborts a slow
query at 60 seconds, and a lower client timeout would race
that server-side abort and misclassify a slow-but-answerable
query as a client-side timeout instead of letting the
server's own HTTP error response arrive and enter the retry
path."""
__SLEEP_SECONDS: ClassVar[float] = 1.0
"""The delay between consecutive HTTP requests, in
seconds."""
__MAX_ATTEMPTS: ClassVar[int] = 5
"""The maximum number of attempts on a transient error."""
__RETRY_SECONDS: ClassVar[float] = 15.0
"""The back-off unit on a transient error, in seconds;
multiplied by the attempt number already made."""
__RETRY_STATUSES: ClassVar[frozenset[int]] \
= frozenset({429, 500, 502, 503})
"""The HTTP statuses that are retried with a back-off."""
__MAX_STAGE1_TITLES: ClassVar[int] = 3
"""The maximum number of charted titles used for the
stage-1 song corroboration."""
__HUMAN_QID: ClassVar[str] = "Q5"
"""The Wikidata item ID of "human"."""
__ENSEMBLE_QID: ClassVar[str] = "Q2088357"
"""The Wikidata item ID of "musical ensemble"."""
__ORIGINAL_CAST_QID: ClassVar[str] = "Q106497009"
"""The Wikidata item ID of "original cast"."""
__GROUP_KEYWORDS: ClassVar[Sequence[str]] \
= ("band", "group", "duo", "trio")
"""The label keywords that suggest a musical ensemble,
covering labels like "boy band" and "girl group"."""
__MIXED_GENDER: ClassVar[str] = "mixed"
"""The gender recorded for a group whose members do not
share one gender."""
__TIME_YEAR_PATTERN: ClassVar[re.Pattern[str]] \
= re.compile(r"^[+-]?\d+")
"""The leading year of a Wikidata time value."""
__PINNED_QIDS: ClassVar[dict[str, str]] = {}
"""The last-resort pinned item IDs, keyed by the artist name.
An entry is for an artist the algorithm documented on
``ArtistFetcher`` is structurally unable to resolve, with its
justification recorded here. Currently empty: the only pin
ever needed, "Pinkfong" (typed as a brand, which the type
gate excludes by design), became moot when the store's
artist entity behind that credit was identified as Hope
Segoine.
A pinned name skips the candidate retrieval and corroboration
steps; its item ID is used directly."""
def __init__(self) -> None: def __init__(self) -> None:
"""Construct the fetcher.""" """Construct the fetcher."""
self.__sent: int = 0 self.__sent: int = 0
@@ -342,8 +327,8 @@ class ArtistFetcher:
transient error are exhausted. transient error are exhausted.
:raises ValueError: On a JSON decoding error. :raises ValueError: On a JSON decoding error.
""" """
if name in PINNED_QIDS: if name in self.__PINNED_QIDS:
return PINNED_QIDS[name] return self.__PINNED_QIDS[name]
candidates: list[str] = self.__candidates(name) candidates: list[str] = self.__candidates(name)
if len(candidates) == 0: if len(candidates) == 0:
return None return None
@@ -373,11 +358,11 @@ class ArtistFetcher:
{{ ?item rdfs:label ?name }} {{ ?item rdfs:label ?name }}
UNION {{ ?item skos:altLabel ?name }} UNION {{ ?item skos:altLabel ?name }}
{{ {{
?item wdt:P31 wd:{HUMAN_QID} ?item wdt:P31 wd:{self.__HUMAN_QID}
}} UNION {{ }} UNION {{
?item wdt:P31/wdt:P279* wd:{ENSEMBLE_QID} ?item wdt:P31/wdt:P279* wd:{self.__ENSEMBLE_QID}
}} UNION {{ }} UNION {{
?item wdt:P31 wd:{ORIGINAL_CAST_QID} ?item wdt:P31 wd:{self.__ORIGINAL_CAST_QID}
}} }}
}} }}
""" """
@@ -403,7 +388,7 @@ class ArtistFetcher:
transient error are exhausted. transient error are exhausted.
:raises ValueError: On a JSON decoding error. :raises ValueError: On a JSON decoding error.
""" """
subset: Sequence[str] = titles[:MAX_STAGE1_TITLES] subset: Sequence[str] = titles[:self.__MAX_STAGE1_TITLES]
if len(subset) == 0: if len(subset) == 0:
return None return None
query: str = f""" query: str = f"""
@@ -532,7 +517,7 @@ class ArtistFetcher:
qid: str qid: str
for qid in qids: for qid in qids:
member: MemberClaims = claims.get(qid, MemberClaims()) member: MemberClaims = claims.get(qid, MemberClaims())
if HUMAN_QID not in member.instance_of_ids: if self.__HUMAN_QID not in member.instance_of_ids:
continue continue
if len(member.gender_ids) == 0: if len(member.gender_ids) == 0:
return return
@@ -543,7 +528,7 @@ class ArtistFetcher:
[x[1] for x in genders], any_language=True) [x[1] for x in genders], any_language=True)
unique: set[str] = {x[1] for x in genders} unique: set[str] = {x[1] for x in genders}
snapshot.gender = labels[genders[0][1]] \ snapshot.gender = labels[genders[0][1]] \
if len(unique) == 1 else MIXED_GENDER if len(unique) == 1 else self.__MIXED_GENDER
basis: str = "gender derived from members: " + "; ".join( basis: str = "gender derived from members: " + "; ".join(
f"{x} {labels[y]}" for x, y in genders) f"{x} {labels[y]}" for x, y in genders)
snapshot.note = f"{snapshot.note}; {basis}" \ snapshot.note = f"{snapshot.note}; {basis}" \
@@ -677,7 +662,8 @@ class ArtistFetcher:
or not isinstance(value.get("time"), str): or not isinstance(value.get("time"), str):
continue continue
match: re.Match[str] | None \ match: re.Match[str] | None \
= TIME_YEAR_PATTERN.match(value["time"]) = ArtistFetcher.__TIME_YEAR_PATTERN.match(
value["time"])
if match is not None: if match is not None:
years.append(int(match.group())) years.append(int(match.group()))
return years return years
@@ -806,12 +792,13 @@ class ArtistFetcher:
``ArtistType.GROUP`` for a musical ensemble, or the ``ArtistType.GROUP`` for a musical ensemble, or the
empty string for the human to decide. empty string for the human to decide.
""" """
if HUMAN_QID in type_ids: if ArtistFetcher.__HUMAN_QID in type_ids:
return ArtistType.SOLO return ArtistType.SOLO
qid: str qid: str
for qid in type_ids: for qid in type_ids:
label: str = labels.get(qid, "").lower() label: str = labels.get(qid, "").lower()
if any(x in label for x in GROUP_KEYWORDS): if any(x in label
for x in ArtistFetcher.__GROUP_KEYWORDS):
return ArtistType.GROUP return ArtistType.GROUP
return "" return ""
@@ -827,14 +814,15 @@ class ArtistFetcher:
transient error are exhausted. transient error are exhausted.
:raises ValueError: On a JSON decoding error. :raises ValueError: On a JSON decoding error.
""" """
url: str = (f"{SPARQL_URL}?" url: str = (
f"{urllib.parse.urlencode({'query': query})}") f"{self.__SPARQL_URL}?"
f"{urllib.parse.urlencode({'query': query})}")
request: urllib.request.Request = urllib.request.Request( request: urllib.request.Request = urllib.request.Request(
url, headers={ url, headers={
"User-Agent": USER_AGENT, "User-Agent": self.__USER_AGENT,
"Accept": "application/sparql-results+json"}) "Accept": "application/sparql-results+json"})
body: bytes = self.__send( body: bytes = self.__send(
request, timeout=SPARQL_TIMEOUT) request, timeout=self.__SPARQL_TIMEOUT)
data: Any = json.loads(body) data: Any = json.loads(body)
bindings: Any = None bindings: Any = None
if isinstance(data, dict) \ if isinstance(data, dict) \
@@ -868,13 +856,14 @@ class ArtistFetcher:
transient error are exhausted. transient error are exhausted.
:raises ValueError: On a JSON decoding error. :raises ValueError: On a JSON decoding error.
""" """
url: str = f"{API_URL}?{urllib.parse.urlencode(params)}" url: str \
= f"{self.__API_URL}?{urllib.parse.urlencode(params)}"
request: urllib.request.Request = urllib.request.Request( request: urllib.request.Request = urllib.request.Request(
url, headers={"User-Agent": USER_AGENT}) url, headers={"User-Agent": self.__USER_AGENT})
return json.loads(self.__send(request)) return json.loads(self.__send(request))
def __send(self, request: urllib.request.Request, def __send(self, request: urllib.request.Request,
timeout: float = TIMEOUT) -> bytes: timeout: float = __TIMEOUT) -> bytes:
"""Send an HTTP request, retrying on a transient error. """Send an HTTP request, retrying on a transient error.
Consecutive requests are separated by a fixed delay. A Consecutive requests are separated by a fixed delay. A
@@ -892,7 +881,7 @@ class ArtistFetcher:
transient error are exhausted. transient error are exhausted.
""" """
if self.__sent > 0: if self.__sent > 0:
time.sleep(SLEEP_SECONDS) time.sleep(self.__SLEEP_SECONDS)
self.__sent += 1 self.__sent += 1
attempt: int = 1 attempt: int = 1
reason: str | None reason: str | None
@@ -905,10 +894,10 @@ class ArtistFetcher:
reason = self.__retry_reason(error) reason = self.__retry_reason(error)
if reason is None: if reason is None:
raise raise
if attempt >= MAX_ATTEMPTS: if attempt >= self.__MAX_ATTEMPTS:
raise RetryExhausted( raise RetryExhausted(
f"retries exhausted ({reason})") from error f"retries exhausted ({reason})") from error
time.sleep(RETRY_SECONDS * attempt) time.sleep(self.__RETRY_SECONDS * attempt)
attempt += 1 attempt += 1
@staticmethod @staticmethod
@@ -923,7 +912,7 @@ class ArtistFetcher:
the error is not transient and must not be retried. the error is not transient and must not be retried.
""" """
if isinstance(error, urllib.error.HTTPError): if isinstance(error, urllib.error.HTTPError):
if error.code not in RETRY_STATUSES: if error.code not in ArtistFetcher.__RETRY_STATUSES:
return None return None
return str(error) return str(error)
if isinstance(error, TimeoutError): if isinstance(error, TimeoutError):
@@ -968,92 +957,216 @@ class ArtistFetcher:
return uri.rsplit("/", 1)[-1] return uri.rsplit("/", 1)[-1]
def read_snapshot_rows(file: TextIO) -> list[dict[str, str]]: @dataclass(frozen=True)
"""Read the current rows of a snapshot CSV file handle. class FetchCounts:
"""The outcome counts of one snapshot update run."""
:param file: The open, seekable snapshot CSV file. fetched: int
:return: The rows, keyed by the column name. """The number of artists newly resolved."""
:raises OSError: When the file cannot be read. not_found: int
"""The number of artists left unresolved."""
errors: int
"""The number of artists that ended in an error."""
class ArtistSnapshotUpdater:
"""The updater of the Wikidata artist snapshot CSV file.
Fetches the metadata of every artist of the working store
that the snapshot does not resolve yet, appends a row for
each to the snapshot as it is fetched, and rewrites the
snapshot sorted by artist name with its stale rows dropped.
""" """
file.seek(0)
reader: csv.DictReader[str] = csv.DictReader(file)
return list(reader)
__SNAPSHOT_FIELDS: ClassVar[Sequence[str]] = tuple(
x.name for x in fields(ArtistSnapshot))
"""The header columns of the Wikidata artist snapshot CSV file."""
def read_artist_titles(session: Session, def __init__(self, wikidata_csv: Path) -> None:
artist_id: int) -> list[str]: """Set up the updater.
"""Read the charted song titles credited to an artist.
:param session: The database session. :param wikidata_csv: The Wikidata artist snapshot CSV
:param artist_id: The artist ID. file.
:return: The song titles credited to the artist, ordered by """
the song ID, with the duplicate titles removed. self.__wikidata_csv: Path = wikidata_csv
""" """The Wikidata artist snapshot CSV file."""
titles: Sequence[str] = session.scalars(
sa.select(Song.title)
.join(SongArtist, SongArtist.song_id == Song.id)
.where(SongArtist.artist_id == artist_id)
.order_by(Song.id)).all()
return list(dict.fromkeys(titles))
def run(self) -> FetchCounts:
"""Fetch every unresolved artist and update the snapshot.
def ensure_snapshot_header(file: TextIO) -> None: :return: The counts of the run.
"""Write the snapshot CSV header row if the file is empty. :raises OSError: When the snapshot file, or its parent
directory, cannot be read or written.
:raises sqlalchemy.exc.SQLAlchemyError: When the working
store cannot be read.
"""
session: Session = ds.get_db()
try:
return self.__run(session)
finally:
session.close()
:param file: The open, seekable snapshot CSV file. def __run(self, session: Session) -> FetchCounts:
:return: None. """Run the fetch loop with an open database session.
:raises OSError: When the file cannot be written.
""" :param session: The database session.
file.seek(0, os.SEEK_END) :return: The counts of the run.
if file.tell() == 0: :raises OSError: When the snapshot file, or its parent
csv.writer(file).writerow(SNAPSHOT_FIELDS) directory, cannot be read or written.
"""
fetcher: ArtistFetcher = ArtistFetcher()
fetched: int = 0
not_found: int = 0
errors: int = 0
self.__wikidata_csv.parent.mkdir(
parents=True, exist_ok=True)
with open(self.__wikidata_csv, "a+", encoding="utf-8",
newline="") as csv_file:
done: set[str] = {
x["name"] for x in
self.__read_snapshot_rows(csv_file)
if x["gender"] != ""}
self.__ensure_snapshot_header(csv_file)
names: set[str] = set()
artist: Artist
for artist in session.scalars(
sa.select(Artist).order_by(Artist.id)):
names.add(artist.name)
if artist.name in done:
continue
titles: list[str] = self.__read_artist_titles(
session, artist.id)
snapshot: ArtistSnapshot = fetcher.fetch(
artist.name, titles)
self.__append_row(csv_file, snapshot)
status: str = snapshot.qid
if snapshot.note == NOTE_NOT_FOUND:
not_found += 1
status = NOTE_NOT_FOUND
elif snapshot.note.startswith("error: "):
errors += 1
status = snapshot.note
else:
fetched += 1
print(f"artist \"{artist.name}\": {status}",
file=sys.stderr)
self.__write_snapshot(csv_file, names)
return FetchCounts(
fetched=fetched, not_found=not_found, errors=errors)
@staticmethod
def __read_snapshot_rows(file: TextIO) \
-> list[dict[str, str]]:
"""Read the current rows of a snapshot CSV file handle.
:param file: The open, seekable snapshot CSV file.
:return: The rows, keyed by the column name.
:raises OSError: When the file cannot be read.
"""
file.seek(0)
reader: csv.DictReader[str] = csv.DictReader(file)
return list(reader)
@staticmethod
def __read_artist_titles(session: Session,
artist_id: int) -> list[str]:
"""Read the charted song titles credited to an artist.
:param session: The database session.
:param artist_id: The artist ID.
:return: The song titles credited to the artist, ordered
by the song ID, with the duplicate titles removed.
"""
titles: Sequence[str] = session.scalars(
sa.select(Song.title)
.join(SongArtist, SongArtist.song_id == Song.id)
.where(SongArtist.artist_id == artist_id)
.order_by(Song.id)).all()
return list(dict.fromkeys(titles))
@staticmethod
def __ensure_snapshot_header(file: TextIO) -> None:
"""Write the snapshot CSV header row if the file is
empty.
:param file: The open, seekable snapshot CSV file.
:return: None.
:raises OSError: When the file cannot be written.
"""
file.seek(0, os.SEEK_END)
if file.tell() == 0:
csv.writer(file).writerow(
ArtistSnapshotUpdater.__SNAPSHOT_FIELDS)
file.flush()
@staticmethod
def __append_row(file: TextIO,
snapshot: ArtistSnapshot) -> None:
"""Append a snapshot row to a snapshot CSV file handle.
:param file: The open snapshot CSV file, opened for
append.
:param snapshot: The snapshot of an artist.
:return: None.
:raises OSError: When the file cannot be written.
"""
csv.DictWriter(
file, ArtistSnapshotUpdater.__SNAPSHOT_FIELDS).writerow(
snapshot.to_row())
file.flush() file.flush()
@staticmethod
def __write_snapshot(file: TextIO, names: Container[str]) \
-> None:
"""Rewrite a snapshot CSV file handle sorted by artist
name.
def append_row(file: TextIO, snapshot: ArtistSnapshot) -> None: The rows are ordered by the case-folded artist name,
"""Append a snapshot row to a snapshot CSV file handle. matching the convention of the derived ``artists.csv``.
An artist keeps one row only, the last one of the file,
so that a re-fetched artist replaces its earlier row. A
row whose name is not an artist of the store is dropped
and reported on the standard error.
:param file: The open snapshot CSV file, opened for append. :param file: The open, seekable snapshot CSV file.
:param snapshot: The snapshot of an artist. :param names: The artist names of the working store.
:return: None. :return: None.
:raises OSError: When the file cannot be written. :raises OSError: When the file cannot be read or written.
"""
kept: dict[str, dict[str, str]] = {}
row: dict[str, str]
for row in ArtistSnapshotUpdater.__read_snapshot_rows(
file):
if row["name"] not in names:
print(f"dropped stale row \"{row['name']}\":"
" no such artist in the store",
file=sys.stderr)
continue
kept[row["name"]] = row
ordered: list[dict[str, str]] = sorted(
kept.values(), key=lambda x: x["name"].casefold())
file.seek(0)
file.truncate()
writer: csv.DictWriter[str] = csv.DictWriter(
file, ArtistSnapshotUpdater.__SNAPSHOT_FIELDS)
writer.writeheader()
writer.writerows(ordered)
def parse_args(argv: list[str] | None) -> argparse.Namespace:
"""Parse the command-line arguments.
:param argv: The command-line arguments, or None for
``sys.argv``.
:return: The parsed arguments.
""" """
csv.DictWriter(file, SNAPSHOT_FIELDS).writerow( parser: argparse.ArgumentParser = argparse.ArgumentParser(
snapshot.to_row()) description="Fetch the artist metadata from Wikidata"
file.flush() " into the capture layer.")
parser.add_argument(
"wikidata_csv", type=Path,
def write_snapshot(file: TextIO, names: Container[str]) -> None: help="the Wikidata artist snapshot CSV file")
"""Rewrite a snapshot CSV file handle sorted by artist name. return parser.parse_args(argv)
The rows are ordered by the case-folded artist name, matching
the convention of the derived ``artists.csv``. An artist
keeps one row only, the last one of the file, so that a
re-fetched artist replaces its earlier row. A row whose name
is not an artist of the store is dropped and reported on the
standard error.
:param file: The open, seekable snapshot CSV file.
:param names: The artist names of the working store.
:return: None.
:raises OSError: When the file cannot be read or written.
"""
kept: dict[str, dict[str, str]] = {}
row: dict[str, str]
for row in read_snapshot_rows(file):
if row["name"] not in names:
print(f"dropped stale row \"{row['name']}\":"
" no such artist in the store", file=sys.stderr)
continue
kept[row["name"]] = row
ordered: list[dict[str, str]] = sorted(
kept.values(), key=lambda x: x["name"].casefold())
file.seek(0)
file.truncate()
writer: csv.DictWriter[str] = csv.DictWriter(
file, SNAPSHOT_FIELDS)
writer.writeheader()
writer.writerows(ordered)
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
@@ -1066,52 +1179,16 @@ def main(argv: list[str] | None = None) -> int:
""" """
started: float = time.monotonic() started: float = time.monotonic()
args: argparse.Namespace = parse_args(argv) args: argparse.Namespace = parse_args(argv)
fetcher: ArtistFetcher = ArtistFetcher()
fetched: int = 0
not_found: int = 0
errors: int = 0
session: Session = ds.get_db()
try: try:
args.wikidata_csv.parent.mkdir( counts: FetchCounts \
parents=True, exist_ok=True) = ArtistSnapshotUpdater(args.wikidata_csv).run()
with open(args.wikidata_csv, "a+", encoding="utf-8",
newline="") as csv_file:
done: set[str] = {x["name"] for x in
read_snapshot_rows(csv_file)
if x["gender"] != ""}
ensure_snapshot_header(csv_file)
names: set[str] = set()
artist: Artist
for artist in session.scalars(
sa.select(Artist).order_by(Artist.id)):
names.add(artist.name)
if artist.name in done:
continue
titles: list[str] = read_artist_titles(
session, artist.id)
snapshot: ArtistSnapshot = fetcher.fetch(
artist.name, titles)
append_row(csv_file, snapshot)
status: str = snapshot.qid
if snapshot.note == NOTE_NOT_FOUND:
not_found += 1
status = "not found"
elif snapshot.note.startswith("error: "):
errors += 1
status = snapshot.note
else:
fetched += 1
print(f"artist \"{artist.name}\": {status}",
file=sys.stderr)
write_snapshot(csv_file, names)
except (OSError, sa.exc.SQLAlchemyError) as error: except (OSError, sa.exc.SQLAlchemyError) as error:
print(f"error: {error}", file=sys.stderr) print(f"error: {error}", file=sys.stderr)
return 1 return 1
finally: attempted: int = counts.fetched + counts.not_found \
session.close() + counts.errors
attempted: int = fetched + not_found + errors
elapsed: str = format_duration(time.monotonic() - started) elapsed: str = format_duration(time.monotonic() - started)
print(f"Done. Resolved {fetched}/{attempted} artists." print(f"Done. Resolved {counts.fetched}/{attempted}"
f" {elapsed} elapsed.", f" artists. {elapsed} elapsed.",
file=sys.stderr) file=sys.stderr)
return 0 return 0
@@ -30,8 +30,9 @@ import time
import urllib.parse import urllib.parse
import urllib.request import urllib.request
from collections.abc import Sequence from collections.abc import Sequence
from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any, ClassVar
import sqlalchemy as sa import sqlalchemy as sa
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -45,60 +46,6 @@ from ..models import (
) )
from ..utils import format_duration from ..utils import format_duration
PROVENANCE_FIELDS: Sequence[str] = (
"song_id", "source", "method", "acquired_at", "note")
"""The header columns of the lyrics provenance CSV file."""
USER_AGENT: str = ("pop-fem-audit-tools"
" (https://github.com/imacat/pop-fem-audit)")
"""The User-Agent header sent on every HTTP request."""
TIMEOUT: float = 30.0
"""The timeout of an HTTP request, in seconds."""
SLEEP_SECONDS: float = 1.0
"""The delay between consecutive HTTP requests, in seconds."""
def __build_normalization() -> dict[int, str | None]:
"""Build the lyrics normalization translation table.
:return: The codepoint-to-replacement mapping, a replacement
of None meaning removal.
"""
table: dict[int, str | None] = {}
codepoint: int
for codepoint in range(0x80, 0xa0):
try:
table[codepoint] = bytes([codepoint]).decode("cp1252")
except UnicodeDecodeError:
table[codepoint] = None
table[0x0435] = "e"
table[0x03cc] = "ó"
for codepoint in (0x2005, 0x205f, 0x200a):
table[codepoint] = " "
for codepoint in (0x200b, 0x200c, 0x200d, 0xfeff):
table[codepoint] = None
return table
NORMALIZATION: dict[int, str | None] = __build_normalization()
"""The codepoint-to-replacement mapping applied to fetched
lyrics: cp1252-mojibake restoration for U+0080-U+009F (with the
five byte values undefined in cp1252 removed), homoglyph
restoration for the Cyrillic "e" and the Greek "o" with tonos,
ASCII-space restoration for exotic space variants, and removal
of zero-width characters. A replacement of None removes the
codepoint."""
def normalize_lyrics(text: str) -> str:
"""Restore or remove watermark and mojibake characters.
:param text: The lyrics text as fetched from an API.
:return: The text with the codepoints in
:data:`NORMALIZATION` replaced or removed; every other
character is unchanged.
"""
return text.translate(NORMALIZATION)
def parse_args(argv: list[str] | None) -> argparse.Namespace: def parse_args(argv: list[str] | None) -> argparse.Namespace:
"""Parse the command-line arguments. """Parse the command-line arguments.
@@ -122,6 +69,16 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
class LyricsFetcher: class LyricsFetcher:
"""A fetcher of song lyrics from the public lyrics APIs.""" """A fetcher of song lyrics from the public lyrics APIs."""
__USER_AGENT: ClassVar[str] = (
"pop-fem-audit-tools"
" (https://github.com/imacat/pop-fem-audit)")
"""The User-Agent header sent on every HTTP request."""
__TIMEOUT: ClassVar[float] = 30.0
"""The timeout of an HTTP request, in seconds."""
__SLEEP_SECONDS: ClassVar[float] = 1.0
"""The delay between consecutive HTTP requests, in
seconds."""
def __init__(self) -> None: def __init__(self) -> None:
"""Construct the fetcher.""" """Construct the fetcher."""
self.__sent: int = 0 self.__sent: int = 0
@@ -193,78 +150,216 @@ class LyricsFetcher:
network, or decoding error. network, or decoding error.
""" """
if self.__sent > 0: if self.__sent > 0:
time.sleep(SLEEP_SECONDS) time.sleep(self.__SLEEP_SECONDS)
self.__sent += 1 self.__sent += 1
request: urllib.request.Request = urllib.request.Request( request: urllib.request.Request = urllib.request.Request(
url, headers={"User-Agent": USER_AGENT}) url, headers={"User-Agent": self.__USER_AGENT})
try: try:
with urllib.request.urlopen( with urllib.request.urlopen(
request, timeout=TIMEOUT) as response: request, timeout=self.__TIMEOUT) as response:
return json.load(response) return json.load(response)
except (OSError, ValueError): except (OSError, ValueError):
return None return None
def query_artist(session: Session, song_id: int) -> str: @dataclass(frozen=True)
"""Find the artist name to query the APIs with. class LyricsFetchCounts:
"""The outcome of one run of fetching the missing lyrics."""
:param session: The database session. fetched: int
:param song_id: The song ID. """The number of songs newly fetched."""
:return: The name of the primary-role artist with the lowest missed: int
position. """The number of songs every API missed."""
"""
name: str | None = session.scalar(
sa.select(Artist.name)
.join(SongArtist, SongArtist.artist_id == Artist.id)
.where(SongArtist.song_id == song_id,
SongArtist.role == Role.PRIMARY)
.order_by(SongArtist.position)
.limit(1))
assert name is not None
return name
def save_lyrics(lyrics_dir: Path, song_id: int, class LyricsFetchRunner:
lyrics: str) -> None: """The orchestrator of one run of fetching missing lyrics."""
"""Write the lyrics of a song into the cache directory.
The cache directory is created when missing. __PROVENANCE_FIELDS: ClassVar[Sequence[str]] = (
"song_id", "source", "method", "acquired_at", "note")
"""The header columns of the lyrics provenance CSV file."""
The lyrics text is normalized with :func:`normalize_lyrics` @staticmethod
before being written. def __build_normalization() -> dict[int, str | None]:
"""Build the lyrics normalization translation table.
:param lyrics_dir: The lyrics cache directory. :return: The codepoint-to-replacement mapping, a
:param song_id: The song ID. replacement of None meaning removal.
:param lyrics: The lyrics text. """
:return: None. table: dict[int, str | None] = {}
:raises OSError: When the file cannot be written. codepoint: int
""" for codepoint in range(0x80, 0xa0):
lyrics_dir.mkdir(parents=True, exist_ok=True) try:
(lyrics_dir / f"{song_id}.txt").write_text( table[codepoint] = bytes(
normalize_lyrics(lyrics), encoding="utf-8") [codepoint]).decode("cp1252")
except UnicodeDecodeError:
table[codepoint] = None
table[0x0435] = "e"
table[0x03cc] = "ó"
for codepoint in (0x2005, 0x205f, 0x200a):
table[codepoint] = " "
for codepoint in (0x200b, 0x200c, 0x200d, 0xfeff):
table[codepoint] = None
return table
__NORMALIZATION: ClassVar[dict[int, str | None]] \
= __build_normalization()
"""The codepoint-to-replacement mapping applied to fetched
lyrics: cp1252-mojibake restoration for U+0080-U+009F (with
the five byte values undefined in cp1252 removed), homoglyph
restoration for the Cyrillic "e" and the Greek "o" with
tonos, ASCII-space restoration for exotic space variants, and
removal of zero-width characters. A replacement of None
removes the codepoint."""
def append_provenance(path: Path, song_id: int, def __init__(self, lyrics_dir: Path,
source: str) -> None: provenance_csv: Path) -> None:
"""Append a provenance row for a fetched lyrics file. """Set up the fetch run.
The CSV file is created with the header row when missing. :param lyrics_dir: The lyrics cache directory.
:param provenance_csv: The lyrics provenance CSV file.
"""
self.__lyrics_dir: Path = lyrics_dir
"""The lyrics cache directory."""
self.__provenance_csv: Path = provenance_csv
"""The lyrics provenance CSV file."""
self.__fetcher: LyricsFetcher = LyricsFetcher()
"""The fetcher of the public lyrics APIs."""
:param path: The lyrics provenance CSV file. def run(self) -> LyricsFetchCounts:
:param song_id: The song ID. """Fetch the missing lyrics of every song in the store.
:param source: The source name of the fetched lyrics.
:return: None. Every song fetched or missed is reported on the standard
:raises OSError: When the file cannot be written. error as an observable side effect.
"""
is_new: bool = not path.exists() :return: The number of songs fetched and missed.
path.parent.mkdir(parents=True, exist_ok=True) :raises OSError: When a cache file or the provenance CSV
with open(path, "a", encoding="utf-8", cannot be written.
newline="") as file: :raises sqlalchemy.exc.SQLAlchemyError: On a database
writer: Any = csv.writer(file) error.
if is_new: """
writer.writerow(PROVENANCE_FIELDS) fetched: int = 0
writer.writerow([song_id, source, "api-fetch", missed: int = 0
datetime.date.today().isoformat(), ""]) session: Session = ds.get_db()
try:
song: Song
for song in session.scalars(
sa.select(Song).order_by(Song.id)):
if (self.__lyrics_dir
/ f"{song.id}.txt").exists():
continue
if self.__fetch_one(session, song):
fetched += 1
else:
missed += 1
finally:
session.close()
return LyricsFetchCounts(fetched=fetched, missed=missed)
def __fetch_one(self, session: Session, song: Song) -> bool:
"""Fetch and save the lyrics of one song.
The song is queried by its primary-role artist name; when
every API misses and the song's full artist credit
differs from that name, the same APIs are queried again
with the artist credit.
:param session: The database session.
:param song: The song to fetch.
:return: True when a lyrics text was fetched and saved,
False when every API missed on both queries.
:raises OSError: When the cache file or the provenance
CSV cannot be written.
"""
artist: str = self.__query_artist(session, song.id)
result: tuple[str, str] | None = self.__fetcher.fetch(
artist, song.title)
if result is None and song.artist_credit != artist:
result = self.__fetcher.fetch(
song.artist_credit, song.title)
if result is None:
print(f"song {song.id} \"{song.title}\": miss",
file=sys.stderr)
return False
lyrics: str
source: str
lyrics, source = result
self.__save_lyrics(song.id, lyrics)
self.__append_provenance(song.id, source)
print(f"song {song.id} \"{song.title}\": {source}",
file=sys.stderr)
return True
@staticmethod
def __query_artist(session: Session, song_id: int) -> str:
"""Find the artist name to query the APIs with.
:param session: The database session.
:param song_id: The song ID.
:return: The name of the primary-role artist with the
lowest position.
"""
name: str | None = session.scalar(
sa.select(Artist.name)
.join(SongArtist, SongArtist.artist_id == Artist.id)
.where(SongArtist.song_id == song_id,
SongArtist.role == Role.PRIMARY)
.order_by(SongArtist.position)
.limit(1))
assert name is not None
return name
def __save_lyrics(self, song_id: int, lyrics: str) -> None:
"""Write the lyrics of a song into the cache directory.
The cache directory is created when missing.
The lyrics text is normalized with
:meth:`normalize_lyrics` before being written.
:param song_id: The song ID.
:param lyrics: The lyrics text.
:return: None.
:raises OSError: When the file cannot be written.
"""
self.__lyrics_dir.mkdir(parents=True, exist_ok=True)
(self.__lyrics_dir / f"{song_id}.txt").write_text(
self.normalize_lyrics(lyrics), encoding="utf-8")
def __append_provenance(self, song_id: int,
source: str) -> None:
"""Append a provenance row for a fetched lyrics file.
The CSV file is created with the header row when
missing.
:param song_id: The song ID.
:param source: The source name of the fetched lyrics.
:return: None.
:raises OSError: When the file cannot be written.
"""
is_new: bool = not self.__provenance_csv.exists()
self.__provenance_csv.parent.mkdir(
parents=True, exist_ok=True)
with open(self.__provenance_csv, "a", encoding="utf-8",
newline="") as file:
writer: Any = csv.writer(file)
if is_new:
writer.writerow(self.__PROVENANCE_FIELDS)
writer.writerow(
[song_id, source, "api-fetch",
datetime.date.today().isoformat(), ""])
@classmethod
def normalize_lyrics(cls, text: str) -> str:
"""Restore or remove watermark and mojibake characters.
:param text: The lyrics text as fetched from an API.
:return: The text with the codepoints of the
normalization table replaced or removed; every other
character is unchanged.
"""
return text.translate(cls.__NORMALIZATION)
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
@@ -277,44 +372,15 @@ def main(argv: list[str] | None = None) -> int:
""" """
started: float = time.monotonic() started: float = time.monotonic()
args: argparse.Namespace = parse_args(argv) args: argparse.Namespace = parse_args(argv)
fetcher: LyricsFetcher = LyricsFetcher()
fetched: int = 0
missed: int = 0
session: Session = ds.get_db()
try: try:
song: Song counts: LyricsFetchCounts = LyricsFetchRunner(
for song in session.scalars( args.lyrics_dir, args.provenance_csv).run()
sa.select(Song).order_by(Song.id)):
if (args.lyrics_dir / f"{song.id}.txt").exists():
continue
artist: str = query_artist(session, song.id)
result: tuple[str, str] | None = fetcher.fetch(
artist, song.title)
if result is None and song.artist_credit != artist:
result = fetcher.fetch(
song.artist_credit, song.title)
if result is None:
missed += 1
print(f"song {song.id} \"{song.title}\": miss",
file=sys.stderr)
continue
lyrics: str
source: str
lyrics, source = result
save_lyrics(args.lyrics_dir, song.id, lyrics)
append_provenance(args.provenance_csv, song.id,
source)
fetched += 1
print(f"song {song.id} \"{song.title}\": {source}",
file=sys.stderr)
except (OSError, sa.exc.SQLAlchemyError) as error: except (OSError, sa.exc.SQLAlchemyError) as error:
print(f"error: {error}", file=sys.stderr) print(f"error: {error}", file=sys.stderr)
return 1 return 1
finally: attempted: int = counts.fetched + counts.missed
session.close()
attempted: int = fetched + missed
elapsed: str = format_duration(time.monotonic() - started) elapsed: str = format_duration(time.monotonic() - started)
print(f"Done. Fetched lyrics for {fetched}/{attempted}" print(f"Done. Fetched lyrics for {counts.fetched}/"
f" songs. {elapsed} elapsed.", f"{attempted} songs. {elapsed} elapsed.",
file=sys.stderr) file=sys.stderr)
return 0 return 0
+431 -330
View File
@@ -28,27 +28,13 @@ import time
from dataclasses import asdict, dataclass from dataclasses import asdict, dataclass
from datetime import datetime from datetime import datetime
from pathlib import Path from pathlib import Path
from typing import Any, Self from typing import Any, ClassVar, Self
import anthropic import anthropic
from ..config import get_settings from ..config import get_settings
from ..utils import format_duration from ..utils import format_duration
# claude-fable-5 accepts neither "temperature" nor "thinking";
# a model's entry holds exactly the extra request parameters it
# accepts.
MODELS: dict[str, dict[str, Any]] = {
"claude-sonnet-4-6": {
"temperature": 0.0,
"thinking": {"type": "disabled"},
},
"claude-fable-5": {},
}
DEFAULT_MODEL: str = "claude-sonnet-4-6"
SCRIPT_VERSION: str = "run_llm.py 3.1.0"
POLL_INTERVAL_SECONDS: float = 60.0
class InputFormatError(Exception): class InputFormatError(Exception):
"""An error in the JSONL input file.""" """An error in the JSONL input file."""
@@ -136,13 +122,23 @@ class BatchResult:
if x.type == "text") if x.type == "text")
return cls(id=entry.custom_id, text=text, return cls(id=entry.custom_id, text=text,
stop_reason=message.stop_reason, stop_reason=message.stop_reason,
usage=usage_to_dict(message.usage)) usage=cls.__usage_to_dict(message.usage))
case "errored": case "errored":
return cls(id=entry.custom_id, return cls(id=entry.custom_id,
error=result.error.error.type) error=result.error.error.type)
case other: case other:
return cls(id=entry.custom_id, error=str(other)) return cls(id=entry.custom_id, error=str(other))
@staticmethod
def __usage_to_dict(usage: Any) -> dict[str, Any]:
"""Convert a usage object to a plain dictionary.
:param usage: The usage object of a message.
:return: The usage as a dictionary, without null entries.
"""
return {k: v for k, v in usage.model_dump().items()
if v is not None}
def to_record(self) -> dict[str, Any]: def to_record(self) -> dict[str, Any]:
"""Return this result as an archive JSONL record. """Return this result as an archive JSONL record.
@@ -169,10 +165,409 @@ class BatchInfo:
is still processing.""" is still processing."""
@dataclass(frozen=True)
class ExecutionOutcome:
"""The outcome of submitting and awaiting one batch."""
batch: BatchInfo
"""The submitted batch's bookkeeping."""
results: Results
"""The batch's results, keyed by item ID."""
@dataclass(frozen=True)
class RunOutcome:
"""The outcome of one LLM definition file run."""
item_count: int
"""The number of loaded input items."""
dry_run: bool
"""Whether this was a dry run."""
dry_run_request: dict[str, Any] | None
"""The first item's preview request, for a dry run; None for
an actual run."""
failed: list[str]
"""The failed item IDs, in item order; always empty for a dry
run."""
class LLMRunner:
"""The orchestrator of one LLM definition file run."""
# claude-fable-5 accepts neither "temperature" nor "thinking";
# a model's entry holds exactly the extra request parameters
# it accepts.
MODELS: ClassVar[dict[str, dict[str, Any]]] = {
"claude-sonnet-4-6": {
"temperature": 0.0,
"thinking": {"type": "disabled"},
},
"claude-fable-5": {},
}
"""The supported model IDs and their extra request
parameters."""
DEFAULT_MODEL: ClassVar[str] = "claude-sonnet-4-6"
"""The default model ID."""
__SCRIPT_VERSION: ClassVar[str] = "run_llm.py 3.1.0"
"""The script version recorded into the archive metadata."""
__POLL_INTERVAL_SECONDS: ClassVar[float] = 60.0
"""The interval between batch status polls."""
def __init__(self, prompt: Path, input_path: Path,
archive_dir: Path, model: str, max_tokens: int,
dry_run: bool, replace: bool) -> None:
"""Set up the run of one LLM definition file.
:param prompt: The prompt definition file, used as the
system prompt.
:param input_path: The JSONL input file with "id" and
"content".
:param archive_dir: The destination archive directory.
:param model: The model ID, a key of :attr:`MODELS`.
:param max_tokens: The maximum output tokens per request.
:param dry_run: Whether to validate and archive without
calling the API.
:param replace: Whether to replace an already existing
archive directory.
"""
self.__prompt: Path = prompt
"""The prompt definition file."""
self.__input: Path = input_path
"""The JSONL input file."""
self.__archive_dir: Path = archive_dir
"""The destination archive directory."""
self.__model: str = model
"""The model ID."""
self.__max_tokens: int = max_tokens
"""The maximum output tokens per request."""
self.__dry_run: bool = dry_run
"""Whether to validate and archive without calling the
API."""
self.__replace: bool = replace
"""Whether to replace an already existing archive
directory."""
def run(self) -> RunOutcome:
"""Load the input, archive the prompt, and run the batch.
Always writes ``prompt.md`` and ``meta.json`` into the
archive directory. A dry run stops there, previewing the
first item's request; an actual run also submits the
batch, awaits it, and writes ``output.jsonl``.
:return: The outcome of the run.
:raises InputFormatError: When the input file is
malformed.
:raises OSError: When the input or prompt file cannot be
read, the archive directory already exists without
``replace``, or an output file cannot be written.
"""
items: list[InputItem] = self.__load_items()
prompt_text: str = self.__prompt.read_text(encoding="utf-8")
archive_dir: Path = self.__create_archive_dir()
(archive_dir / "prompt.md").write_bytes(
self.__prompt.read_bytes())
meta: dict[str, Any] = self.__build_meta(items)
meta_path: Path = archive_dir / "meta.json"
if self.__dry_run:
self.__write_json(meta_path, meta)
request: dict[str, Any] = self.__build_request(
items[0], prompt_text)
return RunOutcome(
item_count=len(items), dry_run=True,
dry_run_request=request, failed=[])
client: anthropic.Anthropic = anthropic.Anthropic(
api_key=get_settings().ANTHROPIC_API_KEY)
outcome: ExecutionOutcome = self.__execute_run(
client, items, prompt_text)
item_ids: list[str] = [x.id for x in items]
self.__write_jsonl(
archive_dir / "output.jsonl",
[outcome.results[x].to_record() for x in item_ids
if x in outcome.results])
meta["batch"] = outcome.batch
meta["usage"] = self.__sum_usage(outcome.results)
self.__write_meta(meta_path, meta)
failed: list[str] = self.__find_failures(
item_ids, outcome.results)
return RunOutcome(
item_count=len(items), dry_run=False,
dry_run_request=None, failed=failed)
def __load_items(self) -> list[InputItem]:
"""Load and validate the JSONL input items.
:return: The input items, in file order.
:raises InputFormatError: When a line is malformed, an ID
is duplicated, or the file contains no item.
:raises OSError: When the file cannot be read.
"""
items: list[InputItem] = []
seen: set[str] = set()
with open(self.__input, encoding="utf-8") as file:
for number, line in enumerate(file, start=1):
if line.strip() == "":
continue
data: Any
try:
data = json.loads(line)
except json.JSONDecodeError as error:
raise InputFormatError(
f"{self.__input}: line {number}: malformed"
f" JSON: {error}")
item: InputItem = InputItem.get_instance(
data, self.__input, number)
if item.id in seen:
raise InputFormatError(
f"{self.__input}: line {number}:"
f" duplicated ID \"{item.id}\"")
seen.add(item.id)
items.append(item)
if len(items) == 0:
raise InputFormatError(f"{self.__input}: no input items")
return items
def __create_archive_dir(self) -> Path:
"""Create the archive directory.
Only this directory is ever created or removed; no other
directory is ever touched.
:return: The created archive directory.
:raises FileExistsError: When the archive directory
already exists and ``replace`` is False.
"""
if self.__archive_dir.exists():
if not self.__replace:
raise FileExistsError(
f"{self.__archive_dir} already exists; pass"
" --replace to replace it")
shutil.rmtree(self.__archive_dir)
self.__archive_dir.mkdir(parents=True)
return self.__archive_dir
def __build_meta(self, items: list[InputItem]) -> dict[str, Any]:
"""Build the initial archive metadata.
:param items: The loaded input items.
:return: The metadata, "batch" and "usage" not yet filled
in for an actual run.
"""
return {
"script_version": self.__SCRIPT_VERSION,
"model": self.__model,
"temperature": self.MODELS[self.__model].get(
"temperature"),
"thinking": self.MODELS[self.__model].get("thinking"),
"max_tokens": self.__max_tokens,
"prompt_path": str(self.__prompt),
"prompt_sha256": self.__sha256_of(self.__prompt),
"input_path": str(self.__input),
"input_sha256": self.__sha256_of(self.__input),
"item_count": len(items),
"dry_run": self.__dry_run,
"started_at": self.__now_iso(),
"batch": None,
"usage": {},
}
def __build_request(self, item: InputItem, system_prompt: str) \
-> dict[str, Any]:
"""Build one Message Batches request for an input item.
:param item: The input item.
:param system_prompt: The system prompt text.
:return: The batch request with "custom_id" and "params".
"""
return {
"custom_id": item.id,
"params": {
"model": self.__model,
"max_tokens": self.__max_tokens,
**self.MODELS[self.__model],
"system": system_prompt,
"messages": [
{"role": "user", "content": item.content},
],
},
}
def __execute_run(
self, client: anthropic.Anthropic,
items: list[InputItem], system_prompt: str) \
-> ExecutionOutcome:
"""Submit the batch of this run and await its results.
:param client: The Anthropic client.
:param items: The input items.
:param system_prompt: The system prompt text.
:return: The submitted batch's bookkeeping and its
results.
"""
requests: list[dict[str, Any]] = [
self.__build_request(x, system_prompt) for x in items]
info: BatchInfo = BatchInfo(
batch_id=self.__submit_batch(client, requests),
submitted_at=self.__now_iso())
print(f"submitted batch {info.batch_id}", file=sys.stderr)
batches: dict[str, Any] = self.__poll_batches(
client, [info.batch_id])
info.ended_at = batches[info.batch_id].ended_at.isoformat()
results: Results = self.__collect_results(
client, info.batch_id)
return ExecutionOutcome(batch=info, results=results)
@staticmethod
def __submit_batch(client: anthropic.Anthropic,
requests: list[dict[str, Any]]) -> str:
"""Submit one message batch.
:param client: The Anthropic client.
:param requests: The batch requests.
:return: The batch ID.
"""
return client.messages.batches.create(requests=requests).id
@classmethod
def __poll_batches(cls, client: anthropic.Anthropic,
batch_ids: list[str]) -> dict[str, Any]:
"""Poll the batches until every one of them has ended.
Progress is printed to the standard error every poll.
:param client: The Anthropic client.
:param batch_ids: The batch IDs to poll.
:return: The final batch object of each batch, keyed by
batch ID.
"""
while True:
batches: dict[str, Any] = {
x: client.messages.batches.retrieve(x)
for x in batch_ids}
pending: list[str] = [
x for x in batch_ids
if batches[x].processing_status != "ended"]
for batch_id in batch_ids:
status: str = batches[batch_id].processing_status
print(f"batch {batch_id}: {status}", file=sys.stderr)
if len(pending) == 0:
return batches
time.sleep(cls.__POLL_INTERVAL_SECONDS)
@staticmethod
def __collect_results(client: anthropic.Anthropic,
batch_id: str) -> Results:
"""Collect the results of an ended batch.
:param client: The Anthropic client.
:param batch_id: The batch ID.
:return: The result records, keyed by custom ID.
"""
results: Results = {}
for entry in client.messages.batches.results(batch_id):
results[entry.custom_id] = BatchResult.get_instance(
entry)
return results
@staticmethod
def __find_failures(item_ids: list[str],
results: Results) -> list[str]:
"""Find the item IDs that failed in a result set.
An item failed when it is missing from the results or when
its record is a failure.
:param item_ids: The item IDs to check, in order.
:param results: The result records, keyed by item ID.
:return: The failed item IDs, in the given order.
"""
return [x for x in item_ids
if x not in results or results[x].is_failure]
@staticmethod
def __sum_usage(results: Results) -> dict[str, int]:
"""Sum the token usage of every succeeded result.
:param results: The result records, keyed by item ID.
:return: The summed integer usage fields.
"""
totals: dict[str, int] = {}
for result in results.values():
if result.usage is None:
continue
for key, value in result.usage.items():
if isinstance(value, int):
totals[key] = totals.get(key, 0) + value
return totals
@staticmethod
def __write_jsonl(path: Path,
records: list[dict[str, Any]]) -> None:
"""Write records to a file as JSON Lines.
:param path: The path of the file to write.
:param records: The records, one per line.
:return: None.
:raises OSError: When the file cannot be written.
"""
with open(path, "w", encoding="utf-8") as file:
for record in records:
file.write(
json.dumps(record, ensure_ascii=False) + "\n")
@staticmethod
def __write_json(path: Path, data: dict[str, Any]) -> None:
"""Write data to a file as pretty-printed JSON.
:param path: The path of the file to write.
:param data: The data to write.
:return: None.
:raises OSError: When the file cannot be written.
"""
path.write_text(
json.dumps(data, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8")
@classmethod
def __write_meta(cls, path: Path, meta: dict[str, Any]) -> None:
"""Write the metadata to the ``meta.json`` file.
The ``BatchInfo`` value under "batch" is written as a
plain JSON object.
:param path: The path of the ``meta.json`` file.
:param meta: The metadata to write.
:return: None.
:raises OSError: When the file cannot be written.
"""
cls.__write_json(
path, {**meta, "batch": asdict(meta["batch"])})
@staticmethod
def __sha256_of(path: Path) -> str:
"""Calculate the SHA-256 digest of a file.
:param path: The path of the file.
:return: The hexadecimal SHA-256 digest.
"""
with open(path, "rb") as file:
return hashlib.file_digest(file, "sha256").hexdigest()
@staticmethod
def __now_iso() -> str:
"""Return the current local time in ISO 8601 format.
:return: The current local time with the timezone offset.
"""
return datetime.now().astimezone().isoformat(
timespec="seconds")
def parse_args(argv: list[str] | None) -> argparse.Namespace: def parse_args(argv: list[str] | None) -> argparse.Namespace:
"""Parse the command-line arguments. """Parse the command-line arguments.
:param argv: The command-line arguments, or None for ``sys.argv``. :param argv: The command-line arguments, or None for
``sys.argv``.
:return: The parsed arguments. :return: The parsed arguments.
""" """
parser: argparse.ArgumentParser = argparse.ArgumentParser( parser: argparse.ArgumentParser = argparse.ArgumentParser(
@@ -180,7 +575,8 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
" and archive the result.") " and archive the result.")
parser.add_argument( parser.add_argument(
"prompt", type=Path, "prompt", type=Path,
help="the prompt definition file, used as the system prompt") help="the prompt definition file, used as the system"
" prompt")
parser.add_argument( parser.add_argument(
"input", type=Path, "input", type=Path,
help="the JSONL input file with \"id\" and \"content\"") help="the JSONL input file with \"id\" and \"content\"")
@@ -188,11 +584,13 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
"archive_dir", type=Path, "archive_dir", type=Path,
help="the destination archive directory") help="the destination archive directory")
parser.add_argument( parser.add_argument(
"--model", choices=sorted(MODELS), default=DEFAULT_MODEL, "--model", choices=sorted(LLMRunner.MODELS),
help=f"the model ID (default {DEFAULT_MODEL})") default=LLMRunner.DEFAULT_MODEL,
help=f"the model ID (default {LLMRunner.DEFAULT_MODEL})")
parser.add_argument( parser.add_argument(
"--max-tokens", type=int, default=2048, "--max-tokens", type=int, default=2048,
help="the maximum output tokens per request (default 2048)") help="the maximum output tokens per request (default"
" 2048)")
parser.add_argument( parser.add_argument(
"--dry-run", action="store_true", "--dry-run", action="store_true",
help="validate and archive without calling the API") help="validate and archive without calling the API")
@@ -202,327 +600,30 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
return parser.parse_args(argv) return parser.parse_args(argv)
def load_items(path: Path) -> list[InputItem]:
"""Load and validate the JSONL input items.
:param path: The path of the JSONL input file.
:return: The input items, in file order.
:raises InputFormatError: When a line is malformed, an ID is
duplicated, or the file contains no item.
:raises OSError: When the file cannot be read.
"""
items: list[InputItem] = []
seen: set[str] = set()
with open(path, encoding="utf-8") as file:
for number, line in enumerate(file, start=1):
if line.strip() == "":
continue
try:
data: Any = json.loads(line)
except json.JSONDecodeError as error:
raise InputFormatError(
f"{path}: line {number}: malformed JSON: {error}")
item: InputItem = InputItem.get_instance(
data, path, number)
if item.id in seen:
raise InputFormatError(
f"{path}: line {number}: duplicated ID"
f" \"{item.id}\"")
seen.add(item.id)
items.append(item)
if len(items) == 0:
raise InputFormatError(f"{path}: no input items")
return items
def build_request(item: InputItem, system_prompt: str,
max_tokens: int, model: str) -> dict[str, Any]:
"""Build one Message Batches request for an input item.
:param item: The input item.
:param system_prompt: The system prompt text.
:param max_tokens: The maximum output tokens.
:param model: The model ID, a key of ``MODELS``.
:return: The batch request with "custom_id" and "params".
"""
return {
"custom_id": item.id,
"params": {
"model": model,
"max_tokens": max_tokens,
**MODELS[model],
"system": system_prompt,
"messages": [
{"role": "user", "content": item.content},
],
},
}
def submit_batch(client: anthropic.Anthropic,
requests: list[dict[str, Any]]) -> str:
"""Submit one message batch.
:param client: The Anthropic client.
:param requests: The batch requests.
:return: The batch ID.
"""
return client.messages.batches.create(requests=requests).id
def poll_batches(client: anthropic.Anthropic,
batch_ids: list[str]) -> dict[str, Any]:
"""Poll the batches until every one of them has ended.
Progress is printed to the standard error every poll.
:param client: The Anthropic client.
:param batch_ids: The batch IDs to poll.
:return: The final batch object of each batch, keyed by batch ID.
"""
while True:
batches: dict[str, Any] = {
x: client.messages.batches.retrieve(x) for x in batch_ids}
pending: list[str] = [
x for x in batch_ids
if batches[x].processing_status != "ended"]
for batch_id in batch_ids:
status: str = batches[batch_id].processing_status
print(f"batch {batch_id}: {status}", file=sys.stderr)
if len(pending) == 0:
return batches
time.sleep(POLL_INTERVAL_SECONDS)
def usage_to_dict(usage: Any) -> dict[str, Any]:
"""Convert a usage object to a plain dictionary.
:param usage: The usage object of a message.
:return: The usage as a dictionary, without null entries.
"""
return {k: v for k, v in usage.model_dump().items()
if v is not None}
def sum_usage(results: Results) -> dict[str, int]:
"""Sum the token usage of every succeeded result.
:param results: The result records, keyed by item ID.
:return: The summed integer usage fields.
"""
totals: dict[str, int] = {}
for result in results.values():
if result.usage is None:
continue
for key, value in result.usage.items():
if isinstance(value, int):
totals[key] = totals.get(key, 0) + value
return totals
def collect_results(client: anthropic.Anthropic,
batch_id: str) -> Results:
"""Collect the results of an ended batch.
:param client: The Anthropic client.
:param batch_id: The batch ID.
:return: The result records, keyed by custom ID.
"""
results: Results = {}
for entry in client.messages.batches.results(batch_id):
results[entry.custom_id] = BatchResult.get_instance(entry)
return results
def find_failures(item_ids: list[str],
results: Results) -> list[str]:
"""Find the item IDs that failed in a result set.
An item failed when it is missing from the results or when its
record is a failure.
:param item_ids: The item IDs to check, in order.
:param results: The result records, keyed by item ID.
:return: The failed item IDs, in the given order.
"""
return [x for x in item_ids
if x not in results or results[x].is_failure]
def create_archive_dir(directory: Path, replace: bool) -> Path:
"""Create the archive directory.
Only this directory is ever created or removed; no other
directory is ever touched.
:param directory: The destination archive directory.
:param replace: Whether to remove an already existing archive
directory before creating it.
:return: The created archive directory.
:raises FileExistsError: When the archive directory already
exists and ``replace`` is False.
"""
if directory.exists():
if not replace:
raise FileExistsError(
f"{directory} already exists; pass --replace to"
" replace it")
shutil.rmtree(directory)
directory.mkdir(parents=True)
return directory
def write_jsonl(path: Path, records: list[dict[str, Any]]) -> None:
"""Write records to a file as JSON Lines.
:param path: The path of the file to write.
:param records: The records, one per line.
:return: None.
"""
with open(path, "w", encoding="utf-8") as file:
for record in records:
file.write(json.dumps(record, ensure_ascii=False) + "\n")
def write_json(path: Path, data: dict[str, Any]) -> None:
"""Write data to a file as pretty-printed JSON.
:param path: The path of the file to write.
:param data: The data to write.
:return: None.
"""
path.write_text(
json.dumps(data, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8")
def write_meta(path: Path, meta: dict[str, Any]) -> None:
"""Write the metadata to the ``meta.json`` file.
The ``BatchInfo`` value under ``batch`` is written as a plain
JSON object.
:param path: The path of the ``meta.json`` file.
:param meta: The metadata to write.
:return: None.
"""
write_json(path, {**meta, "batch": asdict(meta["batch"])})
def sha256_of(path: Path) -> str:
"""Calculate the SHA-256 digest of a file.
:param path: The path of the file.
:return: The hexadecimal SHA-256 digest.
"""
with open(path, "rb") as file:
return hashlib.file_digest(file, "sha256").hexdigest()
def now_iso() -> str:
"""Return the current local time in ISO 8601 format.
:return: The current local time with the timezone offset.
"""
return datetime.now().astimezone().isoformat(timespec="seconds")
def execute_run(
client: anthropic.Anthropic, items: list[InputItem],
system_prompt: str, max_tokens: int, model: str,
meta: dict[str, Any],
) -> Results:
"""Submit the batch of this run and await its results.
The batch ID and timestamps are recorded into the metadata as an
observable side effect.
:param client: The Anthropic client.
:param items: The input items.
:param system_prompt: The system prompt text.
:param max_tokens: The maximum output tokens per request.
:param model: The model ID, a key of ``MODELS``.
:param meta: The metadata to record the batch bookkeeping into.
:return: The results of this run, keyed by item ID.
"""
requests: list[dict[str, Any]] = [
build_request(x, system_prompt, max_tokens, model)
for x in items]
info: BatchInfo = BatchInfo(
batch_id=submit_batch(client, requests),
submitted_at=now_iso())
meta["batch"] = info
print(f"submitted batch {info.batch_id}", file=sys.stderr)
batches: dict[str, Any] = poll_batches(client, [info.batch_id])
info.ended_at = batches[info.batch_id].ended_at.isoformat()
return collect_results(client, info.batch_id)
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
"""Run one LLM definition file against one input and archive it. """Run one LLM definition file against one input and archive it.
:param argv: The command-line arguments, or None for ``sys.argv``. :param argv: The command-line arguments, or None for
``sys.argv``.
:return: The exit status: 0 on success, non-zero on failure. :return: The exit status: 0 on success, non-zero on failure.
""" """
started: float = time.monotonic() started: float = time.monotonic()
args: argparse.Namespace = parse_args(argv) args: argparse.Namespace = parse_args(argv)
try: try:
items: list[InputItem] = load_items(args.input) outcome: RunOutcome = LLMRunner(
prompt_text: str = args.prompt.read_text(encoding="utf-8") args.prompt, args.input, args.archive_dir, args.model,
except (OSError, InputFormatError) as error: args.max_tokens, args.dry_run, args.replace).run()
except (InputFormatError, OSError) as error:
print(f"error: {error}", file=sys.stderr) print(f"error: {error}", file=sys.stderr)
return 1 return 1
try: if not outcome.dry_run and len(outcome.failed) > 0:
archive_dir: Path = create_archive_dir( print(f"error: failed items: {', '.join(outcome.failed)}",
args.archive_dir, args.replace)
except FileExistsError as error:
print(f"error: {error}", file=sys.stderr)
return 1
meta_path: Path = archive_dir / "meta.json"
(archive_dir / "prompt.md").write_bytes(args.prompt.read_bytes())
meta: dict[str, Any] = {
"script_version": SCRIPT_VERSION,
"model": args.model,
"temperature": MODELS[args.model].get("temperature"),
"thinking": MODELS[args.model].get("thinking"),
"max_tokens": args.max_tokens,
"prompt_path": str(args.prompt),
"prompt_sha256": sha256_of(args.prompt),
"input_path": str(args.input),
"input_sha256": sha256_of(args.input),
"item_count": len(items),
"dry_run": args.dry_run,
"started_at": now_iso(),
"batch": None,
"usage": {},
}
if args.dry_run:
write_json(meta_path, meta)
print(json.dumps(
build_request(items[0], prompt_text, args.max_tokens,
args.model),
ensure_ascii=False, indent=2))
elapsed: str = format_duration(time.monotonic() - started)
print(f"Done. {len(items)} jobs finished."
f" {elapsed} elapsed.", file=sys.stderr)
return 0
client: anthropic.Anthropic = anthropic.Anthropic(
api_key=get_settings().ANTHROPIC_API_KEY)
results: Results = execute_run(
client, items, prompt_text, args.max_tokens, args.model,
meta)
item_ids: list[str] = [x.id for x in items]
write_jsonl(
archive_dir / "output.jsonl",
[results[x].to_record() for x in item_ids if x in results])
meta["usage"] = sum_usage(results)
write_meta(meta_path, meta)
failed: list[str] = find_failures(item_ids, results)
if len(failed) > 0:
print(f"error: failed items: {', '.join(failed)}",
file=sys.stderr) file=sys.stderr)
return 1 return 1
if outcome.dry_run:
print(json.dumps(
outcome.dry_run_request, ensure_ascii=False, indent=2))
elapsed: str = format_duration(time.monotonic() - started) elapsed: str = format_duration(time.monotonic() - started)
print(f"Done. {len(items)} jobs finished." print(f"Done. {outcome.item_count} jobs finished."
f" {elapsed} elapsed.", file=sys.stderr) f" {elapsed} elapsed.", file=sys.stderr)
return 0 return 0

Some files were not shown because too many files have changed in this diff Show More