114 Commits
Author SHA1 Message Date
imacat 0ae0a90750 License the project under CC BY 4.0 2026-08-17 22:48:57 +08:00
imacatandClaude Fable 5 9a48446129 Add the reliability measurement record to the docs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:44:14 +08:00
imacatandClaude Fable 5 20fa83576c Update the project structure document to the completed pipeline
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:44:14 +08:00
imacat 5990c15dde Finalize camera-ready version of conference paper 2026-08-17 22:44:11 +08:00
imacatandClaude Fable 5 46835bfb77 Add the wide pattern matrix of the women-power songs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:43 +08:00
imacatandClaude Fable 5 76a90cb8ae Rewrite the research plan to the executed pipeline
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:43 +08:00
imacatandClaude Fable 5 e5e323f938 Import the settled patterns and annotations into the working store
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:42 +08:00
imacatandClaude Fable 5 60a1803a06 Add the tally-annotations subcommand settling the pattern matrix
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:42 +08:00
imacatandClaude Fable 5 71fd8ce9c6 Archive the step 5-4 pattern-annotation runs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:42 +08:00
imacatandClaude Fable 5 5731782fe6 Archive the step 5-3 grouped synthesis runs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:42 +08:00
imacatandClaude Fable 5 9756c07cad Add the step 5 grouped synthesis and pattern-annotation design
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:42 +08:00
imacatandClaude Fable 5 e9c29e5867 Correct three performer genders via build-db --gender-corrections
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:42 +08:00
imacatandClaude Fable 5 171d2f81fe Archive the step 5 reading, consolidation, and synthesis runs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:41 +08:00
imacatandClaude Fable 5 df363e14f0 Add the step 5 qualitative deep-reading design and definition files
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:41 +08:00
imacatandClaude Fable 5 d591931101 Record the Fable 5 auxiliary deep-reading experiments in the decision log
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:41 +08:00
imacatandClaude Fable 5 03ee51912d Import the settled code groups into the working store
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:41 +08:00
imacatandClaude Fable 5 87ad31568a Add the tally-groups subcommand settling results/groups.csv
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:41 +08:00
imacatandClaude Fable 5 ed54a1248f Replace the quote newline escape with the lyric line-break convention
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:40 +08:00
imacatandClaude Fable 5 963504e9c3 Add step 4 semantic code grouping by majority vote on claude-fable-5
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:40 +08:00
imacatandClaude Fable 5 64dd9b0cc3 Add a model option to run-llm for the claude-fable-5 model
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:40 +08:00
imacat 41d2822063 Add conference paper first draft. 2026-08-17 22:38:40 +08:00
imacatandClaude Fable 5 70578b5d77 Record the blind semantic code grouping in the decision log
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:40 +08:00
imacatandClaude Fable 5 ff2a3a217f Record the full k-value comparison table in the decision log
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:39 +08:00
imacatandClaude Opus 5 2c15c0bb8b Derive the performing group genders from their Wikidata members
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:39 +08:00
imacatandClaude Opus 5 5d10383dcb Derive the song performer gender and identify Pinkfong as Hope Segoine
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:39 +08:00
imacatandClaude Opus 5 99f5c1c260 Drop the redundant HTTP error class from the retry catch
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:39 +08:00
imacatandClaude Opus 5 d370ac51dc Pin the parenthesized members pattern to a single split point
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 c7b630c632 Make the module-level helpers private instead of protected
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 5194d4246b Extract the protected name restoration out of the credit parser
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 36b3456414 Classify the transient request errors in one place
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 511abd0a55 Extract the record content builder out of the export line builder
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 047f322bf3 Stop binding the unread role in the parsed credit check
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:38 +08:00
imacatandClaude Opus 5 2d8a34ea9e Remove the unused skipped counter of the artist fetcher
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:37 +08:00
imacatandClaude Opus 5 1c78f4a918 Capture the command output in the parent-directory export test
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:37 +08:00
imacatandClaude Opus 5 71f9c713f0 Merge the two identical chart-position formatters of the CSV exporter
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:37 +08:00
imacatandClaude Opus 5 95b843f76e Unify the export-llm-input and run-llm summary lines with build-db
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:37 +08:00
imacatandClaude Opus 5 76d5df3a87 Close the HTTP error fixtures when the test ends
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:37 +08:00
imacatandClaude Opus 5 a21e407e2f Run the tests on an in-memory store and dispose the engine afterwards
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:36 +08:00
imacatandClaude Opus 5 7a5176b8b3 Trim the build-db summary to songs, artists, codings and elapsed time
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:36 +08:00
imacatandClaude Opus 5 045086f9ca Import the settled coding into the working store
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:36 +08:00
imacatandClaude Opus 5 87352a2530 Settle the coding of the 883 songs against the 101 keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:36 +08:00
imacatandClaude Opus 5 bfb5208133 Repair the coding output with a reviewed correction table
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:35 +08:00
imacatandClaude Opus 5 5018f12cd1 Record how far the coding output departs from its stated contract
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:35 +08:00
imacatandClaude Opus 5 10fccca108 Add the tally-codings subcommand for the majority vote
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:35 +08:00
imacatandClaude Opus 5 4dd82867fa Archive the third coding run
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:34 +08:00
imacatandClaude Opus 5 8c06406b3a Settle the coding by a majority of three runs instead of an arbitration
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:34 +08:00
imacatandClaude Opus 5 3a2419f43a Import each command module only when its subcommand runs
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:34 +08:00
imacatandClaude Opus 5 8299906b51 Restructure cluster-keywords into builders and result records
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:34 +08:00
imacatandClaude Opus 5 adbd9c1a24 Archive the arbitration pass over the coding disagreements
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:34 +08:00
imacatandClaude Opus 5 94971eb07d Quote one line per kept keyword in the arbitration
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:33 +08:00
imacatandClaude Opus 5 85db4dac12 Build the arbitration input with a per-ID extras merge
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:33 +08:00
imacatandClaude Opus 5 c2ffa9d247 Add the compare-codings subcommand for the arbitration handoff
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:33 +08:00
imacatandClaude Opus 5 c997cc8d74 Drop the source-provenance artifact from cluster-keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:33 +08:00
imacatandClaude Opus 5 c1e099901d Rename the arbitration definition file to 03-02-arbitration
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:32 +08:00
imacatandClaude Opus 5 787ca10f29 Archive the second coding pass against the 101 keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:32 +08:00
imacatandClaude Opus 5 1db46039fe Archive the first coding pass against the 101 keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:32 +08:00
imacatandClaude Opus 5 d4c313734a Keep the medoid group names after testing LLM naming
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:32 +08:00
imacatandClaude Opus 5 fbb80180af Report the clustering result in a sentence
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:32 +08:00
imacatandClaude Opus 5 def2affa95 Rebuild the coding keywords with 100 groups
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:31 +08:00
imacatandClaude Opus 5 c5ad09524b Record the clustering invocation in a meta file
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:31 +08:00
imacatandClaude Opus 5 5b376e001d Raise the clustering to 100 groups
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:31 +08:00
imacatandClaude Opus 5 59f62cd4d4 Quote one line per assigned keyword
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:31 +08:00
imacatandClaude Opus 5 864735352a Take the extra coding keywords from the command line
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:30 +08:00
imacatandClaude Opus 5 1c4d5c5ed7 Merge the keyword pooling into cluster-keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:30 +08:00
imacatandClaude Opus 5 066ae15800 Separate the clustered keyword list from the keywords to merge
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:30 +08:00
imacatandClaude Opus 5 bf2b4970fd Pass the coding keywords as input instead of baking them into the prompt
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:29 +08:00
imacatandClaude Opus 5 896ee09b21 Archive the clustered coding vocabulary
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:29 +08:00
imacatandClaude Opus 5 a14e2a878a Add the cluster-keywords subcommand for the coding vocabulary
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:29 +08:00
imacatandClaude Opus 5 b0a79af1a8 Archive the pooled keywords of the two tagging runs
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:29 +08:00
imacatandClaude Opus 5 0b995e5a51 Add the pool-keywords subcommand for the tagging runs' keywords
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:28 +08:00
imacatandClaude Opus 5 4eb278164b Rebuild the vocabulary step on embeddings and drop the screen track
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:38:28 +08:00
imacatandClaude Fable 5 d9014cf1f3 Report the elapsed time when run-llm finishes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:28 +08:00
imacatandClaude Fable 5 4a0a4ceead Archive the tag run2 execution
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:27 +08:00
imacatandClaude Fable 5 2e5ecaaad0 Archive the tag run1 execution and add the run-cost ledger
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:27 +08:00
imacatandClaude Fable 5 e858f7a38f Require valid JSON with escaped quotes in the definition files
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:27 +08:00
imacatandClaude Fable 5 5050b16812 Forbid Markdown code fences in the definition-file outputs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:27 +08:00
imacatandClaude Fable 5 93b8afc547 Record the song-775 refusal ablation and rule it unanalyzable
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:27 +08:00
imacatandClaude Fable 5 cba73a6ace Add watermark normalization to fetch-lyrics
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 bc98fdf171 Changed the natural-coding definition files to single-pass convergence
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 9ef84cfe54 Add the natural-coding definition files and the convergence algorithms
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 09a8381519 Settle women-power as the a-priori screen label term
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 cf80a6f51e Reduce run-llm to a pure batch executor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 0b77310d86 Record the natural-coding pipeline design in the project documents
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 5092e04d34 Gather the CLI command modules into a commands sub-package
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 376916fb8c Add the export-llm-input subcommand for the lyrics-only LLM input
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 ce8a217e94 Regenerate the artists report with the Wikidata snapshot applied
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:26 +08:00
imacatandClaude Fable 5 14cc80e662 Rename the Markdown documents to dash-connected names
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:38:20 +08:00
imacatandClaude Fable 5 21c5ae5481 Drop the engine preparation from build-db
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:36 +08:00
imacatandClaude Fable 5 383670ab2b Move the invariant checks into the importers and drop find_violations
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:36 +08:00
imacatandClaude Fable 5 c59a023095 Package the capture-layer application into a CaptureImporter class
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:36 +08:00
imacatandClaude Fable 5 10dc4cb693 Repackage the build-db import steps into job classes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:36 +08:00
imacatandClaude Fable 5 449675e7a2 Capture the song lyrics from the pilot study and the public APIs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:36 +08:00
imacatandClaude Fable 5 a7806c0c9d Align the fetch-lyrics summary with fetch-artists via shared utils
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:34:26 +08:00
imacatandClaude Fable 5 105664e93b Remove the missing-lyrics report from fetch-lyrics
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:15:49 +08:00
imacatandClaude Fable 5 1220c21807 Fall back to the original artist credit when fetching lyrics
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:15:49 +08:00
imacatandClaude Fable 5 86914de850 Remove the retired artist overrides layer and the mixed artist type
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:15:49 +08:00
imacat 8463cfc906 Keep the artist snapshot sorted by name 2026-08-04 15:15:49 +08:00
imacat ef4112c61e Show resolved count and elapsed time in the fetch-artists summary 2026-08-04 15:15:49 +08:00
imacat 1eaebeb5fb Resolve artists on Wikidata via exact-match SPARQL with song corroboration 2026-08-04 15:15:49 +08:00
imacat 6fc4b7db17 Restore official artist name spellings in the canonical table 2026-08-04 15:15:49 +08:00
imacat c94b6f2589 Record the complete-Wikidata-then-capture workflow decision 2026-08-04 15:15:49 +08:00
imacat b3ac12c88d Export human-readable songs and artists reports to the derived data layer 2026-08-04 15:15:49 +08:00
imacat 5b2c192827 Split artist credits with documented exceptions and normalize artist names 2026-08-04 15:15:49 +08:00
imacat 991ab5d93a Merge same-song chart rows via a canonical artist credit table 2026-08-04 15:15:49 +08:00
imacat 78cf5e6ce0 Store the source chart CSV with its original CRLF line endings 2026-08-04 15:15:49 +08:00
imacat aa78a54d12 Align the source chart CSV with the archived Billboard year-end charts 2026-08-04 15:15:49 +08:00
imacatandClaude Fable 5 3d472d5c15 Reorganize data/ into source, captures, and manual layers
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:15:49 +08:00
imacatandClaude Fable 5 2f048dee36 Make every subcommand input an explicit CLI argument or option
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:13:31 +08:00
imacatandClaude Fable 5 a3062e848c Refactor internal data passing to dataclasses and enums, and remove unneeded defensive code
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:13:31 +08:00
imacatandClaude Fable 5 1b2d46cb42 Add the data layer with the build-db, fetch-lyrics, and fetch-artists subcommands
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 15:12:22 +08:00
imacatandClaude Fable 5 f2ccca60b5 Use the unified pydantic-settings configuration in run_llm
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 14:03:37 +08:00
imacatandClaude Fable 5 db7b5f6679 Record the data storage architecture and phase-1 plan in the project documents
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 14:03:37 +08:00
imacatandClaude Fable 5 98b665537f Add the tools sub-project with the run_llm batch runner
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 14:03:28 +08:00
imacatandClaude Fable 5 9a089f6c6b Initialize project structure, workflow rules, and research plan
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-30 18:51:10 +08:00
109 changed files with 3303 additions and 3283 deletions
+68
View File
@@ -0,0 +1,68 @@
# Project Conventions
## Analysis pipeline
- LLM analysis runs via Python scripts calling the Anthropic
Messages API, Batch API where possible. Steps 1 and 3 run
on `claude-sonnet-4-6` with `temperature=0` and thinking
disabled; steps 4 and 5 run on `claude-fable-5`, which
accepts neither parameter -- step 4 absorbs its sampling
variance by the majority vote, step 5 by consolidating the
three readings.
- Prompt definition files live in
`prompts/<step>-<substep>-<task>.md` (e.g. 01-tag.md; no
version suffix -- versions live in git history) and are
passed verbatim as the system prompt. The number names a
step of the research procedure, not the file: the
deterministic vocabulary step (step 2) has no definition
file yet holds its own number. Zero padding is for
sorting only -- prose says "step 1", "step 3".
- Itemwise LLM judgments (per-song coding in step 3,
per-keyword group selection in step 4) run the same
definition file three times, independently, over the same
input; a deterministic tally then assigns an item (a
(song, keyword) or (group, keyword) pair) when at least
two of the three runs assign it ("3 runs + majority
vote"). Free-generation steps run
twice and both outputs are pooled. The step-5 qualitative
readings are neither: three independent readings per song,
consolidated per song and synthesized across songs by
their own definition files -- a qualitative protocol, not
a vote (see docs/methodology.md). The vocabulary is
built by a deterministic subcommand (embedding +
clustering), not by an LLM. If a validation outcome is
unexpected, revise the definition file and repeat that
cycle; never patch results by hand.
- Each run of a step is archived self-contained under the
destination directory given explicitly on the `run-llm`
command line (by convention `runs/<step>/run<N>/`):
prompt snapshot, raw output, and `meta.json` (model ID,
parameters, timestamps, batch ID). The runs of a step are
that many separate invocations of `run-llm`. Replacing
an existing run archive requires an explicit flag;
superseded runs live in git history. Deterministic steps
archive under `runs/<step>/` with no `run<N>` level.
- Token usage and cost of every `run-llm` execution are
recorded in `docs/run-costs.md` in the same commit as the
run archive.
- Scripts read the API key from the `ANTHROPIC_API_KEY`
environment variable (`.env`, gitignored).
## Data rules
- `data/source/` holds the immutable hand-placed raw files;
`data/captures/` is written only by the fetch commands and the
private import script; `data/manual/` is written only by the
user's own hand; `data/derived/` is written only by the
`build-db` subcommand.
- Full lyrics are copyrighted: they stay in `data/captures/lyrics/`
(gitignored) and must never be committed or reproduced in
full anywhere in the repo.
## Documents
- `results/` holds the final tallied tables (what the paper
cites); `runs/` holds raw audit records. The paper cites
`results/` only.
- Any change to a definition file, the codebook, or the plan is
recorded in `docs/decision-log.md` with date and reason.
+29 -22
View File
@@ -1,26 +1,39 @@
# 流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End Hot 1002016-2025)為例的內容分析 # 流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End Hot 10020182025)為例的內容分析
這是研究論文《流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End Hot 1002016-2025)為例的內容分析》的專案資料,包括論文本身、文件、紀錄、資料、工具程式、LLM提示詞,等等。 流行音樂中「女性力量」語彙的挪用與污染——以 Billboard Year-End
Hot 10020162025)為例的內容分析。
## 論文和摘要 台灣女性學學會 2026 年會論文之研究資料與分析程式。
研究論文全文和摘要,請參閱 paper/ 資料夾。 ## 目錄結構
## 文件說明 `docs/project-structure.md`。研究步驟規劃見
`docs/research-plan.md`;方法細節見 `docs/methodology.md`
人工編碼手冊見 `docs/codebook.md`;決策日誌見
`docs/decision-log.md`
研究方法、記錄等文件,請參閱 docs/ 資料夾。 ## 重現方式
## LLM提示 1. 準備 Python 3.14+ 環境,安裝分析管線套件:
`pip install -e tools/`
2.`tools/.env.example` 建立 `tools/.env`,寫入
Anthropic API 金鑰。
3.`docs/research-plan.md` 的階段順序,於 `tools/` 目錄下
執行子命令,必要輸入以位置引數、選擇性輸入以選項給定(如
`pop-fem-audit-tools build-db
../data/source/yearend_hot100_2016_2025.csv
../data/derived
--lyrics-dir ../data/captures/lyrics
--wikidata-csv ../data/captures/artists-wikidata.csv`)。
LLM 步驟使用 `claude-sonnet-4-6`、temperature=0、
thinking 關閉;輸出可逐項比對的步驟獨立執行三次,由
程式取三票多數決定案(「三次執行+多數決」協定),
自由生成步驟兩次執行進池。
4. 每次執行的完整紀錄(定義檔快照、原始輸出、參數)存於
`runs/`,可逐筆稽核。論文引用的最終資料表在 `results/`
LLM提示請參閱 prompts/ 資料夾。 注意:歌詞受版權保護,`data/captures/lyrics/` 不隨 repo 發布,須自行
`tools/` 中的抓取程式重建。
## 輔助工具程式
輔助工具程式請參閱 tools/ 資料夾。
## 研究資料
研究資料請參閱 data/ 資料夾。
## 授權 ## 授權
@@ -31,9 +44,3 @@ LLM提示請參閱 prompts/ 資料夾。
排除物件: 排除物件:
1. 本專案引用之第三方歌詞,相應權利歸各該權利人所有; 1. 本專案引用之第三方歌詞,相應權利歸各該權利人所有;
2. tools/ 資料夾內之工具程式,依該工具程式授權條款提供。 2. tools/ 資料夾內之工具程式,依該工具程式授權條款提供。
## 作者
楊士青<br>
imacat@mail.imacat.idv.tw<br>
2026/8/18<br>
-69
View File
@@ -1,69 +0,0 @@
# 常設工作規範
本專案的常設工作規範。原專案 `CLAUDE.md`;移置於此,
使 Claude Code subagent 不將其繼承入 context(盲判型
agent 不得見之)。凡於本專案工作的主會話,動手管線、
資料或文件之前,先讀本檔。
## 分析管線
管線已跑畢;本規範適用於任何重跑或擴充。
- LLM 分析以 Python 腳本呼叫 Anthropic Messages API
執行,能用 Batch API 處即用之。步驟 1 與步驟 3 以
`claude-sonnet-4-6` 執行,`temperature=0`、thinking
停用;步驟 4 與步驟 5 以 `claude-fable-5` 執行,該
模型兩個參數皆不受理——步驟 4 的取樣變異由多數決
吸收,步驟 5 由整合三份閱讀吸收。
- 定義檔置於 `prompts/<步><次步>-<task>.md`(如
1-tag.md、5a-read.md;次步以字母標示,與論文正文的
步驟編號一致;不帶版本號——版本即 git 歷史),逐字
作為 system prompt。
- 逐項的 LLM 判斷(步驟 3 的逐首編碼、步驟 4 的逐碼
入群判斷)以同一份定義檔、同一份輸入獨立執行三次;
再由確定性計票將三次執行中至少兩次指派的項目(一個
(歌,關鍵字)或(群,關鍵字)配對)收入定案
(「三次執行+多數決」)。自由生成步驟執行兩次,
兩份輸出進池。步驟 5 的質性閱讀兩者皆非:逐首三次
獨立閱讀,由各自的定義檔逐首整合、跨首統整——質性
協定,不是投票(見 docs/methodology.md)。詞彙表由
確定性子命令建構(嵌入+分群),不經 LLM。驗證結果
不符預期時,修訂定義檔並重複該循環;絕不手改結果。
- 一步的每次執行皆自我完備歸檔於 `run-llm` 命令列上
明示指定的目的目錄下(慣例為
`data/runs/<步驟>/run<N>/`):定義檔快照、原始輸出與
`meta.json`model ID、參數、時間戳、batch ID)。
一步的 N 次執行即 N 次各自的 `run-llm` 呼叫。覆蓋
既有執行歸檔須明示旗標;被取代的執行留在 git 歷史。
確定性步驟歸檔於 `data/runs/<步驟>/`,不分 `run<N>` 層。
- 每次 `run-llm` 執行的 token 用量與費用記入
`docs/run-costs.md`,與執行歸檔同一 commit。
- 腳本自環境變數 `ANTHROPIC_API_KEY` 讀取 API key
`.env`gitignored)。
## 資料規則
- `data/source/` 存手放後不動的原始檔;
`data/captures/` 只由 fetch 命令與私人匯入腳本
寫入;`data/manual/` 只由研究者親手寫入;
`data/derived/` 只由 `build-db` 子命令寫入;
`data/runs/` 存 LLM 執行的原始歸檔,只由執行程序
寫入;`data/results/` 存論文引用的定案表,只由
計票程序寫入。
- 歌詞全文有版權:一律置於 `data/captures/lyrics/`
gitignored),絕不 commit,亦絕不於 repo 任何處
全文重現。
## 文件
- `data/results/` 存計票後的定案表(論文所引);
`data/runs/` 存原始稽核紀錄。論文只引 `data/results/`
- **Commit 判準**:凡能由「committed 的輸入+committed
的程式」決定性再生者不 commit;凡不能者一律以文字
格式 commit,格式跟著上文「資料規則」一節所定的
層次走。例外:`data/results/` 定案表與
`data/derived/` 人讀報表雖可再生仍 commit——理由是
引用穩定性、審稿人零門檻、撰稿期數字變動可 diff;
兩者皆與工作儲存同一動作產出,稽核鏈無中間空缺。
- 凡定義檔或研究規劃之更動,皆記入
`docs/decision-log.md`,註明日期與原因。
+18 -62
View File
@@ -99,8 +99,8 @@
## 2026-08-02 ## 2026-08-02
- **`import-lyrics` 不設為子命令,pilot 歌詞改以私人腳本 - **`import-lyrics` 不設為子命令,pilot 歌詞改以 `excludes/`
匯入**。理由:pilot 捕捉檔不隨論文發布,子命令 私人腳本匯入**。理由:pilot 捕捉檔不隨論文發布,子命令
形式會在發布的 CLI 裡留下讀者無法執行的死命令——要交待的 形式會在發布的 CLI 裡留下讀者無法執行的死命令——要交待的
是「沿用 pilot 捕捉」的事實(記於 lyrics-provenance.csv 與 是「沿用 pilot 捕捉」的事實(記於 lyrics-provenance.csv 與
論文方法節),不是工具本身;工具移出專案,發布管線即 論文方法節),不是工具本身;工具移出專案,發布管線即
@@ -178,8 +178,8 @@
來源公開可稽核,快照乾淨、overrides 縮小;論文方法節揭露 來源公開可稽核,快照乾淨、overrides 縮小;論文方法節揭露
「捕捉前經研究者查證補完」。性別以公開自我認同為準, 「捕捉前經研究者查證補完」。性別以公開自我認同為準,
推測不確定且無佐證者列疑慮清單;查證屬資料策展而非 推測不確定且無佐證者列疑慮清單;查證屬資料策展而非
分析,以 Claude Code 輔助、不走分析 APIQS 批次 分析,以 Claude Code 輔助、不走分析 APIQS 批次
私人工作檔留存,不隨論文發布。 `excludes/`私人工作檔,不隨論文發布
## 2026-08-04 ## 2026-08-04
@@ -227,7 +227,7 @@
- **Markdown 檔名一律以 dash 連接**decision-log.md、 - **Markdown 檔名一律以 dash 連接**decision-log.md、
research-plan.md、project-structure.md 等;與 data/ 層 research-plan.md、project-structure.md 等;與 data/ 層
CSV 檔名慣例一致),定義檔命名慣例同步改為 CSV 檔名慣例一致),定義檔命名慣例同步改為
`prompts/<task>-v<N>.md`(如 screen-v1.md),版本庫外的 `prompts/<task>-v<N>.md`(如 screen-v1.md),`excludes/`
私人工作檔一併改名;`conference_abstract.md` 改名並搬入 私人工作檔一併改名;`conference_abstract.md` 改名並搬入
`paper/`——它是本次年會實際送出的摘要,與全文同屬投稿 `paper/`——它是本次年會實際送出的摘要,與全文同屬投稿
血脈,不是 `docs/` 的內部工作文件。全 repo 指涉同步更新。 血脈,不是 `docs/` 的內部工作文件。全 repo 指涉同步更新。
@@ -583,8 +583,8 @@
Billboard 署名 Pinkfong 為品牌(Wikidata Q55735607,型態 Billboard 署名 Pinkfong 為品牌(Wikidata Q55735607,型態
brand),非演唱者;其唯一上榜曲 Baby Shark2019#75 brand),非演唱者;其唯一上榜曲 Baby Shark2019#75
的實際演唱者為 Hope SegoineKTVB 2019-03-06 報導、 的實際演唱者為 Hope SegoineKTVB 2019-03-06 報導、
Songfacts、經紀簡介,出處詳私人留存的 Songfacts、經紀簡介,出處詳
QuickStatements 草稿)。藉既有的 `excludes/quickstatements/hope-segoine.md`)。藉既有的
署名正規化機制(`ArtistImporter.CANONICAL_ARTIST_NAMES` 署名正規化機制(`ArtistImporter.CANONICAL_ARTIST_NAMES`
指認:歌曲的署名字串維持榜單所印的 "Pinkfong",解析出 指認:歌曲的署名字串維持榜單所印的 "Pinkfong",解析出
的演出者實體為 Hope Segoine。品牌無性別可言,人有; 的演出者實體為 Hope Segoine。品牌無性別可言,人有;
@@ -637,9 +637,10 @@
sentence-transformers/all-mpnet-base-v2, revision sentence-transformers/all-mpnet-base-v2, revision
e8c3b32e)對 `runs/02-cluster/source-keywords.txt` e8c3b32e)對 `runs/02-cluster/source-keywords.txt`
5,999 個關鍵字重新計算。組內一致性定義為組內成員兩兩 5,999 個關鍵字重新計算。組內一致性定義為組內成員兩兩
餘弦相似度之平均。k=50 直接取自私人留存的分群結果 餘弦相似度之平均。k=50 直接取自留存的分群結果
k=30 以同參數重跑(確定性演算法),重算所得最大組恰為 `excludes/k50/02-cluster/groups.csv`);k=30 以同參數
491 詞,與當時粗算紀錄吻合,佐證重算與原實驗一致。數表: 重跑(確定性演算法),重算所得最大組恰為 491 詞,與
當時粗算紀錄吻合,佐證重算與原實驗一致。數表:
| | k=30 | k=50 | k=100 | | | k=30 | k=50 | k=100 |
|---|---|---|---| |---|---|---|---|
@@ -702,8 +703,8 @@
力量語意即入選,women-power 群 11 碼,並產詞彙表外 力量語意即入選,women-power 群 11 碼,並產詞彙表外
幻覺碼一筆),claude-fable-5 讀為詞彙化概念(要求女性 幻覺碼一筆),claude-fable-5 讀為詞彙化概念(要求女性
標記,women-power 群 2 碼),與草稿盲選及深度閱讀輔助 標記,women-power 群 2 碼),與草稿盲選及深度閱讀輔助
判讀同讀法;sonnet 對照執行歸檔另行私人備份,不入 判讀同讀法;sonnet 對照執行歸檔備份於
版本庫,支出留帳。claude-fable-5 不 `excludes/experiments/`,支出留帳。claude-fable-5 不
受理 temperature 與 thinking 參數(均不送出),無法釘 受理 temperature 與 thinking 參數(均不送出),無法釘
temperature=0;執行間變異實測存在(run1/run2 於陽剛、 temperature=0;執行間變異實測存在(run1/run2 於陽剛、
脆弱邊緣碼分歧),由三票多數決吸收;詞彙表外輸出項 脆弱邊緣碼分歧),由三票多數決吸收;詞彙表外輸出項
@@ -750,9 +751,9 @@
編碼、票數),納入重建摘要與清空範圍,供群層次查詢; 編碼、票數),納入重建摘要與清空範圍,供群層次查詢;
重建命令自此帶 `--groups results/groups.csv` 重建命令自此帶 `--groups results/groups.csv`
- **Fable 5 輔助判讀實驗總錄(非正式編碼;結果檔私人 - **Fable 5 輔助判讀實驗總錄(非正式編碼;結果檔
留存、不入版本庫,本條為其版本庫內的程序錨點)**: gitignored 之 `excludes/`,本條為其版本庫內的程序
2026-08-08 起以 Claude Code subagent(模型 錨點)**:2026-08-08 起以 Claude Code subagent(模型
Fable 5)進行七項深度閱讀實驗。共同程序:每首歌一個 Fable 5)進行七項深度閱讀實驗。共同程序:每首歌一個
獨立會話、提示最少化且逐字統一、判讀者互不知情、不經 獨立會話、提示最少化且逐字統一、判讀者互不知情、不經
三票制;定位為研究者的輔助判讀(初篩),不改動任何 三票制;定位為研究者的輔助判讀(初篩),不改動任何
@@ -782,7 +783,7 @@
`male-mixed-wp-fe-feminist-reading.md` 與同名 .ods)。 `male-mixed-wp-fe-feminist-reading.md` 與同名 .ods)。
此系列即論文方法節「最後編碼的結果,再由研究者與 LLM 此系列即論文方法節「最後編碼的結果,再由研究者與 LLM
(Claude Code)輔助判讀」之所指;結果檔含大量歌詞引文, (Claude Code)輔助判讀」之所指;結果檔含大量歌詞引文,
依著作權紀律私人留置,不入版本庫。 依著作權紀律留置 `excludes/`,不入版本庫。
- **步驟 5:女性主義問題之質性深讀(設計定案)**:論文 - **步驟 5:女性主義問題之質性深讀(設計定案)**:論文
題目「挪用與污染」需要框架層的系統性證據;47 首 題目「挪用與污染」需要框架層的系統性證據;47 首
@@ -898,49 +899,4 @@
performer_gender 手工修正)、四份 results 定案表、時程 performer_gender 手工修正)、四份 results 定案表、時程
展延一至二日與剩餘撰寫工作。仍然有效的原則(先導不 展延一至二日與剩餘撰寫工作。仍然有效的原則(先導不
比較、提示只定格式、歌手背景防火牆、commit 判準、 比較、提示只定格式、歌手背景防火牆、commit 判準、
版權規則)保留。 版權規則、positionality)保留。
## 2026-08-17
- **先導研究的地位另立常設紀錄 `docs/pilot-study.md`**:論文
交件後重讀投稿摘要,認清先導研究係由當時協作的 Claude Code
設計並執行,研究者檢視的是研究問題與答案,未逐步檢視中間
過程;摘要中的具體數字未經稽核,亦無歸檔可回溯。此地位
先前僅零星散見於本日誌各條(歌詞沿用、women-power 來歷
考據、粒度繼承、簿記容量),投稿摘要本身則只是產物,不足以
記述過程。裁定:另立 `docs/pilot-study.md`,完整記述先導
研究的執行方式、沿用與棄用的成果,以及它如何促成正式研究
的可稽核設計;投稿摘要不於樹中另存副本,其內容留在 git
歷史(`paper/abstract.md` 的前身)。
## 2026-08-18
- **工序編號改與論文正文一致**:論文正文將五個步驟的細分
工序以字母標示(表三「步驟3a」、表四「步驟5d」),repo
的檔名與歸檔目錄卻沿用 `05-01``05-02` 這種數字次步,
兩邊對不上。裁定:改為 `1-tag``2-cluster``3a-code`
`4-group``5a-read``5b-consolidate``5c-synthesize`
`5d-annotate`(`prompts/``runs/` 同步,以 `git mv`
保留歷史),補零一併取消。`runs/*/meta.json`
內記的 `prompt_path` 不追改——它是執行當下的實況記錄;
本日誌的既有條目同理,維持當時的名稱。`run-costs.md`
相反:它的「步驟」欄是查閱歸檔的索引,故一律換為現行
編號(`01-01-01-tag``1-tag``03-01-code`
`03-code``3a-code`),並新增末欄「原步驟名」保留執行
當時的名稱——索引要指得到現在的歸檔,紀錄要留得住當時
的形狀,兩者以兩欄並存解決。已刪除的工序(步驟 2 的
LLM 合併嘗試 `01-02-01-merge`、步驟 3 的仲裁
`03-02-arbitration`)其歸檔已不存在,無現行編號可指,
不另發新名,步驟欄逕標「已廢棄」,原名存於末欄。
- **CLAUDE.md 移為 `docs/conventions.md`**:專案 CLAUDE.md 會
全文自動注入每一個 Claude Code subagent 的 context(2026-07-30
實測條),步驟 6 的 883 個盲判 agent 因此都看到了工作規範。
規範本身無研究語意(無 women-power、無假說),污染有限,但
「防脈絡污染」的原則應貫徹到自家工具鏈:docs/ 底下的檔案
不會自動注入,搬移後 subagent 除非主動翻閱否則不會看到,
盲判 agent 的定義檔並明令不得查閱專案材料。代價是主會話
也不再自動載入,須自行先讀 `docs/conventions.md`。內容
順帶微調:註明管線已完成、規範適用於重跑;刪去已不存在
的 codebook 一詞。進論文的數據一律走 Anthropic API(無
檔案系統,機制上不可能讀到),此層防線不受本次搬移影響。
+202 -101
View File
@@ -1,12 +1,12 @@
# 方法細節 # 方法細節
(全文方法節底稿。演算法在執行前寫定;任何修訂記入 (全文方法節底稿。演算法在執行前寫定;任何修訂記入
`decision-log.md` `decision-log.md`定義檔全文見 `prompts/`,執行紀錄見
`runs/`。)
## 自然編碼管線總覽 ## 自然編碼管線總覽
五個步驟:步驟 1 自由標註(`claude-sonnet-4-6` 五個步驟:步驟 1 自由標註(兩次執行進池)→ 步驟 2 詞彙表
temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
建構(詞向量分群,確定性)→ 步驟 3 全量編碼(三次執行+ 建構(詞向量分群,確定性)→ 步驟 3 全量編碼(三次執行+
多數決)→ 步驟 4 語意編碼群(三次執行+多數決)→ 步驟 5 多數決)→ 步驟 4 語意編碼群(三次執行+多數決)→ 步驟 5
女性主義問題之質性深讀(三次閱讀+逐首整合+樣態統整)。 女性主義問題之質性深讀(三次閱讀+逐首整合+樣態統整)。
@@ -14,6 +14,13 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
不接觸歌詞,步驟 2 亦不呼叫 LLM。設計原則見 不接觸歌詞,步驟 2 亦不呼叫 LLM。設計原則見
`research-plan.md`;本檔記載可重現的演算法細節。 `research-plan.md`;本檔記載可重現的演算法細節。
編號的所指為**研究程序的工序**,不是定義檔:步驟 1、
步驟 3、步驟 4 與步驟 5 有定義檔(`prompts/`;步驟 5 依
子工序有三份),步驟 2 沒有——它是單一確定性計算,由
`cluster-keywords` 一個子命令完成。有無定義檔的區別即
「該步是否為 LLM 判斷」,由 `prompts/` 是否存在同號檔案
直接可見。
## 步驟 2 詞彙表建構——詞向量分群 ## 步驟 2 詞彙表建構——詞向量分群
詞彙表由確定性程序產生,不經 LLM。完整分割(每個關鍵字 詞彙表由確定性程序產生,不經 LLM。完整分割(每個關鍵字
@@ -23,7 +30,8 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
### 進池 ### 進池
兩次標註執行的全部關鍵字取聯集、逐字串精確去重、字典序 兩次標註執行的全部關鍵字取聯集、逐字串精確去重、字典序
排列。失敗與拒答的記錄跳過(其歌曲不貢獻關鍵字) 排列。失敗與拒答的記錄跳過(其歌曲不貢獻關鍵字);解析
時偵測重複鍵,違規即失敗。
### 分群 ### 分群
@@ -31,10 +39,14 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
(釘定 revision),關鍵字的連字號先還原為空格再編碼, (釘定 revision),關鍵字的連字號先還原為空格再編碼,
輸出 768 維向量並 L2 正規化。 輸出 768 維向量並 L2 正規化。
- **分群**:階層式聚合分群(Ward linkage),k=100。 - **分群**:階層式聚合分群(Ward linkage),k=100。
向量既已正規化,歐氏距離與餘弦相似度單調對應 向量既已正規化,歐氏距離與餘弦相似度單調對應;三種
linkage 實測比較,Ward 於各個 k 的組內一致性均最高
average 與 complete 皆產生吞噬半數語料的巨大異質
組)。
- **組數的取捨**:k 太小則壓縮比過高,樹上層被迫併入 - **組數的取捨**:k 太小則壓縮比過高,樹上層被迫併入
不相干的詞,組雖大而無主題k 太大則人工難以通覽。 不相干的詞,組雖大而無主題k=30 最大組 491 詞、
定於 100,理由是實測顯示雜物櫃 組內一致性 0.41,成員橫跨籃球、海灘、外星人綁架);
k 太大則人工難以通覽。定於 100,理由是實測顯示雜物櫃
組於此始裂解為有主題的組,且編碼實測未見碼數過多的 組於此始裂解為有主題的組,且編碼實測未見碼數過多的
副作用——全量三次執行下 101 個碼全數用到;模型另行 副作用——全量三次執行下 101 個碼全數用到;模型另行
造出的碼共 13 筆,佔 44,149 筆標籤指派的 0.03%。 造出的碼共 13 筆,佔 44,149 筆標籤指派的 0.03%。
@@ -43,23 +55,39 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
自己產出過的關鍵字,非任何人事後撰寫。已知限制: 自己產出過的關鍵字,非任何人事後撰寫。已知限制:
組越大越異質時,medoid 只是折衷詞,可能代表不了組內 組越大越異質時,medoid 只是折衷詞,可能代表不了組內
內容(實測 `mutual-individuality` 組內一致 0.70 而 內容(實測 `mutual-individuality` 組內一致 0.70 而
編碼從未使用);此類碼於結果中呈現為零使用 編碼從未使用);此類碼於結果中呈現為零使用,據實
- **取捨紀錄**:詞彙表分群曾比較的替代法與棄用理由,見 報告,不事後改名。
`decision-log.md` 2026-08-05 條。 - **取捨紀錄**:曾以 LLM 單發收斂(mergecap 兩步)
- **產物**:分群輸出中,只有實際交給模型的碼表——組名 實作本步,四種模型六次執行全部無法維持完整分割,
加上先驗主題詞——含研究者的介入;其餘皆為分群過程本身 已棄用(詳見 `decision-log.md` 2026-08-05;棄用的
的機械紀錄 定義檔止於 git 歷史,見 `git log -- prompts/`
- **產物**:五份,前綴分別標示來源與結果。
`source-keywords.txt`(進池後的關鍵字,一行一個)
記錄進來的是什麼;`result-keywords.txt`(組名,一行
一個)與 `groups.csv`(欄位 Group、Keyword,一列一個
成員)記錄算出來的分割;`keywords-to-merge.json`
`{"keywords": [...]}`)是實際交給模型的碼,即組名
加上先驗主題詞——五份中只有這一份含研究者的介入。
`meta.json` 記錄執行本身:進池的兩份執行歸檔與其有效
筆數、嵌入模型與釘定 revision、分群參數與組數、外加
的先驗詞、關鍵字總數,以及產生數字的套件版本。凡命令
列上的選擇與環境事實皆在此,不記時間戳與輸入雜湊
——前者使同環境重跑逐位元組可再生,後者只會重述 git
已保證的事。
- **可重現性**:同一輸入、同一釘定模型、同一參數逐次 - **可重現性**:同一輸入、同一釘定模型、同一參數逐次
重現。不同 CPU/BLAS 實作的浮點尾數差異可能使邊界 重現。不同 CPU/BLAS 實作的浮點尾數差異可能使邊界
詞的歸屬翻動,屬已揭露的限制 詞的歸屬翻動,屬已揭露的限制;論文所用碼表逐字
commit,引用單位為該份定案檔案。
### women-power 的注入 ### women-power 的注入
定案詞彙表為 100 個分群組名再加上 `women-power` 一詞, 定案詞彙表為 100 個分群組名再加上 `women-power` 一詞,
共 101 個碼。`women-power` 是研究者任意決定的先驗主題(即本 共 101 個碼。`women-power` 是研究者任意決定的先驗主題(即本
論文的主題本身),不由資料產生,屬揭露的儀器介入。該詞 論文的主題本身),不由資料產生,屬揭露的儀器介入。該詞
的加入於每次執行皆為明示、可稽核的介入,而非無聲內建於 於執行時以 `--extra-keyword` 明示加入,不寫死在程式裏
判斷邏輯;分群結果本身不含它,只有交給模型的碼表含它。 ——研究者的介入因此每次都出現在重現命令上,而非無聲
發生;分群結果的兩份產物不含它,只有交給模型的碼表含
它。
注入而非另設篩選軌的理由:讓研究者的主題詞與模型自己 注入而非另設篩選軌的理由:讓研究者的主題詞與模型自己
收斂出的類別(分群已自行長出 `female-empowerment` 等組) 收斂出的類別(分群已自行長出 `female-empowerment` 等組)
@@ -69,17 +97,24 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
## 步驟 3 編碼的三次執行與多數決 ## 步驟 3 編碼的三次執行與多數決
- **模型**`claude-sonnet-4-6`temperature=0、thinking 關閉。
- **三次執行**:同一份定義檔、同一份輸入檔,獨立執行 - **三次執行**:同一份定義檔、同一份輸入檔,獨立執行
三次,三份歸檔並列,彼此無先後主從之別。 三次,三份歸檔並列`runs/03-code/run1``run2`
`run3`),彼此無先後主從之別。
- **多數決**:一首歌的一個標籤,三次執行中至少兩次標出 - **多數決**:一首歌的一個標籤,三次執行中至少兩次標出
即收入定案編碼。三票不平手,裁決規則因此無例外條款, 即收入定案編碼。三票不平手,裁決規則因此無例外條款,
計票由確定性程序完成(見交接契約)。 計票由確定性子命令完成(見交接契約)。
- **第三票取全量**第三次執行同為全量——全部歌曲、 - **第三票取全量**只對前兩次分歧的標籤補問第三票,
全部關鍵字——使三票在同一條件下取得。 計票結果相同;仍採全量執行——全部歌曲、全部關鍵字
——使三票在同一條件下取得。
- **對邊緣標籤的作用**:兩次執行只分得出「兩次皆標」與 - **對邊緣標籤的作用**:兩次執行只分得出「兩次皆標」與
「僅一次標」;三次執行還分得出 3-0 與 2-1,故「定案 「僅一次標」;三次執行還分得出 3-0 與 2-1,故「定案
編碼中有多少比例僅以一票之差成立」成為可報告的量。 編碼中有多少比例僅以一票之差成立」成為可報告的量。
至於判定本身,兩次執行相左的標籤在何種協定下都由第三
個判斷定奪,票數不使不確定性消失:某標籤於單次執行被
標出的傾向若恰為一半,任何票數皆為擲幣。三票之效在
傾向偏離一半處——多數決將判定推向該傾向本身(單次
0.7 者為 0.780.9 者為 0.97),程序重跑的一致性因而
高於單次執行,唯獨恰半處無從改善。
## 步驟 4 語意編碼群 ## 步驟 4 語意編碼群
@@ -90,79 +125,103 @@ temperature=0、thinking 關閉,兩次執行進池)→ 步驟 2 詞彙表
由 LLM 依編碼名的字面語意判斷。 由 LLM 依編碼名的字面語意判斷。
- **任務**:每筆輸入為一個群名加 101 個編碼的字母序 - **任務**:每筆輸入為一個群名加 101 個編碼的字母序
清單,輸出為入選編碼的單層 JSON 陣列;定義檔 清單,輸出為入選編碼的單層 JSON 陣列;定義檔
只定格式,不含任何群的語意定義。 `prompts/04-group.md` 只定格式,不含任何群的語意定義。
- **模型**`claude-fable-5`;取樣變異由多數決吸收; - **模型**`claude-fable-5`(步驟 1、3 為
`claude-sonnet-4-6`)。該模型不受理 `temperature`
`thinking` 參數,兩者均不送出;取樣變異由多數決吸收。
模型裁定的理由與對照實驗見決策日誌。 模型裁定的理由與對照實驗見決策日誌。
- **三次執行**:同一份定義檔、同一份輸入檔,獨立執行 - **三次執行**:同一份定義檔、同一份輸入檔,獨立執行
三次,歸檔並列。 三次,歸檔並列`runs/04-group/run1``run2``run3`
- **多數決**:一個(群,編碼)配對,三次執行中至少兩次 - **多數決**:一個(群,編碼)配對,三次執行中至少兩次
入選即屬該群;不在 101 碼詞彙表內的輸出項無效, 入選即屬該群;不在 101 碼詞彙表內的輸出項無效,
記錄後丟棄。計票由確定性計票程序完成,合法碼 丟棄印於標準錯誤。計票由確定性子命令 `tally-groups`
清單之產法同步驟 3,結果為定案分群表。 完成:`tally-groups <執行歸檔 1> <執行歸檔 2> <執行歸檔
3> <合法碼清單> <輸出 CSV>`,合法碼清單之產法同步驟 3。
定案分群寫入 `results/groups.csv`,欄位 `Group`
`Keyword``Votes`,列序先依群名、再依編碼,一律以
Unicode 碼位比較,換行為 CRLF。
- **工作儲存**`build-db --groups <定案分群 CSV>` 將定案
分群逐欄照存入 `groups` 資料表(群、編碼、票數),供
群層次查詢。
## 步驟 5 女性主義問題之質性深讀 ## 步驟 5 女性主義問題之質性深讀
本步驟之 5a 至 5c 為**質性閱讀非編碼**輸出為自由 本步驟之 5-1 至 5-3 為**質性閱讀,非編碼**:輸出為自由
文字的問題閱讀報告無可逐項機械比對的單位故不適用 文字的問題閱讀報告,無可逐項機械比對的單位,故不適用
三票多數決與仲裁 三票多數決與仲裁;
三次獨立閱讀為分析者三角檢核逐首整合為整合而非裁決 三次獨立閱讀為分析者三角檢核,逐首整合為整合而非裁決,
跨首統整之產出為草稿終審與詮釋由研究者為之。論文引用 跨首統整之產出為草稿,終審與詮釋由研究者為之。論文引用
本步驟時不作次數宣稱。 本步驟時不作次數宣稱。
- **對象**定案編碼含 `women-power` - **對象**:定案編碼含 `women-power`
`female-empowerment` 的 145 首歌。 `female-empowerment` 的 145 首歌。
- **5a 逐首閱讀**每筆輸入為一首歌的完整歌詞逐字全文 - **5-1 逐首閱讀**:每筆輸入為一首歌的完整歌詞逐字全文,
不含歌名與演唱者盲讀)。同一份定義檔、同一份輸入檔, 不含歌名與演唱者(盲讀);定義檔
獨立執行三次,歸檔並列。 `prompts/05-01-read.md`。同一份定義檔、同一份輸入檔,
- **5b 逐首整合**:每筆輸入為該首歌的三份閱讀報告 獨立執行三次,歸檔並列(`runs/05-01-read/run1`
(不含歌詞);以問題機制為單位保守合併,標收斂註記 `run2``run3`)。
((3/3)、(2/3)),主清單僅列兩讀以上提出者,單讀發現 - **5-2 逐首整合**:每筆輸入為該首歌的三份閱讀報告
以一行存目;定義檔,執行一次。 (不含歌詞);以問題機制為單位保守合併,標收斂註記
- **5c 樣態統整**:單筆輸入為一批整合報告;歸納問題 ((3/3)、(2/3)),主清單僅列兩讀以上提出者,單讀發現
**樣態**——問題呈現與運作的重複形態,非問題分類, 以一行存目;定義檔 `prompts/05-02-consolidate.md`,
代表引句僅取自主清單。四種輸入範圍各執行 執行一次,歸檔 `runs/05-02-consolidate/run1`
一次:全 145 首之基底統整,及依演唱聲音之性別切分 - **5-3 樣態統整**:單筆輸入為一批整合報告;歸納問題
之三個發話脈絡統整——男聲(male)、女聲(female)、 **樣態**——問題呈現與運作的重複形態,非問題分類,
混合(mixed);基底看橫貫各 代表引句僅取自主清單;定義檔
脈絡之樣態,分組看各權力脈絡下之樣態。genderfluid `prompts/05-03-synthesize.md`。四種輸入範圍各執行
與 non-binary 共 3 首不設群——樣本數不支持歸納—— 一次:全 145 首之基底統整(歸檔
僅入基底統整。同一輸入不重複 `runs/05-03-synthesize/run1`),及依 `performer_gender`
執行:自由歸納之產出無機械合併可言,其變異由 5a (演唱聲音之性別)切分之三個發話脈絡統整——男聲
三讀、5b 整合與草稿地位承接,四份草稿互為對照,由 (male,歸檔 `run2`)、女聲(female,歸檔 `run3`)、
研究者終審裁決。 混合(mixed,歸檔 `run4`);基底看橫貫各脈絡之樣態,
- **5d 樣態標註**:以(歌,樣態)對為可逐項機械比對之 分組看各權力脈絡下之樣態。genderfluid 與 non-binary
單位,回歸「三次執行+多數決」協定。對象為「有問題」 共 3 首不設群——樣本數不支持歸納——僅入基底統整,
的歌——三讀中至多一個「無」(多數決精神;恰兩「無」 由研究者以個案閱讀。同一輸入不重複執行:自由歸納之
者其整合報告主清單必為空,與 5b 主清單規則自洽), 產出無機械合併可言,其變異由 5-1 三讀、5-2 整合與
計 111 首(男聲 12、女聲 69、混合 29、genderfluid 1 草稿地位承接,四份草稿互為對照,由研究者終審裁決
樣態表為三份分組統整草稿原文:男聲 13 條(M1–M13)、 - **5-4 樣態標註**:以 (歌, 樣態) 對為可逐項機械比對之
女聲 14 條(F1F14)、混合 16 條(X1–X16);基底 15 條 單位,回歸「三次執行+多數決」協定。對象為「有問題」
的歌——三讀中至多一個「無」(多數決精神;恰兩「無」
者其整合報告主清單必為空,與 5-2 主清單規則自洽),
計 111 首(男聲 12、女聲 69、混合 29、genderfluid 1)。
樣態表為三份分組統整草稿原文:男聲 13 條(M1–M13)、
女聲 14 條(F1F14)、混合 16 條(X1X16);基底 15 條
不入矩陣——全體歸納之一條樣態可能疊合不同方向的 不入矩陣——全體歸納之一條樣態可能疊合不同方向的
權力關係男對女、女對男),實為多個樣態共用一名。 權力關係(男對女、女對男),實為多個樣態共用一名。
檢驗範圍男聲樣態不檢驗純女聲歌、女聲樣態不檢驗 檢驗範圍:男聲樣態不檢驗純女聲歌、女聲樣態不檢驗
純男聲歌發話位置範疇錯置檢查無意義);混合樣態 純男聲歌(發話位置範疇錯置,檢查無意義);混合樣態
檢驗全部其男女聲部無系統化切分方式無意義之標註 檢驗全部(其男女聲部無系統化切分方式,無意義之標註
容忍之);genderfluid 歌三套全查無自身透鏡發話 容忍之);genderfluid 歌三套全查(無自身透鏡,發話
位置無法先驗決定),其歸屬引用維持個案地位。每筆 位置無法先驗決定),其歸屬引用維持個案地位。每筆
輸入為一首歌之整合報告「僅單獨提及」行於組裝時 輸入為一首歌之整合報告(「僅單獨提及」行於組裝時
剝除標註僅依主清單與該首適用之樣態表定義檔 剝除,標註僅依主清單)與該首適用之樣態表;定義檔
獨立執行三次,(歌,樣態)對 `prompts/05-04-annotate.md`,獨立執行三次
(`runs/05-04-annotate/run1`~`run3`),(歌, 樣態) 對
得兩票以上者定案。 得兩票以上者定案。
- **模型**`claude-fable-5` - **模型**:`claude-fable-5`(與先導深讀同儀器;
- **輸入組裝**:各步輸入檔由確定性程序自上游產物組裝 `temperature``thinking` 參數不適用,均不送出)
5d 之樣態條目自分組統整草稿機械切出,代表引句不 - **輸入組裝**:確定性行內腳本。5-1:145 首依歌曲 ID
隨附——引句出自特定歌曲,判該曲時形同預答。 升序,`content` 為歌詞逐字全文;5-2:每筆
`{"reports": [run1 輸出, run2 輸出, run3 輸出]}`;
5-3:單筆以 `song-<ID>` 為鍵、整合報告為值之 JSON
物件,鍵集合為該次統整之範圍(基底為全 145 首,
分組依工作庫 `performer_gender` 切分);5-4:每筆
`{"report": 主清單, "patterns": [{"id", "name",
"description"}]}`,樣態條目自分組統整草稿機械切出,
代表引句不隨附——引句出自特定歌曲,判該曲時形同
預答。各輸入檔之 SHA-256 記入該步 meta。
## 女性力量候選集 ## 女性力量候選集
候選集為定案編碼含 `women-power` `female-empowerment` 候選集為兩類歌曲的合集:定案編碼含 `women-power` 者,
(步驟 4 女性力量群的兩個編碼)之歌曲聯集:wp 66 首、 以及定案編碼含研究者指認之女性力量概念域分群組者。
fe 144 首,聯集 145 首。 指認於詞彙表定案後、黃金標準編碼開始前完成,指認清單
與理由記入決策日誌。
## 軌跡對映(診斷用) ## 軌跡對映(診斷用)
沿收斂軌跡的機械對映:原始關鍵字 →(兩份標註執行歸檔的 沿收斂軌跡的機械對映:原始關鍵字 →(兩份標註執行歸檔的
原始輸出)歌曲、原始關鍵字 →(分群)組,純機械查表, `output.jsonl`)歌曲、原始關鍵字 →(分群)組,純程式查表,
決定性。以其結果與步驟 3 直接編碼的差異率作為「收斂軌跡 決定性。以其結果與步驟 3 直接編碼的差異率作為「收斂軌跡
扭曲」的診斷量,不作主結果。 扭曲」的診斷量,不作主結果。
@@ -171,36 +230,78 @@ fe 144 首,聯集 145 首。
每一步的輸出如何變成下一步的輸入,皆為確定性程序,規則 每一步的輸出如何變成下一步的輸入,皆為確定性程序,規則
明定如下: 明定如下:
- **歌詞輸入檔(步驟 1**由確定性的匯出程序自工作 - **歌詞輸入檔(步驟 1**`export-llm-input` 自工作
儲存產出,一筆一首歌,依歌曲 ID 升序。步驟 3 的輸入 儲存產出,每筆 `{"id": "song-<ID>", "content": <歌詞>}`
由同一匯出程序、同一工作儲存產出(見下),兩步的語料 依歌曲 ID 升序。步驟 3 的輸入由同一子命令、同一工作
同一性由此成立 儲存產出(見下),兩步的語料同一性由此成立;各步
- **步驟 1 → 2**:確定性的分群程序讀兩份執行歸檔的 輸入檔的 SHA-256 記入該步 meta。
執行紀錄,進池後直接分群,產出詞彙表與交給 - **步驟 1 → 2**`cluster-keywords` 讀兩份執行歸檔的
模型的碼表。 `output.jsonl`(一律以換行字元 `\n` 切行——歌詞含
- **步驟 2 → 3 輸入檔**:同一匯出程序自工作儲存產出 U+0085 等控制字元時,`str.splitlines()` 類的通用切行
步驟 3 的輸入,一筆一首歌,兼含歌詞與定案碼表,依 會截斷 JSON 字串,實測踩中),進池後直接分群,一次
歌曲 ID 升序。碼表以參數傳入而非填進定義檔——定義 產出上列五份檔案。
- **步驟 2 → 3 輸入檔**`export-llm-input --extras
<定案碼表>` 自工作儲存產出步驟 3 的輸入,每筆
`{"id": "song-<ID>", "content": <字串>}``content` 為
固定鍵序序列化的 `{"lyrics": …, "keywords": [...]}`
依歌曲 ID 升序。碼表以參數傳入而非填進定義檔——定義
檔只規定任務形狀,換詞彙表、換演算法都不必改它。 檔只規定任務形狀,換詞彙表、換演算法都不必改它。
- **步驟 3 定案**確定性的計票程序讀三份執行歸檔 - **步驟 3 定案**`tally-codings <執行歸檔 1> <執行歸檔
執行紀錄,依序套用更正表、驗證所有標籤皆在合法碼 2> <執行歸檔 3> <輸出 CSV> --corrections <更正表>
--valid-keywords <合法碼清單>` 讀三份執行歸檔的
`output.jsonl`,依序套用更正表、驗證所有標籤皆在合法碼
清單之內、計票。 清單之內、計票。
- **更正表**研究者逐列校定的人工著作,逐筆更正標籤 - **更正表**`data/manual/coding-corrections.csv`,研究者
或引述——以替代字串取代,或刪去該筆標籤指派或該句 逐列校定的人工著作,欄位 `Song ID`、`Run`、`Type`、
引述。一筆引述更正套用於該首歌該次執行的所有出現處。 `To Be Replaced`、`Correct Term`。`Type` 為 `keyword`
表中任一列若在資料中找不到對應者,即中止;校定的 或 `evidence`,分別更正標籤與引述;`Correct Term` 為
判準記於 `decision-log.md` 替代字串,或 `**REMOVE**` 表示刪去該筆標籤指派(`keyword`
- **合法碼清單**:自詞彙表產出:分群組名加上 或該句引述(`evidence`)。一筆 `evidence` 更正套用於該
`women-power` 首歌該次執行的所有出現處。兩個文字欄以歌詞慣例「 / 」
- **定案編碼表**:歌名與演出者名銜逐首查工作儲存取得。 表示換行(與載入後的執行紀錄同一表示法,逐字比對、不再
每個定案標籤隨附其在計票中各份執行所引的 轉換),故一列一行,純文字工具可逐列處理。表中任一列若
歌詞行,供逐碼查核(三份執行彼此無先後主從之別)。 在資料中找不到對應者,即中止;校定的判準記於
`decision-log.md`。
- **合法碼清單**:純文字、一行一個碼,自詞彙表產出:
`{ cat runs/02-cluster/result-keywords.txt; echo
women-power; } | sort`。
- **定案表**`results/codings.csv`,欄位 `Song`、
`Artist Credit`、`Keyword`、`Quote`,一列一個標籤。歌名
與演出者名銜逐首查工作儲存取得,故本子命令須在
`build-db` 之後執行。`Quote` 為該標籤在計票中各份執行
所引的歌詞行:各份的引述串接後逐字去重,按 Unicode
碼位排序,以單一 `|` 相接(三份執行彼此無先後主從之
別,引述之序取決於引述本身);引述內的換行於執行紀錄
載入時一次換成歌詞慣例「 / 」,此後更正表、定案表與
工作儲存全鏈路同一表示法,不再還原。「 / 」的無歧義性
是語料事實而非結構保證:全 883 首歌詞經窮舉查核不含
「 / 」;換語料須重查。列序依印出的前三欄依序排:
歌名、演出者名銜、
標籤,一律以 Unicode 碼位比較,換行為 CRLF(同專案
其他 CSV)。
- **序列化通則**:所有中間檔為 UTF-8,欄序、鍵序與元素
序皆依上列規則明定,無時間戳、無隨機成分;JSON 解析
一律偵測重複鍵,違規即失敗。人讀為主的產物採純文字或
CSVCSV 依 RFC 4180,標題列字首大寫),機器交接檔採
JSON。給定相同的 LLM 執行輸出,全部交接產物逐位元組
可再生。
## 執行與稽核 ## 執行與稽核
- LLM 步驟以批次執行程序執行,一份定義檔配一份 - LLM 步驟以 `run-llm <定義檔> <輸入檔> <歸檔目錄>`
輸入檔;一步的 N 次執行為 N 次各自獨立的呼叫,各自 執行;一步的 N 次執行=重現命令清單上的 N 行命令,
歸檔自我完備。 各自歸檔(`runs/<步驟>/run1`、`run2`,三票制步驟另有
`run3`)。
- 確定性步驟(進池、分群、計票、對映)為子命令,其
輸入輸出檔同隨 `runs/` 歸檔;因無執行變異,歸檔目錄
下不分 `run<N>` 層。
- Batch API 的每筆請求自含全部脈絡且互不可見(平台 - Batch API 的每筆請求自含全部脈絡且互不可見(平台
契約),歌與歌之間的獨立性由此成立;各次執行的獨立 契約),歌與歌之間的獨立性由此成立;各次執行的獨立
性由「一次呼叫、一個批次、一份歸檔」的執行結構自明。 性由「一次呼叫、一個批次、一份歸檔」的執行結構自明。
- 每次 `run-llm` 執行的 token 用量與費用記入
`run-costs.md`,被取代的執行一併保留供總支出核算。
## 映射分析方法
(依 2026-07-30 決策,於看到結果前寫定;待黃金標準
編碼展開前補入。)
+5 -4
View File
@@ -1,11 +1,11 @@
# LLM 輸出的契約查核 # LLM 輸出的契約查核
2026-08-06 量測。對象為步驟 3 的三份執行歸檔 2026-08-06 量測。對象為步驟 3 的三份執行歸檔
`data/runs/3a-code/run1``run3`,共 883 首歌、44,149 筆標籤 `runs/03-code/run1``run3`,共 883 首歌、44,149 筆標籤
指派、44,146 句引述。歌詞以模型實際看到的那一份為準,即 指派、44,146 句引述。歌詞以模型實際看到的那一份為準,即
`tools/instance/llm-input-code.jsonl`。) `tools/instance/llm-input-code.jsonl`。)
定義檔 `prompts/3a-code.md` 對輸出下了四條明確要求:只用 定義檔 `prompts/03-code.md` 對輸出下了四條明確要求:只用
給定的碼且拼寫照給、每個標出的碼恰引一行歌詞、引述逐字、 給定的碼且拼寫照給、每個標出的碼恰引一行歌詞、引述逐字、
輸出為合法 JSON 且無其他文字。本檔記錄這四條各自被遵守到 輸出為合法 JSON 且無其他文字。本檔記錄這四條各自被遵守到
什麼程度。 什麼程度。
@@ -46,8 +46,9 @@ song-750 run3: -use=[] -abuse=["I got a thing for the hard
liquor on ice"] liquor on ice"]
``` ```
song-750 的 run1 與 run2 則整筆使用錯拼的鍵並附上引述 模型寫錯後綴、已輸出的 token 收不回,遂以空陣列收束該鍵
故該首的三票分裂於兩種拼寫之間。 再於正確的鍵補上引述。song-750 的 run1 與 run2 則整筆使用
錯拼的鍵並附上引述,故該首的三票分裂於兩種拼寫之間。
## 引述的存在 ## 引述的存在
-85
View File
@@ -1,85 +0,0 @@
# 先導研究的來歷與地位
2026-08-17 認清並記錄。本檔記述正式研究之前的先導研究:
它做了什麼、研究者檢視到哪一層、哪些成果被沿用、哪些被
棄用,以及它如何促成正式研究的設計。)
## 一、先導研究是什麼
正式研究之前,研究者曾以 Claude Code 對 Billboard Year-End
Hot 100(20182025、684 首)的歌詞做過一輪探索性分析,檢視
「女性力量」語彙的使用狀況,並附帶分析 pussy 一詞作為女性
代稱的修辭。2026 年 4 月投出的研討會摘要即根據該輪分析撰寫。
## 二、它實際的執行方式
**研究問題與方向由研究者提出**:女性力量語彙被父權框架取用、
pussy 作為代稱的跨性別使用,都是研究者長期關注的問題。
**設計與執行由 Claude Code 進行**:分析管線、分類架構(genuine
peripheral/fake 三分類)、五種「假女性力量」類型、以及
frame-aware 提示修正實驗,均由當時協作的 Claude Code 設計並
執行。
**研究者檢視的是問題與答案,不是中間過程**:研究者確認研究
問題的方向正確、答案值得追究,即據以投稿;分類類目如何產生、
提示詞如何下、數字如何算出,並未逐步檢視。投稿摘要中的具體
數字(如「44% 屬於假女性力量」、62 首的三分類、53 首的 pussy
修辭分類、假率由 44% 降至 5% 以下)研究者未曾稽核,其產生
過程亦無歸檔可回溯。
**當時的判斷**:投稿未上即罷;若上,正式研究由研究者自行
設計與執行。事實上正式研究即是如此重做的。
## 三、被沿用的成果
- **歌詞捕捉檔**:先導研究蒐集的 lyrics.json(684 首,
2018–2025)以私人腳本(不入版本庫)匯入歌詞快取,只取
識別欄位與歌詞本文,先導的分析欄位一概不匯入;出處記於
`data/captures/lyrics-provenance.csv`
(見 `decision-log.md` 2026-07-31、08-02 條)。
- **假說方向**:女性力量語彙的挪用與污染,成為正式研究的
研究問題。
- **粒度選擇**:正式研究鎖定 thematic keywords 這一粒度,
係繼承先導研究三種粒度的比較結果(keywords 過碎、themes
過早抽象)。
- **`women-power` 一詞的來歷**:考據先導研究的 local agent
存檔可知,其第一步指令含數十個範例 thematic keywords,
其中即有 women-power——為當時協作的 Claude Code 依研究者
長期表達的關注主動加入(研究者端播種,非明示指定);先導
的標籤 `women-power-and-empowerment` 則是第三步強制合併
兩個關鍵字的管線人工產物(見 `decision-log.md` 2026-08-04
條)。
- **簿記容量的教訓**:先導研究九百餘詞可以在單一回應內完成
分組;此法未沿用,詞彙表改由確定性程序產生(見
`decision-log.md` 2026-08-06 條)。
## 四、被棄用的成果
以下先導研究的產物未進入正式研究,論文亦未引用:
- **genuineperipheralfake 三分類與「44% 假女性力量」**:
構念與分母均未經稽核回溯,未沿用。
- **五種「假女性力量」類型(A–E)**:未沿用。
- **pussy 一詞的修辭分類**:未沿用。
- **frame-aware(框架感知)提示修正法**:先導研究以框架判準
寫進提示以提高準確率;未沿用。
- **三個獨立 LLM subagent+人工仲裁的流程**:未沿用,原因見
`decision-log.md` 2026-07-30 條。
## 五、它促成了可稽核的設計
先導研究的根本限制不在結論對錯,而在**不可稽核**:沒有定義檔
快照、沒有原始輸出歸檔、沒有參數紀錄,因此任何一個數字都無法
回溯到產生它的那一次執行。正式研究的可稽核設計,正是針對這一點
而立。
換言之,正式研究對 LLM 輸出所採取的「不信任、須查核」立場,
其第一個案例就是先導研究本身。
## 六、現存的紀錄
- 投稿時的先導研究摘要:已於 git 歷史中(`paper/abstract.md`
的前身),樹中不另存副本——摘要只是產物,不足以記述過程,
故另立本檔。
- 先導研究的歌詞捕捉檔沿用事實:`data/captures/lyrics-provenance.csv`
+134
View File
@@ -0,0 +1,134 @@
# 專案目錄結構
2026-07-30 討論定案;2026-07-31 更新為 tools/ 子專案與
SQLite 工作儲存架構;2026-08-17 依完成後的現況更新)
```
pop-fem-audit/
├── README.md # 專案說明、重現步驟
├── CLAUDE.md # 極簡工作規範(subagent 會讀到,
│ # 絕不放理論、codebook、預期結果)
├── .gitignore # captures/lyrics/、.env、scratch
├── data/ # 依生命週期分層(文字格式)
│ ├── source/ # 源頭:手放後不動
│ │ └── yearend_hot100_2016_2025.csv # 原始榜單
│ ├── captures/ # 外部捕捉:只由 fetch 命令與
│ │ │ # 私人匯入腳本寫入
│ │ ├── artists-wikidata.csv # Wikidata 快照
│ │ ├── lyrics-provenance.csv # 歌詞出處
│ │ └── lyrics/ # 歌詞 .txt 快取
│ │ # gitignored,版權)
│ ├── manual/ # 人工著作:只由研究者手寫
│ │ ├── coding-corrections.csv # 編碼與引述的校對表
│ │ └── performer-gender-corrections.csv # 演唱聲音性別的
│ │ # 手工修正
│ └── derived/ # 衍生:只由 build-db 寫入
│ ├── songs.csv # 歌曲報表(人讀;進 git)
│ └── artists.csv # 歌手報表(人讀;進 git)
├── prompts/ # LLM 定義檔(逐字作為 system prompt
│ └── <步>-<次步>-<task>.md # 01-tag.md、03-code.md、
│ # 04-group.md、05-01-read.md、
│ # 05-02-consolidate.md、
│ # 05-03-synthesize.md、
│ # 05-04-annotate.md
│ # (步內僅一個執行時省略次步)
│ # 不帶版本號,版本即 git 歷史
│ # (編號的所指是工序:確定性
│ # 的步驟 2 無定義檔仍佔一號;
│ # 補零只為排序)
├── tools/ # 輔助工具子專案(src-layout
│ ├── pyproject.toml # 發行名 pop-fem-audit-tools
│ │ # pip install -e tools/ 安裝
│ ├── README.rst LICENSE MANIFEST.in .env.example .gitignore
│ ├── docs/ # Sphinx API 文件
│ ├── instance/ # SQLite 工作儲存(generated、
│ │ # gitignored;含歌詞全文)
│ ├── src/pop_fem_audit_tools/
│ │ ├── __main__.py # 套件 CLI 進入點(分派子命令)
│ │ ├── commands/ # CLI 子命令模組(登記於 __init__)
│ │ │ ├── build_db.py # build the SQLite working store
│ │ │ │ # from the inputs
│ │ │ ├── export_llm_input.py # export the LLM input JSONL
│ │ │ │ # (lyrics only) from the
│ │ │ │ # working store
│ │ │ ├── fetch_artists.py # fetch artist metadata from
│ │ │ │ # Wikidata into the snapshot CSV
│ │ │ ├── fetch_lyrics.py # fetch missing lyrics from the
│ │ │ │ # public APIs into the lyrics dir
│ │ │ ├── cluster_keywords.py # pool the tagging runs'
│ │ │ │ # keywords and cluster them
│ │ │ │ # into the codes (step 2)
│ │ │ ├── tally_codings.py # settle step 3 by majority
│ │ │ ├── tally_groups.py # settle step 4 by majority
│ │ │ ├── tally_annotations.py # settle step 5-4 by majority
│ │ │ └── run_llm.py # API 執行器:一份定義檔+一份輸入
│ │ │ # →歸檔至指定目錄(Batch API);
│ │ │ # 多次執行的計票由獨立子命令承擔
│ │ ├── config.py # pydantic-settings 設定(.env
│ │ ├── database.py # SQLAlchemy engine / session / Base
│ │ ├── models.py # SQLAlchemy ORM 資料模型
│ │ └── utils.py # 共用工具(format_duration
│ └── tests/ # 單元測試(unittest
├── runs/ # 現行執行的完整稽核紀錄(進 git;
│ │ # 重跑同一 run 須明示 --replace
│ ├── <步驟名>/ # 一步一個目錄(01-tag、03-code、
│ │ # 04-group、05-01-read、
│ │ # 05-02-consolidate、
│ │ # 05-03-synthesize、05-04-annotate
│ │ └── run<N>/ # LLM 步驟:每個 run 一份自我
│ │ ├── prompt.md # 完備歸檔(定義檔快照)
│ │ ├── output.jsonl # 該次執行原始輸出
│ │ └── meta.json # model ID、temperature、時間戳、
│ │ # batch ID、token 用量
│ └── 02-cluster/ # 確定性步驟:無執行變異,
│ # 不分 run<N> 層
├── results/ # 論文引用的定案表 CSV(計票子命令
│ │ # 產出;「可再生仍 commit」的例外)
│ ├── codings.csv # 步驟 3 定案編碼
│ ├── groups.csv # 步驟 4 定案編碼群
│ ├── patterns.csv # 步驟 5-3 定案樣態表
│ ├── annotations.csv # 步驟 5-4 定案歌×樣態
│ └── pattern-matrix.csv # 前四者的人讀寬表
├── docs/
│ ├── research-plan.md # 研究步驟規劃(本檔之姊妹篇)
│ ├── project-structure.md # 本檔
│ ├── output-validation.md # LLM 輸出的契約查核紀錄
│ ├── decision-log.md # 決策日誌:每次改定義檔的原因
│ ├── run-costs.md # 每次執行的 token 用量與費用
│ └── methodology.md # 方法細節(全文方法節底稿;
│ # 映射分析方法須在看結果前寫定)
└── paper/
├── abstract.md # 摘要
└── 流行音樂中「女性力量」….odt # 全文
```
## 設計理由
- **`runs/` 自我完備**:每個執行目錄含定義檔快照 + 原始輸出 +
meta,讀者不需 git 考古即可稽核任一筆結果。
- **`runs/`(原始稽核資料)與 `results/`(最終表)分離**
論文只引 `results/`,其來源可回溯至 `runs/`
- **`prompts/` 檔名不帶版本號**:版本即 git 歷史,失敗的
版本不保留;論文引用的單位是 `runs/` 內隨執行保存的定義檔
快照(每個執行目錄自我完備),不需檔名可指的版本名。
- **工作儲存的資料表**`songs`(含 `performer_gender`=演唱
聲音的性別)、`chart_entries``artists``song_artists`
`codings`(定案編碼:一歌一標籤一列,`quotes` 存該標籤所據的
歌詞引述,多句以 `|` 相接)、`groups`(語意編碼群)、
`patterns`(深讀樣態)、`annotations`(歌×樣態定案矩陣)。
各定案表經 `build-db``--codings``--groups`
`--patterns``--annotations` 匯入,性別修正經
`--gender-corrections` 套用,與其餘資料同一交易,儲存不會
半建;詳見 `research-plan.md`「資料儲存與模型」。
- **Commit 判準**:能由「committed 輸入+程式」決定性再生者不
commitSQLite 工作儲存、LLM 輸入檔);源頭、捕捉、人工著作
一律以文字 commit。「可再生仍 commit」的例外有二:
`results/` 報表(引用穩定性、審稿人零門檻、撰稿期可 diff)
`data/derived/` 人讀報表(與工作儲存同一動作產出,稽核
鏈無中間空缺)。詳見 `research-plan.md`「資料儲存與模型」。
- **設定**經 pydantic-settings 統一:`.env`gitignored,範本
`tools/.env.example`)供應 `SQLALCHEMY_DATABASE_URL`
`ANTHROPIC_API_KEY`,絕不寫入 repo。
- **CLAUDE.md 極簡**:實測證實 Claude Code subagent 會繼承專案
CLAUDE.md 全文,故其中只放工作流程規則,領域知識一律放
`docs/`subagent 不會自動讀到)。
+20 -9
View File
@@ -1,7 +1,8 @@
# 信度:量測方式與結果 # 信度:量測方式與結果
2026-08-16 量測。對象為步驟 3a 與步驟 5d 各三次執行的 2026-08-16 量測。對象為步驟 3 的三份執行歸檔
原始輸出;數字由原始輸出直接計算。) `runs/03-code/run1``run3`,與步驟 5-4 的三份執行歸檔
`runs/05-04-annotate/run1``run3`;數字由原始輸出直接計算。)
## 一、本研究的信度是什麼 ## 一、本研究的信度是什麼
@@ -33,7 +34,11 @@ Krippendorff 依產生資料的設計,把信度分成三型
**精密度(precision)**,而**準確度(accuracy)** 未經量測。 **精密度(precision)**,而**準確度(accuracy)** 未經量測。
若儀器是確定性的,重複性無須量測;但 LLM 不是——同樣的提示、 若儀器是確定性的,重複性無須量測;但 LLM 不是——同樣的提示、
同樣的輸入,三次會給出不同結果,因此重複性是必須報告的儀器 同樣的輸入,三次會給出不同結果,因此重複性是必須報告的儀器
規格。 規格,而非慣例儀式
這一點與 LLM 標註的近期文獻一致:同一模型重複取樣量測的是
自我一致性(self-consistency),而重複執行後取多數決可提升
標註穩定度(如 Prompt Stability Scoring,arXiv:2407.02039)。
## 二、三個常用指標 ## 二、三個常用指標
@@ -62,7 +67,7 @@ Krippendorff 依產生資料的設計,把信度分成三型
## 三、實測結果 ## 三、實測結果
### 步驟 3a(編碼:883 首 × 101 個碼 = 89,183 格,執行三次) ### 步驟 3(編碼:883 首 × 101 個碼 = 89,183 格,執行三次)
| 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard | | 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard |
|---|---:|---:|---:| |---|---:|---:|---:|
@@ -79,7 +84,7 @@ Krippendorff 依產生資料的設計,把信度分成三型
(此處由原始輸出計算,未套用 936 筆人工校對表;論文引用的 (此處由原始輸出計算,未套用 936 筆人工校對表;論文引用的
定案數 14,664 為校對後的結果,差 1 筆。) 定案數 14,664 為校對後的結果,差 1 筆。)
### 步驟 5d(樣態標註:111 首 × 各自適用樣態 = 3,708 格,執行三次) ### 步驟 5-4(樣態標註:111 首 × 各自適用樣態 = 3,708 格,執行三次)
| 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard | | 兩次執行 | 百分比一致率 | Cohen's κ | Jaccard |
|---|---:|---:|---:| |---|---:|---:|---:|
@@ -101,16 +106,22 @@ Jaccard(只看「有標到」的格子,忽略雙方都沒標的)約 0.89–0.91,
比百分比一致率低而更誠實——因為 82% 的格子是雙方都判「無」, 比百分比一致率低而更誠實——因為 82% 的格子是雙方都判「無」,
那些一致並不費力。 那些一致並不費力。
## 四、隨機性的規模 ## 四、隨機性的規模與三票制的作用
信度數字回答的實際問題是:**這台儀器的隨機性,大到會不會 信度數字回答的實際問題是:**這台儀器的隨機性,大到會不會
改變結論?** 改變結論?**
步驟 3a 的 89,183 格中,搖擺的格子共 2,473 格(兩票 1,163、 步驟 3 的 89,183 格中,搖擺的格子共 2,473 格(兩票 1,163、
一票 1,310),佔 2.8%。若只執行一次即定案,這批格子當中約 一票 1,310),佔 2.8%。若只執行一次即定案,這批格子當中約
一半會成為誤收、另一半會成為漏收。三票多數決把兩票以上者 一半會成為誤收、另一半會成為漏收。三票多數決把兩票以上者
收入、一票者剔除,處理的正是這一批。 收入、一票者剔除,處理的正是這一批。
三票制並非消除隨機性,而是**把隨機性往案例原本的傾向推**。
設某個邊緣案例的符合程度為 p,單次執行以機率 p 標出,三次
多數決則以 p³+3p²(1p) 標出:p=0.9 者由 0.9 提高到 0.972,
p=0.1 者由 0.1 壓低到 0.028,而 p=0.5 者仍是 0.5——真正
模稜兩可的案例,任何票制都救不了。
以此規模判斷,結論層的三條帶狀結構(女性力量與陽剛群共現、 以此規模判斷,結論層的三條帶狀結構(女性力量與陽剛群共現、
與脆弱群互斥、與厭女群獨立)不可能由這個量級的雜訊翻轉。 與脆弱群互斥、與厭女群獨立)不可能由這個量級的雜訊翻轉。
反過來說,若一致率只有 0.6,同一組結論就不能採信——信度 反過來說,若一致率只有 0.6,同一組結論就不能採信——信度
@@ -135,8 +146,8 @@ Jaccard(只看「有標到」的格子,忽略雙方都沒標的)約 0.89–0.91,
## 六、計算方式與依據 ## 六、計算方式與依據
- **分析單位**:每一個「(歌曲, 編碼)」格為一個單位,是非題 - **分析單位**:每一個「(歌曲, 編碼)」格為一個單位,是非題
(有標/無標)。步驟 3a 為 883 首 × 101 碼 = 89,183 格; (有標/無標)。步驟 3 為 883 首 × 101 碼 = 89,183 格;
步驟 5d 為各歌適用樣態數合計 3,708 格。 步驟 5-4 為各歌適用樣態數合計 3,708 格。
- **Cohen's κ**:兩次執行的 2×2 表,κ=(p_op_e)/(1p_e), - **Cohen's κ**:兩次執行的 2×2 表,κ=(p_op_e)/(1p_e),
p_e 由兩次各自的邊際比例相乘求得。 p_e 由兩次各自的邊際比例相乘求得。
- **Krippendorff's α(名目、三位「編碼者」、無缺漏)**: - **Krippendorff's α(名目、三位「編碼者」、無缺漏)**:
+87 -23
View File
@@ -11,33 +11,81 @@
**不與先導研究做比較** **不與先導研究做比較**
全文數字一律以正式研究結果為準。先導研究僅作為假說的 全文數字一律以正式研究結果為準。先導研究僅作為假說的
內部來源,記於決策日誌,不進入論文敘事。 內部來源,記於決策日誌,不進入論文敘事。
- **執行原則**:主會話只做討論,分析一律由確定性程序 - **執行原則**:主會話只做討論;所有分析由 deterministic
執行。可逐項機械比對的判斷(步驟 3 編碼、步驟 4 選群、 script 執行。LLM 步驟以 Python script 呼叫 Anthropic
步驟 5d 樣態標註)採「三次獨立執行+多數決」定案, Messages API(個人 Console 帳號、Batch API 五折),定義
自由生成(步驟 1 自由標註)採兩次執行全數進池,步驟 檔逐字作為 system prompt。可逐項機械比對的判斷(步驟 3
5a 至 5c 定為質性閱讀協定;程序細節見 `methodology.md` 編碼、步驟 4 選群、步驟 5-4 樣態標註)採「同一定義檔
工作規範見 `conventions.md` 獨立執行三次+多數決」,計票由確定性子命令完成。自由
生成(步驟 1 自由標註)兩次執行全數進池。步驟 5-1 至
5-3 為質性閱讀協定:三次獨立閱讀為分析者三角檢核、
逐首整合、樣態統整產出草稿,不適用投票與仲裁。詞彙表
不經 LLM,由詞向量嵌入+確定性分群產生。驗證結果不符
預期則修訂定義檔重跑該循環,絕不手改結果。
- **提示詞只定格式、不定語意**:研究對象是通用 LLM 以其 - **提示詞只定格式、不定語意**:研究對象是通用 LLM 以其
網路語料知識背景所做的自然編碼與閱讀,其結果本身是 網路語料知識背景所做的自然編碼與閱讀,其結果本身是
批判對象。定義檔只規定任務形狀(輸入、數量範圍、輸出 批判對象。定義檔只規定任務形狀(輸入、數量範圍、輸出
格式),不給任何主題的定義、判準或範例。LLM 判斷一律 格式),不給任何主題的定義、判準或範例。LLM 判斷一律
要求逐項引述歌詞原句,作為檢視偏差的依據。 要求逐項引述歌詞原句,作為檢視偏差的依據。
- **模型**:步驟 1、3 用 `claude-sonnet-4-6`;步驟 4、5 - **模型**:步驟 1、3 用 `claude-sonnet-4-6`temperature=0、
`claude-fable-5`——實測發現 sonnet 將「Women Power」 thinking 關閉);步驟 4、5 用 `claude-fable-5`(兩參數
拆讀為 womenpower 的組合語意,fable-5 讀為詞彙化 不適用,均不送出;取樣變異由多數決或整合吸收)——實測
概念,語意層任務因此換用 fable-5(經過見決策日誌)。 發現 sonnet 將「Women Power」拆讀為 womenpower 的組合
語意,fable-5 讀為詞彙化概念,語意層任務因此換用
fable-5(經過見決策日誌)。
- **信度與效度**:三次獨立執行量測穩定性(intra-rater - **信度與效度**:三次獨立執行量測穩定性(intra-rater
reliability,可報告兩兩一致率);封閉母體全量檢查取代 reliability,可報告兩兩一致率);封閉母體全量檢查取代
抽樣防衛。可重現性定義為「程序透明+可稽核」:公開定義 抽樣防衛。可重現性定義為「程序透明+可稽核」:公開定義
檔、記錄 model ID 與執行時間、保存全部原始輸出。 檔、記錄 model ID 與執行時間、保存全部原始輸出。
## 範圍性定案 ## 資料儲存與模型
- **Commit 判準**:凡能由「committed 的輸入+committed 的
程式」決定性再生者,不 commit;凡不能者——源頭資料、
外部世界的捕捉(Wikidata 快照、LLM 原始輸出)、人工
著作——一律以文字格式 commit。格式跟著層次走。
- **分層**
- 源頭:原始榜單 CSV(進 git)。
- 捕捉:Wikidata 快照 CSV、`runs/` JSONL(皆進 git);
歌詞 `.txt` 快取(版權因素 gitignored,為已知的稽核
缺口)。
- 人工:`data/manual/`(僅研究者親手寫入:編碼修正、
演唱者性別修正)。
- 工作儲存:SQLite 單檔(`tools/instance/`generated、
不進 git),SQLAlchemy 2.0 typed ORM 定義 schema
設定經 pydantic-settings`.env` 供應
`SQLALCHEMY_DATABASE_URL``ANTHROPIC_API_KEY`)。
歌詞全文入 DB(不進 git 故無版權疑慮)。
- 衍生:`build-db` 建置工作儲存的同一動作產出人讀報表
`data/derived/`,進 git),與 SQLite 同交易語意。
- 報表:論文引用的定案表進 `results/`——
`codings.csv`(步驟 3 定案編碼)、`groups.csv`
(步驟 4 定案編碼群)、`patterns.csv`(步驟 5-3 定案
樣態表)、`annotations.csv`(步驟 5-4 定案歌×樣態
矩陣)。衍生與報表為「可再生仍 commit」的例外,理由:
引用穩定性、審稿人零門檻、撰稿期數字變動可 diff。
- **資料模型**`songs`(含 lyrics、`performer_gender`——
演唱聲音之性別,由署名藝人之 Wikidata 性別推導後套用
`data/manual/performer-gender-corrections.csv` 手工修正)、
`chart_entries``artists``song_artists``codings`
`groups`(語意編碼群)、`patterns`(深讀樣態)、
`annotations`(歌×樣態定案矩陣)。領域不變量(恰 1000
筆榜單、每歌至少一 primary 歌手等)檢查內建於
`build-db`,違規即建置失敗。
- **歌手背景防火牆**:歌手背景資料只進人工解讀階段, - **歌手背景防火牆**:歌手背景資料只進人工解讀階段,
**絕不進 LLM 輸入**——LLM 任務的 user message 維持 **絕不進 LLM 輸入**——LLM 任務的 user message 維持
歌詞-only 或管線中間產物-only,避免光環偏誤。 歌詞-only 或管線中間產物-only,避免光環偏誤。
- **Pilot 歌詞沿用**:先導歌詞沿用之來歷與細節詳見該檔 - **Pilot 歌詞沿用(私人匯入,不進發布管線)**:先導研究
`pilot-study.md`)。 捕捉檔(lyrics.json684 首,20182025)以 `excludes/`
的私人腳本匯入歌詞快取;讀者的重現路徑純粹是
`fetch-lyrics`;沿用之事實記於
`data/captures/lyrics-provenance.csv`(進 git)。
- **子命令**`pop-fem-audit-tools <cmd>`):`build-db`
`--codings``--groups``--gender-corrections`
`--patterns``--annotations`)、`cluster-keywords`
`export-llm-input``fetch-artists``fetch-lyrics`
`run-llm``--model` 於模型登錄表中擇一)、
`tally-codings``tally-groups``tally-annotations`
## 分析管線(五步驟,全部完成) ## 分析管線(五步驟,全部完成)
@@ -48,19 +96,26 @@
階層式聚合分群 k=100,組名取 medoid;再併入研究者先驗 階層式聚合分群 k=100,組名取 medoid;再併入研究者先驗
主題詞 `women-power`(據實揭露的儀器介入),共 101 碼。 主題詞 `women-power`(據實揭露的儀器介入),共 101 碼。
3. **編碼(步驟 3**:以定稿詞彙表對全 883 首編碼,逐 3. **編碼(步驟 3**:以定稿詞彙表對全 883 首編碼,逐
標籤附引述,×3+多數決,產出定案編碼表。「女性力量」 標籤附引述,×3+多數決`tally-codings`),定案
候選集(wp 66 首、fe 144 首、wpfe 145 首)由此浮現。 `results/codings.csv`。「女性力量」候選集(wp 66 首、
fe 144 首、wpfe 145 首)由此浮現。
4. **語意編碼群(步驟 4**:LLM 依編碼字面語意將 101 碼 4. **語意編碼群(步驟 4**:LLM 依編碼字面語意將 101 碼
選入研究者指定的四個主題群(women-powermisogyny 選入研究者指定的四個主題群(women-powermisogyny
masculinevulnerable),×3+多數決,產出定案分群表; masculinevulnerable),×3+多數決`tally-groups`),
wp/fe 與各編碼、各編碼群之關聯統計(BH-FDR 校正)入 定案 `results/groups.csv`wp/fe 與各編碼、各編碼群之
論文。 關聯統計(BH-FDR 校正)入論文。
5. **女性主義問題之質性深讀(步驟 5**:對 wpfe 145 首 5. **女性主義問題之質性深讀(步驟 5**:對 wpfe 145 首
——5a 逐首盲讀(僅歌詞全文)×35b 逐首整合(收斂 ——5-1 逐首盲讀(僅歌詞全文)×35-2 逐首整合(收斂
註記、主清單限兩讀以上);5c 樣態統整(全體基底+ 註記、主清單限兩讀以上);5-3 樣態統整(全體基底+
男聲/女聲/混合三個發話脈絡分組);5d 樣態標註—— 男聲/女聲/混合三個發話脈絡分組);5-4 樣態標註——
以分組樣態表逐首標註「有問題」的 111 首,×3+多數決 以分組樣態表逐首標註「有問題」的 111 首,×3+多數決
產出定案樣態表與歌×樣態定案矩陣。 `tally-annotations`),定案 `results/patterns.csv`
`results/annotations.csv`
定義檔命名 `prompts/<步>-<次步>-<task>.md`,編號的所指是
工序而非定義檔(確定性的第 2 步無定義檔仍佔編號);檔名
不帶版本號——版本即 git 歷史;每次執行的定義檔快照隨
`runs/` 自我完備,token 費用逐筆記於 `docs/run-costs.md`
## 時程與剩餘工作 ## 時程與剩餘工作
@@ -69,3 +124,12 @@
污染」結論(步驟 5 素材已備)、信度說明、fe 與新自由 污染」結論(步驟 5 素材已備)、信度說明、fe 與新自由
主義敘事之詮釋標註。 主義敘事之詮釋標註。
## 其他已定案事項
- 歌詞受版權保護:完整歌詞不進 git
`data/captures/lyrics/` gitignored),論文與 repo 只留
分析所引摘錄。
- 工程支援(寫 script、style check、審稿)用公司訂閱的
Claude Code;進論文的分析 token 由個人 API 帳號支付。
- Positionality statement 寫入論文(53 歲、長年婦運者、
資深工程師、離開學術圈多年、因整理榜單而起)。
+51 -55
View File
@@ -10,61 +10,57 @@ $3$15、opus-4-6 $5$25、opus-5 與 fable-5 $10$50
`--replace` 取代的執行以「已取代」標記,數字保留供 `--replace` 取代的執行以「已取代」標記,數字保留供
總支出核算。 總支出核算。
「步驟」欄的名稱於 2026-08-18 一律換為現行編號(見 | 日期 | 步驟 | 執行 | 模型 | 批次 ID | 耗時 | input | output | 費用 (USD) | 狀態 |
`decision-log.md` 當日條目),執行當時的名稱記於末欄 |---|---|---|---|---|---|---:|---:|---:|---|
「原步驟名」,未曾改名者留空。已刪除的工序——步驟 2 的 | 2026-08-04 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01PTACDQMr8M6ahnshbedjtB | 6 分 27 秒 | 763,318 | 338,661 | $3.68 | 已取代(浮水印清洗與防圍欄修訂後重跑) |
LLM 合併嘗試(`01-02-01-merge`,詞彙表改採詞向量分群時 | 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01TikJNd2pZVxzQ8SaybVthu | 4 分 57 秒 | 774,604 | 323,651 | $3.59 | 已取代(合法 JSON 修訂後重跑) |
放棄)與步驟 3 的仲裁(`03-02-arbitration`,改採三票 | 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01JFBCNqnu1cwXyqmEKLHQYF | 4 分 30 秒 | 790,370 | 328,227 | $3.65 | 已取代(song-288 平台失敗,整批重跑驗證) |
多數決時刪除)——不另發新名,步驟欄標「已廢棄」,僅存 | 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01VSDneWuSbf8mShA32jiWrX | 6 分 15 秒 | 790,370 | 326,193 | $3.63 | 已取代(措辭修訂後全體重跑) |
本表供總支出核算。 | 2026-08-05 | 01-01-01-tag | run1-rescue-288 | claude-sonnet-4-6 | msgbatch_019Hq6bNVXVmp4DjRZ2cgVda | 2 分 5 秒 | 1,015 | 376 | $0.01 | 已取代(措辭修訂後全體重跑,該首原生通過)|
| 2026-08-05 | 01-01-01-tag | run1 | claude-sonnet-4-6 | msgbatch_01VgZ77KAPGWmuu3PnQFqZ7Q | 7 分 1 秒 | 794,913 | 326,435 | $3.64 | 現行 |
| 2026-08-05 | 01-01-01-tag | run2 | claude-sonnet-4-6 | msgbatch_01TLFey3L4fimKxcebTZQGYn | 4 分 21 秒 | 794,913 | 328,324 | $3.65 | 現行 |
| 2026-08-05 | 01-02-01-merge | run1 | claude-sonnet-4-6 | msgbatch_01BvYMFmH8zrUq9SNxWSNba7 | 9 分 14 秒 | 46,454 | 60,974 | $0.53 | 已取代(完整分割驗證不過:漏 366、重複分派 511、撞名 4) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-5 | msgbatch_018TkYpdT3DsZqq2q94FQCUN | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-4-8 | msgbatch_018Wuux4adz5mWSvzJjfgLxb | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-4-6 | msgbatch_01Lmt2j1Txyof9CctUK7CbHA | 12 分 19 秒 | 46,454 | 54,864 | $0.80 | 已取代(圍欄違規;漏 190、發明 151、重複 11) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-5 | msgbatch_012SEfebTnjU5uWN6hhfnhhh | 8 分 45 秒 | 63,368 | 64,000 | $1.92 | 已取代(64k 截斷) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-opus-5 | msgbatch_01Wi9xEoQKCy1nZK6myVNg77 | 11 分 34 秒 | 63,368 | 70,175 | $2.07 | 已取代(驗證不過:漏 59、發明 140、重複 3) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-fable-5 | msgbatch_01M15hUs8P9FDTqWvAHdc1cA | 32 分 45 秒 | 63,368 | 107,721 | $3.01 | 現行歸檔(JSON 語法毀損,驗證不過) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-fable-5 | msgbatch_012stRP28uuQDQwobm41HMLh | — | 0 | 0 | $0.00 | 拒收(thinking.type.enabled 不支援) |
| 2026-08-05 | 01-02-01-merge | run1 | claude-fable-5 | msgbatch_01KpS7AMx1zAHTcNujgGsouJ | 26 分 28 秒 | 63,368 | 128,000 | $3.52 | 現行歸檔(effort max128k 全耗於推理,正文空白) |
| 2026-08-05 | 03-01-code | run1 | claude-sonnet-4-6 | msgbatch_01L7VepTxSNz5vruvzzjuL2c | 6 分 36 秒 | 1,247,264 | 492,730 | $5.57 | 36 首輸出遭內容過濾攔阻,待修訂重跑 |
| 2026-08-05 | 03-01-code | 引述減量實驗(36 首)| claude-sonnet-4-6 | msgbatch_01Hskdhi2DkudYmgZhhgFts7 | 3 分 51 秒 | 56,458 | 11,074 | $0.17 | 實驗:每碼一行引述,36 首全數通過過濾;歸檔不入 repo |
| 2026-08-05 | 03-01-code | run1 | claude-sonnet-4-6 | msgbatch_01GG5Ez9KT1pPwtQaY4sW7tv | 4 分 35 秒 | 1,293,724 | 269,166 | $3.96 | 過濾零攔阻;song-168、song-590 因餘額用盡失敗,song-775 拒答 |
| 2026-08-05 | 03-01-code | 101 碼樹狀探測(148 首)| claude-sonnet-4-6 | msgbatch_017jo3E5gWVks9b39iWMTqz3 | 2 小時 11 分 | 385,270 | 79,690 | $0.87 | 實驗:k=100 葉碼+women-power;零違規碼; 歸檔不入 repo |
| 2026-08-06 | 命名實驗(100 組)| — | claude-sonnet-4-6 | msgbatch_01XSi1YtWzdVyYUzYh7DQRWg | 3 分 5 秒 | 55,090 | 1,168 | $0.09 | 實驗:LLM 命名對照 medoid,未採用;歸檔不入 repo |
| 2026-08-06 | 命名實驗(100 組)| — | claude-fable-5 | msgbatch_01Y8SJj1h1ZuSQRErhqkvgZE | 3 分 2 秒 | 76,211 | 2,049 | $0.43 | 實驗:同上,加禁用 themes;未採用;歸檔不入 repo |
| 2026-08-06 | 03-01-code | run1 | claude-sonnet-4-6 | msgbatch_01GS1opvurvsf62oknnQxhtx | 3 分 46 秒 | 1,625,458 | 362,759 | $5.16 | 現行(101 碼;883 首全數有效,零攔阻) |
| 2026-08-06 | 03-01-code | run2 | claude-sonnet-4-6 | msgbatch_01CxnwNLWzZbRpZK7UAdpb8i | 5 分 22 秒 | 1,625,458 | 364,098 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) |
| 2026-08-06 | 03-02-arbitration | — | claude-sonnet-4-6 | msgbatch_019xcQXwrbwDGFc9nE5M8AjE | 4 分 0 秒 | 763,193 | 42,389 | $1.46 | 已取代(2 首遭內容過濾攔阻、13 首輸出夾帶散文;定義檔修訂後重跑) |
| 2026-08-06 | 03-02-arbitration | — | claude-sonnet-4-6 | msgbatch_01N7bDbXRSfAVUzzaj2thKeR | 4 分 20 秒 | 781,037 | 39,315 | $1.47 | 現行(644 首全數有效,零攔阻;保留 1,481/送裁 1,699 |
| 2026-08-06 | 03-code | run3 | claude-sonnet-4-6 | msgbatch_01KnkCaGETnFJrPddrxZTYHA | 6 分 26 秒 | 1,625,458 | 363,840 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) |
| 2026-08-14 | 04-group | run1 | claude-sonnet-4-6 | msgbatch_01UPNedog6feQzJ9WVfSAxBD | 1 分 1 秒 | 3,129 | 470 | $0.01 | 已取代(僅 3 群;改納 women-power 群後重跑) |
| 2026-08-14 | 04-group | run1 | claude-sonnet-4-6 | msgbatch_01KntJgxdicStaMjNL12P3zi | 1 分 25 秒 | 4,170 | 558 | $0.01 | 已取代(改以 claude-fable-5 執行;vulnerable 輸出含詞彙表外碼 1 筆;歸檔備份於 excludes/experiments/ |
| 2026-08-14 | 04-group | run1 | claude-fable-5 | msgbatch_01Mr6goBb2Efa4YrqCprbP4U | 55 秒 | 5,553 | 2,049 | $0.08 | 現行(4 群;零違規碼;temperature 與 thinking 參數不適用於本模型,未送出) |
| 2026-08-14 | 04-group | run2 | claude-fable-5 | msgbatch_01C17xW3YBefThTYZ83g7KkL | 1 分 21 秒 | 5,553 | 1,891 | $0.08 | 現行(4 群;零違規碼) |
| 2026-08-14 | 04-group | run3 | claude-fable-5 | msgbatch_01DveEMYyjCAYe6wxpcCD87V | 2 分 9 秒 | 5,553 | 1,975 | $0.08 | 現行(4 群;零違規碼) |
| 2026-08-15 | 05-01-read | run1 | claude-fable-5 | msgbatch_018NJejke75o7kzKyjZeeKwi | 2 分 12 秒 | 189,607 | 181,299 | $5.48 | 現行(144/145 有效;song-444 平台錯誤,單筆補送) |
| 2026-08-15 | 05-01-read | run1-rescue-444 | claude-fable-5 | msgbatch_01DgygMm6iYn2qEnyqHkTMfa | 1 分 52 秒 | 1,852 | 1,557 | $0.05 | 現行(run1 之 song-444 單筆補送,成功) |
| 2026-08-15 | 05-01-read | run2 | claude-fable-5 | msgbatch_011txWpWDtdkNXNiPA74ZUfQ | 15 分 25 秒 | 191,459 | 185,630 | $5.60 | 現行(145/145 有效) |
| 2026-08-15 | 05-01-read | run3 | claude-fable-5 | msgbatch_01Unn7C7WCnuVoLraJRXksoG | 7 分 59 秒 | 191,459 | 190,433 | $5.72 | 現行(145/145 有效) |
| 2026-08-15 | 05-02-consolidate | run1 | claude-fable-5 | msgbatch_01WPyH4pCCiieHs4yU3xtHzN | 2 分 28 秒 | 326,287 | 242,785 | $7.70 | 現行(145/145 有效) |
| 2026-08-15 | 05-03-synthesize | run1 | claude-fable-5 | msgbatch_01XbNQb1SFrUxhk2ZWF7epPA | 3 分 0 秒 | 103,888 | 9,982 | $0.77 | 現行(15 個樣態;研究者審定用草稿) |
| 日期 | 步驟 | 執行 | 模型 | 批次 ID | 耗時 | input | output | 費用 (USD) | 狀態 | 原步驟名 | | 2026-08-15 | 05-03-synthesize | run2 | claude-fable-5 | msgbatch_01FKqq8U6m5HLLsx7KpQES26 | 3 分 4 秒 | 13,573 | 6,883 | $0.24 | 現行(男聲群 14 首;13 個樣態) |
|---|---|---|---|---|---|---:|---:|---:|---|---| | 2026-08-15 | 05-03-synthesize | run3 | claude-fable-5 | msgbatch_017tfhiRTnaxqCp6YFyoivsu | 4 分 4 秒 | 60,727 | 8,519 | $0.52 | 現行(女聲群 98 首;14 個樣態) |
| 2026-08-04 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01PTACDQMr8M6ahnshbedjtB | 627 秒 | 763,318 | 338,661 | $3.68 | 已取代(浮水印清洗與防圍欄修訂後重跑) | 01-01-01-tag | | 2026-08-15 | 05-03-synthesize | run4 | claude-fable-5 | msgbatch_01SNjTLiNPEba4LW2tcUAcTJ | 43 秒 | 29,958 | 9,793 | $0.39 | 現行(混合群 30 首;16 個樣態) |
| 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01TikJNd2pZVxzQ8SaybVthu | 4 分 57 秒 | 774,604 | 323,651 | $3.59 | 已取代(合法 JSON 修訂後重跑) | 01-01-01-tag |
| 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01JFBCNqnu1cwXyqmEKLHQYF | 430 秒 | 790,370 | 328,227 | $3.65 | 已取代(song-288 平台失敗,整批重跑驗證) | 01-01-01-tag | | 2026-08-15 | 05-04-annotate | run1 | claude-fable-5 | msgbatch_01H9gXXhNbhZG6gzZtvMHxhh | 39 秒 | 815,492 | 300,901 | $11.60 | 現行(109/111 有效;song-177、song-199 遭 max_tokens 截斷,單筆補送) |
| 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01VSDneWuSbf8mShA32jiWrX | 615 秒 | 790,370 | 326,193 | $3.63 | 已取代(措辭修訂後全體重跑) | 01-01-01-tag | | 2026-08-15 | 05-04-annotate | run1-rescue-177-199 | claude-fable-5 | msgbatch_01YEdroZ51b8btPfG1aqPFCD | 3 分 5 秒 | 17,802 | 11,601 | $0.38 | 現行(run1 之 2 筆補送,max_tokens 提為 16000,成功) |
| 2026-08-05 | 1-tag | run1-rescue-288 | claude-sonnet-4-6 | msgbatch_019Hq6bNVXVmp4DjRZ2cgVda | 25 秒 | 1,015 | 376 | $0.01 | 已取代(措辭修訂後全體重跑,該首原生通過)| 01-01-01-tag | | 2026-08-15 | 05-04-annotate | — | claude-fable-5 | msgbatch_01G67STqZsHpnhedat2HRU85 | 30 秒 | 17,802 | 14,162 | $0.44 | 誤送(補送批次輪詢中斷後誤判死亡而重送;原批次自行完成並歸檔為現行,本批次取消不及、兩筆皆完成,結果棄用) |
| 2026-08-05 | 1-tag | run1 | claude-sonnet-4-6 | msgbatch_01VgZ77KAPGWmuu3PnQFqZ7Q | 7 分 1 秒 | 794,913 | 326,435 | $3.64 | 現行 | 01-01-01-tag |
| 2026-08-05 | 1-tag | run2 | claude-sonnet-4-6 | msgbatch_01TLFey3L4fimKxcebTZQGYn | 421 秒 | 794,913 | 328,324 | $3.65 | 現行 | 01-01-01-tag | | 2026-08-15 | 05-04-annotate | run2 | claude-fable-5 | msgbatch_01VVJpsfidNtF66GnHQdWnGR | 9 分 16 秒 | 815,492 | 306,582 | $11.74 | 現行(111/111 有效;max_tokens 16000,零截斷) |
| 2026-08-05 | 已廢棄 | run1 | claude-sonnet-4-6 | msgbatch_01BvYMFmH8zrUq9SNxWSNba7 | 9 分 14 秒 | 46,454 | 60,974 | $0.53 | 已取代(完整分割驗證不過:漏 366、重複分派 511、撞名 4 | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-opus-5 | msgbatch_018TkYpdT3DsZqq2q94FQCUN | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) | 01-02-01-merge | | 2026-08-15 | 05-04-annotate | run3 | claude-fable-5 | msgbatch_01AsYvZd8rsTYVfUYLzWfeMK | 4 分 12 秒 | 815,492 | 298,512 | $11.54 | 現行(111/111 有效;max_tokens 16000,零截斷) |
| 2026-08-05 | 已廢棄 | run1 | claude-opus-4-8 | msgbatch_018Wuux4adz5mWSvzJjfgLxb | — | 0 | 0 | $0.00 | 拒收(temperature 已棄用) | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-opus-4-6 | msgbatch_01Lmt2j1Txyof9CctUK7CbHA | 12 分 19 秒 | 46,454 | 54,864 | $0.80 | 已取代(圍欄違規;漏 190、發明 151、重複 11 | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-opus-5 | msgbatch_012SEfebTnjU5uWN6hhfnhhh | 8 分 45 秒 | 63,368 | 64,000 | $1.92 | 已取代(64k 截斷) | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-opus-5 | msgbatch_01Wi9xEoQKCy1nZK6myVNg77 | 11 分 34 秒 | 63,368 | 70,175 | $2.07 | 已取代(驗證不過:漏 59、發明 140、重複 3 | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-fable-5 | msgbatch_01M15hUs8P9FDTqWvAHdc1cA | 32 分 45 秒 | 63,368 | 107,721 | $3.01 | 現行歸檔(JSON 語法毀損,驗證不過) | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-fable-5 | msgbatch_012stRP28uuQDQwobm41HMLh | — | 0 | 0 | $0.00 | 拒收(thinking.type.enabled 不支援) | 01-02-01-merge |
| 2026-08-05 | 已廢棄 | run1 | claude-fable-5 | msgbatch_01KpS7AMx1zAHTcNujgGsouJ | 26 分 28 秒 | 63,368 | 128,000 | $3.52 | 現行歸檔(effort max:128k 全耗於推理,正文空白) | 01-02-01-merge |
| 2026-08-05 | 3a-code | run1 | claude-sonnet-4-6 | msgbatch_01L7VepTxSNz5vruvzzjuL2c | 6 分 36 秒 | 1,247,264 | 492,730 | $5.57 | 36 首輸出遭內容過濾攔阻,待修訂重跑 | 03-01-code |
| 2026-08-05 | 3a-code | 引述減量實驗(36 首)| claude-sonnet-4-6 | msgbatch_01Hskdhi2DkudYmgZhhgFts7 | 3 分 51 秒 | 56,458 | 11,074 | $0.17 | 實驗:每碼一行引述,36 首全數通過過濾;歸檔不入 repo | 03-01-code |
| 2026-08-05 | 3a-code | run1 | claude-sonnet-4-6 | msgbatch_01GG5Ez9KT1pPwtQaY4sW7tv | 4 分 35 秒 | 1,293,724 | 269,166 | $3.96 | 過濾零攔阻;song-168、song-590 因餘額用盡失敗,song-775 拒答 | 03-01-code |
| 2026-08-05 | 3a-code | 101 碼樹狀探測(148 首)| claude-sonnet-4-6 | msgbatch_017jo3E5gWVks9b39iWMTqz3 | 2 小時 11 分 | 385,270 | 79,690 | $0.87 | 實驗:k=100 葉碼+women-power;零違規碼; 歸檔不入 repo | 03-01-code |
| 2026-08-06 | 命名實驗(100 組)| — | claude-sonnet-4-6 | msgbatch_01XSi1YtWzdVyYUzYh7DQRWg | 3 分 5 秒 | 55,090 | 1,168 | $0.09 | 實驗:LLM 命名對照 medoid,未採用;歸檔不入 repo | |
| 2026-08-06 | 命名實驗(100 組)| — | claude-fable-5 | msgbatch_01Y8SJj1h1ZuSQRErhqkvgZE | 3 分 2 秒 | 76,211 | 2,049 | $0.43 | 實驗:同上,加禁用 themes;未採用;歸檔不入 repo | |
| 2026-08-06 | 3a-code | run1 | claude-sonnet-4-6 | msgbatch_01GS1opvurvsf62oknnQxhtx | 3 分 46 秒 | 1,625,458 | 362,759 | $5.16 | 現行(101 碼;883 首全數有效,零攔阻) | 03-01-code |
| 2026-08-06 | 3a-code | run2 | claude-sonnet-4-6 | msgbatch_01CxnwNLWzZbRpZK7UAdpb8i | 5 分 22 秒 | 1,625,458 | 364,098 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) | 03-01-code |
| 2026-08-06 | 已廢棄 | — | claude-sonnet-4-6 | msgbatch_019xcQXwrbwDGFc9nE5M8AjE | 4 分 0 秒 | 763,193 | 42,389 | $1.46 | 已取代(2 首遭內容過濾攔阻、13 首輸出夾帶散文;定義檔修訂後重跑) | 03-02-arbitration |
| 2026-08-06 | 已廢棄 | — | claude-sonnet-4-6 | msgbatch_01N7bDbXRSfAVUzzaj2thKeR | 4 分 20 秒 | 781,037 | 39,315 | $1.47 | 現行(644 首全數有效,零攔阻;保留 1,481/送裁 1,699 | 03-02-arbitration |
| 2026-08-06 | 3a-code | run3 | claude-sonnet-4-6 | msgbatch_01KnkCaGETnFJrPddrxZTYHA | 6 分 26 秒 | 1,625,458 | 363,840 | $5.17 | 現行(101 碼;883 首全數有效,零攔阻) | 03-code |
| 2026-08-14 | 4-group | run1 | claude-sonnet-4-6 | msgbatch_01UPNedog6feQzJ9WVfSAxBD | 1 分 1 秒 | 3,129 | 470 | $0.01 | 已取代(僅 3 群;改納 women-power 群後重跑) | 04-group |
| 2026-08-14 | 4-group | run1 | claude-sonnet-4-6 | msgbatch_01KntJgxdicStaMjNL12P3zi | 1 分 25 秒 | 4,170 | 558 | $0.01 | 已取代(改以 claude-fable-5 執行;vulnerable 輸出含詞彙表外碼 1 筆;歸檔另行私人備份,不入版本庫) | 04-group |
| 2026-08-14 | 4-group | run1 | claude-fable-5 | msgbatch_01Mr6goBb2Efa4YrqCprbP4U | 55 秒 | 5,553 | 2,049 | $0.08 | 現行(4 群;零違規碼;temperature 與 thinking 參數不適用於本模型,未送出) | 04-group |
| 2026-08-14 | 4-group | run2 | claude-fable-5 | msgbatch_01C17xW3YBefThTYZ83g7KkL | 1 分 21 秒 | 5,553 | 1,891 | $0.08 | 現行(4 群;零違規碼) | 04-group |
| 2026-08-14 | 4-group | run3 | claude-fable-5 | msgbatch_01DveEMYyjCAYe6wxpcCD87V | 2 分 9 秒 | 5,553 | 1,975 | $0.08 | 現行(4 群;零違規碼) | 04-group |
| 2026-08-15 | 5a-read | run1 | claude-fable-5 | msgbatch_018NJejke75o7kzKyjZeeKwi | 2 分 12 秒 | 189,607 | 181,299 | $5.48 | 現行(144/145 有效;song-444 平台錯誤,單筆補送) | 05-01-read |
| 2026-08-15 | 5a-read | run1-rescue-444 | claude-fable-5 | msgbatch_01DgygMm6iYn2qEnyqHkTMfa | 1 分 52 秒 | 1,852 | 1,557 | $0.05 | 現行(run1 之 song-444 單筆補送,成功) | 05-01-read |
| 2026-08-15 | 5a-read | run2 | claude-fable-5 | msgbatch_011txWpWDtdkNXNiPA74ZUfQ | 15 分 25 秒 | 191,459 | 185,630 | $5.60 | 現行(145/145 有效) | 05-01-read |
| 2026-08-15 | 5a-read | run3 | claude-fable-5 | msgbatch_01Unn7C7WCnuVoLraJRXksoG | 7 分 59 秒 | 191,459 | 190,433 | $5.72 | 現行(145/145 有效) | 05-01-read |
| 2026-08-15 | 5b-consolidate | run1 | claude-fable-5 | msgbatch_01WPyH4pCCiieHs4yU3xtHzN | 2 分 28 秒 | 326,287 | 242,785 | $7.70 | 現行(145/145 有效) | 05-02-consolidate |
| 2026-08-15 | 5c-synthesize | run1 | claude-fable-5 | msgbatch_01XbNQb1SFrUxhk2ZWF7epPA | 3 分 0 秒 | 103,888 | 9,982 | $0.77 | 現行(15 個樣態;研究者審定用草稿) | 05-03-synthesize |
| 2026-08-15 | 5c-synthesize | run2 | claude-fable-5 | msgbatch_01FKqq8U6m5HLLsx7KpQES26 | 3 分 4 秒 | 13,573 | 6,883 | $0.24 | 現行(男聲群 14 首;13 個樣態) | 05-03-synthesize |
| 2026-08-15 | 5c-synthesize | run3 | claude-fable-5 | msgbatch_017tfhiRTnaxqCp6YFyoivsu | 4 分 4 秒 | 60,727 | 8,519 | $0.52 | 現行(女聲群 98 首;14 個樣態) | 05-03-synthesize |
| 2026-08-15 | 5c-synthesize | run4 | claude-fable-5 | msgbatch_01SNjTLiNPEba4LW2tcUAcTJ | 4 分 3 秒 | 29,958 | 9,793 | $0.39 | 現行(混合群 30 首;16 個樣態) | 05-03-synthesize |
| 2026-08-15 | 5d-annotate | run1 | claude-fable-5 | msgbatch_01H9gXXhNbhZG6gzZtvMHxhh | 3 分 9 秒 | 815,492 | 300,901 | $11.60 | 現行(109/111 有效;song-177、song-199 遭 max_tokens 截斷,單筆補送) | 05-04-annotate |
| 2026-08-15 | 5d-annotate | run1-rescue-177-199 | claude-fable-5 | msgbatch_01YEdroZ51b8btPfG1aqPFCD | 3 分 5 秒 | 17,802 | 11,601 | $0.38 | 現行(run1 之 2 筆補送,max_tokens 提為 16000,成功) | 05-04-annotate |
| 2026-08-15 | 5d-annotate | — | claude-fable-5 | msgbatch_01G67STqZsHpnhedat2HRU85 | 3 分 0 秒 | 17,802 | 14,162 | $0.44 | 誤送(補送批次輪詢中斷後誤判死亡而重送;原批次自行完成並歸檔為現行,本批次取消不及、兩筆皆完成,結果棄用) | 05-04-annotate |
| 2026-08-15 | 5d-annotate | run2 | claude-fable-5 | msgbatch_01VVJpsfidNtF66GnHQdWnGR | 9 分 16 秒 | 815,492 | 306,582 | $11.74 | 現行(111/111 有效;max_tokens 16000,零截斷) | 05-04-annotate |
| 2026-08-15 | 5d-annotate | run3 | claude-fable-5 | msgbatch_01AsYvZd8rsTYVfUYLzWfeMK | 4 分 12 秒 | 815,492 | 298,512 | $11.54 | 現行(111/111 有效;max_tokens 16000,零截斷) | 05-04-annotate |
累計支出:$125.65。 累計支出:$125.65。
Can't render this file because it is too large.
-10
View File
@@ -1,10 +0,0 @@
Change Log
==========
version 1.0.0
-------------
Released 2026/8/19
Initial release.
-2
View File
@@ -13,8 +13,6 @@ This is a collection of supporting tools for the conference paper "流行音樂
:maxdepth: 2 :maxdepth: 2
:caption: Contents: :caption: Contents:
changelog
Indices and tables Indices and tables
================== ==================
+1 -1
View File
@@ -6,5 +6,5 @@
"""Tools for A Feminist Audit of Pop Music.""" """Tools for A Feminist Audit of Pop Music."""
VERSION: str = "1.0.0" VERSION: str = "0.0.0"
"""The package version.""" """The package version."""
File diff suppressed because it is too large Load Diff
@@ -24,7 +24,8 @@ keyword set for ``export-llm-input --extras`` is written as a JSON
file holding the group name keywords plus every extra a-priori file holding the group name keywords plus every extra a-priori
keyword the caller gives with the repeatable ``--extra-keyword`` keyword the caller gives with the repeatable ``--extra-keyword``
command-line option, as command-line option, as
:attr:`KeywordsToMerge.KEYWORDS_TO_MERGE_JSON`. No default :attr:`KeywordsToMerge.KEYWORDS_TO_MERGE_JSON`; with no
``--extra-keyword``, it holds the group names alone. No default
extra keyword is ever injected; the caller supplies each one extra keyword is ever injected; the caller supplies each one
consciously. Finally, the command-line choices and the consciously. Finally, the command-line choices and the
environment that produced the numbers -- neither recoverable from environment that produced the numbers -- neither recoverable from
@@ -159,8 +160,15 @@ class KeywordPooler:
if line.strip() == "": if line.strip() == "":
continue continue
record: Any = json.loads(line) record: Any = json.loads(line)
if not isinstance(record, dict) or "id" not in record:
raise ValueError(
f"{path}: record without \"id\": {line}")
if "error" in record: if "error" in record:
continue continue
if "text" not in record:
raise ValueError(
f"{path}: id {record['id']}: record without"
" \"text\" or \"error\"")
song_id: int = cls.__parse_song_id(record["id"], path) song_id: int = cls.__parse_song_id(record["id"], path)
try: try:
keywords: Any = json.loads( keywords: Any = json.loads(
@@ -289,8 +297,7 @@ class KeywordGroups:
class KeywordClusterer: class KeywordClusterer:
"""The clusterer of the pooled keywords into coding groups.""" """The clusterer of the pooled keywords into coding groups."""
DEFAULT_MODEL: ClassVar[str] \ DEFAULT_MODEL: str = "sentence-transformers/all-mpnet-base-v2"
= "sentence-transformers/all-mpnet-base-v2"
"""The sentence embedding model used when the caller names """The sentence embedding model used when the caller names
none.""" none."""
@@ -661,7 +668,11 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser.add_argument( parser.add_argument(
"output_dir", type=Path, "output_dir", type=Path,
help="the output directory, created if missing, that" help="the output directory, created if missing, that"
" receives the run's output artifacts") f" receives {PooledKeywords.SOURCE_KEYWORDS_TXT},"
f" {KeywordGroups.RESULT_KEYWORDS_TXT},"
f" {KeywordGroups.RESULT_GROUPS_CSV},"
f" {KeywordsToMerge.KEYWORDS_TO_MERGE_JSON},"
f" and {RunMeta.META_JSON}")
parser.add_argument( parser.add_argument(
"--model", default=model, "--model", default=model,
help=f"the sentence embedding model (default \"{model}\")") help=f"the sentence embedding model (default \"{model}\")")
@@ -687,9 +698,16 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
"""Pool the two tagging runs' keywords and cluster them. """Pool the two tagging runs' keywords and cluster them.
Creates the output directory (with parents) if it does not Writes the five fixed-named artifacts under the output
exist. Each output artifact is written as soon as its directory, creating it (with parents) if it does not exist:
content is computed, so when the input is rejected, or an the pooled keyword text file; then the group membership CSV
file, holding the clustering result alone; the group name
keyword text file, holding the same group names as a readable
list; the coding keyword set JSON file, holding the group
names plus every extra keyword given via ``--extra-keyword``;
and the run metadata JSON file, recording the command-line
choices and the environment. Each file is written as soon as
its content is computed, so when the input is rejected, or an
extra keyword duplicates a group name or another extra extra keyword duplicates a group name or another extra
keyword, the output directory holds whatever the steps before keyword, the output directory holds whatever the steps before
the failing one produced, and the error message names what the failing one produced, and the error message names what
@@ -701,8 +719,8 @@ def main(argv: list[str] | None = None) -> int:
""" """
started: float = time.monotonic() started: float = time.monotonic()
args: argparse.Namespace = parse_args(argv) args: argparse.Namespace = parse_args(argv)
args.output_dir.mkdir(parents=True, exist_ok=True)
try: try:
args.output_dir.mkdir(parents=True, exist_ok=True)
source: PooledKeywords = KeywordPooler( source: PooledKeywords = KeywordPooler(
args.run_dir_1, args.run_dir_2, args.output_dir).run() args.run_dir_1, args.run_dir_2, args.output_dir).run()
clusters: KeywordGroups = KeywordClusterer( clusters: KeywordGroups = KeywordClusterer(
@@ -717,7 +735,7 @@ def main(argv: list[str] | None = None) -> int:
f"Done. Clustered {len(source.keywords)} keywords into" f"Done. Clustered {len(source.keywords)} keywords into"
f" {len(clusters.names)}. {elapsed} elapsed.", f" {len(clusters.names)}. {elapsed} elapsed.",
file=sys.stderr) file=sys.stderr)
except (ClusterError, OSError) as error: except ClusterError as error:
print(f"error: {error}", file=sys.stderr) print(f"error: {error}", file=sys.stderr)
return 1 return 1
return 0 return 0
@@ -11,17 +11,26 @@ project's lyrics-only firewall: the output carries only the
lyrics text of each song, identified by an opaque song key; no lyrics text of each song, identified by an opaque song key; no
title, artist, or chart data crosses into the LLM input. title, artist, or chart data crosses into the LLM input.
With ``--extras`` and ``--extras-per-id``, a record's content may With ``--extras``, each record's ``content`` becomes a JSON
carry extra parameters alongside the lyrics, so a step that needs object serialized as a string, its ``lyrics`` key holding the
them can get them without this module knowing what they mean; see song's lyrics followed by the keys of the given extras file in
the exporter's content-building step for how the two merge. their file order, so a step that needs parameters alongside the
lyrics can carry them without this module knowing what they mean.
With ``--extras-per-id``, the same merge happens per song: the
given file maps a song ID to the extra keys of that one song, and
the export is restricted to the song IDs the file names, so a step
that revisits only some of the songs, each with its own parameters,
gets exactly those records. The two options may be given together,
in which case a record's keys are ``lyrics``, the shared extras'
keys, then that song's own keys, each group in its file order.
""" """
import argparse import argparse
import json import json
import sys import sys
import time import time
from pathlib import Path from pathlib import Path
from typing import Any, ClassVar from typing import Any
import sqlalchemy as sa import sqlalchemy as sa
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -31,270 +40,6 @@ from ..models import Song
from ..utils import format_duration from ..utils import format_duration
class LlmInputExporter:
"""The exporter of the LLM input JSONL file."""
__LYRICS_KEY: ClassVar[str] = "lyrics"
"""The key holding the lyrics in a record's merged content,
and the key forbidden in an extras file."""
def __init__(
self, output_jsonl: Path, extras: Path | None = None,
extras_per_id: Path | None = None) -> None:
"""Set up the exporter of the LLM input JSONL file.
:param output_jsonl: The JSONL output file.
:param extras: The extras JSON file, or None for none.
:param extras_per_id: The per-ID extras JSON file, or
None for none.
"""
self.__output_jsonl: Path = output_jsonl
"""The JSONL output file."""
self.__extras_path: Path | None = extras
"""The extras JSON file, or None for none."""
self.__extras_per_id_path: Path | None = extras_per_id
"""The per-ID extras JSON file, or None for none."""
def run(self) -> int:
"""Export the songs' lyrics to the output JSONL file.
:return: The number of songs exported.
:raises OSError: When a file cannot be read or written.
:raises sqlalchemy.exc.SQLAlchemyError: When the working
store cannot be read.
:raises ValueError: When an extras file is malformed, an
exported song has no lyrics, or the per-ID extras
name a song the working store does not have.
"""
session: Session = ds.get_db()
try:
extras: dict[str, Any] | None = None
if self.__extras_path is not None:
extras = self.__load_extras(self.__extras_path)
extras_per_id: dict[str, dict[str, Any]] | None = None
if self.__extras_per_id_path is not None:
extras_per_id = self.__load_extras_per_id(
self.__extras_per_id_path)
lines: list[str] = self.__build_lines(
session, extras, extras_per_id)
finally:
session.close()
self.__write_output(lines)
return len(lines)
@staticmethod
def __no_duplicate_keys(
pairs: list[tuple[str, Any]]) -> dict[str, Any]:
"""Build a dict from JSON object pairs, rejecting
duplicates.
:param pairs: The key-value pairs of a JSON object, in
file order.
:return: The pairs as a dict, in file order.
:raises ValueError: When a key appears more than once.
"""
result: dict[str, Any] = {}
key: str
value: Any
for key, value in pairs:
if key in result:
raise ValueError(
f"duplicate key \"{key}\" in extras")
result[key] = value
return result
@classmethod
def __load_json_object(
cls, path: Path, label: str) -> dict[str, Any]:
"""Load a single JSON object from a file, in file order.
:param path: The JSON file.
:param label: The kind of file, for the error messages.
:return: The object, in file order.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is
not a JSON object, or has duplicate keys.
"""
with open(path, encoding="utf-8") as file:
text: str = file.read()
try:
data: Any = json.loads(
text, object_pairs_hook=cls.__no_duplicate_keys)
except json.JSONDecodeError as error:
raise ValueError(
f"invalid JSON in {label} file {path}: {error}") \
from error
if not isinstance(data, dict):
raise ValueError(
f"{label} file {path} must contain a JSON object")
return data
@classmethod
def __load_extras(cls, path: Path) -> dict[str, Any]:
"""Load the extras object from a JSON file.
:param path: The extras JSON file.
:return: The extras, in file order.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is
not a JSON object, has duplicate keys, or has a
"lyrics" key.
"""
data: dict[str, Any] = cls.__load_json_object(
path, "extras")
if cls.__LYRICS_KEY in data:
raise ValueError(
f"extras file {path} must not have a"
f" \"{cls.__LYRICS_KEY}\" key")
return data
@classmethod
def __load_extras_per_id(
cls, path: Path) -> dict[str, dict[str, Any]]:
"""Load the per-ID extras object from a JSON file.
:param path: The per-ID extras JSON file, mapping a song
ID, as ``song-<N>``, to the extras of that one song.
:return: The extras of each song ID, in file order, every
song's own extras in their file order too.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is
not a JSON object, has duplicate keys, has a song
whose value is not a JSON object, or has a song with
a "lyrics" key.
"""
data: dict[str, Any] = cls.__load_json_object(
path, "per-ID extras")
song_id: str
extras: Any
for song_id, extras in data.items():
if not isinstance(extras, dict):
raise ValueError(
f"per-ID extras file {path}: id {song_id}"
" must have a JSON object")
if cls.__LYRICS_KEY in extras:
raise ValueError(
f"per-ID extras file {path}: id {song_id}"
f" must not have a \"{cls.__LYRICS_KEY}\""
" key")
return data
def __build_lines(
self, session: Session,
extras: dict[str, Any] | None = None,
extras_per_id: dict[str, dict[str, Any]] | None
= None) -> list[str]:
"""Build the JSONL lines of the exported songs' lyrics.
Every song is exported, unless per-ID extras are given,
in which case only the songs they name are; see
:meth:`__build_content` for how the extras merge into a
record's content.
:param session: The database session.
:param extras: The extra parameters merged into every
record's content alongside the lyrics, in the order
they are to appear, or None for none.
:param extras_per_id: The extra parameters merged into
the content of one record alone, keyed by that
record's song ID and in the order they are to appear,
restricting the export to the song IDs they name, or
None for no such extras and no such restriction.
:return: The JSON lines, one per exported song, ordered by
song ID.
:raises ValueError: When an exported song has no lyrics,
or the per-ID extras name a song the working store
does not have.
"""
lines: list[str] = []
exported: set[str] = set()
song: Song
for song in session.scalars(
sa.select(Song).order_by(Song.id)):
song_id: str = f"song-{song.id}"
if extras_per_id is not None \
and song_id not in extras_per_id:
continue
if song.lyrics is None:
raise ValueError(
f"song {song.id} \"{song.title}\": no lyrics")
song_extras: dict[str, Any] | None = None \
if extras_per_id is None \
else extras_per_id[song_id]
content: str = self.__build_content(
song.lyrics, extras, song_extras)
record: dict[str, str] = {
"id": song_id, "content": content}
lines.append(json.dumps(record, ensure_ascii=False))
exported.add(song_id)
if extras_per_id is not None:
missing: list[str] = sorted(
set(extras_per_id) - exported)
if len(missing) > 0:
raise ValueError(
"the per-ID extras name songs the working"
f" store does not have: {', '.join(missing)}")
return lines
@classmethod
def __build_content(
cls, lyrics: str, extras: dict[str, Any] | None,
song_extras: dict[str, Any] | None) -> str:
"""Build the content of one exported record.
Without extras of either kind, a record's content is the
bare lyrics string. With ``--extras``, the content
becomes a JSON object serialized as a string, its
"lyrics" key holding the song's lyrics followed by the
keys of the given extras file, in their file order. With
``--extras-per-id``, the same merge happens per song: the
song's own extra keys follow the lyrics instead. When
both are given, a record's keys are "lyrics", the shared
extras' keys, then that song's own keys, each group in
its file order.
:param lyrics: The lyrics of the song.
:param extras: The extra parameters shared by every
record, in the order they are to appear, or None for
none.
:param song_extras: The extra parameters of this record
alone, in the order they are to appear, or None for
none.
:return: The bare lyrics when there are no extras of
either kind, or otherwise a JSON object serialized as
a string, whose first key is "lyrics" holding the
lyrics, followed by the shared extras' keys and then
this record's own keys, each group in its given
order.
"""
if extras is None and song_extras is None:
return lyrics
payload: dict[str, Any] = {cls.__LYRICS_KEY: lyrics}
if extras is not None:
payload.update(extras)
if song_extras is not None:
payload.update(song_extras)
return json.dumps(payload, ensure_ascii=False)
def __write_output(self, lines: list[str]) -> None:
"""Write the exported lines to the output JSONL file.
Creates the parent directory when it does not exist.
:param lines: The JSONL lines, in the output order.
:return: None.
:raises OSError: When the file cannot be written.
"""
self.__output_jsonl.parent.mkdir(
parents=True, exist_ok=True)
with open(
self.__output_jsonl, "w",
encoding="utf-8") as file:
line: str
for line in lines:
file.write(line + "\n")
def parse_args(argv: list[str] | None) -> argparse.Namespace: def parse_args(argv: list[str] | None) -> argparse.Namespace:
"""Parse the command-line arguments. """Parse the command-line arguments.
@@ -311,33 +56,227 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser.add_argument( parser.add_argument(
"--extras", type=Path, default=None, "--extras", type=Path, default=None,
help="a JSON file holding a single JSON object of extra" help="a JSON file holding a single JSON object of extra"
" parameters merged into every record's content") " parameters; when given, each record's \"content\""
" becomes a JSON object string with a \"lyrics\" key"
" followed by the extras' keys, instead of the bare"
" lyrics string")
parser.add_argument( parser.add_argument(
"--extras-per-id", type=Path, default=None, "--extras-per-id", type=Path, default=None,
help="a JSON file holding a single JSON object that maps a" help="a JSON file holding a single JSON object that maps a"
" song ID, as \"song-<N>\", to a JSON object of extra" " song ID, as \"song-<N>\", to a JSON object of extra"
" parameters for that one song, restricting the" " parameters for that one song; the song's object is"
" export to the song IDs the file names") " merged into its \"content\" the same way as with"
" --extras, and the export is restricted to the song"
" IDs the file names")
return parser.parse_args(argv) return parser.parse_args(argv)
def __no_duplicate_keys(
pairs: list[tuple[str, Any]]) -> dict[str, Any]:
"""Build a dict from JSON object pairs, rejecting duplicates.
:param pairs: The key-value pairs of a JSON object, in file
order.
:return: The pairs as a dict, in file order.
:raises ValueError: When a key appears more than once.
"""
result: dict[str, Any] = {}
key: str
value: Any
for key, value in pairs:
if key in result:
raise ValueError(f"duplicate key \"{key}\" in extras")
result[key] = value
return result
def __load_json_object(path: Path, label: str) -> dict[str, Any]:
"""Load a single JSON object from a file, in file order.
:param path: The JSON file.
:param label: The kind of file, for the error messages.
:return: The object, in file order.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is not
a JSON object, or has duplicate keys.
"""
with open(path, encoding="utf-8") as file:
text: str = file.read()
try:
data: Any = json.loads(
text, object_pairs_hook=__no_duplicate_keys)
except json.JSONDecodeError as error:
raise ValueError(
f"invalid JSON in {label} file {path}: {error}") \
from error
if not isinstance(data, dict):
raise ValueError(
f"{label} file {path} must contain a JSON object")
return data
def load_extras(path: Path) -> dict[str, Any]:
"""Load the extras object from a JSON file.
:param path: The extras JSON file.
:return: The extras, in file order.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is not
a JSON object, has duplicate keys, or has a "lyrics" key.
"""
data: dict[str, Any] = __load_json_object(path, "extras")
if "lyrics" in data:
raise ValueError(
f"extras file {path} must not have a \"lyrics\" key")
return data
def load_extras_per_id(path: Path) -> dict[str, dict[str, Any]]:
"""Load the per-ID extras object from a JSON file.
:param path: The per-ID extras JSON file, mapping a song ID,
as ``song-<N>``, to the extras of that one song.
:return: The extras of each song ID, in file order, every
song's own extras in their file order too.
:raises OSError: When the file cannot be read.
:raises ValueError: When the file is not valid JSON, is not
a JSON object, has duplicate keys, has a song whose value
is not a JSON object, or has a song with a "lyrics" key.
"""
data: dict[str, Any] = __load_json_object(path, "per-ID extras")
song_id: str
extras: Any
for song_id, extras in data.items():
if not isinstance(extras, dict):
raise ValueError(
f"per-ID extras file {path}: id {song_id} must"
" have a JSON object")
if "lyrics" in extras:
raise ValueError(
f"per-ID extras file {path}: id {song_id} must not"
" have a \"lyrics\" key")
return data
def __build_content(
lyrics: str,
extras: dict[str, Any] | None,
song_extras: dict[str, Any] | None) -> str:
"""Build the content of one exported record.
:param lyrics: The lyrics of the song.
:param extras: The extra parameters shared by every record,
in the order they are to appear, or None for none.
:param song_extras: The extra parameters of this record
alone, in the order they are to appear, or None for none.
:return: The bare lyrics when there are no extras of either
kind, or otherwise a JSON object serialized as a string,
whose first key is ``"lyrics"`` holding the lyrics,
followed by the shared extras' keys and then this
record's own keys, each group in its given order.
"""
if extras is None and song_extras is None:
return lyrics
payload: dict[str, Any] = {"lyrics": lyrics}
if extras is not None:
payload.update(extras)
if song_extras is not None:
payload.update(song_extras)
return json.dumps(payload, ensure_ascii=False)
def build_lines(
session: Session,
extras: dict[str, Any] | None = None,
extras_per_id: dict[str, dict[str, Any]] | None = None) \
-> list[str]:
"""Build the JSONL lines of the exported songs' lyrics.
Without extras of either kind, each record's ``content`` is
the bare lyrics string. With extras, ``content`` is a JSON
object serialized as a string, whose first key is ``"lyrics"``
holding the lyrics string, followed by the shared extras' keys
and then the song's own per-ID extras' keys, each group in its
given order.
Every song is exported, unless per-ID extras are given, in
which case only the songs they name are.
:param session: The database session.
:param extras: The extra parameters merged into every
record's content alongside the lyrics, in the order they
are to appear, or None for none.
:param extras_per_id: The extra parameters merged into the
content of one record alone, keyed by that record's song
ID and in the order they are to appear, restricting the
export to the song IDs they name, or None for no such
extras and no such restriction.
:return: The JSON lines, one per exported song, ordered by
song ID.
:raises ValueError: When an exported song has no lyrics, or
the per-ID extras name a song the working store does not
have.
"""
lines: list[str] = []
exported: set[str] = set()
song: Song
for song in session.scalars(sa.select(Song).order_by(Song.id)):
song_id: str = f"song-{song.id}"
if extras_per_id is not None and song_id not in extras_per_id:
continue
if song.lyrics is None:
raise ValueError(
f"song {song.id} \"{song.title}\": no lyrics")
song_extras: dict[str, Any] | None = None \
if extras_per_id is None else extras_per_id[song_id]
content: str = __build_content(
song.lyrics, extras, song_extras)
record: dict[str, str] = {
"id": song_id, "content": content}
lines.append(json.dumps(record, ensure_ascii=False))
exported.add(song_id)
if extras_per_id is not None:
missing: list[str] = sorted(set(extras_per_id) - exported)
if len(missing) > 0:
raise ValueError(
"the per-ID extras name songs the working store"
f" does not have: {', '.join(missing)}")
return lines
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
"""Export the LLM input JSONL file from the working store. """Export the LLM input JSONL file from the working store.
Every song is exported, unless ``--extras-per-id`` is given,
in which case only the songs its file names are.
:param argv: The command-line arguments, or None for :param argv: The command-line arguments, or None for
``sys.argv``. ``sys.argv``.
:return: The exit status: 0 on success, non-zero on failure. :return: The exit status: 0 on success, non-zero on failure.
""" """
started: float = time.monotonic() started: float = time.monotonic()
args: argparse.Namespace = parse_args(argv) args: argparse.Namespace = parse_args(argv)
session: Session = ds.get_db()
lines: list[str]
try: try:
count: int = LlmInputExporter( extras: dict[str, Any] | None = None
args.output_jsonl, args.extras, if args.extras is not None:
args.extras_per_id).run() extras = load_extras(args.extras)
extras_per_id: dict[str, dict[str, Any]] | None = None
if args.extras_per_id is not None:
extras_per_id = load_extras_per_id(args.extras_per_id)
lines = build_lines(session, extras, extras_per_id)
except (OSError, sa.exc.SQLAlchemyError, ValueError) as error: except (OSError, sa.exc.SQLAlchemyError, ValueError) as error:
print(f"error: {error}", file=sys.stderr) print(f"error: {error}", file=sys.stderr)
return 1 return 1
finally:
session.close()
args.output_jsonl.parent.mkdir(parents=True, exist_ok=True)
with open(args.output_jsonl, "w", encoding="utf-8") as file:
line: str
for line in lines:
file.write(line + "\n")
elapsed: str = format_duration(time.monotonic() - started) elapsed: str = format_duration(time.monotonic() - started)
print(f"Done. {count} songs exported." print(f"Done. {len(lines)} songs exported."
f" {elapsed} elapsed.", file=sys.stderr) f" {elapsed} elapsed.", file=sys.stderr)
return 0 return 0
@@ -12,10 +12,12 @@ layer. The working store is only read, never written; the
``build-db`` subcommand assembles the captured files into the ``build-db`` subcommand assembles the captured files into the
store on the next rebuild. store on the next rebuild.
An unresolved artist or an error on one artist is noted on its Every fetched row is meant for later human verification: the
row and does not fail the run. A row whose name is no longer an description of the resolved item is recorded in the note column
artist of the store is dropped from the snapshot and reported on so that a bad match can be spotted. An unresolved artist or an
the standard error. error on one artist is noted on its row and does not fail the
run. A row whose name is no longer an artist of the store is
dropped from the snapshot and reported on the standard error.
""" """
import argparse import argparse
import csv import csv
@@ -31,7 +33,7 @@ import urllib.request
from collections.abc import Container, Sequence from collections.abc import Container, Sequence
from dataclasses import asdict, dataclass, field, fields from dataclasses import asdict, dataclass, field, fields
from pathlib import Path from pathlib import Path
from typing import Any, ClassVar, Literal, TextIO from typing import Any, Literal, TextIO
import sqlalchemy as sa import sqlalchemy as sa
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -41,6 +43,71 @@ from ..database import ds
from ..models import Artist, Song, SongArtist from ..models import Artist, Song, SongArtist
from ..utils import format_duration from ..utils import format_duration
API_URL: str = "https://www.wikidata.org/w/api.php"
"""The URL of the Wikidata API endpoint."""
SPARQL_URL: str = "https://query.wikidata.org/sparql"
"""The URL of the Wikidata Query Service SPARQL endpoint."""
USER_AGENT: str = (
f"pop-fem-audit-tools/{VERSION}"
" (https://github.com/imacat/pop-fem-audit;"
" mailto:imacat@mail.imacat.idv.tw)")
"""The User-Agent header sent on every HTTP request."""
TIMEOUT: float = 30.0
"""The timeout of an API HTTP request, in seconds."""
SPARQL_TIMEOUT: float = 90.0
"""The timeout of a SPARQL HTTP request, in seconds.
Higher than the API timeout: the WDQS server aborts a slow
query at 60 seconds, and a lower client timeout would race
that server-side abort and misclassify a slow-but-answerable
query as a client-side timeout instead of letting the server's
own HTTP error response arrive and enter the retry path."""
SLEEP_SECONDS: float = 1.0
"""The delay between consecutive HTTP requests, in seconds."""
MAX_ATTEMPTS: int = 5
"""The maximum number of attempts on a transient error."""
RETRY_SECONDS: float = 15.0
"""The back-off unit on a transient error, in seconds;
multiplied by the attempt number already made."""
RETRY_STATUSES: frozenset[int] = frozenset({429, 500, 502, 503})
"""The HTTP statuses that are retried with a back-off."""
MAX_STAGE1_TITLES: int = 3
"""The maximum number of charted titles used for the stage-1 song
corroboration."""
HUMAN_QID: str = "Q5"
"""The Wikidata item ID of "human"."""
ENSEMBLE_QID: str = "Q2088357"
"""The Wikidata item ID of "musical ensemble"."""
ORIGINAL_CAST_QID: str = "Q106497009"
"""The Wikidata item ID of "original cast"."""
GROUP_KEYWORDS: Sequence[str] = ("band", "group", "duo", "trio")
"""The label keywords that suggest a musical ensemble, covering
labels like "boy band" and "girl group"."""
NOTE_NOT_FOUND: str = "not found"
"""The note sentinel of an artist without a resolved Wikidata
item, written to the snapshot and read back for the
classification."""
CORPUS_START_YEAR: int = 2016
"""The first year of the corpus window: a member who left a group
before it never performed a corpus song."""
MIXED_GENDER: str = "mixed"
"""The gender recorded for a group whose members do not share one
gender."""
TIME_YEAR_PATTERN: re.Pattern[str] = re.compile(r"^[+-]?\d+")
"""The leading year of a Wikidata time value."""
PINNED_QIDS: dict[str, str] = {}
"""The last-resort pinned item IDs, keyed by the artist name.
An entry is for an artist the algorithm documented on
``ArtistFetcher`` is structurally unable to resolve, with its
justification recorded here. Currently empty: the only pin ever
needed, "Pinkfong" (typed as a brand, which the type gate
excludes by design), became moot when the store's artist entity
behind that credit was identified as Hope Segoine.
A pinned name skips the candidate retrieval and corroboration
steps; its item ID is used directly."""
class ArtistType(enum.StrEnum): class ArtistType(enum.StrEnum):
"""The decided artist type of a snapshot row.""" """The decided artist type of a snapshot row."""
@@ -80,14 +147,15 @@ class ArtistSnapshot:
return asdict(self) return asdict(self)
SNAPSHOT_FIELDS: Sequence[str] = tuple(
x.name for x in fields(ArtistSnapshot))
"""The header columns of the Wikidata artist snapshot CSV file."""
@dataclass @dataclass
class GroupMember: class GroupMember:
"""One has-part member of a Wikidata group item.""" """One has-part member of a Wikidata group item."""
__CORPUS_START_YEAR: ClassVar[int] = 2016
"""The first year of the corpus window: a member who left a
group before it never performed a corpus song."""
qid: str qid: str
"""The item ID of the member.""" """The item ID of the member."""
start_years: list[int] = field(default_factory=list) start_years: list[int] = field(default_factory=list)
@@ -113,7 +181,7 @@ class GroupMember:
if len(self.end_years) == 0: if len(self.end_years) == 0:
return True return True
last_end: int = max(self.end_years) last_end: int = max(self.end_years)
if last_end >= self.__CORPUS_START_YEAR: if last_end >= CORPUS_START_YEAR:
return True return True
if len(self.start_years) == 0: if len(self.start_years) == 0:
return False return False
@@ -161,8 +229,20 @@ class RetryExhausted(Exception):
""" """
NOTE_NOT_FOUND: str = "not found" def parse_args(argv: list[str] | None) -> argparse.Namespace:
"""The note marking an artist that could not be resolved.""" """Parse the command-line arguments.
:param argv: The command-line arguments, or None for
``sys.argv``.
:return: The parsed arguments.
"""
parser: argparse.ArgumentParser = argparse.ArgumentParser(
description="Fetch the artist metadata from Wikidata"
" into the capture layer.")
parser.add_argument(
"wikidata_csv", type=Path,
help="the Wikidata artist snapshot CSV file")
return parser.parse_args(argv)
class ArtistFetcher: class ArtistFetcher:
@@ -211,71 +291,6 @@ class ArtistFetcher:
in the note. in the note.
""" """
__API_URL: ClassVar[str] = "https://www.wikidata.org/w/api.php"
"""The URL of the Wikidata API endpoint."""
__SPARQL_URL: ClassVar[str] \
= "https://query.wikidata.org/sparql"
"""The URL of the Wikidata Query Service SPARQL endpoint."""
__USER_AGENT: ClassVar[str] = (
f"pop-fem-audit-tools/{VERSION}"
" (https://github.com/imacat/pop-fem-audit;"
" mailto:imacat@mail.imacat.idv.tw)")
"""The User-Agent header sent on every HTTP request."""
__TIMEOUT: ClassVar[float] = 30.0
"""The timeout of an API HTTP request, in seconds."""
__SPARQL_TIMEOUT: ClassVar[float] = 90.0
"""The timeout of a SPARQL HTTP request, in seconds.
Higher than the API timeout: the WDQS server aborts a slow
query at 60 seconds, and a lower client timeout would race
that server-side abort and misclassify a slow-but-answerable
query as a client-side timeout instead of letting the
server's own HTTP error response arrive and enter the retry
path."""
__SLEEP_SECONDS: ClassVar[float] = 1.0
"""The delay between consecutive HTTP requests, in
seconds."""
__MAX_ATTEMPTS: ClassVar[int] = 5
"""The maximum number of attempts on a transient error."""
__RETRY_SECONDS: ClassVar[float] = 15.0
"""The back-off unit on a transient error, in seconds;
multiplied by the attempt number already made."""
__RETRY_STATUSES: ClassVar[frozenset[int]] \
= frozenset({429, 500, 502, 503})
"""The HTTP statuses that are retried with a back-off."""
__MAX_STAGE1_TITLES: ClassVar[int] = 3
"""The maximum number of charted titles used for the
stage-1 song corroboration."""
__HUMAN_QID: ClassVar[str] = "Q5"
"""The Wikidata item ID of "human"."""
__ENSEMBLE_QID: ClassVar[str] = "Q2088357"
"""The Wikidata item ID of "musical ensemble"."""
__ORIGINAL_CAST_QID: ClassVar[str] = "Q106497009"
"""The Wikidata item ID of "original cast"."""
__GROUP_KEYWORDS: ClassVar[Sequence[str]] \
= ("band", "group", "duo", "trio")
"""The label keywords that suggest a musical ensemble,
covering labels like "boy band" and "girl group"."""
__MIXED_GENDER: ClassVar[str] = "mixed"
"""The gender recorded for a group whose members do not
share one gender."""
__TIME_YEAR_PATTERN: ClassVar[re.Pattern[str]] \
= re.compile(r"^[+-]?\d+")
"""The leading year of a Wikidata time value."""
__PINNED_QIDS: ClassVar[dict[str, str]] = {}
"""The last-resort pinned item IDs, keyed by the artist name.
An entry is for an artist the algorithm documented on
``ArtistFetcher`` is structurally unable to resolve, with its
justification recorded here. Currently empty: the only pin
ever needed, "Pinkfong" (typed as a brand, which the type
gate excludes by design), became moot when the store's
artist entity behind that credit was identified as Hope
Segoine.
A pinned name skips the candidate retrieval and corroboration
steps; its item ID is used directly."""
def __init__(self) -> None: def __init__(self) -> None:
"""Construct the fetcher.""" """Construct the fetcher."""
self.__sent: int = 0 self.__sent: int = 0
@@ -327,8 +342,8 @@ class ArtistFetcher:
transient error are exhausted. transient error are exhausted.
:raises ValueError: On a JSON decoding error. :raises ValueError: On a JSON decoding error.
""" """
if name in self.__PINNED_QIDS: if name in PINNED_QIDS:
return self.__PINNED_QIDS[name] return PINNED_QIDS[name]
candidates: list[str] = self.__candidates(name) candidates: list[str] = self.__candidates(name)
if len(candidates) == 0: if len(candidates) == 0:
return None return None
@@ -358,11 +373,11 @@ class ArtistFetcher:
{{ ?item rdfs:label ?name }} {{ ?item rdfs:label ?name }}
UNION {{ ?item skos:altLabel ?name }} UNION {{ ?item skos:altLabel ?name }}
{{ {{
?item wdt:P31 wd:{self.__HUMAN_QID} ?item wdt:P31 wd:{HUMAN_QID}
}} UNION {{ }} UNION {{
?item wdt:P31/wdt:P279* wd:{self.__ENSEMBLE_QID} ?item wdt:P31/wdt:P279* wd:{ENSEMBLE_QID}
}} UNION {{ }} UNION {{
?item wdt:P31 wd:{self.__ORIGINAL_CAST_QID} ?item wdt:P31 wd:{ORIGINAL_CAST_QID}
}} }}
}} }}
""" """
@@ -388,7 +403,7 @@ class ArtistFetcher:
transient error are exhausted. transient error are exhausted.
:raises ValueError: On a JSON decoding error. :raises ValueError: On a JSON decoding error.
""" """
subset: Sequence[str] = titles[:self.__MAX_STAGE1_TITLES] subset: Sequence[str] = titles[:MAX_STAGE1_TITLES]
if len(subset) == 0: if len(subset) == 0:
return None return None
query: str = f""" query: str = f"""
@@ -517,7 +532,7 @@ class ArtistFetcher:
qid: str qid: str
for qid in qids: for qid in qids:
member: MemberClaims = claims.get(qid, MemberClaims()) member: MemberClaims = claims.get(qid, MemberClaims())
if self.__HUMAN_QID not in member.instance_of_ids: if HUMAN_QID not in member.instance_of_ids:
continue continue
if len(member.gender_ids) == 0: if len(member.gender_ids) == 0:
return return
@@ -528,7 +543,7 @@ class ArtistFetcher:
[x[1] for x in genders], any_language=True) [x[1] for x in genders], any_language=True)
unique: set[str] = {x[1] for x in genders} unique: set[str] = {x[1] for x in genders}
snapshot.gender = labels[genders[0][1]] \ snapshot.gender = labels[genders[0][1]] \
if len(unique) == 1 else self.__MIXED_GENDER if len(unique) == 1 else MIXED_GENDER
basis: str = "gender derived from members: " + "; ".join( basis: str = "gender derived from members: " + "; ".join(
f"{x} {labels[y]}" for x, y in genders) f"{x} {labels[y]}" for x, y in genders)
snapshot.note = f"{snapshot.note}; {basis}" \ snapshot.note = f"{snapshot.note}; {basis}" \
@@ -662,8 +677,7 @@ class ArtistFetcher:
or not isinstance(value.get("time"), str): or not isinstance(value.get("time"), str):
continue continue
match: re.Match[str] | None \ match: re.Match[str] | None \
= ArtistFetcher.__TIME_YEAR_PATTERN.match( = TIME_YEAR_PATTERN.match(value["time"])
value["time"])
if match is not None: if match is not None:
years.append(int(match.group())) years.append(int(match.group()))
return years return years
@@ -792,13 +806,12 @@ class ArtistFetcher:
``ArtistType.GROUP`` for a musical ensemble, or the ``ArtistType.GROUP`` for a musical ensemble, or the
empty string for the human to decide. empty string for the human to decide.
""" """
if ArtistFetcher.__HUMAN_QID in type_ids: if HUMAN_QID in type_ids:
return ArtistType.SOLO return ArtistType.SOLO
qid: str qid: str
for qid in type_ids: for qid in type_ids:
label: str = labels.get(qid, "").lower() label: str = labels.get(qid, "").lower()
if any(x in label if any(x in label for x in GROUP_KEYWORDS):
for x in ArtistFetcher.__GROUP_KEYWORDS):
return ArtistType.GROUP return ArtistType.GROUP
return "" return ""
@@ -814,15 +827,14 @@ class ArtistFetcher:
transient error are exhausted. transient error are exhausted.
:raises ValueError: On a JSON decoding error. :raises ValueError: On a JSON decoding error.
""" """
url: str = ( url: str = (f"{SPARQL_URL}?"
f"{self.__SPARQL_URL}?" f"{urllib.parse.urlencode({'query': query})}")
f"{urllib.parse.urlencode({'query': query})}")
request: urllib.request.Request = urllib.request.Request( request: urllib.request.Request = urllib.request.Request(
url, headers={ url, headers={
"User-Agent": self.__USER_AGENT, "User-Agent": USER_AGENT,
"Accept": "application/sparql-results+json"}) "Accept": "application/sparql-results+json"})
body: bytes = self.__send( body: bytes = self.__send(
request, timeout=self.__SPARQL_TIMEOUT) request, timeout=SPARQL_TIMEOUT)
data: Any = json.loads(body) data: Any = json.loads(body)
bindings: Any = None bindings: Any = None
if isinstance(data, dict) \ if isinstance(data, dict) \
@@ -856,14 +868,13 @@ class ArtistFetcher:
transient error are exhausted. transient error are exhausted.
:raises ValueError: On a JSON decoding error. :raises ValueError: On a JSON decoding error.
""" """
url: str \ url: str = f"{API_URL}?{urllib.parse.urlencode(params)}"
= f"{self.__API_URL}?{urllib.parse.urlencode(params)}"
request: urllib.request.Request = urllib.request.Request( request: urllib.request.Request = urllib.request.Request(
url, headers={"User-Agent": self.__USER_AGENT}) url, headers={"User-Agent": USER_AGENT})
return json.loads(self.__send(request)) return json.loads(self.__send(request))
def __send(self, request: urllib.request.Request, def __send(self, request: urllib.request.Request,
timeout: float = __TIMEOUT) -> bytes: timeout: float = TIMEOUT) -> bytes:
"""Send an HTTP request, retrying on a transient error. """Send an HTTP request, retrying on a transient error.
Consecutive requests are separated by a fixed delay. A Consecutive requests are separated by a fixed delay. A
@@ -881,7 +892,7 @@ class ArtistFetcher:
transient error are exhausted. transient error are exhausted.
""" """
if self.__sent > 0: if self.__sent > 0:
time.sleep(self.__SLEEP_SECONDS) time.sleep(SLEEP_SECONDS)
self.__sent += 1 self.__sent += 1
attempt: int = 1 attempt: int = 1
reason: str | None reason: str | None
@@ -894,10 +905,10 @@ class ArtistFetcher:
reason = self.__retry_reason(error) reason = self.__retry_reason(error)
if reason is None: if reason is None:
raise raise
if attempt >= self.__MAX_ATTEMPTS: if attempt >= MAX_ATTEMPTS:
raise RetryExhausted( raise RetryExhausted(
f"retries exhausted ({reason})") from error f"retries exhausted ({reason})") from error
time.sleep(self.__RETRY_SECONDS * attempt) time.sleep(RETRY_SECONDS * attempt)
attempt += 1 attempt += 1
@staticmethod @staticmethod
@@ -912,7 +923,7 @@ class ArtistFetcher:
the error is not transient and must not be retried. the error is not transient and must not be retried.
""" """
if isinstance(error, urllib.error.HTTPError): if isinstance(error, urllib.error.HTTPError):
if error.code not in ArtistFetcher.__RETRY_STATUSES: if error.code not in RETRY_STATUSES:
return None return None
return str(error) return str(error)
if isinstance(error, TimeoutError): if isinstance(error, TimeoutError):
@@ -957,216 +968,92 @@ class ArtistFetcher:
return uri.rsplit("/", 1)[-1] return uri.rsplit("/", 1)[-1]
@dataclass(frozen=True) def read_snapshot_rows(file: TextIO) -> list[dict[str, str]]:
class FetchCounts: """Read the current rows of a snapshot CSV file handle.
"""The outcome counts of one snapshot update run."""
fetched: int :param file: The open, seekable snapshot CSV file.
"""The number of artists newly resolved.""" :return: The rows, keyed by the column name.
not_found: int :raises OSError: When the file cannot be read.
"""The number of artists left unresolved."""
errors: int
"""The number of artists that ended in an error."""
class ArtistSnapshotUpdater:
"""The updater of the Wikidata artist snapshot CSV file.
Fetches the metadata of every artist of the working store
that the snapshot does not resolve yet, appends a row for
each to the snapshot as it is fetched, and rewrites the
snapshot sorted by artist name with its stale rows dropped.
""" """
file.seek(0)
reader: csv.DictReader[str] = csv.DictReader(file)
return list(reader)
__SNAPSHOT_FIELDS: ClassVar[Sequence[str]] = tuple(
x.name for x in fields(ArtistSnapshot))
"""The header columns of the Wikidata artist snapshot CSV file."""
def __init__(self, wikidata_csv: Path) -> None: def read_artist_titles(session: Session,
"""Set up the updater. artist_id: int) -> list[str]:
"""Read the charted song titles credited to an artist.
:param wikidata_csv: The Wikidata artist snapshot CSV :param session: The database session.
file. :param artist_id: The artist ID.
""" :return: The song titles credited to the artist, ordered by
self.__wikidata_csv: Path = wikidata_csv the song ID, with the duplicate titles removed.
"""The Wikidata artist snapshot CSV file.""" """
titles: Sequence[str] = session.scalars(
sa.select(Song.title)
.join(SongArtist, SongArtist.song_id == Song.id)
.where(SongArtist.artist_id == artist_id)
.order_by(Song.id)).all()
return list(dict.fromkeys(titles))
def run(self) -> FetchCounts:
"""Fetch every unresolved artist and update the snapshot.
:return: The counts of the run. def ensure_snapshot_header(file: TextIO) -> None:
:raises OSError: When the snapshot file, or its parent """Write the snapshot CSV header row if the file is empty.
directory, cannot be read or written.
:raises sqlalchemy.exc.SQLAlchemyError: When the working
store cannot be read.
"""
session: Session = ds.get_db()
try:
return self.__run(session)
finally:
session.close()
def __run(self, session: Session) -> FetchCounts: :param file: The open, seekable snapshot CSV file.
"""Run the fetch loop with an open database session. :return: None.
:raises OSError: When the file cannot be written.
:param session: The database session. """
:return: The counts of the run. file.seek(0, os.SEEK_END)
:raises OSError: When the snapshot file, or its parent if file.tell() == 0:
directory, cannot be read or written. csv.writer(file).writerow(SNAPSHOT_FIELDS)
"""
fetcher: ArtistFetcher = ArtistFetcher()
fetched: int = 0
not_found: int = 0
errors: int = 0
self.__wikidata_csv.parent.mkdir(
parents=True, exist_ok=True)
with open(self.__wikidata_csv, "a+", encoding="utf-8",
newline="") as csv_file:
done: set[str] = {
x["name"] for x in
self.__read_snapshot_rows(csv_file)
if x["gender"] != ""}
self.__ensure_snapshot_header(csv_file)
names: set[str] = set()
artist: Artist
for artist in session.scalars(
sa.select(Artist).order_by(Artist.id)):
names.add(artist.name)
if artist.name in done:
continue
titles: list[str] = self.__read_artist_titles(
session, artist.id)
snapshot: ArtistSnapshot = fetcher.fetch(
artist.name, titles)
self.__append_row(csv_file, snapshot)
status: str = snapshot.qid
if snapshot.note == NOTE_NOT_FOUND:
not_found += 1
status = NOTE_NOT_FOUND
elif snapshot.note.startswith("error: "):
errors += 1
status = snapshot.note
else:
fetched += 1
print(f"artist \"{artist.name}\": {status}",
file=sys.stderr)
self.__write_snapshot(csv_file, names)
return FetchCounts(
fetched=fetched, not_found=not_found, errors=errors)
@staticmethod
def __read_snapshot_rows(file: TextIO) \
-> list[dict[str, str]]:
"""Read the current rows of a snapshot CSV file handle.
:param file: The open, seekable snapshot CSV file.
:return: The rows, keyed by the column name.
:raises OSError: When the file cannot be read.
"""
file.seek(0)
reader: csv.DictReader[str] = csv.DictReader(file)
return list(reader)
@staticmethod
def __read_artist_titles(session: Session,
artist_id: int) -> list[str]:
"""Read the charted song titles credited to an artist.
:param session: The database session.
:param artist_id: The artist ID.
:return: The song titles credited to the artist, ordered
by the song ID, with the duplicate titles removed.
"""
titles: Sequence[str] = session.scalars(
sa.select(Song.title)
.join(SongArtist, SongArtist.song_id == Song.id)
.where(SongArtist.artist_id == artist_id)
.order_by(Song.id)).all()
return list(dict.fromkeys(titles))
@staticmethod
def __ensure_snapshot_header(file: TextIO) -> None:
"""Write the snapshot CSV header row if the file is
empty.
:param file: The open, seekable snapshot CSV file.
:return: None.
:raises OSError: When the file cannot be written.
"""
file.seek(0, os.SEEK_END)
if file.tell() == 0:
csv.writer(file).writerow(
ArtistSnapshotUpdater.__SNAPSHOT_FIELDS)
file.flush()
@staticmethod
def __append_row(file: TextIO,
snapshot: ArtistSnapshot) -> None:
"""Append a snapshot row to a snapshot CSV file handle.
:param file: The open snapshot CSV file, opened for
append.
:param snapshot: The snapshot of an artist.
:return: None.
:raises OSError: When the file cannot be written.
"""
csv.DictWriter(
file, ArtistSnapshotUpdater.__SNAPSHOT_FIELDS).writerow(
snapshot.to_row())
file.flush() file.flush()
@staticmethod
def __write_snapshot(file: TextIO, names: Container[str]) \
-> None:
"""Rewrite a snapshot CSV file handle sorted by artist
name.
The rows are ordered by the case-folded artist name, def append_row(file: TextIO, snapshot: ArtistSnapshot) -> None:
matching the convention of the derived ``artists.csv``. """Append a snapshot row to a snapshot CSV file handle.
An artist keeps one row only, the last one of the file,
so that a re-fetched artist replaces its earlier row. A
row whose name is not an artist of the store is dropped
and reported on the standard error.
:param file: The open, seekable snapshot CSV file. :param file: The open snapshot CSV file, opened for append.
:param names: The artist names of the working store. :param snapshot: The snapshot of an artist.
:return: None. :return: None.
:raises OSError: When the file cannot be read or written. :raises OSError: When the file cannot be written.
"""
kept: dict[str, dict[str, str]] = {}
row: dict[str, str]
for row in ArtistSnapshotUpdater.__read_snapshot_rows(
file):
if row["name"] not in names:
print(f"dropped stale row \"{row['name']}\":"
" no such artist in the store",
file=sys.stderr)
continue
kept[row["name"]] = row
ordered: list[dict[str, str]] = sorted(
kept.values(), key=lambda x: x["name"].casefold())
file.seek(0)
file.truncate()
writer: csv.DictWriter[str] = csv.DictWriter(
file, ArtistSnapshotUpdater.__SNAPSHOT_FIELDS)
writer.writeheader()
writer.writerows(ordered)
def parse_args(argv: list[str] | None) -> argparse.Namespace:
"""Parse the command-line arguments.
:param argv: The command-line arguments, or None for
``sys.argv``.
:return: The parsed arguments.
""" """
parser: argparse.ArgumentParser = argparse.ArgumentParser( csv.DictWriter(file, SNAPSHOT_FIELDS).writerow(
description="Fetch the artist metadata from Wikidata" snapshot.to_row())
" into the capture layer.") file.flush()
parser.add_argument(
"wikidata_csv", type=Path,
help="the Wikidata artist snapshot CSV file") def write_snapshot(file: TextIO, names: Container[str]) -> None:
return parser.parse_args(argv) """Rewrite a snapshot CSV file handle sorted by artist name.
The rows are ordered by the case-folded artist name, matching
the convention of the derived ``artists.csv``. An artist
keeps one row only, the last one of the file, so that a
re-fetched artist replaces its earlier row. A row whose name
is not an artist of the store is dropped and reported on the
standard error.
:param file: The open, seekable snapshot CSV file.
:param names: The artist names of the working store.
:return: None.
:raises OSError: When the file cannot be read or written.
"""
kept: dict[str, dict[str, str]] = {}
row: dict[str, str]
for row in read_snapshot_rows(file):
if row["name"] not in names:
print(f"dropped stale row \"{row['name']}\":"
" no such artist in the store", file=sys.stderr)
continue
kept[row["name"]] = row
ordered: list[dict[str, str]] = sorted(
kept.values(), key=lambda x: x["name"].casefold())
file.seek(0)
file.truncate()
writer: csv.DictWriter[str] = csv.DictWriter(
file, SNAPSHOT_FIELDS)
writer.writeheader()
writer.writerows(ordered)
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
@@ -1179,16 +1066,52 @@ def main(argv: list[str] | None = None) -> int:
""" """
started: float = time.monotonic() started: float = time.monotonic()
args: argparse.Namespace = parse_args(argv) args: argparse.Namespace = parse_args(argv)
fetcher: ArtistFetcher = ArtistFetcher()
fetched: int = 0
not_found: int = 0
errors: int = 0
session: Session = ds.get_db()
try: try:
counts: FetchCounts \ args.wikidata_csv.parent.mkdir(
= ArtistSnapshotUpdater(args.wikidata_csv).run() parents=True, exist_ok=True)
with open(args.wikidata_csv, "a+", encoding="utf-8",
newline="") as csv_file:
done: set[str] = {x["name"] for x in
read_snapshot_rows(csv_file)
if x["gender"] != ""}
ensure_snapshot_header(csv_file)
names: set[str] = set()
artist: Artist
for artist in session.scalars(
sa.select(Artist).order_by(Artist.id)):
names.add(artist.name)
if artist.name in done:
continue
titles: list[str] = read_artist_titles(
session, artist.id)
snapshot: ArtistSnapshot = fetcher.fetch(
artist.name, titles)
append_row(csv_file, snapshot)
status: str = snapshot.qid
if snapshot.note == NOTE_NOT_FOUND:
not_found += 1
status = "not found"
elif snapshot.note.startswith("error: "):
errors += 1
status = snapshot.note
else:
fetched += 1
print(f"artist \"{artist.name}\": {status}",
file=sys.stderr)
write_snapshot(csv_file, names)
except (OSError, sa.exc.SQLAlchemyError) as error: except (OSError, sa.exc.SQLAlchemyError) as error:
print(f"error: {error}", file=sys.stderr) print(f"error: {error}", file=sys.stderr)
return 1 return 1
attempted: int = counts.fetched + counts.not_found \ finally:
+ counts.errors session.close()
attempted: int = fetched + not_found + errors
elapsed: str = format_duration(time.monotonic() - started) elapsed: str = format_duration(time.monotonic() - started)
print(f"Done. Resolved {counts.fetched}/{attempted}" print(f"Done. Resolved {fetched}/{attempted} artists."
f" artists. {elapsed} elapsed.", f" {elapsed} elapsed.",
file=sys.stderr) file=sys.stderr)
return 0 return 0
@@ -30,9 +30,8 @@ import time
import urllib.parse import urllib.parse
import urllib.request import urllib.request
from collections.abc import Sequence from collections.abc import Sequence
from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Any, ClassVar from typing import Any
import sqlalchemy as sa import sqlalchemy as sa
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -46,6 +45,60 @@ from ..models import (
) )
from ..utils import format_duration from ..utils import format_duration
PROVENANCE_FIELDS: Sequence[str] = (
"song_id", "source", "method", "acquired_at", "note")
"""The header columns of the lyrics provenance CSV file."""
USER_AGENT: str = ("pop-fem-audit-tools"
" (https://github.com/imacat/pop-fem-audit)")
"""The User-Agent header sent on every HTTP request."""
TIMEOUT: float = 30.0
"""The timeout of an HTTP request, in seconds."""
SLEEP_SECONDS: float = 1.0
"""The delay between consecutive HTTP requests, in seconds."""
def __build_normalization() -> dict[int, str | None]:
"""Build the lyrics normalization translation table.
:return: The codepoint-to-replacement mapping, a replacement
of None meaning removal.
"""
table: dict[int, str | None] = {}
codepoint: int
for codepoint in range(0x80, 0xa0):
try:
table[codepoint] = bytes([codepoint]).decode("cp1252")
except UnicodeDecodeError:
table[codepoint] = None
table[0x0435] = "e"
table[0x03cc] = "ó"
for codepoint in (0x2005, 0x205f, 0x200a):
table[codepoint] = " "
for codepoint in (0x200b, 0x200c, 0x200d, 0xfeff):
table[codepoint] = None
return table
NORMALIZATION: dict[int, str | None] = __build_normalization()
"""The codepoint-to-replacement mapping applied to fetched
lyrics: cp1252-mojibake restoration for U+0080-U+009F (with the
five byte values undefined in cp1252 removed), homoglyph
restoration for the Cyrillic "e" and the Greek "o" with tonos,
ASCII-space restoration for exotic space variants, and removal
of zero-width characters. A replacement of None removes the
codepoint."""
def normalize_lyrics(text: str) -> str:
"""Restore or remove watermark and mojibake characters.
:param text: The lyrics text as fetched from an API.
:return: The text with the codepoints in
:data:`NORMALIZATION` replaced or removed; every other
character is unchanged.
"""
return text.translate(NORMALIZATION)
def parse_args(argv: list[str] | None) -> argparse.Namespace: def parse_args(argv: list[str] | None) -> argparse.Namespace:
"""Parse the command-line arguments. """Parse the command-line arguments.
@@ -69,16 +122,6 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
class LyricsFetcher: class LyricsFetcher:
"""A fetcher of song lyrics from the public lyrics APIs.""" """A fetcher of song lyrics from the public lyrics APIs."""
__USER_AGENT: ClassVar[str] = (
"pop-fem-audit-tools"
" (https://github.com/imacat/pop-fem-audit)")
"""The User-Agent header sent on every HTTP request."""
__TIMEOUT: ClassVar[float] = 30.0
"""The timeout of an HTTP request, in seconds."""
__SLEEP_SECONDS: ClassVar[float] = 1.0
"""The delay between consecutive HTTP requests, in
seconds."""
def __init__(self) -> None: def __init__(self) -> None:
"""Construct the fetcher.""" """Construct the fetcher."""
self.__sent: int = 0 self.__sent: int = 0
@@ -150,216 +193,78 @@ class LyricsFetcher:
network, or decoding error. network, or decoding error.
""" """
if self.__sent > 0: if self.__sent > 0:
time.sleep(self.__SLEEP_SECONDS) time.sleep(SLEEP_SECONDS)
self.__sent += 1 self.__sent += 1
request: urllib.request.Request = urllib.request.Request( request: urllib.request.Request = urllib.request.Request(
url, headers={"User-Agent": self.__USER_AGENT}) url, headers={"User-Agent": USER_AGENT})
try: try:
with urllib.request.urlopen( with urllib.request.urlopen(
request, timeout=self.__TIMEOUT) as response: request, timeout=TIMEOUT) as response:
return json.load(response) return json.load(response)
except (OSError, ValueError): except (OSError, ValueError):
return None return None
@dataclass(frozen=True) def query_artist(session: Session, song_id: int) -> str:
class LyricsFetchCounts: """Find the artist name to query the APIs with.
"""The outcome of one run of fetching the missing lyrics."""
fetched: int :param session: The database session.
"""The number of songs newly fetched.""" :param song_id: The song ID.
missed: int :return: The name of the primary-role artist with the lowest
"""The number of songs every API missed.""" position.
"""
name: str | None = session.scalar(
sa.select(Artist.name)
.join(SongArtist, SongArtist.artist_id == Artist.id)
.where(SongArtist.song_id == song_id,
SongArtist.role == Role.PRIMARY)
.order_by(SongArtist.position)
.limit(1))
assert name is not None
return name
class LyricsFetchRunner: def save_lyrics(lyrics_dir: Path, song_id: int,
"""The orchestrator of one run of fetching missing lyrics.""" lyrics: str) -> None:
"""Write the lyrics of a song into the cache directory.
__PROVENANCE_FIELDS: ClassVar[Sequence[str]] = ( The cache directory is created when missing.
"song_id", "source", "method", "acquired_at", "note")
"""The header columns of the lyrics provenance CSV file."""
@staticmethod The lyrics text is normalized with :func:`normalize_lyrics`
def __build_normalization() -> dict[int, str | None]: before being written.
"""Build the lyrics normalization translation table.
:return: The codepoint-to-replacement mapping, a :param lyrics_dir: The lyrics cache directory.
replacement of None meaning removal. :param song_id: The song ID.
""" :param lyrics: The lyrics text.
table: dict[int, str | None] = {} :return: None.
codepoint: int :raises OSError: When the file cannot be written.
for codepoint in range(0x80, 0xa0): """
try: lyrics_dir.mkdir(parents=True, exist_ok=True)
table[codepoint] = bytes( (lyrics_dir / f"{song_id}.txt").write_text(
[codepoint]).decode("cp1252") normalize_lyrics(lyrics), encoding="utf-8")
except UnicodeDecodeError:
table[codepoint] = None
table[0x0435] = "e"
table[0x03cc] = "ó"
for codepoint in (0x2005, 0x205f, 0x200a):
table[codepoint] = " "
for codepoint in (0x200b, 0x200c, 0x200d, 0xfeff):
table[codepoint] = None
return table
__NORMALIZATION: ClassVar[dict[int, str | None]] \
= __build_normalization()
"""The codepoint-to-replacement mapping applied to fetched
lyrics: cp1252-mojibake restoration for U+0080-U+009F (with
the five byte values undefined in cp1252 removed), homoglyph
restoration for the Cyrillic "e" and the Greek "o" with
tonos, ASCII-space restoration for exotic space variants, and
removal of zero-width characters. A replacement of None
removes the codepoint."""
def __init__(self, lyrics_dir: Path, def append_provenance(path: Path, song_id: int,
provenance_csv: Path) -> None: source: str) -> None:
"""Set up the fetch run. """Append a provenance row for a fetched lyrics file.
:param lyrics_dir: The lyrics cache directory. The CSV file is created with the header row when missing.
:param provenance_csv: The lyrics provenance CSV file.
"""
self.__lyrics_dir: Path = lyrics_dir
"""The lyrics cache directory."""
self.__provenance_csv: Path = provenance_csv
"""The lyrics provenance CSV file."""
self.__fetcher: LyricsFetcher = LyricsFetcher()
"""The fetcher of the public lyrics APIs."""
def run(self) -> LyricsFetchCounts: :param path: The lyrics provenance CSV file.
"""Fetch the missing lyrics of every song in the store. :param song_id: The song ID.
:param source: The source name of the fetched lyrics.
Every song fetched or missed is reported on the standard :return: None.
error as an observable side effect. :raises OSError: When the file cannot be written.
"""
:return: The number of songs fetched and missed. is_new: bool = not path.exists()
:raises OSError: When a cache file or the provenance CSV path.parent.mkdir(parents=True, exist_ok=True)
cannot be written. with open(path, "a", encoding="utf-8",
:raises sqlalchemy.exc.SQLAlchemyError: On a database newline="") as file:
error. writer: Any = csv.writer(file)
""" if is_new:
fetched: int = 0 writer.writerow(PROVENANCE_FIELDS)
missed: int = 0 writer.writerow([song_id, source, "api-fetch",
session: Session = ds.get_db() datetime.date.today().isoformat(), ""])
try:
song: Song
for song in session.scalars(
sa.select(Song).order_by(Song.id)):
if (self.__lyrics_dir
/ f"{song.id}.txt").exists():
continue
if self.__fetch_one(session, song):
fetched += 1
else:
missed += 1
finally:
session.close()
return LyricsFetchCounts(fetched=fetched, missed=missed)
def __fetch_one(self, session: Session, song: Song) -> bool:
"""Fetch and save the lyrics of one song.
The song is queried by its primary-role artist name; when
every API misses and the song's full artist credit
differs from that name, the same APIs are queried again
with the artist credit.
:param session: The database session.
:param song: The song to fetch.
:return: True when a lyrics text was fetched and saved,
False when every API missed on both queries.
:raises OSError: When the cache file or the provenance
CSV cannot be written.
"""
artist: str = self.__query_artist(session, song.id)
result: tuple[str, str] | None = self.__fetcher.fetch(
artist, song.title)
if result is None and song.artist_credit != artist:
result = self.__fetcher.fetch(
song.artist_credit, song.title)
if result is None:
print(f"song {song.id} \"{song.title}\": miss",
file=sys.stderr)
return False
lyrics: str
source: str
lyrics, source = result
self.__save_lyrics(song.id, lyrics)
self.__append_provenance(song.id, source)
print(f"song {song.id} \"{song.title}\": {source}",
file=sys.stderr)
return True
@staticmethod
def __query_artist(session: Session, song_id: int) -> str:
"""Find the artist name to query the APIs with.
:param session: The database session.
:param song_id: The song ID.
:return: The name of the primary-role artist with the
lowest position.
"""
name: str | None = session.scalar(
sa.select(Artist.name)
.join(SongArtist, SongArtist.artist_id == Artist.id)
.where(SongArtist.song_id == song_id,
SongArtist.role == Role.PRIMARY)
.order_by(SongArtist.position)
.limit(1))
assert name is not None
return name
def __save_lyrics(self, song_id: int, lyrics: str) -> None:
"""Write the lyrics of a song into the cache directory.
The cache directory is created when missing.
The lyrics text is normalized with
:meth:`normalize_lyrics` before being written.
:param song_id: The song ID.
:param lyrics: The lyrics text.
:return: None.
:raises OSError: When the file cannot be written.
"""
self.__lyrics_dir.mkdir(parents=True, exist_ok=True)
(self.__lyrics_dir / f"{song_id}.txt").write_text(
self.normalize_lyrics(lyrics), encoding="utf-8")
def __append_provenance(self, song_id: int,
source: str) -> None:
"""Append a provenance row for a fetched lyrics file.
The CSV file is created with the header row when
missing.
:param song_id: The song ID.
:param source: The source name of the fetched lyrics.
:return: None.
:raises OSError: When the file cannot be written.
"""
is_new: bool = not self.__provenance_csv.exists()
self.__provenance_csv.parent.mkdir(
parents=True, exist_ok=True)
with open(self.__provenance_csv, "a", encoding="utf-8",
newline="") as file:
writer: Any = csv.writer(file)
if is_new:
writer.writerow(self.__PROVENANCE_FIELDS)
writer.writerow(
[song_id, source, "api-fetch",
datetime.date.today().isoformat(), ""])
@classmethod
def normalize_lyrics(cls, text: str) -> str:
"""Restore or remove watermark and mojibake characters.
:param text: The lyrics text as fetched from an API.
:return: The text with the codepoints of the
normalization table replaced or removed; every other
character is unchanged.
"""
return text.translate(cls.__NORMALIZATION)
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
@@ -372,15 +277,44 @@ def main(argv: list[str] | None = None) -> int:
""" """
started: float = time.monotonic() started: float = time.monotonic()
args: argparse.Namespace = parse_args(argv) args: argparse.Namespace = parse_args(argv)
fetcher: LyricsFetcher = LyricsFetcher()
fetched: int = 0
missed: int = 0
session: Session = ds.get_db()
try: try:
counts: LyricsFetchCounts = LyricsFetchRunner( song: Song
args.lyrics_dir, args.provenance_csv).run() for song in session.scalars(
sa.select(Song).order_by(Song.id)):
if (args.lyrics_dir / f"{song.id}.txt").exists():
continue
artist: str = query_artist(session, song.id)
result: tuple[str, str] | None = fetcher.fetch(
artist, song.title)
if result is None and song.artist_credit != artist:
result = fetcher.fetch(
song.artist_credit, song.title)
if result is None:
missed += 1
print(f"song {song.id} \"{song.title}\": miss",
file=sys.stderr)
continue
lyrics: str
source: str
lyrics, source = result
save_lyrics(args.lyrics_dir, song.id, lyrics)
append_provenance(args.provenance_csv, song.id,
source)
fetched += 1
print(f"song {song.id} \"{song.title}\": {source}",
file=sys.stderr)
except (OSError, sa.exc.SQLAlchemyError) as error: except (OSError, sa.exc.SQLAlchemyError) as error:
print(f"error: {error}", file=sys.stderr) print(f"error: {error}", file=sys.stderr)
return 1 return 1
attempted: int = counts.fetched + counts.missed finally:
session.close()
attempted: int = fetched + missed
elapsed: str = format_duration(time.monotonic() - started) elapsed: str = format_duration(time.monotonic() - started)
print(f"Done. Fetched lyrics for {counts.fetched}/" print(f"Done. Fetched lyrics for {fetched}/{attempted}"
f"{attempted} songs. {elapsed} elapsed.", f" songs. {elapsed} elapsed.",
file=sys.stderr) file=sys.stderr)
return 0 return 0
+330 -431
View File
@@ -28,13 +28,27 @@ import time
from dataclasses import asdict, dataclass from dataclasses import asdict, dataclass
from datetime import datetime from datetime import datetime
from pathlib import Path from pathlib import Path
from typing import Any, ClassVar, Self from typing import Any, Self
import anthropic import anthropic
from ..config import get_settings from ..config import get_settings
from ..utils import format_duration from ..utils import format_duration
# claude-fable-5 accepts neither "temperature" nor "thinking";
# a model's entry holds exactly the extra request parameters it
# accepts.
MODELS: dict[str, dict[str, Any]] = {
"claude-sonnet-4-6": {
"temperature": 0.0,
"thinking": {"type": "disabled"},
},
"claude-fable-5": {},
}
DEFAULT_MODEL: str = "claude-sonnet-4-6"
SCRIPT_VERSION: str = "run_llm.py 3.1.0"
POLL_INTERVAL_SECONDS: float = 60.0
class InputFormatError(Exception): class InputFormatError(Exception):
"""An error in the JSONL input file.""" """An error in the JSONL input file."""
@@ -122,23 +136,13 @@ class BatchResult:
if x.type == "text") if x.type == "text")
return cls(id=entry.custom_id, text=text, return cls(id=entry.custom_id, text=text,
stop_reason=message.stop_reason, stop_reason=message.stop_reason,
usage=cls.__usage_to_dict(message.usage)) usage=usage_to_dict(message.usage))
case "errored": case "errored":
return cls(id=entry.custom_id, return cls(id=entry.custom_id,
error=result.error.error.type) error=result.error.error.type)
case other: case other:
return cls(id=entry.custom_id, error=str(other)) return cls(id=entry.custom_id, error=str(other))
@staticmethod
def __usage_to_dict(usage: Any) -> dict[str, Any]:
"""Convert a usage object to a plain dictionary.
:param usage: The usage object of a message.
:return: The usage as a dictionary, without null entries.
"""
return {k: v for k, v in usage.model_dump().items()
if v is not None}
def to_record(self) -> dict[str, Any]: def to_record(self) -> dict[str, Any]:
"""Return this result as an archive JSONL record. """Return this result as an archive JSONL record.
@@ -165,409 +169,10 @@ class BatchInfo:
is still processing.""" is still processing."""
@dataclass(frozen=True)
class ExecutionOutcome:
"""The outcome of submitting and awaiting one batch."""
batch: BatchInfo
"""The submitted batch's bookkeeping."""
results: Results
"""The batch's results, keyed by item ID."""
@dataclass(frozen=True)
class RunOutcome:
"""The outcome of one LLM definition file run."""
item_count: int
"""The number of loaded input items."""
dry_run: bool
"""Whether this was a dry run."""
dry_run_request: dict[str, Any] | None
"""The first item's preview request, for a dry run; None for
an actual run."""
failed: list[str]
"""The failed item IDs, in item order; always empty for a dry
run."""
class LLMRunner:
"""The orchestrator of one LLM definition file run."""
# claude-fable-5 accepts neither "temperature" nor "thinking";
# a model's entry holds exactly the extra request parameters
# it accepts.
MODELS: ClassVar[dict[str, dict[str, Any]]] = {
"claude-sonnet-4-6": {
"temperature": 0.0,
"thinking": {"type": "disabled"},
},
"claude-fable-5": {},
}
"""The supported model IDs and their extra request
parameters."""
DEFAULT_MODEL: ClassVar[str] = "claude-sonnet-4-6"
"""The default model ID."""
__SCRIPT_VERSION: ClassVar[str] = "run_llm.py 3.1.0"
"""The script version recorded into the archive metadata."""
__POLL_INTERVAL_SECONDS: ClassVar[float] = 60.0
"""The interval between batch status polls."""
def __init__(self, prompt: Path, input_path: Path,
archive_dir: Path, model: str, max_tokens: int,
dry_run: bool, replace: bool) -> None:
"""Set up the run of one LLM definition file.
:param prompt: The prompt definition file, used as the
system prompt.
:param input_path: The JSONL input file with "id" and
"content".
:param archive_dir: The destination archive directory.
:param model: The model ID, a key of :attr:`MODELS`.
:param max_tokens: The maximum output tokens per request.
:param dry_run: Whether to validate and archive without
calling the API.
:param replace: Whether to replace an already existing
archive directory.
"""
self.__prompt: Path = prompt
"""The prompt definition file."""
self.__input: Path = input_path
"""The JSONL input file."""
self.__archive_dir: Path = archive_dir
"""The destination archive directory."""
self.__model: str = model
"""The model ID."""
self.__max_tokens: int = max_tokens
"""The maximum output tokens per request."""
self.__dry_run: bool = dry_run
"""Whether to validate and archive without calling the
API."""
self.__replace: bool = replace
"""Whether to replace an already existing archive
directory."""
def run(self) -> RunOutcome:
"""Load the input, archive the prompt, and run the batch.
Always writes ``prompt.md`` and ``meta.json`` into the
archive directory. A dry run stops there, previewing the
first item's request; an actual run also submits the
batch, awaits it, and writes ``output.jsonl``.
:return: The outcome of the run.
:raises InputFormatError: When the input file is
malformed.
:raises OSError: When the input or prompt file cannot be
read, the archive directory already exists without
``replace``, or an output file cannot be written.
"""
items: list[InputItem] = self.__load_items()
prompt_text: str = self.__prompt.read_text(encoding="utf-8")
archive_dir: Path = self.__create_archive_dir()
(archive_dir / "prompt.md").write_bytes(
self.__prompt.read_bytes())
meta: dict[str, Any] = self.__build_meta(items)
meta_path: Path = archive_dir / "meta.json"
if self.__dry_run:
self.__write_json(meta_path, meta)
request: dict[str, Any] = self.__build_request(
items[0], prompt_text)
return RunOutcome(
item_count=len(items), dry_run=True,
dry_run_request=request, failed=[])
client: anthropic.Anthropic = anthropic.Anthropic(
api_key=get_settings().ANTHROPIC_API_KEY)
outcome: ExecutionOutcome = self.__execute_run(
client, items, prompt_text)
item_ids: list[str] = [x.id for x in items]
self.__write_jsonl(
archive_dir / "output.jsonl",
[outcome.results[x].to_record() for x in item_ids
if x in outcome.results])
meta["batch"] = outcome.batch
meta["usage"] = self.__sum_usage(outcome.results)
self.__write_meta(meta_path, meta)
failed: list[str] = self.__find_failures(
item_ids, outcome.results)
return RunOutcome(
item_count=len(items), dry_run=False,
dry_run_request=None, failed=failed)
def __load_items(self) -> list[InputItem]:
"""Load and validate the JSONL input items.
:return: The input items, in file order.
:raises InputFormatError: When a line is malformed, an ID
is duplicated, or the file contains no item.
:raises OSError: When the file cannot be read.
"""
items: list[InputItem] = []
seen: set[str] = set()
with open(self.__input, encoding="utf-8") as file:
for number, line in enumerate(file, start=1):
if line.strip() == "":
continue
data: Any
try:
data = json.loads(line)
except json.JSONDecodeError as error:
raise InputFormatError(
f"{self.__input}: line {number}: malformed"
f" JSON: {error}")
item: InputItem = InputItem.get_instance(
data, self.__input, number)
if item.id in seen:
raise InputFormatError(
f"{self.__input}: line {number}:"
f" duplicated ID \"{item.id}\"")
seen.add(item.id)
items.append(item)
if len(items) == 0:
raise InputFormatError(f"{self.__input}: no input items")
return items
def __create_archive_dir(self) -> Path:
"""Create the archive directory.
Only this directory is ever created or removed; no other
directory is ever touched.
:return: The created archive directory.
:raises FileExistsError: When the archive directory
already exists and ``replace`` is False.
"""
if self.__archive_dir.exists():
if not self.__replace:
raise FileExistsError(
f"{self.__archive_dir} already exists; pass"
" --replace to replace it")
shutil.rmtree(self.__archive_dir)
self.__archive_dir.mkdir(parents=True)
return self.__archive_dir
def __build_meta(self, items: list[InputItem]) -> dict[str, Any]:
"""Build the initial archive metadata.
:param items: The loaded input items.
:return: The metadata, "batch" and "usage" not yet filled
in for an actual run.
"""
return {
"script_version": self.__SCRIPT_VERSION,
"model": self.__model,
"temperature": self.MODELS[self.__model].get(
"temperature"),
"thinking": self.MODELS[self.__model].get("thinking"),
"max_tokens": self.__max_tokens,
"prompt_path": str(self.__prompt),
"prompt_sha256": self.__sha256_of(self.__prompt),
"input_path": str(self.__input),
"input_sha256": self.__sha256_of(self.__input),
"item_count": len(items),
"dry_run": self.__dry_run,
"started_at": self.__now_iso(),
"batch": None,
"usage": {},
}
def __build_request(self, item: InputItem, system_prompt: str) \
-> dict[str, Any]:
"""Build one Message Batches request for an input item.
:param item: The input item.
:param system_prompt: The system prompt text.
:return: The batch request with "custom_id" and "params".
"""
return {
"custom_id": item.id,
"params": {
"model": self.__model,
"max_tokens": self.__max_tokens,
**self.MODELS[self.__model],
"system": system_prompt,
"messages": [
{"role": "user", "content": item.content},
],
},
}
def __execute_run(
self, client: anthropic.Anthropic,
items: list[InputItem], system_prompt: str) \
-> ExecutionOutcome:
"""Submit the batch of this run and await its results.
:param client: The Anthropic client.
:param items: The input items.
:param system_prompt: The system prompt text.
:return: The submitted batch's bookkeeping and its
results.
"""
requests: list[dict[str, Any]] = [
self.__build_request(x, system_prompt) for x in items]
info: BatchInfo = BatchInfo(
batch_id=self.__submit_batch(client, requests),
submitted_at=self.__now_iso())
print(f"submitted batch {info.batch_id}", file=sys.stderr)
batches: dict[str, Any] = self.__poll_batches(
client, [info.batch_id])
info.ended_at = batches[info.batch_id].ended_at.isoformat()
results: Results = self.__collect_results(
client, info.batch_id)
return ExecutionOutcome(batch=info, results=results)
@staticmethod
def __submit_batch(client: anthropic.Anthropic,
requests: list[dict[str, Any]]) -> str:
"""Submit one message batch.
:param client: The Anthropic client.
:param requests: The batch requests.
:return: The batch ID.
"""
return client.messages.batches.create(requests=requests).id
@classmethod
def __poll_batches(cls, client: anthropic.Anthropic,
batch_ids: list[str]) -> dict[str, Any]:
"""Poll the batches until every one of them has ended.
Progress is printed to the standard error every poll.
:param client: The Anthropic client.
:param batch_ids: The batch IDs to poll.
:return: The final batch object of each batch, keyed by
batch ID.
"""
while True:
batches: dict[str, Any] = {
x: client.messages.batches.retrieve(x)
for x in batch_ids}
pending: list[str] = [
x for x in batch_ids
if batches[x].processing_status != "ended"]
for batch_id in batch_ids:
status: str = batches[batch_id].processing_status
print(f"batch {batch_id}: {status}", file=sys.stderr)
if len(pending) == 0:
return batches
time.sleep(cls.__POLL_INTERVAL_SECONDS)
@staticmethod
def __collect_results(client: anthropic.Anthropic,
batch_id: str) -> Results:
"""Collect the results of an ended batch.
:param client: The Anthropic client.
:param batch_id: The batch ID.
:return: The result records, keyed by custom ID.
"""
results: Results = {}
for entry in client.messages.batches.results(batch_id):
results[entry.custom_id] = BatchResult.get_instance(
entry)
return results
@staticmethod
def __find_failures(item_ids: list[str],
results: Results) -> list[str]:
"""Find the item IDs that failed in a result set.
An item failed when it is missing from the results or when
its record is a failure.
:param item_ids: The item IDs to check, in order.
:param results: The result records, keyed by item ID.
:return: The failed item IDs, in the given order.
"""
return [x for x in item_ids
if x not in results or results[x].is_failure]
@staticmethod
def __sum_usage(results: Results) -> dict[str, int]:
"""Sum the token usage of every succeeded result.
:param results: The result records, keyed by item ID.
:return: The summed integer usage fields.
"""
totals: dict[str, int] = {}
for result in results.values():
if result.usage is None:
continue
for key, value in result.usage.items():
if isinstance(value, int):
totals[key] = totals.get(key, 0) + value
return totals
@staticmethod
def __write_jsonl(path: Path,
records: list[dict[str, Any]]) -> None:
"""Write records to a file as JSON Lines.
:param path: The path of the file to write.
:param records: The records, one per line.
:return: None.
:raises OSError: When the file cannot be written.
"""
with open(path, "w", encoding="utf-8") as file:
for record in records:
file.write(
json.dumps(record, ensure_ascii=False) + "\n")
@staticmethod
def __write_json(path: Path, data: dict[str, Any]) -> None:
"""Write data to a file as pretty-printed JSON.
:param path: The path of the file to write.
:param data: The data to write.
:return: None.
:raises OSError: When the file cannot be written.
"""
path.write_text(
json.dumps(data, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8")
@classmethod
def __write_meta(cls, path: Path, meta: dict[str, Any]) -> None:
"""Write the metadata to the ``meta.json`` file.
The ``BatchInfo`` value under "batch" is written as a
plain JSON object.
:param path: The path of the ``meta.json`` file.
:param meta: The metadata to write.
:return: None.
:raises OSError: When the file cannot be written.
"""
cls.__write_json(
path, {**meta, "batch": asdict(meta["batch"])})
@staticmethod
def __sha256_of(path: Path) -> str:
"""Calculate the SHA-256 digest of a file.
:param path: The path of the file.
:return: The hexadecimal SHA-256 digest.
"""
with open(path, "rb") as file:
return hashlib.file_digest(file, "sha256").hexdigest()
@staticmethod
def __now_iso() -> str:
"""Return the current local time in ISO 8601 format.
:return: The current local time with the timezone offset.
"""
return datetime.now().astimezone().isoformat(
timespec="seconds")
def parse_args(argv: list[str] | None) -> argparse.Namespace: def parse_args(argv: list[str] | None) -> argparse.Namespace:
"""Parse the command-line arguments. """Parse the command-line arguments.
:param argv: The command-line arguments, or None for :param argv: The command-line arguments, or None for ``sys.argv``.
``sys.argv``.
:return: The parsed arguments. :return: The parsed arguments.
""" """
parser: argparse.ArgumentParser = argparse.ArgumentParser( parser: argparse.ArgumentParser = argparse.ArgumentParser(
@@ -575,8 +180,7 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
" and archive the result.") " and archive the result.")
parser.add_argument( parser.add_argument(
"prompt", type=Path, "prompt", type=Path,
help="the prompt definition file, used as the system" help="the prompt definition file, used as the system prompt")
" prompt")
parser.add_argument( parser.add_argument(
"input", type=Path, "input", type=Path,
help="the JSONL input file with \"id\" and \"content\"") help="the JSONL input file with \"id\" and \"content\"")
@@ -584,13 +188,11 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
"archive_dir", type=Path, "archive_dir", type=Path,
help="the destination archive directory") help="the destination archive directory")
parser.add_argument( parser.add_argument(
"--model", choices=sorted(LLMRunner.MODELS), "--model", choices=sorted(MODELS), default=DEFAULT_MODEL,
default=LLMRunner.DEFAULT_MODEL, help=f"the model ID (default {DEFAULT_MODEL})")
help=f"the model ID (default {LLMRunner.DEFAULT_MODEL})")
parser.add_argument( parser.add_argument(
"--max-tokens", type=int, default=2048, "--max-tokens", type=int, default=2048,
help="the maximum output tokens per request (default" help="the maximum output tokens per request (default 2048)")
" 2048)")
parser.add_argument( parser.add_argument(
"--dry-run", action="store_true", "--dry-run", action="store_true",
help="validate and archive without calling the API") help="validate and archive without calling the API")
@@ -600,30 +202,327 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
return parser.parse_args(argv) return parser.parse_args(argv)
def load_items(path: Path) -> list[InputItem]:
"""Load and validate the JSONL input items.
:param path: The path of the JSONL input file.
:return: The input items, in file order.
:raises InputFormatError: When a line is malformed, an ID is
duplicated, or the file contains no item.
:raises OSError: When the file cannot be read.
"""
items: list[InputItem] = []
seen: set[str] = set()
with open(path, encoding="utf-8") as file:
for number, line in enumerate(file, start=1):
if line.strip() == "":
continue
try:
data: Any = json.loads(line)
except json.JSONDecodeError as error:
raise InputFormatError(
f"{path}: line {number}: malformed JSON: {error}")
item: InputItem = InputItem.get_instance(
data, path, number)
if item.id in seen:
raise InputFormatError(
f"{path}: line {number}: duplicated ID"
f" \"{item.id}\"")
seen.add(item.id)
items.append(item)
if len(items) == 0:
raise InputFormatError(f"{path}: no input items")
return items
def build_request(item: InputItem, system_prompt: str,
max_tokens: int, model: str) -> dict[str, Any]:
"""Build one Message Batches request for an input item.
:param item: The input item.
:param system_prompt: The system prompt text.
:param max_tokens: The maximum output tokens.
:param model: The model ID, a key of ``MODELS``.
:return: The batch request with "custom_id" and "params".
"""
return {
"custom_id": item.id,
"params": {
"model": model,
"max_tokens": max_tokens,
**MODELS[model],
"system": system_prompt,
"messages": [
{"role": "user", "content": item.content},
],
},
}
def submit_batch(client: anthropic.Anthropic,
requests: list[dict[str, Any]]) -> str:
"""Submit one message batch.
:param client: The Anthropic client.
:param requests: The batch requests.
:return: The batch ID.
"""
return client.messages.batches.create(requests=requests).id
def poll_batches(client: anthropic.Anthropic,
batch_ids: list[str]) -> dict[str, Any]:
"""Poll the batches until every one of them has ended.
Progress is printed to the standard error every poll.
:param client: The Anthropic client.
:param batch_ids: The batch IDs to poll.
:return: The final batch object of each batch, keyed by batch ID.
"""
while True:
batches: dict[str, Any] = {
x: client.messages.batches.retrieve(x) for x in batch_ids}
pending: list[str] = [
x for x in batch_ids
if batches[x].processing_status != "ended"]
for batch_id in batch_ids:
status: str = batches[batch_id].processing_status
print(f"batch {batch_id}: {status}", file=sys.stderr)
if len(pending) == 0:
return batches
time.sleep(POLL_INTERVAL_SECONDS)
def usage_to_dict(usage: Any) -> dict[str, Any]:
"""Convert a usage object to a plain dictionary.
:param usage: The usage object of a message.
:return: The usage as a dictionary, without null entries.
"""
return {k: v for k, v in usage.model_dump().items()
if v is not None}
def sum_usage(results: Results) -> dict[str, int]:
"""Sum the token usage of every succeeded result.
:param results: The result records, keyed by item ID.
:return: The summed integer usage fields.
"""
totals: dict[str, int] = {}
for result in results.values():
if result.usage is None:
continue
for key, value in result.usage.items():
if isinstance(value, int):
totals[key] = totals.get(key, 0) + value
return totals
def collect_results(client: anthropic.Anthropic,
batch_id: str) -> Results:
"""Collect the results of an ended batch.
:param client: The Anthropic client.
:param batch_id: The batch ID.
:return: The result records, keyed by custom ID.
"""
results: Results = {}
for entry in client.messages.batches.results(batch_id):
results[entry.custom_id] = BatchResult.get_instance(entry)
return results
def find_failures(item_ids: list[str],
results: Results) -> list[str]:
"""Find the item IDs that failed in a result set.
An item failed when it is missing from the results or when its
record is a failure.
:param item_ids: The item IDs to check, in order.
:param results: The result records, keyed by item ID.
:return: The failed item IDs, in the given order.
"""
return [x for x in item_ids
if x not in results or results[x].is_failure]
def create_archive_dir(directory: Path, replace: bool) -> Path:
"""Create the archive directory.
Only this directory is ever created or removed; no other
directory is ever touched.
:param directory: The destination archive directory.
:param replace: Whether to remove an already existing archive
directory before creating it.
:return: The created archive directory.
:raises FileExistsError: When the archive directory already
exists and ``replace`` is False.
"""
if directory.exists():
if not replace:
raise FileExistsError(
f"{directory} already exists; pass --replace to"
" replace it")
shutil.rmtree(directory)
directory.mkdir(parents=True)
return directory
def write_jsonl(path: Path, records: list[dict[str, Any]]) -> None:
"""Write records to a file as JSON Lines.
:param path: The path of the file to write.
:param records: The records, one per line.
:return: None.
"""
with open(path, "w", encoding="utf-8") as file:
for record in records:
file.write(json.dumps(record, ensure_ascii=False) + "\n")
def write_json(path: Path, data: dict[str, Any]) -> None:
"""Write data to a file as pretty-printed JSON.
:param path: The path of the file to write.
:param data: The data to write.
:return: None.
"""
path.write_text(
json.dumps(data, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8")
def write_meta(path: Path, meta: dict[str, Any]) -> None:
"""Write the metadata to the ``meta.json`` file.
The ``BatchInfo`` value under ``batch`` is written as a plain
JSON object.
:param path: The path of the ``meta.json`` file.
:param meta: The metadata to write.
:return: None.
"""
write_json(path, {**meta, "batch": asdict(meta["batch"])})
def sha256_of(path: Path) -> str:
"""Calculate the SHA-256 digest of a file.
:param path: The path of the file.
:return: The hexadecimal SHA-256 digest.
"""
with open(path, "rb") as file:
return hashlib.file_digest(file, "sha256").hexdigest()
def now_iso() -> str:
"""Return the current local time in ISO 8601 format.
:return: The current local time with the timezone offset.
"""
return datetime.now().astimezone().isoformat(timespec="seconds")
def execute_run(
client: anthropic.Anthropic, items: list[InputItem],
system_prompt: str, max_tokens: int, model: str,
meta: dict[str, Any],
) -> Results:
"""Submit the batch of this run and await its results.
The batch ID and timestamps are recorded into the metadata as an
observable side effect.
:param client: The Anthropic client.
:param items: The input items.
:param system_prompt: The system prompt text.
:param max_tokens: The maximum output tokens per request.
:param model: The model ID, a key of ``MODELS``.
:param meta: The metadata to record the batch bookkeeping into.
:return: The results of this run, keyed by item ID.
"""
requests: list[dict[str, Any]] = [
build_request(x, system_prompt, max_tokens, model)
for x in items]
info: BatchInfo = BatchInfo(
batch_id=submit_batch(client, requests),
submitted_at=now_iso())
meta["batch"] = info
print(f"submitted batch {info.batch_id}", file=sys.stderr)
batches: dict[str, Any] = poll_batches(client, [info.batch_id])
info.ended_at = batches[info.batch_id].ended_at.isoformat()
return collect_results(client, info.batch_id)
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
"""Run one LLM definition file against one input and archive it. """Run one LLM definition file against one input and archive it.
:param argv: The command-line arguments, or None for :param argv: The command-line arguments, or None for ``sys.argv``.
``sys.argv``.
:return: The exit status: 0 on success, non-zero on failure. :return: The exit status: 0 on success, non-zero on failure.
""" """
started: float = time.monotonic() started: float = time.monotonic()
args: argparse.Namespace = parse_args(argv) args: argparse.Namespace = parse_args(argv)
try: try:
outcome: RunOutcome = LLMRunner( items: list[InputItem] = load_items(args.input)
args.prompt, args.input, args.archive_dir, args.model, prompt_text: str = args.prompt.read_text(encoding="utf-8")
args.max_tokens, args.dry_run, args.replace).run() except (OSError, InputFormatError) as error:
except (InputFormatError, OSError) as error:
print(f"error: {error}", file=sys.stderr) print(f"error: {error}", file=sys.stderr)
return 1 return 1
if not outcome.dry_run and len(outcome.failed) > 0: try:
print(f"error: failed items: {', '.join(outcome.failed)}", archive_dir: Path = create_archive_dir(
args.archive_dir, args.replace)
except FileExistsError as error:
print(f"error: {error}", file=sys.stderr)
return 1
meta_path: Path = archive_dir / "meta.json"
(archive_dir / "prompt.md").write_bytes(args.prompt.read_bytes())
meta: dict[str, Any] = {
"script_version": SCRIPT_VERSION,
"model": args.model,
"temperature": MODELS[args.model].get("temperature"),
"thinking": MODELS[args.model].get("thinking"),
"max_tokens": args.max_tokens,
"prompt_path": str(args.prompt),
"prompt_sha256": sha256_of(args.prompt),
"input_path": str(args.input),
"input_sha256": sha256_of(args.input),
"item_count": len(items),
"dry_run": args.dry_run,
"started_at": now_iso(),
"batch": None,
"usage": {},
}
if args.dry_run:
write_json(meta_path, meta)
print(json.dumps(
build_request(items[0], prompt_text, args.max_tokens,
args.model),
ensure_ascii=False, indent=2))
elapsed: str = format_duration(time.monotonic() - started)
print(f"Done. {len(items)} jobs finished."
f" {elapsed} elapsed.", file=sys.stderr)
return 0
client: anthropic.Anthropic = anthropic.Anthropic(
api_key=get_settings().ANTHROPIC_API_KEY)
results: Results = execute_run(
client, items, prompt_text, args.max_tokens, args.model,
meta)
item_ids: list[str] = [x.id for x in items]
write_jsonl(
archive_dir / "output.jsonl",
[results[x].to_record() for x in item_ids if x in results])
meta["usage"] = sum_usage(results)
write_meta(meta_path, meta)
failed: list[str] = find_failures(item_ids, results)
if len(failed) > 0:
print(f"error: failed items: {', '.join(failed)}",
file=sys.stderr) file=sys.stderr)
return 1 return 1
if outcome.dry_run:
print(json.dumps(
outcome.dry_run_request, ensure_ascii=False, indent=2))
elapsed: str = format_duration(time.monotonic() - started) elapsed: str = format_duration(time.monotonic() - started)
print(f"Done. {outcome.item_count} jobs finished." print(f"Done. {len(items)} jobs finished."
f" {elapsed} elapsed.", file=sys.stderr) f" {elapsed} elapsed.", file=sys.stderr)
return 0 return 0

Some files were not shown because too many files have changed in this diff Show More