pop-fem-audit/docs/conventions.md

Project Conventions

The standing working rules of this project. Formerly the project CLAUDE.md; moved here so that Claude Code subagents do not inherit it into their context (blind-reading agents must not see it). A main session working on this project reads this file before touching the pipeline, the data, or the documents.

Analysis pipeline

The pipeline has run to completion; these conventions govern any rerun or extension.

• LLM analysis runs via Python scripts calling the Anthropic Messages API, Batch API where possible. Steps 1 and 3 run on claude-sonnet-4-6 with temperature=0 and thinking disabled; steps 4 and 5 run on claude-fable-5, which accepts neither parameter -- step 4 absorbs its sampling variance by the majority vote, step 5 by consolidating the three readings.
• Prompt definition files live in prompts/<step><substep>-<task>.md (e.g. 1-tag.md, 5a-read.md; substeps are lettered, matching the step numbering of the paper; no version suffix -- versions live in git history) and are passed verbatim as the system prompt. The number names a step of the research procedure, not the file: the deterministic vocabulary step (step 2) has no definition file yet holds its own number.
• Itemwise LLM judgments (per-song coding in step 3, per-keyword group selection in step 4) run the same definition file three times, independently, over the same input; a deterministic tally then assigns an item (a (song, keyword) or (group, keyword) pair) when at least two of the three runs assign it ("3 runs + majority vote"). Free-generation steps run twice and both outputs are pooled. The step-5 qualitative readings are neither: three independent readings per song, consolidated per song and synthesized across songs by their own definition files -- a qualitative protocol, not a vote (see docs/methodology.md). The vocabulary is built by a deterministic subcommand (embedding + clustering), not by an LLM. If a validation outcome is unexpected, revise the definition file and repeat that cycle; never patch results by hand.
• Each run of a step is archived self-contained under the destination directory given explicitly on the run-llm command line (by convention runs/<step>/run<N>/): prompt snapshot, raw output, and meta.json (model ID, parameters, timestamps, batch ID). The runs of a step are that many separate invocations of run-llm. Replacing an existing run archive requires an explicit flag; superseded runs live in git history. Deterministic steps archive under runs/<step>/ with no run<N> level.
• Token usage and cost of every run-llm execution are recorded in docs/run-costs.md in the same commit as the run archive.
• Scripts read the API key from the ANTHROPIC_API_KEY environment variable (.env, gitignored).

Data rules

• data/source/ holds the immutable hand-placed raw files; data/captures/ is written only by the fetch commands and the private import script; data/manual/ is written only by the user's own hand; data/derived/ is written only by the build-db subcommand.
• Full lyrics are copyrighted: they stay in data/captures/lyrics/ (gitignored) and must never be committed or reproduced in full anywhere in the repo.

Documents

• results/ holds the final tallied tables (what the paper cites); runs/ holds raw audit records. The paper cites results/ only.
• Any change to a definition file or the plan is recorded in docs/decision-log.md with date and reason.