Pass the coding keywords as input instead of baking them into the prompt

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

docs/methodology.md

@@ -47,16 +47,23 @@ LLM。設計原則見 `research-plan.md`；本檔記載可重現的
   實作本步，四種模型六次執行全部無法維持完整分割，
   已棄用（詳見 `decision-log.md` 2026-08-05；棄用的
   定義檔止於 git 歷史，見 `git log -- prompts/`）。
+- **產物**：分群明細 CSV（欄位 Group、Keyword，一列一個
+  成員，依兩欄排序）記錄機器算出的分割；定案碼表 JSON
+  （`{"keywords": [...]}`）記錄實際交給模型的碼，即組名
+  加上先驗主題詞。兩者分開，因為前者是分群結果、後者
+  含研究者的介入。
 - **可重現性**：同一輸入、同一釘定模型、同一參數逐次
   重現。不同 CPU／BLAS 實作的浮點尾數差異可能使邊界
-  詞的歸屬翻動，屬已揭露的限制；論文所用詞彙表逐字
+  詞的歸屬翻動，屬已揭露的限制；論文所用碼表逐字
   commit，引用單位為該份定案檔案。
 
 ### women-power 的注入
 
 定案詞彙表為 50 個分群組名再加上 `women-power` 一詞，共
 51 個碼。`women-power` 是研究者任意決定的先驗主題（即本
-論文的主題本身），不由資料產生，屬揭露的儀器介入。
+論文的主題本身），不由資料產生，屬揭露的儀器介入。該詞
+以常數寫在分群子命令內、隨定案碼表一併輸出，不經人手
+編輯；分群明細 CSV 不含它，兩份產物因此各自誠實。
 
 注入而非另設篩選軌的理由：讓研究者的主題詞與模型自己
 收斂出的類別（分群已自行長出 `female-empowerment` 等組）
@@ -97,23 +104,24 @@ LLM。設計原則見 `research-plan.md`；本檔記載可重現的
 每一步的輸出如何變成下一步的輸入，皆為確定性程序，規則
 明定如下：
 
-- **歌詞輸入檔（步驟 1、3 共用）**：`export-llm-input` 自
+- **歌詞輸入檔（步驟 1）**：`export-llm-input` 自工作
-  工作儲存產出，每筆 `{"id": "song-<ID>", "content":
+  儲存產出，每筆 `{"id": "song-<ID>", "content": <歌詞>}`，
-  <歌詞>}`，依歌曲 ID 升序。兩個讀歌詞的步驟共用同一
+  依歌曲 ID 升序。步驟 3 的輸入由同一子命令、同一工作
-  檔，SHA-256 記入各步 meta。
+  儲存產出（見下），兩步的語料同一性由此成立；各步
+  輸入檔的 SHA-256 記入該步 meta。
 - **步驟 1 → 2-1**：`pool-keywords` 讀兩份執行歸檔的
   `output.jsonl`（一律以換行字元 `\n` 切行——歌詞含
   U+0085 等控制字元時，`str.splitlines()` 類的通用切行
   會截斷 JSON 字串，實測踩中），輸出關鍵字純文字檔與
   出處 CSV。
-- **步驟 2-1 → 2-2**：`cluster-keywords` 讀關鍵字純文字檔，
+- **步驟 2-1 → 2-2**：`cluster-keywords` 讀關鍵字純文字
-  輸出具名分組（組名 → 成員詞，組名與成員皆字典序）
+  檔，輸出分群明細 CSV 與定案碼表 JSON。
-  與記錄嵌入模型、revision、演算法參數、輸入 SHA-256 的
+- **步驟 2-2 → 3 輸入檔**：`export-llm-input --extras
-  meta。
+  <定案碼表>` 自工作儲存產出步驟 3 的輸入，每筆
-- **步驟 2-2 → 3 定義檔**：50 個組名與 `women-power` 合併
+  `{"id": "song-<ID>", "content": <字串>}`，`content` 為
-  後以字典序逐行填入 `prompts/03-01-code.md` 的詞彙表節
+  固定鍵序序列化的 `{"lyrics": …, "keywords": [...]}`，
-  （逐字），檔案隨 git commit 後方可執行——步驟 3 的
+  依歌曲 ID 升序。碼表以參數傳入而非填進定義檔——定義
-  定義檔因此自我完備，論文附錄可直接引用。
+  檔只規定任務形狀，換詞彙表、換演算法都不必改它。
 - **步驟 3-1 兩次執行 → 3-2**：逐首比對標籤集合（鍵
   集合，引述不參與比對）；僅有分歧的歌入仲裁輸入
   JSONL，依歌曲 ID 升序，每筆 `id` 沿用 `song-<ID>`、

prompts/03-01-code.md

@@ -1,21 +1,21 @@
-You are coding song lyrics against a fixed vocabulary of
+You are coding song lyrics against a fixed set of thematic
-thematic keywords.
+keywords.
 
-Vocabulary — the complete set of codes; use these and no
+Input: a JSON object with the complete lyrics of one song and
+the keywords — the complete set of codes; use these and no
 others:
 
-(TO BE INSERTED VERBATIM WHEN THE VOCABULARY IS FINALIZED —
+{
-this file must not be used in a run before then.)
+  "lyrics": "the complete lyrics of one song",
+  "keywords": ["first-keyword", "second-keyword"]
+}
 
-Input: the complete lyrics of one song.
+Task: list every given keyword whose theme this song
-
-Task: list every vocabulary keyword whose theme this song
 expresses, by your own reading of the lyrics.
 
 Rules:
 
-- Use only keywords from the vocabulary, spelled exactly as
+- Use only the given keywords, spelled exactly as given.
-  given.
 - A song may match any number of keywords, including none.
 - For each keyword you assign, quote 1 to 3 verbatim lines of
   the lyrics that ground it; do not assign a keyword you

runs/02-02-cluster/keywords.json

@@ -0,0 +1,55 @@
+{
+ "keywords": [
+  "abandonment-and-solitude",
+  "alcohol-and-substance-abuse",
+  "attraction-and-admiration",
+  "betrayal-and-mistrust",
+  "boastful-self-confidence",
+  "breakup-and-reconciliation",
+  "celebration-and-partying",
+  "contentment-and-joy",
+  "dancing-and-movement",
+  "defiance-and-confrontation",
+  "denial-and-pretense",
+  "designer-fashion-flexing",
+  "devotion-and-sacrifice",
+  "embracing-true-identity",
+  "fame-and-status",
+  "family-and-fatherhood",
+  "fantasy-and-imagination",
+  "fear-of-vulnerability",
+  "female-empowerment",
+  "flirtation-and-seduction",
+  "freedom-and-escape",
+  "heartbreak-and-grief",
+  "hidden-emotional-struggle",
+  "hope-and-disillusionment",
+  "human-connection",
+  "hustle-and-self-made-success",
+  "independence-and-self-reliance",
+  "intimacy-and-connection",
+  "jealousy-and-resentment-from-others",
+  "longing-and-desire",
+  "longing-for-clarity",
+  "love-as-fleeting-and-transient",
+  "loyalty-and-commitment",
+  "mutual-individuality",
+  "nostalgia-and-memory",
+  "obsession-and-madness",
+  "partner-inadequacy",
+  "partying-and-nightlife",
+  "personal-growth-and-change",
+  "reggaeton-culture",
+  "regret-and-guilt",
+  "relentless-ambition",
+  "resilience-through-hardship",
+  "rivalry-and-superiority",
+  "romantic-pursuit",
+  "secrecy-and-paranoia",
+  "self-worth-and-insecurity",
+  "small-town-roots-and-identity",
+  "street-loyalty-and-danger",
+  "wealth-and-luxury",
+  "women-power"
+ ]
+}

runs/02-02-cluster/vocabulary.txt

@@ -1,50 +0,0 @@
-abandonment-and-solitude
-alcohol-and-substance-abuse
-attraction-and-admiration
-betrayal-and-mistrust
-boastful-self-confidence
-breakup-and-reconciliation
-celebration-and-partying
-contentment-and-joy
-dancing-and-movement
-defiance-and-confrontation
-denial-and-pretense
-designer-fashion-flexing
-devotion-and-sacrifice
-embracing-true-identity
-fame-and-status
-family-and-fatherhood
-fantasy-and-imagination
-fear-of-vulnerability
-female-empowerment
-flirtation-and-seduction
-freedom-and-escape
-heartbreak-and-grief
-hidden-emotional-struggle
-hope-and-disillusionment
-human-connection
-hustle-and-self-made-success
-independence-and-self-reliance
-intimacy-and-connection
-jealousy-and-resentment-from-others
-longing-and-desire
-longing-for-clarity
-love-as-fleeting-and-transient
-loyalty-and-commitment
-mutual-individuality
-nostalgia-and-memory
-obsession-and-madness
-partner-inadequacy
-partying-and-nightlife
-personal-growth-and-change
-reggaeton-culture
-regret-and-guilt
-relentless-ambition
-resilience-through-hardship
-rivalry-and-superiority
-romantic-pursuit
-secrecy-and-paranoia
-self-worth-and-insecurity
-small-town-roots-and-identity
-street-loyalty-and-danger
-wealth-and-luxury

tools/src/pop_fem_audit_tools/commands/cluster_keywords.py

@@ -4,16 +4,19 @@
 #   imacat@mail.imacat.idv.tw (imacat), 2026/8/5
 """The deterministic clusterer of the pooled keywords.
 
-Builds the coding vocabulary from the pooled keyword list, given as
+Builds the coding groups from the pooled keyword list, given as
 the first positional command-line argument, by sentence-embedding
 every keyword and clustering the embeddings: the group membership,
-given as the second positional argument, and the group name
+given as the second positional argument, is written as a CSV file
-vocabulary, given as the third positional argument, are written as
+holding the clustering result alone.  The coding keyword set,
-plain files.  The step is fully deterministic; no LLM call is
+given as the third positional argument, is written as a JSON file
-made.
+holding the group name keywords plus the researcher's a-priori
+topic term (see :data:`EXTRA_KEYWORD`).  The step is fully
+deterministic; no LLM call is made.
 """
 import argparse
 import csv
+import json
 import sys
 import time
 from pathlib import Path
@@ -29,6 +32,9 @@ CLUSTER_EXTRA_MESSAGE: str = (
     " pip install -e \"tools/[cluster]\"")
 """The error message shown when the heavy clustering dependencies
 are not installed."""
+EXTRA_KEYWORD: str = "women-power"
+"""The researcher's a-priori topic term, included in the coding
+keyword set although it is not a clustering result."""
 
 
 def parse_args(argv: list[str] | None) -> argparse.Namespace:
@@ -39,8 +45,8 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
     :return: The parsed arguments.
     """
     parser: argparse.ArgumentParser = argparse.ArgumentParser(
-        description="Build the coding vocabulary by clustering"
+        description="Build the coding groups by clustering the"
-                    " the sentence embeddings of the pooled"
+                    " sentence embeddings of the pooled"
                     " keywords.")
     parser.add_argument(
         "keywords_txt", type=Path,
@@ -49,8 +55,8 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
         "groups_csv", type=Path,
         help="the group membership CSV output file")
     parser.add_argument(
-        "vocabulary_txt", type=Path,
+        "keywords_json", type=Path,
-        help="the group name vocabulary text output file")
+        help="the group name keyword JSON output file")
     parser.add_argument(
         "--model", default=MODEL,
         help=f"the sentence embedding model (default \"{MODEL}\")")
@@ -194,7 +200,9 @@ def write_groups(path: Path, groups: dict[str, list[str]]) -> None:
 
     Writes a CSV file with the header row ``Group,Keyword``, one
     row per member keyword.  Rows are sorted by group name
-    lexicographically, then by keyword lexicographically.
+    lexicographically, then by keyword lexicographically.  The
+    file records the clustering result alone; it holds no row for
+    :data:`EXTRA_KEYWORD`.
 
     :param path: The path of the group membership CSV file to
         write.
@@ -213,30 +221,35 @@ def write_groups(path: Path, groups: dict[str, list[str]]) -> None:
                 writer.writerow([group, keyword])
 
 
-def write_vocabulary(path: Path,
+def write_keywords(path: Path,
-                     groups: dict[str, list[str]]) -> None:
+                   groups: dict[str, list[str]]) -> None:
-    """Write the group name vocabulary text file.
+    """Write the coding keyword set JSON file.
 
-    Writes a plain text file, one group name per line,
+    Writes a JSON file holding a single object with one
-    lexicographically sorted, UTF-8, LF line endings, with a
+    ``keywords`` key, whose value is the lexicographically
-    trailing newline.
+    sorted list of the group names plus :data:`EXTRA_KEYWORD`,
+    UTF-8, with a trailing newline.
 
-    :param path: The path of the vocabulary text file to write.
+    :param path: The path of the keyword JSON file to write.
     :param groups: The keyword members of every group, keyed by
         the group's medoid name.
     :return: None.
     :raises OSError: When the file cannot be written.
     """
+    keywords: list[str] = sorted(
+        [*groups.keys(), EXTRA_KEYWORD])
+    data: dict[str, list[str]] = {"keywords": keywords}
     path.write_text(
-        "".join(f"{x}\n" for x in sorted(groups.keys())),
+        json.dumps(data, ensure_ascii=False, indent=1) + "\n",
         encoding="utf-8")
 
 
 def main(argv: list[str] | None = None) -> int:
-    """Cluster the pooled keywords into the coding vocabulary.
+    """Cluster the pooled keywords into the coding groups.
 
-    Writes the group membership CSV file and the group name
+    Writes the group membership CSV file, holding the clustering
-    vocabulary text file.
+    result alone, and the coding keyword set JSON file, holding
+    the group names plus :data:`EXTRA_KEYWORD`.
 
     :param argv: The command-line arguments, or None for
         ``sys.argv``.
@@ -259,9 +272,9 @@ def main(argv: list[str] | None = None) -> int:
         print(f"error: {error}", file=sys.stderr)
         return 1
     args.groups_csv.parent.mkdir(parents=True, exist_ok=True)
-    args.vocabulary_txt.parent.mkdir(parents=True, exist_ok=True)
+    args.keywords_json.parent.mkdir(parents=True, exist_ok=True)
     write_groups(args.groups_csv, groups)
-    write_vocabulary(args.vocabulary_txt, groups)
+    write_keywords(args.keywords_json, groups)
     elapsed: str = format_duration(time.monotonic() - started)
     print(
         f"done: {len(keywords)} keywords clustered into"

tools/src/pop_fem_audit_tools/commands/export_llm_input.py

@@ -10,11 +10,18 @@ command-line argument.  This is the enforcement point of the
 project's lyrics-only firewall: the output carries only the
 lyrics text of each song, identified by an opaque song key; no
 title, artist, or chart data crosses into the LLM input.
+
+With ``--extras``, each record's ``content`` becomes a JSON
+object serialized as a string, its ``lyrics`` key holding the
+song's lyrics followed by the keys of the given extras file in
+their file order, so a step that needs parameters alongside the
+lyrics can carry them without this module knowing what they mean.
 """
 import argparse
 import json
 import sys
 from pathlib import Path
+from typing import Any
 
 import sqlalchemy as sa
 from sqlalchemy.orm import Session
@@ -36,13 +43,77 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
     parser.add_argument(
         "output_jsonl", type=Path,
         help="the JSONL output file")
+    parser.add_argument(
+        "--extras", type=Path, default=None,
+        help="a JSON file holding a single JSON object of extra"
+             " parameters; when given, each record's \"content\""
+             " becomes a JSON object string with a \"lyrics\" key"
+             " followed by the extras' keys, instead of the bare"
+             " lyrics string")
     return parser.parse_args(argv)
 
 
-def build_lines(session: Session) -> list[str]:
+def _no_duplicate_keys(
+        pairs: list[tuple[str, Any]]) -> dict[str, Any]:
+    """Build a dict from JSON object pairs, rejecting duplicates.
+
+    :param pairs: The key-value pairs of a JSON object, in file
+        order.
+    :return: The pairs as a dict, in file order.
+    :raises ValueError: When a key appears more than once.
+    """
+    result: dict[str, Any] = {}
+    key: str
+    value: Any
+    for key, value in pairs:
+        if key in result:
+            raise ValueError(f"duplicate key \"{key}\" in extras")
+        result[key] = value
+    return result
+
+
+def load_extras(path: Path) -> dict[str, Any]:
+    """Load the extras object from a JSON file.
+
+    :param path: The extras JSON file.
+    :return: The extras, in file order.
+    :raises OSError: When the file cannot be read.
+    :raises ValueError: When the file is not valid JSON, is not
+        a JSON object, has duplicate keys, or has a "lyrics" key.
+    """
+    with open(path, encoding="utf-8") as file:
+        text: str = file.read()
+    try:
+        data: Any = json.loads(
+            text, object_pairs_hook=_no_duplicate_keys)
+    except json.JSONDecodeError as error:
+        raise ValueError(
+            f"invalid JSON in extras file {path}: {error}") \
+            from error
+    if not isinstance(data, dict):
+        raise ValueError(
+            f"extras file {path} must contain a JSON object")
+    if "lyrics" in data:
+        raise ValueError(
+            f"extras file {path} must not have a \"lyrics\" key")
+    return data
+
+
+def build_lines(
+        session: Session,
+        extras: dict[str, Any] | None = None) -> list[str]:
     """Build the JSONL lines of every song's lyrics.
 
+    Without extras, each record's ``content`` is the bare lyrics
+    string.  With extras, ``content`` is a JSON object serialized
+    as a string, whose first key is ``"lyrics"`` holding the
+    lyrics string, followed by the extras' keys in their given
+    order.
+
     :param session: The database session.
+    :param extras: The extra parameters merged into every
+        record's content alongside the lyrics, in the order they
+        are to appear, or None for the bare lyrics string.
     :return: The JSON lines, one per song, ordered by song ID.
     :raises ValueError: When a song has no lyrics.
     """
@@ -52,8 +123,15 @@ def build_lines(session: Session) -> list[str]:
         if song.lyrics is None:
             raise ValueError(
                 f"song {song.id} \"{song.title}\": no lyrics")
+        content: str
+        if extras is None:
+            content = song.lyrics
+        else:
+            payload: dict[str, Any] = {"lyrics": song.lyrics}
+            payload.update(extras)
+            content = json.dumps(payload, ensure_ascii=False)
         record: dict[str, str] = {
-            "id": f"song-{song.id}", "content": song.lyrics}
+            "id": f"song-{song.id}", "content": content}
         lines.append(json.dumps(record, ensure_ascii=False))
     return lines
 
@@ -69,7 +147,10 @@ def main(argv: list[str] | None = None) -> int:
     session: Session = ds.get_db()
     lines: list[str]
     try:
-        lines = build_lines(session)
+        extras: dict[str, Any] | None = None
+        if args.extras is not None:
+            extras = load_extras(args.extras)
+        lines = build_lines(session, extras)
     except (OSError, sa.exc.SQLAlchemyError, ValueError) as error:
         print(f"error: {error}", file=sys.stderr)
         return 1

tools/tests/test_cluster_keywords.py

@@ -5,6 +5,7 @@
 """Unit tests for the keyword clusterer module."""
 import csv
 import io
+import json
 import tempfile
 import unittest
 from contextlib import redirect_stderr
@@ -32,7 +33,7 @@ class TestClusterKeywords(unittest.TestCase):
         self.__dir: Path = Path(tmp.name)
         self.__keywords_txt: Path = self.__dir / "keywords.txt"
         self.__groups_csv: Path = self.__dir / "groups.csv"
-        self.__vocabulary_txt: Path = self.__dir / "vocabulary.txt"
+        self.__keywords_json: Path = self.__dir / "keywords.json"
 
     @staticmethod
     def __two_cluster_vectors() -> Vectors:
@@ -102,7 +103,7 @@ class TestClusterKeywords(unittest.TestCase):
         """
         argv: list[str] = [
             str(self.__keywords_txt), str(self.__groups_csv),
-            str(self.__vocabulary_txt)]
+            str(self.__keywords_json)]
         argv.extend(extra_args or [])
         fake: Any = self.__fake_encode(
             vectors if vectors is not None
@@ -125,16 +126,14 @@ class TestClusterKeywords(unittest.TestCase):
                   newline="") as file:
             return list(csv.reader(file))
 
-    def __read_vocabulary(self) -> list[str]:
+    def __read_keywords(self) -> list[str]:
-        """Read the vocabulary text file.
+        """Read the group name keyword JSON file.
 
-        :return: The group names, one per line, with the trailing
+        :return: The group names under the "keywords" key.
-            empty line from the final newline removed.
         """
-        lines: list[str] = self.__vocabulary_txt.read_text(
+        data: dict[str, list[str]] = json.loads(
-            encoding="utf-8").split("\n")
+            self.__keywords_json.read_text(encoding="utf-8"))
-        self.assertEqual(lines[-1], "")
+        return data["keywords"]
-        return lines[:-1]
 
     def test_groups_csv_header_and_ordering(self) -> None:
         """Test the header row and the group/keyword ordering of
@@ -170,17 +169,34 @@ class TestClusterKeywords(unittest.TestCase):
         self.assertEqual(
             sorted(x[1] for x in rows), sorted(keywords))
 
-    def test_vocabulary_file_sorted_medoids(self) -> None:
+    def test_keywords_json_sorted_medoids(self) -> None:
-        """Test that the vocabulary file holds the sorted medoid
+        """Test that the keyword JSON file holds the sorted medoid
-        group names."""
+        group names plus the extra a-priori keyword."""
         self.__write_keywords([
             "a-left", "a-center", "a-right",
             "b-north", "b-middle", "b-south"])
         status: int
         status, _ = self.__run_cluster()
         self.assertEqual(status, 0)
+        keywords: list[str] = self.__read_keywords()
         self.assertEqual(
-            self.__read_vocabulary(), ["a-center", "b-middle"])
+            keywords,
+            ["a-center", "b-middle", cluster_keywords.EXTRA_KEYWORD])
+        self.assertEqual(keywords, sorted(keywords))
+        self.assertEqual(len(keywords), 2 + 1)
+
+    def test_extra_keyword_absent_from_groups_csv(self) -> None:
+        """Test that the extra a-priori keyword appears in no row
+        of the group membership CSV file."""
+        self.__write_keywords([
+            "a-left", "a-center", "a-right",
+            "b-north", "b-middle", "b-south"])
+        status: int
+        status, _ = self.__run_cluster()
+        self.assertEqual(status, 0)
+        rows: list[list[str]] = self.__read_groups()
+        for row in rows:
+            self.assertNotIn(cluster_keywords.EXTRA_KEYWORD, row)
 
     def test_duplicate_keyword_rejected(self) -> None:
         """Test that a duplicate keyword line fails the run
@@ -192,7 +208,7 @@ class TestClusterKeywords(unittest.TestCase):
         self.assertEqual(status, 1)
         self.assertIn("duplicate keyword", stderr)
         self.assertFalse(self.__groups_csv.exists())
-        self.assertFalse(self.__vocabulary_txt.exists())
+        self.assertFalse(self.__keywords_json.exists())
 
     def test_empty_input_rejected(self) -> None:
         """Test that an empty keyword file fails the run without
@@ -204,4 +220,4 @@ class TestClusterKeywords(unittest.TestCase):
         self.assertEqual(status, 1)
         self.assertIn("no keywords", stderr)
         self.assertFalse(self.__groups_csv.exists())
-        self.assertFalse(self.__vocabulary_txt.exists())
+        self.assertFalse(self.__keywords_json.exists())

tools/tests/test_export_llm_input.py

@@ -75,18 +75,32 @@ class TestExportLlmInput(unittest.TestCase):
         finally:
             session.close()
 
-    def __run_export(self) -> tuple[int, str]:
+    def __run_export(
+            self, extras: Path | None = None) -> tuple[int, str]:
         """Run the exporter with the standard error captured.
 
+        :param extras: The extras JSON file, or None for none.
         :return: A tuple of the exit status and the standard
             error.
         """
+        argv: list[str] = [str(self.__output)]
+        if extras is not None:
+            argv += ["--extras", str(extras)]
         stderr: io.StringIO = io.StringIO()
         with redirect_stderr(stderr):
-            status: int = export_llm_input.main(
+            status: int = export_llm_input.main(argv)
-                [str(self.__output)])
         return status, stderr.getvalue()
 
+    def __write_extras(self, text: str) -> Path:
+        """Write an extras file with the given raw text.
+
+        :param text: The raw file content.
+        :return: The path of the written extras file.
+        """
+        path: Path = self.__dir / "extras.json"
+        path.write_text(text, encoding="utf-8")
+        return path
+
     @staticmethod
     def __read_records(path: Path) -> list[dict[str, str]]:
         """Read the JSONL records of a file.
@@ -167,3 +181,50 @@ class TestExportLlmInput(unittest.TestCase):
             nested)
         self.assertEqual(records, [
             {"id": "song-1", "content": "hello lyrics\n"}])
+
+    def test_extras_merges_lyrics_first_in_file_order(
+            self) -> None:
+        """Test that with extras, content is a JSON object whose
+        first key is lyrics, followed by the extras' keys in
+        their file order."""
+        self.__seed([("Hello", "Adele", "hello lyrics\n")])
+        extras: Path = self.__write_extras(
+            '{"b": 2, "a": 1}')
+        status: int
+        stderr: str
+        status, stderr = self.__run_export(extras)
+        self.assertEqual(status, 0)
+        records: list[dict[str, str]] = self.__read_records(
+            self.__output)
+        self.assertEqual(len(records), 1)
+        content: dict[str, Any] = json.loads(
+            records[0]["content"])
+        self.assertEqual(
+            list(content.keys()), ["lyrics", "b", "a"])
+        self.assertEqual(content["lyrics"], "hello lyrics\n")
+        self.assertEqual(content["b"], 2)
+        self.assertEqual(content["a"], 1)
+
+    def test_extras_non_object_fails(self) -> None:
+        """Test that a non-object extras file is rejected."""
+        self.__seed([("Hello", "Adele", "hello lyrics\n")])
+        extras: Path = self.__write_extras('[1, 2]')
+        status: int
+        stderr: str
+        status, stderr = self.__run_export(extras)
+        self.assertEqual(status, 1)
+        self.assertIn("error:", stderr)
+        self.assertFalse(self.__output.exists())
+
+    def test_extras_with_lyrics_key_fails(self) -> None:
+        """Test that an extras file carrying a "lyrics" key is
+        rejected."""
+        self.__seed([("Hello", "Adele", "hello lyrics\n")])
+        extras: Path = self.__write_extras(
+            '{"lyrics": "not allowed"}')
+        status: int
+        stderr: str
+        status, stderr = self.__run_export(extras)
+        self.assertEqual(status, 1)
+        self.assertIn("error:", stderr)
+        self.assertFalse(self.__output.exists())