Remove Spanish-specific TSV tooling

README.md

@@ -204,59 +204,36 @@ hablar 9
 Extract vocabulary from an Anki TSV export file instead of using AnkiConnect:
 
 ```shell
-saiki words --lang es --input Español.txt --field 2 --output words_es_content.txt --debug words_es_debug.tsv
-```
-
-Additional file-based options:
-
-```shell
-saiki words --lang es --input Español.txt --field 2 --include-proper-nouns
-saiki words --lang es --input Español.txt --field 2 --function-words
-saiki words --lang es --input Español.txt --field 2 --lemma-corrections my_fixes.tsv
-saiki words --lang es --input Español.txt --field 2 --bad-lemma-file bad_lemmas.txt
+saiki words --lang es --input Español.txt --field 2 --field-section first --output words_es_content.txt --debug words_es_debug.tsv
+saiki words --lang jp --input Japanese.txt --field 2 --output words_jp_content.txt
 ```
 
 When `--input` is provided, `--field` specifies the 1-based column index of the
-Spanish text (default 2). The audio column (index 1) and tags column are
-automatically skipped. File-based NLP extraction is currently Spanish-specific;
-the older AnkiConnect-based `saiki words jp` flow is unchanged.
+text field (default 2). Only that column is mined; other columns such as audio
+or Anki tags are ignored. By default, all blank-line-separated sections inside
+the selected field are kept. Use `--field-section first` when your card format
+stores target-language text before a translation or note in the same field.
 
 The file-based pipeline:
 - Parses Anki `#` header lines (`#separator:tab`, `#html:true`, `#tags column:N`)
 - Uses Python's `csv` module for robust TSV parsing
-- Removes `[sound:...mp3]` markers, HTML tags, and English glosses after
-  `<br><br>` from the field text
-- Applies safe Spanish lemmatisation (multi-word lemmas like `ayudar yo` or
-  `lavar él` are rejected; bad lemmas like `comar` -> `comer` are corrected)
+- Removes `[sound:...mp3]` markers, HTML tags, URLs, and email addresses from
+  the field text
+- Uses the configured language's spaCy model, token filter, and output format
 - Tracks POS counts, surface forms, example sentences, and source line numbers
-- Tracks original spaCy lemmas and correction/fallback reasons for debugging
+- Tracks original spaCy lemmas for debugging
 
 Output files produced:
-- `words_es_content.txt` — cleaned content vocabulary (NOUN, VERB, ADJ, ADV)
-- `words_es_debug.tsv` — per-lemma debug info
-- `words_es_proper_nouns.txt` — proper nouns (only with `--include-proper-nouns`)
-- `words_es_function_words.txt` — function words (only with `--function-words`)
-- `words_es_suspicious_tokens.txt` — lemmas that required correction
+- `words_<lang>_content.txt` — cleaned vocabulary
+- `words_<lang>_debug.tsv` — per-entry debug info (only with `--debug`)
 
 Debug TSV example:
 
 ```text
-lemma	count	pos_counts	top_surface_forms	example_sentences	source_lines	original_lemmas	lemma_statuses	status
-comer	8	VERB:8	como, come, comen	Yo como manzanas.; Ustedes los comen con arroz.	70,979	comar, comer	corrected:comar->comer:1, ok:7
+entry	count	pos_counts	top_surface_forms	example_sentences	source_lines	original_lemmas
+comer	8	VERB:8	como, come, comen	Yo como manzanas.; Ustedes los comen con arroz.	70,979	comer
 ```
 
-### Lint Spanish Cards
-
-Check an Anki TSV export for suspicious or awkward Spanish:
-
-```shell
-saiki lint-anki-es --input Español.txt --field 2 --output suspicious_cards_es.tsv
-```
-
-Detects known errors and rule-based suspicious patterns and produces a TSV
-report with: source line, original text, cleaned text, reason, and suggested
-fix.
-
 ### Compare
 
 Compare deck vocabulary against a target list:
@@ -270,7 +247,7 @@ Normalises case and optionally strips accents for matching, so `cómo` and
 `como` are treated as the same word. Use `--min-frequency` to ignore accidental
 low-frequency words in the deck.
 
-
+### YouTube
 
 Mine vocabulary or sentence rows from YouTube subtitles.