Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,9 @@ The frontend never downloads the whole catalog, however large it is: each page
load asks for a random deck of 200 (`deck(size: 200)`), and a new deck is drawn
when the learner finishes one. The server caps a deck at
`EXERCISE_DECK_MAX_SIZE` (default 500), settable under `env:` like any other
backend variable.
backend variable. A client paging through `exercises` with
`pagination: { offset, limit }` gets at most `STRAWBERRY_PAGINATION_MAX_LIMIT`
(default 100) per page.

### Translations

Expand Down
32 changes: 11 additions & 21 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,34 +137,24 @@ translation of every sentence ingestion can select, and **Ingest the corpus**
loads it with `load_translations` after the corpus. Nothing needs a key or a
network.

The file is made once, on a developer's machine, with the Anthropic API. The key
lives only in that shell. It is not in `.env`, the Secret, or the chart. The
command reads the Common Voice release directly, the same selection ingestion
makes, so it needs neither the cluster nor a database. Start with a trial run:
To add to it, `translate_catalog` runs on a developer's machine against a local
OpenAI-compatible server, such as Lemonade. No key is needed. The command reads
the Common Voice release directly, the same selection ingestion makes, so it
needs neither the cluster nor a database. Start the server, then do a trial run
with the model name it lists:

```bash
cd src/backend
export ANTHROPIC_API_KEY=sk-ant-...
uv run python manage.py translate_catalog "$TYPELEARN_CORPUS_DIR" --limit 20
```

Read what came back, then run it without `--limit`. For the whole catalog (about
21k sentences into 5 languages) that is about 4,300 requests, a one-time spend.
A re-run only requests the languages an entry is missing, and an interrupted run
keeps everything that was saved: the file is written every 30 seconds and on
exit. `--model` and `--effort` trade quality for cost, and `--languages en,ru`
narrows the targets. Review the diff and commit the file.

To use a local OpenAI-compatible server instead, such as Lemonade, start it and
pass the model name it lists. No key is needed:

```bash
uv run python manage.py translate_catalog "$TYPELEARN_CORPUS_DIR" \
--provider openai-compatible --model <model name> --batch-size 10 --limit 20
--model <model name> --batch-size 10 --limit 20
```

`--base-url` defaults to Lemonade's `http://localhost:8000/api/v1`. Local models
are less reliable than Claude, so keep the batches small and read the trial run.
are not always reliable, so keep the batches small and read what came back
before running it without `--limit`. A re-run only requests the languages an
entry is missing, and an interrupted run keeps everything that was saved: the
file is written every 30 seconds and on exit. `--languages en,ru` narrows the
targets. Review the diff and commit the file.

To fix a bad translation, edit its line and commit. Loading replaces the text
and clears that translation's ratings, because they were cast on different
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,37 +46,58 @@ accepts.
```graphql
enum TranslationRatingValue { UP DOWN NONE }

type Mutation {
rateTranslation(translationId: ID!, clientId: String!, value: TranslationRatingValue!): Translation!
proposeTranslation(exerciseId: ID!, language: String!, text: String!, clientId: String!): ProposeTranslationResult!
input OneToManyInput { set: ID }

input TranslationRatingInput {
translation: OneToManyInput!
clientId: String!
value: TranslationRatingValue!
}

input TranslationCreateInput {
exercise: OneToManyInput!
language: String!
text: String!
clientId: String
}

type TranslationMutations {
rateTranslation(data: TranslationRatingInput!): Translation!
proposeTranslation(data: TranslationCreateInput!): Translation!
}

type ProposeTranslationResult { accepted: Boolean! }
type Mutation {
translation: TranslationMutations!
}
```

`clientId` is an opaque random identifier that the browser generates and keeps.
It carries no authority beyond deduplicating ratings and scoping rate limits.
`rateTranslation` with `NONE` withdraws the caller's rating, and it applies only
to published translations. `proposeTranslation` stores a pending suggestion.
Invalid input (unknown ids, unsupported language, blank or over-long text,
malformed client id, a duplicate of an existing translation) SHALL return a
GraphQL error with a stable, machine-readable code and store nothing.
to published translations. `proposeTranslation` stores a pending suggestion and
returns it. Invalid input (unknown ids, unsupported language, blank or
over-long text, malformed or missing client id, a duplicate of an existing
translation) SHALL return a GraphQL error with a stable, machine-readable code
and store nothing.

#### Scenario: Rating is idempotent per client
- **WHEN** the same `clientId` sends `rateTranslation(..., value: UP)` twice
- **WHEN** the same `clientId` sends `translation { rateTranslation(data: {..., value: UP}) }` twice
- **THEN** the translation's `upVotes` increased by one in total, not two

#### Scenario: Rating an unpublished translation
- **WHEN** a client rates a pending or rejected translation
- **THEN** the response is an error and no rating is stored
- **THEN** the response is exactly the error an id no translation has gets,
so it does not reveal that the hidden translation exists, and no rating is
stored

#### Scenario: Proposal stored as pending
- **WHEN** a client proposes a valid new translation
- **THEN** `accepted` is true and a pending learner translation exists, which
the `translation` field does not return
- **THEN** the mutation returns the new translation, with origin `LEARNER`,
and it is stored as pending, which the `translation` field does not return

#### Scenario: Over-long proposal
- **WHEN** a client proposes text longer than the configured maximum
- **WHEN** a client proposes text longer than the longest translation the
catalog stores
- **THEN** the response is an error with a length code and nothing is stored

### Requirement: Write rate limits
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,11 @@
- [x] 6.3 Add the `anthropic` SDK to the dev dependency group and implement `AnthropicProvider` (structured JSON-array output, length and blank validation, key read from `ANTHROPIC_API_KEY` in the developer's shell, fails early when the key is missing; model, effort and refusal fallback passed in, not read from settings). Verify with unit tests using a stubbed client: a good batch, a wrong length, a blank item, and a missing key.
- [x] 6.4 Extract `select_from_release` from `load_corpus`, and implement `translate_catalog CORPUS_ROOT` (design D8: the full selection read from the release with no database, target defaults, `--languages/--locale/--provider/--model/--effort/--batch-size/--limit/--dir`, requesting only the missing languages, the offline provider refusing the committed fixture, saves every 30 s and on exit, failure summary, non-zero exit). Verify with command tests: a fresh run, no database queries, a re-run makes no provider calls and leaves the file byte-identical, a resume after a failed batch, an interrupted run keeps what came back, new sentences and languages only, hand corrections kept, and the source language excluded.
- [x] 6.5 Rewrite `load_translations` to read the fixture (match by sentence, then sentence id; create missing machine rows; update changed text while clearing its ratings; leave learner rows alone; skip over-long text; report counts; a missing file is a no-op, an invalid one fails). Verify with command tests: loading into a fresh catalog, an idempotent re-load that keeps ratings, a changed text clearing ratings, unmatched entries reported, and a database rebuild restoring everything with zero provider calls.
- [ ] 6.6 Run `translate_catalog` against the Thai release with a real key: first `--limit 20`, reviewing the output and the cost, then the full run. Commit `exercises/fixtures/translations/th.yaml`.
- [x] 6.6 Run `translate_catalog` against the Thai release with a real key: first `--limit 20`, reviewing the output and the cost, then the full run. Commit `exercises/fixtures/translations/th.yaml`.

## 7. Chart

- [ ] 7.1 Change the ingest Job command to `load_corpus ... && load_translations`. Verify with `helm template` output, and by ingesting in a local cluster once the fixture exists.
- [x] 7.1 Change the ingest Job command to `load_corpus ... && load_translations`. Verify with `helm template` output, and by ingesting in a local cluster once the fixture exists.
- [x] 7.2 Keep provider keys and translation out of the chart, the Tiltfile, and `.env.example`: no translate Job, no `translate` values, no `_API_KEY` secret rule. Verify that `helm template` renders no Job other than ingest.

## 8. Frontend
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-10-24
Loading
Loading