Feature/ftrack provider

.gitignore

@@ -29,3 +29,4 @@ backend/docker-compose.override.yml
 
 # Cursor files
 *.issue-body-temp.md*
+.dna-dev/

DEPLOYMENT.md

@@ -351,7 +351,7 @@ The following secrets must be configured in GitHub repository settings:
 |----------|-------|
 | `PYTHONUNBUFFERED` | 1 |
 | `STORAGE_PROVIDER` | mongodb |
-| `PRODTRACK_PROVIDER` | shotgrid |
+| `PRODTRACK_PROVIDER` | `shotgrid`, `ftrack` or `mock` |
 | `LLM_PROVIDER` | openai |
 | `AUTH_PROVIDER` | `none` (noop) for local dev; `google` for production |
 | `CORS_ALLOWED_ORIGINS` | Comma-separated list of allowed origins |

QUICKSTART.md

@@ -278,7 +278,16 @@ The React app will be available at `http://localhost:5173`.
 | `SHOTGRID_URL` | Yes\* | - | ShotGrid site URL (required when using ShotGrid) |
 | `SHOTGRID_API_KEY` | Yes\* | - | ShotGrid API key (required when using ShotGrid) |
 | `SHOTGRID_SCRIPT_NAME` | Yes\* | - | ShotGrid script name (required when using ShotGrid) |
-| `PRODTRACK_PROVIDER` | No | `shotgrid` | `shotgrid` or `mock`; set to `mock` to use the read-only mock DB without ShotGrid |
+| `PRODTRACK_PROVIDER` | No | `shotgrid` | `shotgrid`, `ftrack` or `mock`; set to `mock` to use the read-only mock DB without ShotGrid |
+| `FTRACK_SERVER` | Yes\* | - | ftrack server URL (required when `PRODTRACK_PROVIDER=ftrack`) |
+| `FTRACK_API_KEY` | Yes\* | - | ftrack API key (required when `PRODTRACK_PROVIDER=ftrack`) |
+| `FTRACK_API_USER` | Yes\* | - | ftrack API user (required when `PRODTRACK_PROVIDER=ftrack`) |
+| `FTRACK_PLAYLIST_ENTITY` | No | `AssetVersionList` | Which ftrack entity acts as a playlist: `AssetVersionList` (Lists) or `ReviewSession` (Client Reviews) |
+| `FTRACK_ID_MAP` | No | `mongodb` | Where the ftrack UUID-to-int id map is persisted: `mongodb`, `sqlite` or `memory` |
+| `FTRACK_ID_MAP_PATH` | No | `/tmp/dna_ftrack_id_map.db` | SQLite file for the id map when `FTRACK_ID_MAP=sqlite` |
+| `FTRACK_CACHE_SECONDS` | No | `300` | How long ftrack project and version-status lookups are reused; `0` disables |
+| `FTRACK_MOVIE_COMPONENTS` | No | - | Component names to resolve `movie_path` from, most preferred first (e.g. `movie,main`); empty means no path |
+| `FTRACK_FRAME_COMPONENTS` | No | - | Component names to resolve `frame_path` from, most preferred first |
 | `MONGODB_URL` | No | `mongodb://mongo:27017` | MongoDB connection string |
 | `STORAGE_PROVIDER` | No | `mongodb` | Storage provider type |
 | `VEXA_API_KEY` | Yes | - | API key for Vexa transcription service |

backend/README.md

@@ -26,6 +26,8 @@ Production Tracking providers are the services that provide data to the backend
 
 **ShotGrid** is the primary production tracking integration. To run without a ShotGrid seat, set **`PRODTRACK_PROVIDER=mock`**; the mock provider is read-only and backed by a SQLite database under `src/dna/prodtrack_providers/mock_data/`. See [Mock production tracking](#mock-production-tracking) below.
 
+**ftrack** is also supported: set `PRODTRACK_PROVIDER=ftrack` with `FTRACK_SERVER`, `FTRACK_API_KEY` and `FTRACK_API_USER`. See [ftrack production tracking](#ftrack-production-tracking) below.
+
 ### LLM
 
 LLM providers are the services that provide the LLM functionality to the backend.
@@ -80,6 +82,7 @@ To configure ShotGrid and other local settings, create a local docker-compose ov
 
 2. Edit `docker-compose.local.yml` and set at least:
    - **ShotGrid:** To use ShotGrid, set `PRODTRACK_PROVIDER=shotgrid` (or leave unset) and set `SHOTGRID_URL`, `SHOTGRID_API_KEY`, and `SHOTGRID_SCRIPT_NAME`. To run without ShotGrid, set `PRODTRACK_PROVIDER=mock`; see [Mock production tracking](#mock-production-tracking).
+   - **ftrack:** To use ftrack instead, set `PRODTRACK_PROVIDER=ftrack` and `FTRACK_SERVER`, `FTRACK_API_KEY`, `FTRACK_API_USER`; see [ftrack production tracking](#ftrack-production-tracking).
    - **Auth (local dev):** Keep `AUTH_PROVIDER=none` so the noop provider is used and you can sign in with any email. Change to `AUTH_PROVIDER=google` only if you need to test Google OAuth locally.
    - **LLM:** Choose an LLM provider and matching credentials. Examples:
 
@@ -119,6 +122,19 @@ To configure ShotGrid and other local settings, create a local docker-compose ov
 
 3. The `docker-compose.local.yml` file is gitignored, so your credentials will not be committed to the repository.
 
+### ftrack production tracking
+
+Set **`PRODTRACK_PROVIDER=ftrack`** together with `FTRACK_SERVER`, `FTRACK_API_KEY` and `FTRACK_API_USER`.
+
+- **Playlists:** ftrack has two entities a studio might treat as a playlist. `FTRACK_PLAYLIST_ENTITY` picks which one: `AssetVersionList` (ftrack Lists, the default) or `ReviewSession` (ftrack Client Reviews). Both the schema names and the UI names (`Lists`, `ClientReviews`) are accepted. The setting is re-read on every playlist operation, so it can change without a restart — this is the seam a per-user preference will plug into.
+- **Entity ids:** ftrack keys entities by UUID while DNA ids are ints, so the provider assigns a stable int per UUID and persists the pairing. `FTRACK_ID_MAP` selects where: `mongodb` (default, shared by every backend instance), `sqlite` (single host, path from `FTRACK_ID_MAP_PATH`) or `memory` (tests). **The map must outlive the process** — draft notes, transcripts and stored segments all reference these ids, and losing the map orphans them.
+- **Flat queries, fetched in layers:** projections never use a dotted path like `asset.parent.object_type.name`. A deep projection makes the ftrack server build the joins and is reliably slower than asking for each layer on its own, so `_hydrate_versions` walks them — versions, then their assets, then those assets' parents, then the parents' object types, plus statuses, users, tasks and projects alongside — and stitches the nested shape the converters expect. Each layer is one `id in (...)` query for the whole batch, so a playlist costs about ten queries whatever its size. Filters use flat foreign keys (`project_id`) for the same reason; the only join left is a version's `entity`, which has to reach through its asset. Two tests hold this line: one rejects a dotted path in any projection constant, another rejects one in any query a playlist load issues.
+- **Batching elsewhere:** id pairings for a batch are warmed in one store read rather than one per reference, and cached in process (safe — a pairing never changes). Component paths go through `pick_locations`/`get_filesystem_paths`: one availability request plus one per location, not two per version. Note recipients resolve in a single query. Project and version-status lookups are cached for `FTRACK_CACHE_SECONDS` (default 300, `0` disables) because one publish round asks for them repeatedly; the TTL exists because the provider is a process-lifetime singleton and a schema edit should not need a redeploy. Adding a new read path is worth the same treatment: fetch ids, hydrate in layers, `_warm_ids(...)`, then convert.
+- **Notes:** ftrack notes attach to exactly one parent and have no subject or cc field. DNA's subject is kept in the note's metadata, cc recipients are merged into the recipient list, and links beyond the primary one are dropped.
+- **Thumbnails:** ftrack's own thumbnail URL carries the API key in its query string, so thumbnails are proxied through `GET /api/ftrack-thumbnails/{version_id}` instead of being handed to the browser.
+- **Media paths:** `movie_path` and `frame_path` are empty unless you opt in, because neither kind of ftrack component yields a path DNA can use. The reviewable encodings (`ftrackreview-mp4`, `ftrackreview-webm`) live in the ftrack.server location, which has no filesystem path at all; `movie` and `main` live on studio disk locations that a containerised DNA almost certainly cannot read. Resolving also costs a location round trip per component, per version in a playlist. If DNA runs inside the studio network with the storage mounted, set `FTRACK_MOVIE_COMPONENTS` (e.g. `movie,main`) and `FTRACK_FRAME_COMPONENTS` — comma-separated, most preferred first, since ftrack often carries several encodings of the same version.
+- **Transcripts:** ftrack has no equivalent of ShotGrid's spare custom-entity slots, so a published transcript is a note on the version, tagged through note metadata.
+
 ### Mock production tracking
 
 When **`PRODTRACK_PROVIDER=mock`** is set, the backend uses a read-only mock provider backed by `src/dna/prodtrack_providers/mock_data/mock.db`. The mock must be explicitly selected; there is no automatic fallback when ShotGrid credentials are missing. This allows the full stack to run without ShotGrid access.