From 6009bb3cb651299f5ae7ad095e42cfa5b118a961 Mon Sep 17 00:00:00 2001 From: dorukustay1 Date: Mon, 12 Oct 2026 16:12:07 +0100 Subject: [PATCH 1/5] Update query language and SDK docs for TuringDB v3 Split the single Cypher page into a Query Language group (overview, matching, paths, results/aggregation, subqueries, functions/types, writing, procedures/commands) and refresh the cheatsheet. Examples were run against turingdb 3.0.1. Also update pages that contradicted v3: LOAD CSV + MATCH now works, CREATE ... SET works, edgeType() is now type(), UNWIND/aggregates are unrestricted, JSONL ids may have gaps and arrays load as lists, GML property naming, shortestPath output format, vector search (score, HNSW, use ids directly), LOAD PARQUET types, and the SDK's native binary protocol (USE_TURING_PROTO), auth, embedded data_dir (no ~ expansion) and error handling. --- claude-skill.mdx | 11 +- commands.mdx | 57 +++ docs.json | 7 + graph_dev/create_change.mdx | 5 +- graph_dev/create_nodes_edges.mdx | 11 +- graph_dev/submit_change.mdx | 8 +- graph_dev/time_travelling.mdx | 8 + graph_dev/update_properties.mdx | 36 +- import_data/csv.mdx | 33 +- import_data/gml.mdx | 10 +- import_data/jsonl.mdx | 38 +- import_data/parquet.mdx | 93 ++++- index.mdx | 2 +- pythonsdk/get_started.mdx | 2 +- pythonsdk/reference.mdx | 107 ++++- query/aggregation.mdx | 151 +++++++ query/cheatsheet.mdx | 599 +++++++++------------------ query/cypher_subset.mdx | 690 +++++-------------------------- query/functions.mdx | 121 ++++++ query/matching.mdx | 203 +++++++++ query/paths.mdx | 98 +++++ query/procedures.mdx | 140 +++++++ query/subqueries.mdx | 46 +++ query/writing.mdx | 236 +++++++++++ shortest-path.mdx | 79 ++-- troubles/troubles.mdx | 43 ++ tutorials/create_graph.mdx | 6 +- vector-search.mdx | 132 ++++-- 28 files changed, 1832 insertions(+), 1140 deletions(-) create mode 100644 query/aggregation.mdx create mode 100644 query/functions.mdx create mode 100644 query/matching.mdx create mode 100644 query/paths.mdx create mode 100644 query/procedures.mdx create mode 100644 query/subqueries.mdx create mode 100644 query/writing.mdx diff --git a/claude-skill.mdx b/claude-skill.mdx index 3b2cc30..e76243f 100644 --- a/claude-skill.mdx +++ b/claude-skill.mdx @@ -41,11 +41,12 @@ The skill is structured so Claude only loads the reference it needs: | Module | Covers | | --- | --- | -| `startup` | Installation, starting the server, connecting, loading a graph | -| `querying` | `MATCH`, `WHERE`, joins, built-in functions | -| `writing` | `CREATE`, `SET`, the change and commit workflow | -| `algorithms` | Shortest path, vector search | -| `introspection` | Schema exploration, Python SDK reference | +| `startup` | Installation, starting the server, connecting (HTTP, authentication, embedded), loading a graph | +| `querying` | `MATCH`, `OPTIONAL MATCH`, paths, `WITH`, aggregation, `UNWIND`, subqueries, built-in functions, `LOAD CSV` | +| `writing` | `CREATE`, `MERGE`, `SET`, `DELETE`, the change and commit workflow, indexes | +| `importing` | CSV, Parquet, JSONL, GML and Neo4j imports | +| `algorithms` | Shortest path, vector search, embeddings, GNN sampling | +| `introspection` | Schema exploration, versioning and time travel, Python SDK reference | ## Links diff --git a/commands.mdx b/commands.mdx index 5080d6f..b047ecc 100644 --- a/commands.mdx +++ b/commands.mdx @@ -25,8 +25,11 @@ turingdb start [options] | `-reset-default` | Delete the default graph before starting | off | | `-ui` | Launch the built-in visualizer proxy | off | | `-ui-port ` | Visualizer proxy port | `8080` | +| `-auth-on` | Require a bearer token on every request. The token is read from the `TURINGDB_AUTH_TOKEN` environment variable, and startup fails if it is not set | off | | `-start-timeout ` | Time to wait for daemon readiness | `500` | +Flags use a **single dash** (`-demon`, not `--demon`). + ### Examples Interactive mode (default): @@ -53,6 +56,34 @@ Load a graph at startup: turingdb -load mygraph ``` +Require authentication: + +```bash +TURINGDB_AUTH_TOKEN=s3cret turingdb start -demon -auth-on +``` + +Serve the binary protocol instead of JSON (see [Environment variables](#environment-variables)): + +```bash +USE_TURING_PROTO=1 turingdb start -demon +``` + + + Server logs are written to `/logs/turingdb.log`. When an import such as `LOAD JSONL` fails, the details are only in that file. + + +### Interactive shell + +Without `-demon`, `turingdb start` serves HTTP **and** opens an interactive Cypher shell. Besides Cypher queries and [engine commands](/query/procedures#commands), the shell has a few local commands: + +| Command | Description | +|---|---| +| `cd ` | Switch the active graph | +| `checkout change-` | Work on a change. `checkout ` opens a past commit, and `checkout` returns to the latest commit of main | +| `read ` | Run the queries in a file | +| `help` | List the available commands | +| `quit` / `exit` | Leave the shell | + --- ## UI @@ -116,3 +147,29 @@ Stop an instance running in a custom directory: ```bash turingdb stop -turing-dir /data/myturing ``` + +--- + +## Environment variables + +| Variable | Read by | Effect | +|---|---|---| +| `USE_TURING_PROTO` | server | `1` makes the server speak TuringDB's **binary protocol** instead of JSON over HTTP. The server is then binary-only: use `TuringDB(type="native")` to connect, and `curl`, the JSON client and the visualizer stop working against it. See [client backends](/pythonsdk/reference#client-backends) | +| `TURINGDB_AUTH_TOKEN` | server and Python SDK | With `-auth-on`, the token the server requires. The SDK sends it as a bearer token when set | +| `TURINGDB_TYPE` | Python SDK | Default client backend (`json`, `native` or `embedded`) when `type` is not passed | + +--- + +## HTTP API + +The Python SDK is a thin client over a single endpoint, so you can query a server from any language. With the default (JSON) server: + +```bash +curl -s -X POST 'http://localhost:6666/query?graph=mygraph' -d "MATCH (n) RETURN count(n)" +``` + +- The request body is the query text. The optional query-string parameters `change=` and `commit=` target a change or a past commit. +- With `-auth-on`, send `Authorization: Bearer `. A missing or wrong token returns HTTP `401` with an empty body. +- A successful response is `{"header": {"column_names": [...], "column_types": [...]}, "data": [chunk, ...], "time": }`, where each chunk is a list of columns. +- An error response is `{"error": "PARSE_ERROR", "error_details": "...", "time": }`. The code is one of `PARSE_ERROR`, `ANALYZE_ERROR`, `PLAN_ERROR`, `EXEC_ERROR`, `CHANGE_NOT_FOUND` or `COMMIT_NOT_FOUND`. +- Queries on a non-HEAD commit need `LOAD COMMIT ''` first when you talk to the API directly. The CLI and the SDK do this for you. diff --git a/docs.json b/docs.json index 7c0ceb1..9b64f3b 100644 --- a/docs.json +++ b/docs.json @@ -64,6 +64,13 @@ "group": "Query Language", "pages": [ "query/cypher_subset", + "query/matching", + "query/paths", + "query/aggregation", + "query/subqueries", + "query/functions", + "query/writing", + "query/procedures", "query/cheatsheet" ] }, diff --git a/graph_dev/create_change.mdx b/graph_dev/create_change.mdx index 37e3e71..a18e682 100644 --- a/graph_dev/create_change.mdx +++ b/graph_dev/create_change.mdx @@ -18,10 +18,11 @@ description: "Creating a change on a TuringDB graph is similar to creating a new ```python - # Create a new change on the graph + # Create a new change on the graph. + # The client is switched onto the new change automatically. change = client.new_change() - # Checkout into the change + # Optional: checkout into a change that already exists client.checkout(change=change) ``` diff --git a/graph_dev/create_nodes_edges.mdx b/graph_dev/create_nodes_edges.mdx index dc0fa80..7370d33 100644 --- a/graph_dev/create_nodes_edges.mdx +++ b/graph_dev/create_nodes_edges.mdx @@ -69,4 +69,13 @@ description: "Once you have a valid change, you can start adding new nodes and e In the case you create the nodes then the edges, you need to commit your changes after node creation. - \ No newline at end of file + + +## Merge, delete and bulk loading + +- Use `MERGE` instead of `CREATE` to avoid duplicates: `MERGE (n:Person {name: 'Jane'})`. `MERGE` cannot see nodes created earlier in the same change until you `COMMIT`. +- Remove data with `DELETE` (edges, or nodes without edges) and `DETACH DELETE` (a node and all its edges). +- Every node needs at least one label, and every edge exactly one edge type. +- For more than a few thousand nodes or edges, build the graph with [`LOAD PARQUET`](/import_data/parquet) instead of Cypher writes. + +See [Writing data](/query/writing) for the full reference. diff --git a/graph_dev/submit_change.mdx b/graph_dev/submit_change.mdx index fc785dc..878d1ea 100644 --- a/graph_dev/submit_change.mdx +++ b/graph_dev/submit_change.mdx @@ -23,4 +23,10 @@ description: "Submit changes done to the database (just like a commit being subm client.checkout() ``` - \ No newline at end of file + + + + `CHANGE SUBMIT` commits any pending writes itself, so you do not need a `COMMIT` right before it. In the Python SDK, `client.checkout()` afterwards is **required**: the client still points at the submitted change, and the next query would fail with `CHANGE_NOT_FOUND`. + + +If two changes edit the same node, the second submit fails with an error such as `This change attempted to update Node 0 (...) which has been modified on main.` The failed change stays open: fix it and submit again, or discard it with `CHANGE DELETE`. See [Writing data](/query/writing#failed-queries-and-conflicts). diff --git a/graph_dev/time_travelling.mdx b/graph_dev/time_travelling.mdx index a457b3d..126c840 100644 --- a/graph_dev/time_travelling.mdx +++ b/graph_dev/time_travelling.mdx @@ -39,6 +39,14 @@ description: "You can time travel in you graph databases and see the commit hist +## About `db.history()` + +- Commits are listed newest first, and the latest one is marked with a literal `(HEAD)` suffix on its hash: strip it before passing the hash to `checkout`. +- Every `COMMIT` and every submitted change adds an entry. +- `nodeCount` and `edgeCount` count what **that commit** added, not the graph totals. +- Use hashes exactly as printed, since they are not zero-padded. An unknown hash passed to `client.checkout(commit=...)` raises `COMMIT_NOT_FOUND`, and `CALL db.describeCommit('')` returns zeros for an unknown hash. +- A past commit is read-only: writes are refused there. Call `client.checkout()` to return to main. + ## Lazy Commit Loading TuringDB lazily loads commits from disk. When a graph is loaded, only the **HEAD commit** (the latest commit on the current branch) is fully loaded into memory. All other commits are registered but their data remains on disk until explicitly requested. diff --git a/graph_dev/update_properties.mdx b/graph_dev/update_properties.mdx index e8fa194..86c972a 100644 --- a/graph_dev/update_properties.mdx +++ b/graph_dev/update_properties.mdx @@ -84,11 +84,25 @@ The `SET` clause is used to update property values on existing nodes and edges. SET n.isMinor = true ``` - - `CREATE ... SET` is not supported. When creating nodes or edges, use brace initialisation to set properties directly in the `CREATE` clause: + + When creating nodes or edges, you can set properties directly in the `CREATE` clause with braces: ```jsx CREATE (:Person {name: 'Alice', age: 30}) ``` + `CREATE ... SET` also works: `CREATE (n:Item {sku: 'A1'}) SET n.price = 9.99`. + + + ## Clearing and removing properties + + Set a property to `null` to clear its value, or use `REMOVE` to remove properties: + + ```jsx + MATCH (n:Person {name: 'Alice'}) SET n.age = null + MATCH (n:Person {name: 'Alice'}) REMOVE n.position, n.surname + ``` + + + Each property name has a single type across the whole graph, so `SET n.age = 'old'` fails if `age` is an Integer. Labels cannot be changed after creation: `SET n:Label` is not supported. `SET n += {...}` and `SET n = {...}` are not supported either. See [Writing data](/query/writing#set-and-remove) for the full rules. @@ -172,11 +186,25 @@ The `SET` clause is used to update property values on existing nodes and edges. """) ``` - - `CREATE ... SET` is not supported. When creating nodes or edges, use brace initialisation to set properties directly in the `CREATE` clause: + + When creating nodes or edges, you can set properties directly in the `CREATE` clause with braces: ```python client.query("CREATE (:Person {name: 'Alice', age: 30})") ``` + `CREATE ... SET` also works: `client.query("CREATE (n:Item {sku: 'A1'}) SET n.price = 9.99")`. + + + ## Clearing and removing properties + + Set a property to `null` to clear its value, or use `REMOVE` to remove properties: + + ```python + client.query("MATCH (n:Person {name: 'Alice'}) SET n.age = null") + client.query("MATCH (n:Person {name: 'Alice'}) REMOVE n.position, n.surname") + ``` + + + Each property name has a single type across the whole graph, so `SET n.age = 'old'` fails if `age` is an Integer. Labels cannot be changed after creation: `SET n:Label` is not supported. See [Writing data](/query/writing#set-and-remove) for the full rules. diff --git a/import_data/csv.mdx b/import_data/csv.mdx index a38413d..e9a61ac 100644 --- a/import_data/csv.mdx +++ b/import_data/csv.mdx @@ -6,7 +6,7 @@ description: "Stream rows from a CSV file in a query with LOAD CSV" `LOAD CSV` streams the rows of a CSV file into a query. Unlike `LOAD JSONL` / `LOAD GML` (which build a whole graph from a file), `LOAD CSV` is a **reading statement** that yields one row at a time, bound to a variable you can reference in `RETURN`, `MATCH`, or `CREATE`. -The file must live in the `data` subdirectory of the TuringDB working directory (set by `-turing-dir`, default `$HOME/.turing`). +The file must live in the `data` subdirectory of the TuringDB working directory (set by `-turing-dir`, default `$HOME/.turing`). Absolute paths are re-rooted inside that directory. ## Syntax @@ -21,7 +21,7 @@ LOAD CSV "" [WITH HEADERS] [ON ERROR SKIP | ON ERROR FAIL] AS | `ON ERROR FAIL` | Abort on the first malformed row (the default) | | `AS ` | Variable bound to each row | -All values come back as **strings** — wrap them in `toInteger(...)` / `toFloat(...)` if you need numbers. +All values come back as **strings**. Wrap them in `toInteger(...)` / `toFloat(...)` / `datetime(...)` if you need other types. Files must be comma-separated: `FIELDTERMINATOR` and the Neo4j form `LOAD CSV WITH HEADERS FROM 'file:///...'` are not supported. ## With headers @@ -38,19 +38,36 @@ RETURN row[0] AS name, row[1] AS age LIMIT 5 ``` +Without `WITH HEADERS`, the header line comes back as an ordinary data row. A row cannot be returned whole: read its fields as `row.col` or `row[]`. + +## Combining with the rest of a query + +`LOAD CSV` composes with `MATCH`, `WITH` / `WHERE`, aggregates, `ORDER BY` and `UNWIND`. For example, join the rows with nodes that are already in the graph: + +```jsx +LOAD CSV "people.csv" WITH HEADERS AS row +MATCH (p:Person {name: row.name}) +RETURN p.name, row.city +``` + + + An `OPTIONAL MATCH` directly after `LOAD CSV` that uses `row.x` inside its pattern hits an internal error. Project the field first: `... AS row WITH row.name AS nm OPTIONAL MATCH (p:Person {name: nm}) ...`. + + ## Building a graph from CSV Because `LOAD CSV` yields rows, you can drive `CREATE` from it inside a change: ```python -change = client.new_change() -client.checkout(change=change) +client.new_change() client.query('LOAD CSV "people.csv" WITH HEADERS AS row CREATE (:Person {name: row.name, age: toInteger(row.age)})') -client.query("COMMIT") +client.query("COMMIT") # make the new nodes visible to the next query client.query("CHANGE SUBMIT") client.checkout() ``` - - `LOAD CSV` followed by `MATCH` in the same read statement is not yet supported. - + + This is meant for small files and for adding to an existing graph. Creating edges with `LOAD CSV` has to `MATCH` both endpoints for every row, which becomes very slow beyond a few thousand edges. For bulk loads, convert the CSV to Parquet and use [`LOAD PARQUET`](/import_data/parquet), which loads 100,000 nodes and 200,000 edges in about 0.2 seconds. + + +To skip malformed rows instead of failing, add `ON ERROR SKIP`. See [Writing data](/query/writing#bulk-writes) for a single-pass `MERGE` pattern that needs no `COMMIT`. diff --git a/import_data/gml.mdx b/import_data/gml.mdx index fc225f6..ea2842e 100644 --- a/import_data/gml.mdx +++ b/import_data/gml.mdx @@ -32,12 +32,20 @@ graph [ | `source` / `target` | Required. Must reference the `id` of a node defined in the same file. | | other fields | Optional. Stored as edge properties. | -**Supported property value types:** String, Integer, Double, Boolean. All property values are stored as strings in the TuringDB graph regardless of their original type in the GML file. +**Property values:** all property values are stored as **strings** in the TuringDB graph, regardless of their original type in the GML file. Convert them with `toInteger` / `toFloat` in your queries. All imported nodes are assigned the label `GMLNode` and all edges the type `GMLEdge`, regardless of any fields present in the file. Any field other than `id`, `source`, and `target` is treated as a plain property. To query your data after import, use `MATCH (n:GMLNode)` and `MATCH ()-[e:GMLEdge]->()`. + + Property names also get a **type suffix**, for example `label (String)` and `weight (String)`. Quote them with backticks in queries, and run `CALL db.propertyTypes()` to see the exact names: + + ```jsx + MATCH (n:GMLNode) RETURN n.`label (String)`, toInteger(n.`age (String)`) + ``` + + ## Import steps diff --git a/import_data/jsonl.mdx b/import_data/jsonl.mdx index 118b44e..8b1041a 100644 --- a/import_data/jsonl.mdx +++ b/import_data/jsonl.mdx @@ -27,7 +27,7 @@ Each line is a self-contained JSON object representing either a node or a relati | Field | Requirement | | ------------ | ------------------------------------------------------------------------------------------------------ | | `type` | Must be the string `"node"` | -| `id` | Required. Integer or string parseable as integer. Must start at `0` and increment by `1` with no gaps. | +| `id` | Required. Integer or string parseable as integer. Ids do not need to be contiguous. | | `labels` | Required. Array with at least one label. | | `properties` | Optional. Dictionary with property keys and values. | @@ -36,20 +36,48 @@ Each line is a self-contained JSON object representing either a node or a relati | Field | Requirement | | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | Must be the string `"relationship"` | -| `id` | Required. Integer or string parseable as integer. Must start at `0` and increment by `1` with no gaps. | +| `id` | Required. Integer or string parseable as integer. | | `label` | Required. Exactly one label. | | `start` / `end` | Required. Either:
- a bare integer (or string parseable as integer) referencing a node `id`
- a dictionary with an `id` key holding an integer (or string parseable as integer) referencing a node `id`. | | `properties` | Optional. Dictionary with property keys and values. | -## Loading embedding properties +## Property values -By default, an array-valued property in the JSONL is stored as a string. To load specific array properties as **embeddings** (type `Embedding`, usable with `cosine_similarity` / `euclidean_distance`), append a `WITH EMBEDDINGS` clause listing each embedding property and its dimension: +- Scalars (strings, integers, doubles, booleans) become properties of the matching type, and `null` values are skipped. +- Arrays load as **List** properties. +- Nested objects are stored as **JSON strings**. +- ISO date strings stay strings, and integer durations stay integers, unless you declare them (see below). + +Each property name has a single type across the whole graph, so a property that is an integer on one node cannot be a string on another. + +## Loading date, duration and embedding properties + +Typed options can be chained after the graph name: + +```jsx +LOAD JSONL 'mydata.jsonl' AS mygraph + WITH EMBEDDINGS [{"emb", 384}, {"summary_vec", 768}] + WITH DATETIMES [born, updatedAt] + WITH DURATIONS [tenure] +``` + +- `WITH DATETIMES [prop, ...]` parses ISO 8601 strings into `DateTime` values. +- `WITH DURATIONS [prop, ...]` reads integers as microseconds and stores them as `Duration` values. +- `WITH EMBEDDINGS [{"prop", dim}, ...]` stores arrays as `Embedding` values, see below. + +### Loading embedding properties + +An array-valued property is stored as a List by default. To load specific array properties as **embeddings** (type `Embedding`, usable with `cosine_similarity` / `euclidean_distance`), append a `WITH EMBEDDINGS` clause listing each embedding property and its dimension: ```jsx LOAD JSONL 'mydata.jsonl' AS mygraph WITH EMBEDDINGS [{"embedding", 384}, {"summary_vec", 768}] ``` -Each spec is `{"", }`. The dimension must be **greater than 1**, and every array under that property must have exactly that many numeric elements (otherwise the load fails). This works for both node and relationship properties. Array properties not listed in `WITH EMBEDDINGS` are still stored as strings. +Each spec is `{"", }`. The dimension must be **greater than 1**, and every array under that property must have exactly that many numeric elements (otherwise the load fails). This works for both node and relationship properties. Array properties not listed in `WITH EMBEDDINGS` are stored as Lists. + + + When an import fails, the details are written to `/logs/turingdb.log`. + This sets embeddings as node/edge **properties**. To bulk-load embeddings onto an *existing* graph from a Parquet file, use [`LOAD EMBEDDING FROM`](/vector-search#embeddings-as-node-properties) instead. diff --git a/import_data/parquet.mdx b/import_data/parquet.mdx index eda7de9..2636acb 100644 --- a/import_data/parquet.mdx +++ b/import_data/parquet.mdx @@ -1,42 +1,91 @@ --- title: "Import Parquet files" sidebarTitle: "Parquet" -description: "Build a TuringDB graph from node and edge Parquet files with the turing-parquet CLI" +description: "Build a TuringDB graph from node and edge Parquet files with LOAD PARQUET or the turing-parquet CLI" --- -To load Parquet files containing columns for node IDs, node properties, edge source and target IDs, and edge properties, see: `LOAD PARQUET`. +To load Parquet files containing columns for node IDs, node properties, edge source and target IDs, and edge properties, see [`LOAD PARQUET`](#load-parquet). - -To load Parquet files containing a JSON string column, which is then converted into a graph structure, see: `turing-parquet`. +To load Parquet files containing a JSON string column, which is then converted into a graph structure, see [`turing-parquet`](#turing-parquet). # `LOAD PARQUET` -`LOAD PARQUET` queries may be used to load a graph into TuringDB. The command is of the following format: +`LOAD PARQUET` is the fastest way to get data into TuringDB, and the recommended path for anything beyond a few thousand entities. Above roughly 5,000 nodes or 1,000 edges, write your data to Parquet and load it with this command: 100,000 nodes and 200,000 edges load in about 0.2 seconds, whereas Cypher-driven loads (`LOAD CSV` or `UNWIND` with `MATCH` and `CREATE` for the edges) take minutes at a few thousand edges. + ``` -LOAD PARQUET " {AS }" +LOAD PARQUET '' [AS ] [WITH DURATIONS [, ...]] ``` -where `{AS }` may be omitted, in which case the name of the graph will be that of the directory. -The directory specified need contain `nodes.parquet` and `edges.parquet`. +The graph name defaults to the name of the directory. The directory is resolved inside the `data` subdirectory of the TuringDB working directory (`-turing-dir`) and must contain `nodes.parquet` and `edges.parquet`. + + + `LOAD PARQUET` always creates a **new graph**. The target graph name must not already exist, and it cannot append to an existing graph. To add a large volume of data to an existing graph, write the existing data plus the new data to Parquet and load it under a new graph name. Use Cypher writes only for small, incremental changes. + + +## Required columns + +`nodes.parquet` has one row per node: + +- `__id`: `INT64`, unique. Your own node key. Edges refer to it. +- `__labels`: a list of strings (`list`), non-empty. Each element is a label of the node. The path of each element must be `__labels.list.element`. + +`edges.parquet` has one row per edge: + +- `__source`: `INT64`, the `__id` of the source node. +- `__target`: `INT64`, the `__id` of the target node. +- `__type`: a string, the edge type. Every edge has exactly one. -Each row in `nodes.parquet` defines a node, and the file has a required format of: -- a column of name `__id` of type `INT64` - defines the relative node ID for the row; -- a column of name `__labels` of type `BYTE_ARRAY` with exactly 1 level of nesting, forming a byte array of byte arrays - each inner byte array is treated as a string (the name of that label), the path of each element *must* be `__labels.list.element`. +## Property columns -Each row in `edges.parquet` defines an edge, and the file has a required format of: -- a column of name `__source` of type `INT64` - defines the node ID (present in `nodes.parquet`) of the source node of this edge; -- a column of name `__target` of type `INT64` - defines the node ID (present in `nodes.parquet`) of the target node of this edge; -- a column of name `__type` of type `BYTE_ARRAY` with exactly 0 levels of nesting - interpreted as a string (the name of this edge type). +Every other column becomes a property of the node or edge on that row. The column name is the property name. Null values are allowed. -As well as the above required columns, `nodes.parquet` and `edges.parquet` may contain any number of extra columns, all of which are interpreted as a property value for the node/edge of that row. These columns may have any name (other than the names of the columns explicitly required above). +| Parquet type | TuringDB property type | +| ------------ | ---------------------- | +| `INT64` | Integer | +| `DOUBLE` | Double | +| `BOOLEAN` | Boolean | +| `BYTE_ARRAY` (string) | String | +| timestamp | DateTime | +| list of the types above | List | -The supported types for property columns are: -- `BOOLEAN` -- `INT64` -- `DOUBLE` -- `BYTE_ARRAY` with exactly 0 levels of nesting +`INT32` and `FLOAT` columns are rejected (for example `Node column was not of type INT64` or `Unsupported column type`), so cast them to `int64` / `float64` first. To load an `INT64` column of microseconds as a `Duration`, name it in `WITH DURATIONS []`. + +## From pandas + +To go from CSV, JSON, SQL extracts or API data, load it into pandas, map your natural keys to `INT64` ids, and write the two files: + +```python +import pandas as pd, pyarrow as pa, pyarrow.parquet as pq +from pathlib import Path + +people = pd.read_csv("people.csv") # columns: email, name, age +knows = pd.read_csv("knows.csv") # columns: src, dst, since (src and dst are emails) + +ids = {key: i for i, key in enumerate(people["email"])} # natural key -> int64 __id +nodes = pd.DataFrame({ + "__id": pd.Series(range(len(people)), dtype="int64"), + "__labels": [["Person"]] * len(people), + "email": people["email"], + "name": people["name"], + "age": people["age"].astype("Int64"), # int64, never int32 +}) +edges = pd.DataFrame({ + "__source": knows["src"].map(ids).astype("int64"), + "__target": knows["dst"].map(ids).astype("int64"), + "__type": "KNOWS", + "since": knows["since"].astype("int64"), +}) + +out = Path.home() / ".turing" / "data" / "people_graph" # /data/ +out.mkdir(parents=True, exist_ok=True) +pq.write_table(pa.Table.from_pandas(nodes, preserve_index=False), out / "nodes.parquet") +pq.write_table(pa.Table.from_pandas(edges, preserve_index=False), out / "edges.parquet") + +client.query("LOAD PARQUET 'people_graph' AS people") +client.set_graph("people") +``` -Note: all property columns of `BYTE_ARRAY` are treated as strings +For several node types, concatenate them into a single `nodes.parquet` with non-overlapping `__id` ranges, each row carrying its own `__labels`. Keep the natural key (`email` above) as a property so you can match on it in later queries. # `turing-parquet` diff --git a/index.mdx b/index.mdx index d18f7be..40ca18e 100644 --- a/index.mdx +++ b/index.mdx @@ -20,7 +20,7 @@ The platform has three main components: the [TuringDB engine](/concepts/overview Learn about the core concepts and architecture of TuringDB - TuringDB query language based on OpenCypher + Cypher in TuringDB v3: most of openCypher, plus versioning and vector search See how TuringDB performs against other graph databases diff --git a/pythonsdk/get_started.mdx b/pythonsdk/get_started.mdx index 5f32949..2268be1 100644 --- a/pythonsdk/get_started.mdx +++ b/pythonsdk/get_started.mdx @@ -130,7 +130,7 @@ from turingdb import TuringDB client = TuringDB(host="http://localhost:6666") # Or run the engine in-process, with no server: -# client = TuringDB(type="embedded", data_dir="~/.turing") +# client = TuringDB(type="embedded", data_dir="/home/me/.turing") # use an absolute path # Create and set working graph client.create_graph("people") diff --git a/pythonsdk/reference.mdx b/pythonsdk/reference.mdx index 4336c47..99e47a6 100644 --- a/pythonsdk/reference.mdx +++ b/pythonsdk/reference.mdx @@ -22,6 +22,8 @@ or using `pip`: pip install turingdb ``` +The SDK targets TuringDB v3: pin `turingdb>=3.0` in your project (older projects may pin `1.x`, which uses the previous engine and a much smaller Cypher subset). The wheel supports Python 3.10 to 3.14 on Linux (x86_64, aarch64) and macOS 15 or later (arm64). It also contains the `turingdb` command-line tool, the `turing-parquet` import tool, and the in-process engine. + ## Getting Started ```python @@ -42,25 +44,81 @@ print(client.list_available_graphs()) | `type` | Encoding / transport | Constructor arguments | | ------ | -------------------- | --------------------- | -| `"json"` *(default)* | JSON payloads over HTTP against a running daemon | `host="http://localhost:6666"` | -| `"native"` | Binary (Turing proto) payloads over HTTP — faster; requires the compiled extension | `host="localhost"`, `port=6666` | -| `"embedded"` | In-process engine, no daemon or socket | `data_dir="/path/to/.turing"` | - -Both `json` and `native` talk to a daemon over HTTP; they differ in the wire encoding (JSON vs. binary). `embedded` runs the engine in-process with no network at all. +| `"json"` *(default)* | JSON over HTTP against a running server | `host="http://localhost:6666"`, optional `token` | +| `"native"` | TuringDB's binary protocol against a running server. Faster for large results | `host="localhost"`, `port=6666`, optional `token` | +| `"embedded"` | In-process engine, no server or socket | `data_dir="/path/to/.turing"` | ```python -# Default HTTP client +# Default HTTP/JSON client client = TuringDB(host="http://localhost:6666") # Native binary protocol client = TuringDB(type="native", host="localhost", port=6666) -# Embedded, in-process — no server needed -client = TuringDB(type="embedded", data_dir="~/.turing") +# Embedded, in-process: no server needed +client = TuringDB(type="embedded", data_dir="/home/me/.turing") +``` + +Use the default `json` backend unless you have a reason not to. The other two are optimisations for specific situations: + +- **`native`** when you control the server and need more throughput on large results. +- **`embedded`** when you want a self-contained engine with no server to start, stop or connect to. + +### The native binary protocol + +The binary protocol is a **server-side mode**, selected once when the server starts with the environment variable `USE_TURING_PROTO=1`: + +```bash +USE_TURING_PROTO=1 turingdb start -demon -p 6666 +``` + +```python +client = TuringDB(type="native", host="localhost", port=6666) +``` + +A server speaks either JSON or the binary protocol, never both on the same port: + +| Server started | `type="json"` client | `type="native"` client | +| -------------- | -------------------- | ---------------------- | +| normally (JSON, the default) | works | fails with `Proto header dataLen does not match chunk payload size` | +| with `USE_TURING_PROTO=1` | fails with `Invalid response from the server` | works | + +A server in binary mode is therefore **binary-only**: `curl`, the JSON client and the browser visualizer (`-ui`) stop working against it. If you need both, run two servers on different ports. + +- `host` accepts `"localhost"` together with `port=`, or a full URL such as `"http://localhost:6666"`. `"localhost:6666"` without a scheme is not understood. +- The query API is identical, and results come back as the same DataFrames. Embeddings are returned as numpy `float32` arrays instead of Python lists. +- `list_available_graphs()` is not available. Use the JSON client to enumerate the graphs on disk, or `list_loaded_graphs()`. +- `token`, `reconnect()` and the S3 helpers work as on the JSON backend. + +### Authentication + +A server started with `-auth-on` requires a bearer token. Pass it with `token=` or set the `TURINGDB_AUTH_TOKEN` environment variable, which the SDK reads automatically. This works on both the `json` and `native` backends: + +```bash +TURINGDB_AUTH_TOKEN=s3cret turingdb start -demon -auth-on +``` + +```python +client = TuringDB(host="http://localhost:6666", token="s3cret") +``` + +A missing or wrong token gets an HTTP `401`, which the JSON client raises as an `httpx.HTTPStatusError` (not a `TuringDBException`). The embedded backend runs in-process and is unauthenticated: passing `token=` to it is an error. + +### The embedded backend + +```python +import os + +client = TuringDB(type="embedded", data_dir=os.path.expanduser("~/.turing")) ``` +- `data_dir` is the TuringDB root directory (the one that holds `graphs/`, `data/` and `vector/`). Omit it to use `~/.turing`. +- **Give it an absolute path.** A leading `~` is not expanded, so use `os.path.expanduser("~/...")`. +- Writes still persist to disk when you submit a change, so your work survives the process ending. +- The embedded backend does **not** support the browser visualizer, `list_available_graphs()`, concurrent access from several processes, the S3 helpers, or `token=`. Use a server for those. + - `list_available_graphs()` and the S3 helpers (`s3_connect`, `transfer`) are only supported on the default JSON backend (`type="json"`). + `list_available_graphs()` is only supported on the default JSON backend. The S3 helpers (`s3_connect`, `transfer`) work on the JSON and native backends. ## Core Methods @@ -125,15 +183,16 @@ Returns whether the current graph is loaded in memory. ### `new_change() → int` -Creates a new isolated change (equivalent to `CHANGE NEW`), makes it the active change, and returns its integer ID. +Creates a new isolated change (equivalent to `CHANGE NEW`), makes it the active change, and returns its integer ID. The client is already on the new change, so a separate `checkout(change=...)` is not needed. ```python change = client.new_change() -client.checkout(change=change) ``` +`new_change()` raises if the client is already on a change or on a past commit. Call `client.checkout()` first. + - There is no `submit()` method — submit a change by running `client.query("CHANGE SUBMIT")`. + There is no `submit()` method. Submit a change by running `client.query("CHANGE SUBMIT")`, then call `client.checkout()` to return to main. Until you do, the client still points at the submitted change and the next query fails with `CHANGE_NOT_FOUND`. ## Version Control Helpers @@ -184,7 +243,7 @@ You can also read the current context through the `current_graph`, `current_comm ### `reconnect()` -Resets the connection. A no-op for the HTTP and embedded backends; reopens the socket for the native backend. +Resets the connection. A no-op for the HTTP and embedded backends. Reopens the socket for the native backend. ### `try_reach(timeout: int = 5)` / `warmup(timeout: int = 5)` @@ -196,7 +255,7 @@ Return the server-side execution time and the full round-trip time (in milliseco ## S3 / Data Transfer -These helpers move graph data between local paths, S3, and the TuringDB server. They are only available on the JSON backend (`type="json"`). +These helpers move graph data between local paths, S3, and the TuringDB server. They are available on the JSON and native backends, but not on the embedded backend. ### `s3_connect(bucket_name, access_key=None, secret_key=None, region=None, use_scratch=True)` @@ -219,7 +278,7 @@ The equivalent engine commands (issued directly via `query`) are `S3 CONNECT " + Over the JSON backend, Doubles are serialised with 6 decimal places. Use the native or embedded backend if you need full precision. + ## Error Handling -All SDK errors raise a custom `TuringDBException`. +Query errors raise `TuringDBException`. The message starts with the error code (`PARSE_ERROR`, `ANALYZE_ERROR`, `PLAN_ERROR`, `EXEC_ERROR`, `CHANGE_NOT_FOUND`, `COMMIT_NOT_FOUND`, ...) and points at the offending part of the query. ```python +from turingdb import TuringDB, TuringDBException + try: - client.query("MATCH (x) RETURN x") + client.query("MATCH (n RETURN n") except TuringDBException as e: print("Query failed:", e) ``` +On the JSON backend, transport failures (the server is down, or HTTP 401 for a bad or missing token) surface as `httpx` exceptions instead. + ## Example Workflow ```python diff --git a/query/aggregation.mdx b/query/aggregation.mdx new file mode 100644 index 0000000..9e26a8d --- /dev/null +++ b/query/aggregation.mdx @@ -0,0 +1,151 @@ +--- +title: "Results, Aggregation and Lists" +sidebarTitle: "Results and Aggregation" +description: "Shape results with WITH, DISTINCT, ORDER BY, SKIP, LIMIT and UNION, then aggregate, unwind and transform lists." +--- + +These examples use the [sample graph](/query/matching) from the matching page. + +## RETURN, DISTINCT and expressions + +```cypher +MATCH (n:Person) RETURN n.name, n.age + 5 AS adjusted +MATCH (n:Person) RETURN DISTINCT n.city +MATCH (n:Person) RETURN n.name AS nm ORDER BY size(nm) DESC, nm +MATCH (n:Person) RETURN * +``` + +`RETURN *` returns every variable in scope (nodes as IDs). A column without an alias is named after its expression, for example `n.name`. + +## ORDER BY, SKIP and LIMIT + +```cypher +MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age +MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age DESC +MATCH (n:Person) RETURN n.name, n.city, n.age ORDER BY n.city, n.age DESC +MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age DESC SKIP 1 LIMIT 2 +``` + +- You can sort by several keys, by aliases and by expressions. Ascending is the default. +- `SKIP` and `LIMIT` combine for pagination: `SKIP 10 LIMIT 10` returns the 11th to 20th rows. +- `SKIP` and `LIMIT` take constant expressions. +- `RETURN DISTINCT ... ORDER BY` can only sort by returned columns. +- Nulls sort last in ascending order and first in descending order. + +## WITH + +`WITH` pipes the result of one part of a query into the next. It is how you filter on aggregates and chain several stages: + +```cypher +MATCH (n:Person) WITH n WHERE n.age > 28 RETURN n.name +MATCH (n:Person) WITH n.name AS name, n.age AS age ORDER BY age DESC LIMIT 2 RETURN name, age +MATCH (n:Person) WITH n WHERE n.age > 26 WITH n WHERE n.age < 36 RETURN n.name +MATCH (n:Person) WITH n.city AS city, count(*) AS c WHERE c > 1 RETURN city, c +``` + + + Expressions in a `WITH` must be aliased: `WITH n.name AS name`. `RETURN` does not need aliases. + + +## UNION + +```cypher +MATCH (p:Person) RETURN p.name AS name UNION MATCH (c:Company) RETURN c.name AS name +MATCH (p:Person) RETURN p.name AS name UNION ALL MATCH (c:Company) RETURN c.name AS name +``` + +`UNION` removes duplicates and `UNION ALL` keeps them. All branches must return the same column names, and each column must keep one value type across the branches. + +## Aggregation + +Grouping is implicit, as in Neo4j: the non-aggregated items in the `RETURN` are the grouping keys. + +```cypher +MATCH (a:Person)-[:WORKS_AT]->(c:Company) +RETURN c.name, count(a) AS employees, collect(a.name) AS who, avg(a.age) AS avgAge +ORDER BY employees DESC +``` + +```cypher +MATCH (n:Person) RETURN count(*), count(n.age), sum(n.age), avg(n.age), min(n.age), max(n.age) +MATCH (a)-[w:WORKS_AT]->(c) RETURN w.role, count(DISTINCT c) AS companies +MATCH (n:Person) RETURN n.city, max(n.age) - min(n.age) AS spread +MATCH (n:Person) RETURN sum(CASE WHEN n.age > 30 THEN 1 ELSE 0 END) AS seniors +MATCH (n:Person) RETURN collect(n.name)[0..3] AS firstThree +MATCH ()-[r]->() RETURN type(r) AS t, count(*) AS c ORDER BY c DESC +MATCH (n) RETURN labels(n) AS l, count(*) AS c ORDER BY c DESC +``` + +| Aggregate | Notes | +| --------- | ----- | +| `count(x)`, `count(*)`, `count(DISTINCT x)` | `count(x)` ignores nulls | +| `collect(x)`, `collect(DISTINCT x)` | Nulls are dropped | +| `sum(x)` | | +| `avg(x)` | Always a Double | +| `min(x)`, `max(x)` | Numbers, strings, booleans, `DateTime` and `Duration` | + +There are no statistical aggregates such as `stdev` or `percentileCont`. Compute those in your application, for example in pandas. + +**Rules:** + +- Aggregates are **not allowed in `WHERE`**. Filter after a `WITH` instead (see above). +- Aggregates cannot be nested: `count(count(*))` is an error. +- A non-aggregated expression next to an aggregate may only use the grouping keys. +- With no grouping keys, an empty input still returns one row: `count` is `0`, `sum` is `0`, `avg` is `null` and `collect` is `[]`. With grouping keys, an empty input returns no rows. + +## UNWIND + +`UNWIND` expands a list into one row per element. It accepts any list expression, including lists you computed earlier in the query: + +```cypher +UNWIND [1, 2, 3] AS x RETURN x * 10 +UNWIND range(1, 10, 3) AS x RETURN x +MATCH (n:Person) WITH collect(n.name) AS names UNWIND names AS nm RETURN nm +MATCH (n:Person) WITH collect(n) AS ns UNWIND ns AS p RETURN p.name +MATCH (n:Person) UNWIND n.tags AS t RETURN t, count(*) +``` + +`range(start, end[, step])` includes both bounds. Unwound nodes stay nodes, so `p.name` works after `UNWIND ns AS p`. You can also `UNWIND` a list of maps and read their keys: `UNWIND [{a: 1}, {a: 2}] AS m RETURN m.a`. + + + Query parameters (`UNWIND $list`) are not supported. Write the list into the query text instead, for example `f"UNWIND {ids} AS x ..."` in Python. + + +## Lists + +```cypher +RETURN [1, 2, 3, 4, 5][0], [1, 2, 3, 4, 5][-1], [1, 2, 3, 4, 5][1..3], [1, 2, 3][10] +RETURN size([1, 2]), head([1, 2]), last([1, 2]), tail([1, 2]), reverse([1, 2, 3]) +MATCH (n:Person {name: 'Alice'}) RETURN n.tags[0], size(n.tags) +RETURN [1, 2] + [3], 'a' || 'b' +``` + +- List literals can contain expressions, mixed types, nested lists and maps. +- Indexing and slicing work on literals, list properties and function results (`labels(n)[0]`). Negative indexes count from the end, and an out-of-range index returns `null`. +- Lists and maps compare by value with `=`. +- `size(null)` and `head([])` return `null`. + +### List comprehensions + +```cypher +RETURN [x IN range(1, 6) WHERE x % 2 = 0 | x * x] +MATCH (a:Person) RETURN a.name, [(a)-[:KNOWS]->(b) | b.name] AS friends +MATCH (n:Person) WHERE any(t IN n.tags WHERE t = 'vip') RETURN n.name +RETURN reduce(total = 0, x IN [1, 2, 3] | total + x) +``` + +- `[x IN list WHERE condition | expression]` filters and maps a list. +- A **pattern comprehension** `[(a)-[:KNOWS]->(b) | b.name]` collects a value for every match of a pattern. +- `all`, `any`, `none` and `single` test a condition over a list: `any(t IN n.tags WHERE t = 'vip')`. +- `reduce(acc = init, x IN list | expression)` folds a list into a single value. + +## CASE + +```cypher +MATCH (n:Person) +RETURN n.name, + CASE WHEN n.age >= 35 THEN 'senior' WHEN n.age IS NULL THEN 'unknown' ELSE 'junior' END AS band, + CASE n.city WHEN 'London' THEN 'UK' WHEN 'Paris' THEN 'FR' ELSE 'other' END AS country +``` + +The simple form (`CASE n.city WHEN 'London' THEN ...`) and the searched form (`CASE WHEN condition THEN ...`) both work. TuringDB also accepts comparison tests in the simple form, such as `CASE n.age WHEN > 30 THEN 'old' WHEN IS NULL THEN '?' END`. diff --git a/query/cheatsheet.mdx b/query/cheatsheet.mdx index 8340edb..139b487 100644 --- a/query/cheatsheet.mdx +++ b/query/cheatsheet.mdx @@ -1,477 +1,270 @@ --- title: "TuringDB Query Language Cheatsheet" sidebarTitle: "CheatSheet" -description: "A compact reference for querying, creating, and exploring graphs in TuringDB using Cypher-like syntax + TuringDB-specific extensions." +description: "A compact reference for querying, writing and exploring graphs in TuringDB v3: Cypher plus TuringDB-specific extensions." --- -## MATCH: Pattern Matching +Each section links to the page that explains it in full. -```jsx +## MATCH: pattern matching + +[Matching patterns](/query/matching) + +```cypher MATCH (n) RETURN n MATCH (n:Person {name: 'Alice'}) RETURN n.age MATCH (a:Person)-->(b:Company) RETURN a.name, b.name +MATCH (a:Person)-[e:KNOWS {since: 2010}]->(b:Person) RETURN a.name, e.since, b.name +MATCH (a)-[r:KNOWS|WORKS_AT]->(b) RETURN a.name, type(r), b.name +MATCH (a:Person)-[e]-(b) RETURN a.name, b.name +MATCH (a:Person WHERE a.age > 30)-[r:KNOWS WHERE r.since > 2012]->(b) RETURN b.name +MATCH (n:Person) ORDER BY n.age LIMIT 2 RETURN n.name ``` -- `()` → Node -- `[]` → Edge -- `-` → Undirected edge -- Node labels: `:Label` - e.g. `(n:Person)` -- Edge types: `:EDGE_TYPE` - e.g. `-[:KNOWS]->` (one edge type per edge) -- Properties: `{key: 'value', age: 30}` +- `()` is a node, `[]` is an edge, `-` ignores direction. +- Node labels use `:Label` and can be stacked: `(n:Person:Employee)`. +- Edge types use `:EDGE_TYPE`: `-[:KNOWS]->`. An edge has exactly one type, and `[:A|B]` matches either. +- `RETURN n` gives the internal node ID. Return properties explicitly: `RETURN n.name`. +- Unknown labels and properties do not error. They match nothing or read as `null`. -## CREATE: Add Nodes & Edges +## OPTIONAL MATCH -```jsx -CREATE (:Person {name: 'Alice', age: 30}) -CREATE (:Person {name: 'Mick'})-[:FRIEND_OF]->(:Person {name: 'John'}) -CREATE (a)-[:EDGE]->(b), (b)-[:EDGE]->(c), (c)-[:EDGE]->(a) - -CREATE (gabby:Person {name: 'Gabby', age: 27}), -(susan:Person {name: 'Susan', age: 35}), -(lynette:Person {name: 'Lynette', age: 41}), -(bree:Person {name: 'Bree', age: 42}), -(gabby)-[FRIEND_OF]->(susan), -(gabby)-[FRIEND_OF]->(lynette), -... +```cypher +MATCH (a:Person) OPTIONAL MATCH (a)-[:WORKS_AT]->(c:Company) RETURN a.name, c.name +MATCH (a:Person) OPTIONAL MATCH (a)-[:KNOWS]->(b) RETURN a.name, count(b) ``` -- Labels **required** for node creation -- No `RETURN` clause -- Use commas to create multiple items at once - -## MATCH + CREATE and MATCH + CREATE + RETURN - -```jsx -// Create two nodes Person -CREATE (:Person {name: 'Alice', age: 24}), - (:Person {name: 'John', age: 27}) - -// Match two nodes to create the edge between them -MATCH (n:Person {name: 'Alice'}), (m:Person {name: 'John'}) -CREATE (n)-[:FRIENDS_WITH]->(m) - -// RETURN clause can also be added -MATCH (n:Person {name: 'Alice'}), (m:Person {name: 'John'}) -CREATE (n)-[:FRIENDS_WITH]->(m) -RETURN n.name, n.age, m.name, m.age +## WHERE: filter + +```cypher +MATCH (n) WHERE n:Person RETURN n.name +MATCH (n:Person) WHERE n.age >= 18 AND n.city = 'London' RETURN n.name +MATCH (n:Person) WHERE n.name STARTS WITH 'A' OR n.name CONTAINS 'ro' RETURN n.name +MATCH (n:Person) WHERE n.name IN ['Alice', 'Bob'] RETURN n.name +MATCH (n:Person) WHERE 'admin' IN n.tags RETURN n.name +MATCH (n:Person) WHERE n.tags IS NOT NULL RETURN n.name +MATCH (n)-[e]->(m) WHERE e:KNOWS AND e.since > 2012 RETURN n.name, m.name +MATCH (p:Person) WHERE NOT (p)-[:KNOWS]->() RETURN p.name ``` -## SET: Update Properties - -```jsx -// Update a single property -MATCH (n:Person {name: 'Alice'}) -SET n.age = 30 +- Comparison: `= <> != < <= > >=`, null tests `IS NULL` / `IS NOT NULL`, boolean `AND OR XOR NOT`. +- Strings and lists: `IN`, `STARTS WITH`, `ENDS WITH`, `CONTAINS`, concatenation with `+` or `||`. +- No `=~` regex and no `exists(n.prop)` (use `IS NOT NULL`). +- `Integer = Double` and `Double = Double` comparisons are rejected. Use ranges or `toInteger`. -// Update multiple properties at once -MATCH (n:Person {name: 'Alice'}) -SET n.position = 'Developer', n.surname = 'Taylor' +## Lookup by native node ID -// Update from an expression -MATCH (p:Product) -SET p.discountPrice = p.price * (1 - 0.15) +[Details](/query/matching#look-up-nodes-by-native-node-id) -// Update edge property -MATCH (n:Person {name: 'Alice'})-[e:KNOWS]->(m:Person {name: 'Bob'}) -SET e.since = 2020 - -// Set an embedding vector -MATCH (n:Person {name: 'Alice'}) -SET n.emb = (1.2, 2.0, 0.0, 12.0) - -// Set an embedding vector by node ID -MATCH (n) WHERE n = 1234 -SET n.emb = (1.2, 2.0, 0.0, 12.0) - -// Conditional update with WHERE -MATCH (n:Person) -WHERE n.age < 18 -SET n.isMinor = true +```cypher +MATCH (n) WHERE n = 0 RETURN n.name +MATCH (n)-[:KNOWS]->(m) WHERE n = 0 RETURN m.name +UNWIND [0, 1, 2] AS x MATCH (n) WHERE n = x RETURN id(n), n.name +EXPLAIN MATCH (n) WHERE n = 0 RETURN n.name ``` -- Always used with `MATCH`, not `CREATE` -- Creates the property if it does not already exist -- Use `COMMIT` to persist changes - -## WHERE: Filter nodes and edges by properties +- Compare the **variable** (`n = 0`), not `id(n) = 0`. The first is a direct seek and the second scans every node. +- For batches, `UNWIND` a literal list of distinct IDs and do not return `x`. -To filter on node label: +## Joins and Cartesian products -```jsx -MATCH (n) -WHERE n:Person -RETURN n, n.age +```cypher +MATCH (p:Person), (c:Company) RETURN p.name, c.name +MATCH (a:Person)-[:KNOWS]->(b), (b)-[:WORKS_AT]->(c) RETURN a.name, b.name, c.name +MATCH (a:Person)-->(i:Interest)<--(b:Person) WHERE a <> b RETURN a.name, b.name, i.name +MATCH (a:Person), (b:Person) WHERE a.city = b.city AND a <> b RETURN a.name, b.name ``` -To filter on node property: +- A comma separates independent patterns. Shared variables create joins, unrelated patterns create Cartesian products (N × M rows). -```jsx -MATCH (n) -WHERE n.name = 'Alice' -RETURN n, n.age -``` +## Variable-length and quantified paths -To filter on edge label: +[Paths](/query/paths) -```jsx -MATCH (n)-[e]->(m) -WHERE e:PLAY_POKER_TOGETHER -RETURN n.name, m.age +```cypher +MATCH (a:Person {name: 'Alice'})-[:KNOWS]->+(b) RETURN DISTINCT b.name +MATCH (a:Person {name: 'Alice'})-[:KNOWS]->*(b) RETURN DISTINCT b.name +MATCH (a:Person {name: 'Alice'})-[:KNOWS]->{1,2}(b) RETURN DISTINCT b.name +MATCH (a:Person {name: 'Alice'})-[:KNOWS*1..3]->(b) RETURN DISTINCT b.name +MATCH (a:Person {name: 'Alice'}) ((x)-[:KNOWS]->(y) WHERE y.age > x.age)+ (b) RETURN b.name +MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) RETURN [n IN nodes(p) | n.name], length(p) ``` -To filter on edge property: - -```jsx -MATCH (n)-[e]->(m:Person) -WHERE n.name = 'Gabby' -AND e.play_poker_together = true -RETURN n.name, m.age -``` +- Postfix `->*` is **0** or more hops. Bracket `-[:KNOWS*]->` is **1** or more. +- Edges never repeat within a path. Bound the length on large graphs. -## Edge type filtering +## WITH, ORDER BY, SKIP, LIMIT, UNION -```jsx -// Match only edges of a given type -MATCH (n:Person)-[:KNOWS]->(m:Person) RETURN n.name, m.name +[Results and aggregation](/query/aggregation) -// Name the edge to also return its properties -MATCH (n:Person)-[e:KNOWS]->(m:Person) RETURN n.name, e.since, m.name - -// One edge type per hop -MATCH (a:Person)-[:KNOWS]->(b:Person)-[:WORKS_AT]->(c:Company) RETURN a.name, b.name, c.name - -// List the edge types in the graph -CALL db.edgeTypes() -``` - -- An edge has exactly one edge type, so a pattern takes a single `[:EDGE_TYPE]` constraint - -## SKIP and LIMIT - -```jsx -// Return first 10 results -MATCH (n) RETURN n LIMIT 10 - -// Pagination: skip 10, return next 10 -MATCH (n) RETURN n SKIP 10 LIMIT 10 +```cypher +MATCH (n:Person) WITH n.city AS city, count(*) AS c WHERE c > 1 RETURN city, c +MATCH (n:Person) RETURN DISTINCT n.city +MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age DESC SKIP 1 LIMIT 2 +MATCH (p:Person) RETURN p.name AS name UNION MATCH (c:Company) RETURN c.name AS name ``` -## ORDER BY +- Expressions in `WITH` must be aliased. +- Nulls sort last ascending and first descending. -```jsx -// Sort ascending (default) -MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age +## Aggregation -// Sort descending -MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age DESC - -// Sort by multiple properties -MATCH (n:Person) RETURN n.name, n.age, n.city ORDER BY n.city, n.age DESC - -// Combined with pagination -MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age DESC SKIP 10 LIMIT 10 +```cypher +MATCH (n:Person) RETURN count(*), count(n.age), sum(n.age), avg(n.age), min(n.age), max(n.age) +MATCH (a:Person)-[:WORKS_AT]->(c:Company) RETURN c.name, count(a) AS employees, collect(a.name) AS who +MATCH (n:Person) RETURN count(DISTINCT n.city) +MATCH ()-[r]->() RETURN type(r) AS t, count(*) AS c ORDER BY c DESC ``` -## Joins and Cartesian Products - -```jsx -// Cartesian product: all combinations of Person × Interest -MATCH (a:Person), (b:Interest) RETURN a.name, b.name - -// Three-way cartesian product -MATCH (p:Person), (i:Interest), (c:Category) RETURN p.name, i.name, c.name - -// Join on shared variable: persons sharing an interest -MATCH (a:Person)-->(b:Interest)<--(c:Person) -WHERE a.name <> c.name -RETURN a.name, b.name, c.name - -// With explicit edge types -MATCH (a:Person)-[:INTERESTED_IN]->(b:Interest)<-[:INTERESTED_IN]-(c:Person) -WHERE a.name <> c.name -RETURN a.name, b.name, c.name - -// Multi-hop join -MATCH (a:Person)-->(i:Interest)-->(c:Category) -RETURN a.name, i.name, c.name +- Grouping is implicit. Aggregates are not allowed in `WHERE`: filter after a `WITH`. +- No `stdev` or `percentile*` aggregates. -// Mix path + cartesian product -MATCH (a:Person)-->(i:Interest), (c:Category) -WHERE c.name = 'Cat1' -RETURN a.name, i.name, c.name +## UNWIND, lists, CASE -// Two independent paths (cartesian of each path's results) -MATCH (a:Person)-->(i1:Interest), (b:Person)-->(i2:Interest) -WHERE a.name <> b.name -RETURN a.name, i1.name, b.name, i2.name +```cypher +UNWIND [1, 2, 3] AS x RETURN x * 10 +MATCH (n:Person) WITH collect(n.name) AS names UNWIND names AS nm RETURN nm +MATCH (n:Person) UNWIND n.tags AS t RETURN t, count(*) +RETURN [x IN range(1, 6) WHERE x % 2 = 0 | x * x] +MATCH (a:Person) RETURN a.name, [(a)-[:KNOWS]->(b) | b.name] AS friends +MATCH (n:Person) WHERE any(t IN n.tags WHERE t = 'vip') RETURN n.name +RETURN reduce(total = 0, x IN [1, 2, 3] | total + x) +MATCH (n:Person) RETURN n.name, CASE WHEN n.age >= 35 THEN 'senior' ELSE 'junior' END AS band ``` -- Comma `,` separates independent patterns -- Shared variables between patterns create joins -- Unrelated patterns create cartesian products (N × M rows) +## Subqueries -## Property Match Operators +[Subqueries](/query/subqueries) -```jsx -MATCH (n {name: 'Alice'}) RETURN n -// exact equivalent of -MATCH (n) WHERE n.name = 'Alice' RETURN n +```cypher +MATCH (p:Person) WHERE EXISTS { (p)-[:WORKS_AT]->(:Company {name: 'Acme'}) } RETURN p.name +MATCH (p:Person) RETURN p.name, COUNT { (p)-[:KNOWS]->() } AS outDegree +MATCH (p:Person) CALL (p) { MATCH (p)-[:KNOWS]->(f) RETURN f.name AS friend LIMIT 1 } RETURN p.name, friend +MATCH (p:Person) OPTIONAL CALL (p) { MATCH (p)-[:WORKS_AT]->(c) RETURN c.name AS co } RETURN p.name, co ``` -- `:` → exact match using property filter in node directly -- `=` → exact match using property filter in WHERE clause - -## Boolean operators - -- `OR` - - ```jsx - MATCH (n:Person) - WHERE n.medication = "Aspirin" - OR n.medication = "Ibuprofen" - RETURN n.name - ``` -- `AND` - - ```jsx - MATCH (n) - WHERE n.name = 'Matt' - AND n.age = 20 - AND n.hasPhD = false - RETURN n - ``` - -## Comparison operators - -- Equal: `=` - - ```jsx - # Find antibodies targeting proteins in Human - MATCH (ab:Antibody)-->(prot:Protein) - WHERE prot.host = 'Human' - RETURN ab.name, prot.name - ``` -- Inequal: `<>` - - ```jsx - # Find antibodies (associated to a protein) used together - # in same publication (2-hop) - MATCH (ab1:Antibody)-->(prot:Protein), (ab2:Antibody)-->(prot:Protein) - WHERE ab1.name <> ab2.name - RETURN ab1.name, ab2.name, prot.name, prot.gene_name - ``` -- Less than: `<` -- Less than or equal to: `<=` -- Greater than: `>` -- Greater than or equal to: `>=` - - ```jsx - # Publications published on 2020 or after - MATCH (pub:Publication) - WHERE pub.published_year >= 2020 - RETURN pub.displayName, pub.pubmedid, pub.published_year, pub.country - ``` -- is null:`IS NULL` -- is not null:`IS NOT NULL` - - ```jsx - # Publications from the United States - MATCH (pub:Publication) - WHERE pub.country = 'United States' - AND pub.institution IS NOT NULL - RETURN pub.displayName, pub.institution, pub.published_year - ``` - -## Data Types - -| Type | Example | -| --------- | ------------------------ | -| String | `'hello'` or `"hello"` | -| Integer | `age=30` | -| Boolean | `flag=true` | -| Double | `score=3.14` | -| Embedding | `emb=(1.2, 2.0, 0.0)` | - -## RETURN Clause - -```jsx -MATCH (n)-->(m) RETURN n.name, n.age, m.name, m.age -``` - -- Select fields with `RETURN n.prop, m.prop` - -## Expression evaluation - -```jsx -// Multiply property by constant -MATCH (n) RETURN n.price * 1.1 - -// Divide two properties -MATCH ()-[r]->() RETURN r.a / r.b +## Functions -// Add two properties -MATCH (n) RETURN n.val + n.tax +[Functions and data types](/query/functions) + +| Category | Functions | +| -------- | --------- | +| Entity | `id`, `labels`, `type`, `startNode`, `endNode` | +| Path and list | `nodes`, `relationships`, `length`, `size`, `head`, `last`, `tail`, `reverse`, `range`, `reduce` | +| Conversion | `toInteger`, `toFloat`, `toString`, `toBoolean`, `coalesce` | +| String | `toUpper`, `toLower`, `trim`, `ltrim`, `rtrim`, `reverse` | +| Math | `abs`, `sign`, `ceil`, `floor`, `round`, `sqrt`, `exp`, `log`, `log10`, `pi`, `e`, `sin`, `cos`, `tan` | +| Temporal | `datetime`, `duration` | +| Embedding | `cosine_similarity`, `euclidean_distance` | +| Aggregate | `count`, `collect`, `sum`, `avg`, `min`, `max` | + +```cypher +MATCH (n)-[e]->(m) RETURN labels(n), type(e), labels(m) +RETURN datetime('2024-03-15T10:30:00Z') + duration({days: 1}) +MATCH (n:Document) RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) ``` -## LOAD graphs +- Function names are case-sensitive, except `count` and `collect`. +- `edgeType()` was removed: use `type(e)`. -### Load TuringDB graph +## Data types -Files have to be available in `graphs` subdirectory of `turing-dir`. - -```jsx -LOAD GRAPH graph_name -``` +| Type | Example | +| ---- | ------- | +| String | `'hello'` or `"hello"` | +| Integer | `30`, `0x1F` | +| Double | `3.14` | +| Boolean | `true` | +| List | `[1, 'a', [2]]` | +| Map | `{a: 1}` | +| DateTime | `datetime('2024-03-15T10:30:00Z')` | +| Duration | `duration({days: 2})` | +| Embedding | `(1.2, 2.0, 0.0)` (parentheses, not brackets) | -### Load external data into TuringDB graph +## Writing (inside a change) -Files have to be available in `data` subdirectory of `turing-dir`. +[Writing data](/query/writing) -```jsx -LOAD GML 'mygml.gml' AS my_graph -``` - -```jsx -LOAD JSONL 'myjsonl.jsonl' AS my_graph +```cypher +CHANGE NEW +CREATE (:Person {name: 'Alice', age: 30}) +CREATE (:Person {name: 'Mick'})-[:FRIEND_OF {since: 2020}]->(:Person {name: 'John'}) +COMMIT +MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Mick'}) CREATE (a)-[:KNOWS]->(b) +MERGE (n:Person {name: 'Alice'}) ON CREATE SET n.created = true ON MATCH SET n.seen = true +MATCH (n:Person {name: 'Alice'}) SET n.age = 31, n.position = 'Developer' +MATCH (n:Person {name: 'Alice'}) REMOVE n.position +MATCH (n:Person {name: 'Alice'}) DETACH DELETE n +MATCH (n:Person)-[e:KNOWS]->(m) DELETE e +CHANGE SUBMIT ``` -## Procedures (`CALL`) - -| Query | Description | -| ------------------------- | ----------------------------------- | -| `CALL db.labels()` | All node labels | -| `CALL db.propertyTypes()` | All node/edge property keys & types | -| `CALL db.edgeTypes()` | All edge types | -| `CALL db.history()` | Commit history | -| `CALL db.describeCommit('')` | Counts for a single commit | -| `CALL db.procedures()` | List available procedures | -| `CALL db.showIndexes()` | List property indexes | +- Every node needs at least one label and every edge exactly one type. Labels cannot be changed later. +- Run `COMMIT` after creating entities that a later query must match. `CHANGE SUBMIT` commits for you. +- In the Python SDK, call `client.checkout()` after `CHANGE SUBMIT`. +- For more than a few thousand entities, use [`LOAD PARQUET`](/import_data/parquet). -Procedure outputs can be `YIELD`ed and chained: `CALL db.labels() YIELD label MATCH (n) WHERE n.name = label RETURN n`. +## Procedures -## DELETE +[Procedures and commands](/query/procedures) -```jsx -// Delete a node -MATCH (n:Person {name: 'Alice'}) DELETE n +| Query | Description | +| ----- | ----------- | +| `CALL db.labels()` | All node labels | +| `CALL db.edgeTypes()` | All edge types | +| `CALL db.propertyTypes()` | All property keys and their types | +| `CALL db.history()` | Commit history | +| `CALL db.describeCommit('')` | Counts for a single commit | +| `CALL db.showIndexes()` | Property indexes | +| `CALL db.getNodes([0, 1])` | Labels and all properties of specific nodes | +| `CALL db.procedures()` | All procedures and signatures | -// Delete an edge -MATCH (n)-[e:KNOWS]->(m) DELETE e -``` +Outputs can be chained: `CALL db.labels() YIELD label MATCH (n) WHERE label IN labels(n) RETURN label, count(n)`. -## Variable-length paths +## Graph algorithms and search -```jsx -// Postfix quantifiers (not Neo4j's -[*1..3]-) -MATCH (a:Person)-[e]->+(b:Person) RETURN a, b // one or more hops -MATCH (a:Person)-[e]->*(b:Person) RETURN a, b // zero or more hops -MATCH (a:Person)-[e]->{2,4}(b:Interest) RETURN a, b // 2 to 4 hops +```cypher +MATCH (a:Station {name: 'Ashchurch'}), (b:Station {name: 'Pershore'}) shortestPath(a, b, distance, dist, path) RETURN dist, path +VECTOR SEARCH IN my_index FOR 5 (1.2, 0.5, 3.0, 0.1) YIELD ids, score RETURN ids, score ``` -## Graph algorithms - -| Function | Description | -| ------------------------------------------ | -------------------------------------------------------------- | -| `shortestPath(n, m, distanceProperty, distOutputVar, pathOutputVar)` | Shortest path between nodes using the Dijkstra processor | - -## Functions - -| Function | Description | -| ----------------- | ------------------------------------------------------------------ | -| `labels(n)` | Node label as a string | -| `edgeType(e)` | Edge type as a string | -| `toInteger(expr)` | Parses a string as an integer | -| `toFloat(expr)` | Parses a string as a float | -| `toBoolean(expr)` | Parses a string as a boolean | -| `cosine_similarity(a, b)` | Cosine similarity of two embeddings | -| `euclidean_distance(a, b)` | Euclidean distance of two embeddings | - -## Aggregate Functions - -| Function | Description | -| ----------------- | ------------------------------------------------------------------ | -| `count(expr)` | Returns the number of non-null rows in the column corresponding to `expr` | -| `avg(expr)` | Returns the average value of non-null rows in the numeric-valued column corresponding to `expr` | - -```jsx -// Introspection -MATCH (n)-[e]->(m) RETURN labels(n), edgeType(e) - -// Type conversion -MATCH (n) WHERE n.year > toInteger("2020") RETURN n.name -MATCH (n) RETURN n.price * toFloat("1.07") - -// Aggregation -MATCH (n:Person) RETURN count(*) -``` - -## Engine Commands - -| Command | Description | -| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| `CREATE GRAPH ` | Create a new named graph | -| `LOAD GRAPH ` | Load an existing graph | -| `LOAD GML 'mygraph.gml' AS my_graph` | Load the specified GML as TuringDB graph. Requires the GML to be accessible in TuringDB directory (`-turing-dir`, `data` subdir) | -| `LOAD JSONL 'mygraph.jsonl' AS my_graph` | Load the specified JSONL as TuringDB graph. Requires the JSONL to be accessible in TuringDB directory (`-turing-dir`, `data` subdir) | -| `COMMIT` | Persist intermediate state within a change (needed between separate node/edge create steps) | -| `LOAD COMMIT ''` | Load a past commit into memory (needed for REST API queries on non-HEAD commits; CLI and SDK do this automatically) | -| `LIST GRAPH` | List all available graphs | -| `HISTORY` | Show commit history (versioning) | -| `CHANGE NEW` | Create a new isolated change | -| `CHANGE SUBMIT` | Merge current change into main | -| `CHANGE DELETE` | Delete current change | -| `CHANGE LIST` | List active uncommitted changes | -| `CREATE VECTOR INDEX WITH DIMENSION METRIC ` | Create a vector index | -| `VECTOR SEARCH IN FOR () YIELD ids` | k-nearest-neighbor vector search | -| `SHOW VECTOR INDEXES` / `DELETE VECTOR INDEX ` | List / delete vector indexes | -| `CREATE INDEX FOR (n) ON n.prop` / `FOR [e] ON e.prop` | Create a property index (write; commit to take effect) | -| `DROP INDEX ` | Drop a property or vector index | -| `MERGE_DATAPARTS` | Compact the current graph's DataParts into one | -| `INSTALL ` / `SHOW EXTENSIONS` / `SHOW PROCEDURES` | Load a procedure extension / list extensions / list procedures | -| `LOAD CSV "" [WITH HEADERS] [ON ERROR SKIP\|FAIL] AS row` | Stream CSV rows (see [CSV import](/import_data/csv)) | - -## UNWIND - -```jsx -// Expand a literal list into rows (literal lists only) -UNWIND [1, 2, 3] AS x RETURN x - -// Return a list literal directly -RETURN [1, 2, 3] AS nums -``` +See [Shortest path](/shortest-path) and [Vector search](/vector-search). ## Property indexes -```jsx -// Indexes are writes — create inside a change, then commit +```cypher CHANGE NEW CREATE INDEX age_index FOR (n) ON n.age +CREATE INDEX since_index FOR [e] ON e.since CHANGE SUBMIT -MATCH (n) WHERE n.age = 30 RETURN n // optimizer uses the index - -DROP INDEX age_index // remove ``` -## Useful Patterns - -```jsx -// Two-hop connection -MATCH (a)-[r1]->(b)-[r2]->(c) RETURN c - -// All people named John -MATCH (n:Person {name: 'John'}) RETURN n -/// exact equivalent of -MATCH (n:Person) WHERE n.name = 'John' RETURN n - -// Find co-authors: two persons linked to the same publication -MATCH (a:Person)-->(p:Publication)<--(b:Person) -WHERE a.name <> b.name -RETURN a.name, b.name, p.title - -// Match existing nodes then create edge between them -MATCH (n:Person {name: 'Alice'}), (m:Person {name: 'Bob'}) -CREATE (n)-[:KNOWS]->(m) - -// Cartesian: pair every person with every city -MATCH (p:Person), (c:City) RETURN p.name, c.name -``` +`DROP INDEX ` is also a write, so run it inside a change too. + +## Engine commands + +| Command | Description | +| ------- | ----------- | +| `CREATE GRAPH ` | Create a new named graph | +| `LOAD GRAPH ` | Load an existing graph | +| `LOAD GML 'f.gml' AS g` | Load a [GML](/import_data/gml) file as a graph (file in `/data`) | +| `LOAD JSONL 'f.jsonl' AS g` | Load a [JSONL](/import_data/jsonl) file as a graph, optionally `WITH EMBEDDINGS`, `WITH DATETIMES`, `WITH DURATIONS` | +| `LOAD PARQUET '' AS g` | Load [Parquet](/import_data/parquet) files as a graph | +| `LOAD CSV 'f.csv' [WITH HEADERS] [ON ERROR SKIP\|FAIL] AS row` | Stream rows from a [CSV](/import_data/csv) file | +| `LIST GRAPH` / `LIST AVAILABLE GRAPHS` | Loaded graphs / graphs on disk | +| `CHANGE NEW` / `SUBMIT` / `DELETE` / `LIST` | Create, merge, discard and list changes | +| `COMMIT` | Seal pending writes inside a change | +| `LOAD COMMIT ''` | Load a past commit (REST API only; the CLI and SDK do it for you) | +| `MERGE_DATAPARTS` | Compact the graph's DataParts into one (renumbers nodes) | +| `CREATE VECTOR INDEX WITH DIMENSION METRIC ` | Create a vector index | +| `SHOW VECTOR INDEXES` / `DELETE VECTOR INDEX ` | List / delete vector indexes | +| `INSTALL ` / `SHOW EXTENSIONS` / `SHOW PROCEDURES` | Load a procedure extension / list extensions / list procedures | +| `EXPLAIN ` | Show the query plan without running it | + +## Not supported + +- Query parameters (`$name`), more than one statement per request, `--` and `/* */` comments. +- `=~` regular expressions, `FOREACH`, `COLLECT { }`, `CREATE CONSTRAINT`. +- Label expressions `(n:A|B)` and `(n:!A)`. Use `WHERE n:A OR n:B`. +- `SET n:Label`, `REMOVE n:Label`, `SET n += {...}`, `SET n = {...}`. +- `properties()`, `keys()`, `elementId()`, map projection `n {.name}`. diff --git a/query/cypher_subset.mdx b/query/cypher_subset.mdx index 3a0d2fa..d0e932d 100644 --- a/query/cypher_subset.mdx +++ b/query/cypher_subset.mdx @@ -1,621 +1,139 @@ --- title: "Query Language in TuringDB" -sidebarTitle: "Query Language" -description: "TuringDB uses Cypher as a query language and has some unique types of queries" +sidebarTitle: "Overview" +description: "TuringDB v3 runs Cypher on a new engine and supports most of openCypher: OPTIONAL MATCH, WITH, UNWIND, UNION, MERGE, variable-length paths, subqueries and more." --- -TuringDB supports most of the **Cypher query language**, extended with versioning, metadata search, and flexible property matching. +TuringDB v3 (`turingdb` 3.0 on PyPI) runs every query on a new MLIR-based engine and supports **most of openCypher**, extended with versioning, vector search and fast lookup by native node ID. -This guide covers some examples of queries types, including `MATCH`, `CREATE`, property filters, procedures, and available data types. +If you know Cypher from Neo4j, you can write the queries you are used to. This page covers the basics and the handful of places where TuringDB differs. The rest of the language is documented in the pages that follow. -# Basics + + + `MATCH`, `OPTIONAL MATCH`, `WHERE`, joins and fast lookup by node ID + + + Variable-length paths, quantified path patterns and named paths + + + `WITH`, `ORDER BY`, `UNION`, aggregates, `UNWIND`, comprehensions, `CASE` + + + `EXISTS`, `COUNT` and `CALL` subqueries + + + Built-in functions, value types, dates and durations, maps + + + `CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`, changes and indexes + + + `CALL db.*`, graph and change commands, extensions + + + A compact reference of everything above + + -Queries are built by referencing nodes and edges. TuringDB supports the standard CYPHER syntax, in that nodes are denoted using parentheses `()`, whilst edges are denoted using square brackets `[]`. +## What is supported -For example, `(n)` would denote a node named `n`, whilst `[e]` would denote an edge named `e`. +| Area | Supported | +| ---- | --------- | +| Reading | `MATCH`, `OPTIONAL MATCH`, `WHERE`, `WITH`, `RETURN [DISTINCT]`, `ORDER BY`, `SKIP`, `LIMIT`, `UNWIND`, `UNION [ALL]`, `LOAD CSV` | +| Patterns | Multiple labels, edge type alternatives (`[:A\|B]`), inline `WHERE`, variable-length paths (`*1..3`, `->+`, `{2,4}`), quantified path patterns, named paths, pattern predicates | +| Expressions | Arithmetic, string and list concatenation, `CASE`, list and pattern comprehensions, `reduce`, `all` / `any` / `none` / `single`, list slicing, maps | +| Aggregation | `count`, `sum`, `avg`, `min`, `max`, `collect`, `DISTINCT`, implicit grouping, aggregates inside expressions | +| Subqueries | `EXISTS { }`, `COUNT { }`, `CALL { }` and `OPTIONAL CALL { }` | +| Writing | `CREATE`, `MERGE` with `ON CREATE` / `ON MATCH`, `SET`, `REMOVE`, `DELETE`, `DETACH DELETE`, property indexes | +| Types | String, Integer, Double, Boolean, List, Map, `DateTime`, `Duration`, Embedding | +| TuringDB extras | Changes, commits and time travel, `shortestPath`, `VECTOR SEARCH`, `gnn.graphSAGE`, lookup by native node ID | -Nodes and edges can have both **label** and **property** constraints. As in standard CYPHER, property constraints are specified using curly brackets `{}`, whilst labels are specified using colon `:` syntax. +## Basics -An example of a node with a label constraint would be `(n:Person)`. The edge equivalent is the **edge type**, written the same way: `[e:FRIENDS_WITH]`. An edge has exactly one edge type. +Queries are built from **nodes** and **edges**: -An example of an edge with a property constraint would be `[e {duration: 10}]`. - -Property and label constraints may be combined, for instance, `(n:Person {name: 'John'})` specifies a node which both has the label `Person`, and a `name` property with the value `John`. - -Nodes and edges can have label and property multiple constraints, which are specified in a comma-separated list: `(n:Person:Man {name: 'John', age: 20})`. These lists can be arbitrarily long. - -By convention: - -- we prefer using single quotes around strings (even if double quotes alsowork ) -- node label are written in Pascal case (no spaces): e.g. `Person`, `BankAccount`, `BloodType` -- edge label are written in upper case (spaces replaced by underscores): e.g. `TRANSACTION`, `FRIENDS_WITH`, `IS_CLIENT_OF` - -### Summary: - -Queries are built around **nodes** and **edges**: - -- **Nodes** are written in parentheses `()` e.g. `(n)` - a node with alias `n` -- **Edges** are written in square brackets `[]` e.g. `[e]` - an edge with alias `e` - -You can add: - -- **Labels** with a colon `:` - e.g. `(n:Person)` -- **Edge types** with a colon `:` - e.g. `[e:FRIENDS_WITH]` or `[:FRIENDS_WITH]` -- **Property constraints** with curly braces `{}` - e.g. `[e {duration: 10}]` -- **Both** at once - e.g. `(n:Person {name: 'John'})` - -Multiple labels and properties can be specified: - -```jsx -(n:Person:Man {name: 'John', age: 20}) -``` - -# Queries - -Queries are built up of combinations of nodes and edges, as specified above. - -## `MATCH` queries - -`MATCH` queries are used to retrieve nodes, edges, and their property values from the database via specification of relationships and properties. - -It is easiest to demonstrate the syntax of a `MATCH` query with a concrete example: - -```jsx -MATCH (n)-[e]->(m) RETURN n, e, m -MATCH (m)<-[e]-(n) RETURN n, e, m // also works -``` - -The above query will look for all the directed edges in the graph. The query will return the internal ID of the node `n`. - - - When using `RETURN n`, other implementations of CYPHER may return all properties of `n`, whilst TuringDB only returns the internal ID of `n`. - - -`MATCH` queries are flexible: they can contain a single node and no edges, or any number of node, edge pairs. For example: - -```jsx -MATCH (n) RETURN n -``` - -will match all nodes in the database. An example of a multi-hop `MATCH` query would be: - -```jsx -MATCH (n)-[e]->(m)-[f]->(p) RETURN n, m, p -``` - - - Queries have _variables_, which in the above examples are those such as `n`, `m`, `e`, etc. Variables are a way to give a name to a node or an edge, so that those nodes or edges can be specified in the `RETURN` clause. However, if you do not want to return an edge or its properties, the edge need not have a name. For example: - - ```jsx - MATCH (n)-->(m) RETURN m - ``` - - Both node and edges can omit a variable name if they specify at least a label constraint: - - ```jsx - MATCH (:Person)-[:FRIENDS_WITH]->(m) RETURN m - ``` - - -You can also return multiple properties using a comma separated list: - -```jsx -MATCH (n:Person)-->(m:Person) RETURN n.name, m.age -``` - -Combining the syntax of `MATCH` queries, and the ability to specify constraints, here are a few examples of some syntactically correct TuringDB `MATCH` queries: - -- `MATCH (n:Person) RETURN n.name` -- `MATCH (:Person)-->(n:Person) RETURN n.name` -- `MATCH (n:Person:Woman:SoftwareEngineer) RETURN n.name` -- `MATCH (n:Person:Woman:SoftwareEngineer)-->(m)-[e]->(p:Man)-[f]->(q) RETURN e, f` - - - Note that whilst all the above the queries are all _syntactically_ valid, if the graph does not have a node property which is used in a query, it will fail to execute. - - -## `CREATE` queries - -`CREATE` queries follow exactly the same syntax as `MATCH` queries when it comes to specifying nodes, edges, and property/label constraints. `CREATE` queries may have `RETURN` clauses, but they do not need them. There is also the additional requirement that all nodes and edges _must have at least one label_. This means a query such as - -```jsx -CREATE (n) -``` - -is not valid, whilst - -```jsx -CREATE (n:Person) -``` - -is a valid query. - -There is no requirement to declare any names for any variables. This means we can have queries such as - -```jsx -CREATE (:Person)-[:FRIENDS_WITH]->(:Person) -``` - -However, naming variables can be useful if you want to create multiple edges to or from a given node. For instance, if you would like to create a triangle pattern, this can be achieved using the following approach - -```jsx -CREATE (a:Corner)-[:EDGE]->(b:Corner), (b)-[:EDGE]->(c:Corner), (c)-[:EDGE]->(a) -``` - -## `MATCH ... CREATE ...` and `MATCH ... CREATE ... RETURN ...` queries - -`MATCH`, `CREATE` and `RETURN` statements can be used together in queries. - -For example, to create a edge between two existing graph, the following queries can be used: - -```jsx -// Create two nodes Person -CREATE (:Person {name: 'Alice', age: 24}) -CREATE (:Person {name: 'John', age: 27}) - -// Match two nodes to create the edge between them -MATCH (n:Person {name: 'Alice'}), (m:Person {name: 'John'}) -CREATE (n)-[:FRIENDS_WITH]->(m) - -// RETURN clause can also be added -MATCH (n:Person {name: 'Alice'}), (m:Person {name: 'John'}) -CREATE (n)-[:FRIENDS_WITH]->(m) -RETURN n.name, n.age, m.name, m.age -``` - -## `WHERE` queries - -The `WHERE` clause allows to filter the results on node and/or edge labels and/or properties. - -To filter on node label: - -```jsx -MATCH (n) -WHERE n:Person -RETURN n, n.age -``` - -To filter on node property: - -```jsx -MATCH (n) -WHERE n.name = 'Alice' -RETURN n, n.age -``` - -To filter on edge label: - -```jsx -MATCH (n)-[e]->(m) -WHERE e:PLAYPOKER -RETURN n.name, m.age -``` - -To filter on edge property: - -```jsx -MATCH (n)-[e]->(m:Person) -WHERE n.name = 'Gabby' -AND e.play_poker = true -RETURN n.name, m.age -``` - -## Filtering on edge types - -`MATCH` queries can be restricted to a given edge type with the `[:EDGE_TYPE]` syntax, the edge counterpart of a node label: +- Nodes are written in parentheses: `(n)` is a node named `n`. +- Edges are written in square brackets: `[e]` is an edge named `e`. +- **Labels** are added with a colon: `(n:Person)`. A node can have several: `(n:Person:Employee)`. +- **Edge types** use the same syntax: `[e:FRIENDS_WITH]`. An edge has exactly one edge type. +- **Properties** are constrained with curly braces: `(n:Person {name: 'John', age: 20})`. ```cypher -// Only KNOWS edges -MATCH (n:Person)-[:KNOWS]->(m:Person) -RETURN n.name, m.name - -// Name the edge to also return its properties -MATCH (n:Person)-[e:KNOWS]->(m:Person) -RETURN n.name, e.since, m.name - -// Each hop can have its own edge type -MATCH (a:Person)-[:KNOWS]->(b:Person)-[:WORKS_AT]->(c:Company) -RETURN a.name, b.name, c.name +MATCH (a:Person {name: 'Alice'})-[e:KNOWS]->(b:Person) +RETURN a.name, e.since, b.name ``` -An edge carries exactly one edge type, so each edge pattern takes a single `[:EDGE_TYPE]` constraint. Use `CALL db.edgeTypes()` to list the edge types available in the graph. - -## Multi-pattern queries (Joins and Cartesian Products) - -TuringDB supports matching multiple patterns in a single query by separating them with commas. Depending on whether the patterns share variables, this results in either a **join** or a **cartesian product**. - -### Cartesian Product +By convention, node labels are written in PascalCase (`Person`, `BankAccount`), edge types in upper case with underscores (`WORKS_AT`, `FRIENDS_WITH`), and strings use single quotes. -When patterns in a `MATCH` clause are separated by commas and do **not** share any variables, TuringDB computes a cartesian product of the results. Every row from the first pattern is combined with every row from the second. +Keywords are case-insensitive (`MATCH` and `match` are the same). Labels, edge types and property names are case-sensitive. Only `//` starts a comment: ```cypher -MATCH (a:Person), (b:Interest) -RETURN a.name, b.name -``` - -This returns all combinations of Person nodes with Interest nodes. If there are 6 persons and 5 interests, the result contains 30 rows. - -You can use `WHERE` to filter the cartesian product: - -```cypher -MATCH (p:Person), (i:Interest) -WHERE p.isFrench = true AND i.name = 'MegaHub' -RETURN p.name, i.name -``` - -Three or more patterns can be combined: - -```cypher -MATCH (p:Person), (i:Interest), (c:Category) -WHERE p.name = 'A' AND i.name = 'Shared' AND c.name = 'Cat1' -RETURN p.name, i.name, c.name +// a line comment +MATCH (n:Person) RETURN n.name // also fine at the end of a line ``` - Cartesian products can produce very large result sets. A product of N nodes by M nodes produces N × M rows. Use `WHERE` filters or `LIMIT` to keep result sizes manageable. + SQL-style `-- comments` and `/* block comments */` are **not** supported. `--` is parsed as part of the query. -### Pattern-based Joins +### Running a query -When two paths converge on a shared node variable, TuringDB performs a **hash join** on that variable. This is useful for finding entities that share a common connection. +Queries run through the Python SDK, the interactive shell, or the HTTP endpoint. Read queries run directly against the current graph; [write queries](/query/writing) must run inside a change. -```cypher -MATCH (a:Person)-->(b:Interest)<--(c:Person) -WHERE a.name <> c.name -RETURN a.name, b.name, c.name -``` +```python +from turingdb import TuringDB -This finds pairs of different persons who share a common interest. The variable `b` acts as the join point. - -Multi-hop joins are also supported: - -```cypher -MATCH (a:Person)-->(i:Interest)-->(c:Category) -RETURN a.name, i.name, c.name +client = TuringDB(host="http://localhost:6666") +df = client.query("MATCH (n:Person) RETURN n.name, n.age") # a pandas DataFrame ``` -You can specify edge types explicitly: - -```cypher -MATCH (a:Person)-[:INTERESTED_IN]->(b:Interest)<-[:INTERESTED_IN]-(c:Person) -WHERE a.name <> c.name -RETURN a.name, b.name, c.name -``` - -### Mixing Paths and Cartesian Products - -Connected paths and independent patterns can be combined in the same query. Shared variables create joins, while unrelated patterns create cartesian products. - -```cypher -MATCH (a:Person)-->(i:Interest), (c:Category) -WHERE c.name = 'Cat1' -RETURN a.name, i.name, c.name -``` - -This returns every Person→Interest connection combined with the Cat1 category node. - -Two independent paths can also be combined: - -```cypher -MATCH (a:Person)-->(i1:Interest), (b:Person)-->(i2:Interest) -WHERE a.name = 'Alice' AND b.name = 'Bob' AND i1.name <> i2.name -RETURN a.name, i1.name, b.name, i2.name -``` - -This finds all combinations of Alice's interests with Bob's interests, excluding pairs where both interests are the same. - -## `LIMIT` keyword - -**LIMIT** restricts the number of returned results. The following query returns only the first 10 results: - -```cypher -MATCH (n) -RETURN n -LIMIT 10 -``` - -## `SKIP` keyword - -**SKIP** skips the first N results before returning the rest. The following query skips the first 10 results: - -```cypher -MATCH (n) -RETURN n -SKIP 10 -``` - -`SKIP` and `LIMIT` can be combined for pagination. The following query will return the 11th to 20th results: - -```cypher -MATCH (n) -RETURN n -SKIP 10 -LIMIT 10 -``` - -## Sorting with `ORDER BY` - -**ORDER BY** sorts the results by one or more properties. By default, results are sorted in ascending order. - -```cypher -MATCH (n:Person) -RETURN n.name, n.age -ORDER BY n.age -``` - -Use `DESC` to sort in descending order: - -```cypher -MATCH (n:Person) -RETURN n.name, n.age -ORDER BY n.age DESC -``` - -You can sort by multiple properties. Results are sorted by the first property, then ties are broken by subsequent properties: - -```cypher -MATCH (n:Person) -RETURN n.name, n.age, n.city -ORDER BY n.city, n.age DESC -``` - -`ORDER BY` can be combined with `SKIP` and `LIMIT` for sorted pagination: - -```cypher -MATCH (n:Person) -RETURN n.name, n.age -ORDER BY n.age DESC -SKIP 10 -LIMIT 10 -``` - -# Data Types - -TuringDB offers the following data types for node and edge properties: - -- String -- Boolean -- Integer (signed) -- Double (decimal) -- Embedding (vector of floats, e.g. `(1.2, 2.0, 0.0)`) - -String properties can be enclosed using double quotes (`"`), single quotes (`'`), or backticks (\`\`\`). - -# Operators - -## Boolean operators - -The `OR` and `AND` operartors are used to filter on multiple conditions: - -```jsx -MATCH (n:Person) -WHERE n.medication = "Aspirin" -OR n.medication = "Ibuprofen" -RETURN n.name -``` - -```jsx -MATCH (n) -WHERE n.name = 'Matt' -AND n.age = 20 -AND n.hasPhD = false -RETURN n -``` - -## Comparison operators - -TuringDB allows you to query against node and edge properties using the `:` operator for exact matching of the property value. - -```jsx -MATCH (n {name: 'Matt', age: 20, hasPhD: false}) RETURN n -``` - -You can also pass through the WHERE clause to do the exact equivalent query: - -```text -MATCH (n) -WHERE n.name = 'Matt' -AND n.age = 20 -AND n.hasPhD = false -RETURN n -``` - -Implemented comparison operators: - -- Equal: `=` - - ```text - # Find antibodies targeting proteins in Human - MATCH (ab:Antibody)-->(prot:Protein) - WHERE prot.host = 'Human' - RETURN ab.name, prot.name - ``` -- Inequal: `<>` - - ```jsx - # Find antibodies (associated to a protein) used together - # in same publication (2-hop) - MATCH (ab1:Antibody)-->(prot:Protein), (ab2:Antibody)-->(prot:Protein) - WHERE ab1.name <> ab2.name - RETURN ab1.name, ab2.name, prot.name, prot.gene_name - ``` -- Less than: `<` -- Less than or equal to: `<=` -- Greater than: `>` -- Greater than or equal to: `>=` - - ```jsx - # Publications published on 2020 or after - MATCH (pub:Publication) - WHERE pub.published_year >= 2020 - RETURN pub.displayName, pub.pubmedid, pub.published_year, pub.country - ``` -- is null:`IS NULL` -- is not null:`IS NOT NULL` - - ```jsx - # Publications from the United States - MATCH (pub:Publication) - WHERE pub.country = 'United States' - AND pub.institution IS NOT NULL - RETURN pub.displayName, pub.institution, pub.published_year - ``` - -# Expression Evaluation in RETURN - -Since v1.22.0, TuringDB supports arithmetic expressions directly in `RETURN` projections. - -**Supported operators:** `+`, `-`, `*`, `/` - -```cypher --- Single property with constant -MATCH (n) -WHERE n.type = 'Patient' -RETURN n.age * 12 - --- Two properties combined -MATCH ()-[r]->() -WHERE r.value_satoshi > 0 -AND r.value_usd > 0 -RETURN r.value_usd / r.value_satoshi * 100000000.0 -``` - -# Built-in Functions - -TuringDB supports built-in functions for type conversion, introspection, and embedding comparison. - -## Introspection: `labels()`, `edgeType()` - -```jsx -MATCH (n)-[e]->(m) RETURN labels(n), edgeType(e), labels(m) -``` - -## Type conversion: `toInteger()`, `toFloat()`, `toBoolean()` - -```jsx -MATCH (n) WHERE n.year > toInteger("2020") RETURN n.name, n.year -MATCH (n) RETURN n.name, n.price * toFloat("1.07") -MATCH (n) RETURN toBoolean("true") -``` - -## Embedding comparison: `cosine_similarity()`, `euclidean_distance()` - -Compare two embeddings (embedding literals use parentheses — see [Vector Search](/vector-search)): - -```jsx -MATCH (n:Document) RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) -MATCH (n:Document) RETURN n.title, euclidean_distance(n.emb, (0.4, 0.3, 0.8, 0.1)) -``` - -## Aggregation: `count()`, `avg()` - -TuringDB supports the `count()` and `avg()` aggregating functions. Aggregates may only be used when the query has a single return item: - -```jsx -MATCH (n:Person) RETURN count(*) -MATCH (n:Person) RETURN count(n.age) -MATCH (n:Person) RETURN avg(n.age) -``` - -Aggregates may be combined within that single return item: - -```jsx -MATCH (n:Person) RETURN count(n) + avg(n.age) -``` - -# Procedures - -On top of supporting queries which return or alter information _in the graph_, TuringDB supports a number of _procedures_ which return information, or metadata, _about the graph_. - -These queries follow the `CALL` syntax, and the following variants are supported: - -1. `CALL db.propertyTypes()` - all node and edge property keys and their types -2. `CALL db.labels()` - all node labels -3. `CALL db.edgeTypes()` - all edge types (edge equivalent of node labels) -4. `CALL db.history()` - the commit history (commit, nodeCount, edgeCount, partCount) -5. `CALL db.describeCommit('')` - node/edge/part counts for a single commit -6. `CALL db.procedures()` - list the available procedures and their signatures -7. `CALL db.showIndexes()` - list property indexes and their sizes - -These procedures are useful for exploring the data which is available in the graph, and using this to plan `MATCH` queries. - -A procedure's output columns can be `YIELD`ed and fed into a following `MATCH`/`WHERE`: - -```jsx -CALL db.propertyTypes() YIELD propertyType, valueType RETURN * -CALL db.labels() YIELD label, id MATCH (n) WHERE n.name = label RETURN n, label -``` - -# Commands - -Outside of CYPHER, there are a number of commands which you can use to interact with the TuringDB engine. - -| Command | Explanation | -| -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `CREATE GRAPH ` | Create a graph with the specified name | -| `LOAD GRAPH ` | Load the specified TuringDB graph. Requires the graph files to be accessible in TuringDB directory (`-turing-dir`, `graphs` subdir) | -| `LOAD GML 'mygraph.gml' AS my_graph` | Load the specified GML as TuringDB graph. Requires the GML to be accessible in TuringDB directory (`-turing-dir`, `data` subdir) | -| `LOAD JSONL 'mygraph.jsonl' AS my_graph` | Load the specified JSONL as TuringDB graph. Requires the JSONL to be accessible in TuringDB directory (`-turing-dir`, `data` subdir) | -| `LOAD JSONL 'mygraph.jsonl' AS my_graph WITH EMBEDDINGS [{"emb", 384}]` | Load JSONL and mark array properties as embeddings (see [JSONL import](/import_data/jsonl#loading-embedding-properties)) | -| `COMMIT` | Persist intermediate state within a change (needed between separate node and edge create steps) | -| `CHANGE NEW` | Creates a new change, returning a column with the ID of the created change | -| `CHANGE SUBMIT` | When checked out on a specific change, submits all changes made to the “master branch” | -| `CHANGE DELETE` | Deletes the currently checked out change | -| `CHANGE LIST` | Lists the currently active (uncommitted) changes | -| `LOAD COMMIT ''` | Load a past commit into memory for querying. Required when sending queries directly to the REST API for non-HEAD commits. The CLI `checkout` and Python SDK handle this automatically. | -| `LIST GRAPH` | Lists the available graphs | -| `CREATE VECTOR INDEX WITH DIMENSION METRIC ` | Create a vector index (see [Vector Search](/vector-search)) | -| `VECTOR SEARCH IN FOR () YIELD ids` | k-nearest-neighbor search over a vector index | -| `CREATE INDEX FOR (n) ON n.prop` | Create a property index on a node property (write — see Indexes below) | -| `CREATE INDEX FOR [e] ON e.prop` | Create a property index on an edge property | -| `DROP INDEX ` | Drop a property or vector index by name | -| `MERGE_DATAPARTS` | Compact all DataParts of the current graph into one (see [DataParts](/concepts/dataparts)) | -| `INSTALL ` | Load a procedure extension (see Extensions below) | -| `SHOW EXTENSIONS` | List installed extensions | -| `SHOW PROCEDURES` | List available procedures and signatures (same data as `CALL db.procedures()`) | - -# UNWIND - -`UNWIND` expands a list into one row per element. It is a reading statement and composes with `RETURN` / `MATCH`. + + `EXPLAIN ` returns the compiled query plan without running the query. It is the easiest way to check that a lookup uses an index or a constant node scan. + -```jsx -UNWIND [1, 2, 3] AS x RETURN x -``` +## What to know if you come from Neo4j + +TuringDB follows openCypher closely. These are the differences you are most likely to hit: + +- **`RETURN n` returns the node's internal ID**, an integer, not a node object with all its properties. Return the properties you need: `RETURN n.name, n.age`. There is no `properties(n)` or `keys(n)`; use [`db.getNodes`](/query/procedures#inspecting-nodes-and-edges) to see every property of a few nodes. +- **No query parameters.** `$name` is a parse error and `client.query()` only takes a string. Put literal values in the query text and escape quotes yourself. +- **One statement per request.** Scripts separated by `;` are rejected (a single trailing `;` is fine). +- **Labels are fixed at creation.** `SET n:Label` and `REMOVE n:Label` are not supported. Every node has at least one label, and every edge has exactly one edge type. +- **Property types are global.** A property name has a single value type across the whole graph. If `age` is an Integer, you cannot store a String `age` on another node. +- **Strict numeric equality.** Comparing an Integer with a Double using `=` (for example `n.age = 30.0`) is rejected. Use a range (`n.score > 1.99 AND n.score < 2.01`) or convert with `toInteger`. `<`, `>` and arithmetic across numeric types are fine. +- **Unknown labels and properties do not raise an error.** A label that does not exist matches nothing, and a misspelled property reads as `null`. Check names with [`CALL db.labels()` and `CALL db.propertyTypes()`](/query/procedures). +- **Label expressions such as `(n:A|B)` or `(n:!A)` are not supported.** Write `WHERE n:A OR n:B` or `WHERE NOT n:A` instead. Edge type alternatives (`[:A|B]`) are supported. +- **No regular expressions.** `=~` is a parse error. Use `STARTS WITH`, `ENDS WITH` and `CONTAINS`. +- **No `FOREACH` and no constraints.** Use `UNWIND` for loops and `MERGE` to avoid duplicates. +- **Shortest paths use a TuringDB statement**, not `shortestPath((a)-[*]-(b))`. See [Shortest path](/shortest-path). +- **Function names are case-sensitive** (`toInteger`, not `TOINTEGER`), except `count` and `collect`. + + + Coming from TuringDB 1.x? The v3 engine replaces the previous one, and many earlier limits are gone: `OPTIONAL MATCH`, `WITH`, `DISTINCT`, `UNION`, `MERGE`, `DELETE`, aggregates in any position, `UNWIND` over any list, `LOAD CSV` followed by `MATCH`, and `CREATE ... SET` all work now. One function changed name: `edgeType(e)` is now `type(e)`. + + +## Data types + +TuringDB offers these types for node and edge properties and for values in queries: + +| Type | Literal | Notes | +| ---- | ------- | ----- | +| String | `'Alice'` or `"Alice"` | Backslash escapes (`\'`). Backticks quote identifiers, not strings | +| Integer | `30`, `0x1F` | Signed 64-bit. Integer division truncates | +| Double | `3.14`, `1e3` | `^` always returns a Double | +| Boolean | `true`, `false` | | +| List | `[1, 'a', [2]]` | May mix types and nest | +| Map | `{a: 1, b: [1, 2]}` | See [Maps](/query/functions#maps) | +| DateTime | `datetime('2024-03-15T10:30:00Z')` | Microsecond precision, always UTC | +| Duration | `duration({days: 2})` | Microsecond precision, no month component | +| Embedding | `(1.2, 2.0, 0.0)` | Parentheses, at least 2 numbers. See [Vector search](/vector-search) | - `UNWIND` currently accepts **literal lists only** — `UNWIND $param`, `UNWIND someVariable`, and `UNWIND collect(...)` are not yet supported. List literals (`[...]`) may also be returned directly, e.g. `RETURN [1, 2, 3] AS nums`, but their elements must be literals. + Embedding literals use **parentheses**. `[1.2, 2.0, 0.0]` is a List, which is a different type. -# Property Indexes - -A property index speeds up equality lookups on a node or edge property. The query optimizer automatically rewrites `WHERE n.prop = ` into an index lookup when a matching index exists. - -Creating an index is a **write**, so it must run inside a change and only takes effect after the change is committed/submitted: - -```jsx -CHANGE NEW -CREATE INDEX age_index FOR (n) ON n.age -- node property index -CREATE INDEX since_index FOR [e] ON e.since -- edge property index -CHANGE SUBMIT - --- now this uses the index instead of a full scan: -MATCH (n) WHERE n.age = 30 RETURN n -``` - -Inspect existing indexes with `CALL db.showIndexes()` (yields `name`, `size`); remove one with `DROP INDEX age_index`. - -# Extensions - -Extensions are native shared libraries that register extra procedure namespaces callable via `CALL`. Install one by name, then call its procedures: - -```jsx -INSTALL greeter -SHOW EXTENSIONS -CALL greeter.hello() YIELD message RETURN message -``` - -`SHOW PROCEDURES` lists every available procedure and its signature (the same information as `CALL db.procedures()`). +More detail on types, conversion and results is in [Functions and data types](/query/functions). -# Roadmap +## Roadmap -Whilst TuringDB currently supports most of CYPHER, 100% of CYPHER can be parsed but we are working on supporting query execution for some rare CYPHER queries types. TuringDB also has a CALL function which over time will contain more and more algorithms. +We keep extending Cypher coverage and growing the library of procedures available through `CALL`. If you need a query form that is not supported yet, please [tell us](/troubles/troubles#contact-us). diff --git a/query/functions.mdx b/query/functions.mdx new file mode 100644 index 0000000..873e6af --- /dev/null +++ b/query/functions.mdx @@ -0,0 +1,121 @@ +--- +title: "Functions and Data Types" +sidebarTitle: "Functions and Types" +description: "Built-in functions, how values come back to the Python SDK, dates and durations, maps, and embedding comparison in TuringDB v3." +--- + +## Built-in functions + +Function names are **case-sensitive** (`toInteger`, not `TOINTEGER`), except `count` and `collect`. + +| Category | Functions | Notes | +| -------- | --------- | ----- | +| Entity | `id(x)`, `labels(n)`, `type(r)`, `startNode(r)`, `endNode(r)` | `labels` returns a **list**. `type` returns a string. `startNode(r).name` works | +| Path | `nodes(p)`, `relationships(p)`, `length(p)` | Lists of IDs. Read properties through a [comprehension](/query/aggregation#list-comprehensions) | +| List | `size`, `length`, `head`, `last`, `tail`, `reverse`, `range(start, end[, step])`, `reduce` | `size`, `length` and `reverse` also work on strings. `range` bounds are inclusive | +| Predicates | `all`, `any`, `none`, `single` `(x IN list WHERE ...)` | | +| Conversion | `toInteger`, `toFloat`, `toString`, `toBoolean`, `coalesce(a, b, ...)` | `toInteger` and `toFloat` accept strings, integers and doubles, and return `null` for a bad string. `toBoolean` accepts strings. `coalesce` arguments must share one type | +| String | `toUpper`, `toLower`, `trim`, `ltrim`, `rtrim`, `reverse` | Plus the operators `STARTS WITH`, `ENDS WITH`, `CONTAINS` and `\|\|` | +| Math | `abs`, `sign`, `ceil`, `floor`, `round`, `sqrt`, `exp`, `log`, `log10`, `pi()`, `e()` | Trigonometry: `sin`, `cos`, `tan`, `cot`, `asin`, `acos`, `atan`, `degrees`, `radians`, `haversin` | +| Temporal | `datetime(isoString)`, `datetime(epochSeconds)`, `datetime()`, `duration({days: 2, hours: 3})`, `duration(micros)` | See [Dates and durations](#dates-and-durations) | +| Embedding | `cosine_similarity(a, b)`, `euclidean_distance(a, b)` | Arguments must be Embeddings, not lists. See [Vector search](/vector-search) | +| Aggregate | `count`, `collect`, `sum`, `avg`, `min`, `max` | See [Aggregation](/query/aggregation#aggregation) | + +```cypher +MATCH (n)-[e]->(m) RETURN labels(n), type(e), labels(m) +MATCH (n:Person) RETURN toUpper(n.name), toString(n.age) +MATCH (n:Person) WHERE n.age > toInteger('29') RETURN n.name +MATCH (n:Person) RETURN n.name, n.age * toFloat('1.07') +RETURN toBoolean('true') +RETURN abs(-3), round(2.5), sqrt(16.0), pi() +MATCH (n:Person) RETURN n.name, coalesce(n.nickname, n.name) +``` + + + **`edgeType()` was removed in v3.** Use `type(e)` to get an edge's type as a string. + + +Calling a function that does not exist fails with `Function 'X' does not exist`. Functions that are **not** available include `substring`, `split`, `replace`, `left`, `right`, `rand`, `date()`, `timestamp()`, `properties`, `keys` and `elementId`. Do that kind of post-processing in your application, and filter strings with `STARTS WITH`, `ENDS WITH` and `CONTAINS`. + +## Arithmetic in `RETURN` and `SET` + +Arithmetic expressions work anywhere an expression is allowed: + +```cypher +MATCH (n:Person) RETURN n.age * 12 +MATCH (n:Person) RETURN n.name, n.age + 5 AS adjusted, n.age / 2 +MATCH ()-[r]->() WHERE r.since > 0 RETURN r.since / 10 * 100 +``` + +`+ - * / % ^` are supported. Integer `/` truncates, and `^` returns a Double. `+` and `||` concatenate strings and lists, and `'x' + 1` gives `'x1'`. + +## Dates and durations + +`DateTime` and `Duration` are first-class types: + +```cypher +RETURN datetime('2024-03-15T10:30:00Z') AS d +RETURN datetime('2024-03-15') AS midnight +RETURN datetime(1700000000) AS fromEpoch +RETURN datetime('2024-03-15T10:30:00Z') + duration({days: 1, hours: 2}) AS later +RETURN datetime('2024-03-16T00:00:00Z') - datetime('2024-03-15T00:00:00Z') AS elapsed +RETURN datetime('2024-03-15T10:30:00Z').year, datetime('2024-03-15T10:30:00Z').month +RETURN duration({days: 2, hours: 3}).hours +``` + +- `datetime` values are stored as microseconds since the Unix epoch, **normalised to UTC**. A date-only string means midnight, and an offset (`+02:00`) is converted to the equivalent UTC instant. +- `duration` is a signed count of microseconds. It has no month component, because a month has no fixed length. `duration()` reads the number as microseconds. +- Arithmetic works: `datetime ± duration`, `datetime − datetime`, `duration × number`. +- Components are available as properties: `d.year`, `d.month`, `d.day`, `d.hour`, and so on. On a duration, `dur.days`, `dur.hours`, ... give the **total** in that unit, so `duration({days: 2, hours: 3}).hours` is `51`. +- `<`, `>`, `min` and `max` work on temporal values. +- When you [import JSONL](/import_data/jsonl) you can declare which properties are `DateTime` or `Duration` with `WITH DATETIMES [...]` and `WITH DURATIONS [...]`. + +## Maps + +Map literals can be built, returned, compared with `=`, and stored as node or edge properties: + +```cypher +RETURN {name: 'Alice', tags: ['a', 'b']} AS m +WITH {a: 1, b: 'x'} AS m RETURN m.a, m.b +UNWIND [{a: 1}, {a: 2}] AS m RETURN m.a +``` + +You can read a key from a map **value** that is in scope, for example one bound by `WITH` or `UNWIND`, with `m.key`. + + + A map **stored as a property** cannot be dereferenced inside a query: `n.meta.level` and `m['key']` are errors, and map projection (`n {.name, .age}`) is not supported. Return the property as a whole, or store the individual values as separate properties. + + +## How results come back + +`client.query()` returns a pandas DataFrame. The column types follow the query's value types: + +| Type | HTTP `column_types` | pandas dtype | +| ---- | ------------------- | ------------ | +| String | `String` | `string` | +| Integer | `Int64` | `Int64` | +| Unsigned integer (IDs, `count`, `length(p)`) | `UInt64` | `UInt64` | +| Double | `Double` | `float64` | +| Boolean | `Bool` | `boolean` | +| List | `List` | object (Python list) | +| Map | `Map` | object (dict) | +| Embedding | `Embedding` | object (list over HTTP, numpy `float32` array over the native and embedded backends) | +| DateTime | `DateTime` | `datetime64[us, UTC]` | +| Duration | `Duration` | `timedelta64[us]` | +| Node or edge (`RETURN n`) | `UInt64` | `UInt64` (internal ID) | +| Named path (`RETURN p`) | `EntityList` | object (list of `{type, id}` dicts) | + + + Over the default HTTP/JSON backend, Doubles are serialised with **6 decimal places** (`3.14159265` becomes `3.141593`). If you need full precision, scale the value in the query, or use the [native or embedded backend](/pythonsdk/reference#client-backends). + + +## Embeddings + +Embeddings are stored as ordinary properties (type `Embedding`) and compared with `cosine_similarity` (true cosine) and `euclidean_distance`. Both work in `RETURN`, `WHERE` and `ORDER BY`: + +```cypher +MATCH (n:Document) RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) AS sim ORDER BY sim DESC LIMIT 10 +MATCH (n:Document) RETURN n.title, euclidean_distance(n.emb, (0.4, 0.3, 0.8, 0.1)) +``` + +This is a brute-force scan over the matched nodes. For large collections, build a [vector index](/vector-search). diff --git a/query/matching.mdx b/query/matching.mdx new file mode 100644 index 0000000..b12d581 --- /dev/null +++ b/query/matching.mdx @@ -0,0 +1,203 @@ +--- +title: "Matching Patterns" +sidebarTitle: "Matching" +description: "Read data with MATCH, OPTIONAL MATCH and WHERE, join patterns, and look up nodes quickly by native node ID." +--- + +Read queries run directly against the current graph. There is no change or transaction to open first. + +The examples on this page use a small graph of people, companies and interests. Create it once in a change (see [Writing data](/query/writing)) if you want to follow along: + +```cypher +CREATE + (alice:Person {name: 'Alice', age: 30, city: 'London', active: true, tags: ['admin', 'vip']}), + (bob:Person {name: 'Bob', age: 35, city: 'London', active: true, tags: ['dev']}), + (carol:Person {name: 'Carol', age: 28, city: 'Paris', active: false, tags: ['dev', 'vip']}), + (dave:Person {name: 'Dave', age: 41, city: 'Paris', active: true}), + (acme:Company {name: 'Acme', founded: 1999}), + (globex:Company {name: 'Globex', founded: 2010}), + (music:Interest {name: 'Music'}), + (alice)-[:KNOWS {since: 2010}]->(bob), + (bob)-[:KNOWS {since: 2015}]->(carol), + (carol)-[:KNOWS {since: 2018}]->(dave), + (alice)-[:WORKS_AT {role: 'CTO'}]->(acme), + (bob)-[:WORKS_AT {role: 'Engineer'}]->(acme), + (carol)-[:WORKS_AT {role: 'Engineer'}]->(globex), + (alice)-[:LIKES]->(music), + (carol)-[:LIKES]->(music) +``` + +## MATCH + +```cypher +MATCH (n) RETURN count(n) +MATCH (n:Person) RETURN n.name, n.age +MATCH (n:Person {name: 'Alice', age: 30}) RETURN n.name +MATCH (n:Person:Employee) RETURN n.name +MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN a.name, b.name +MATCH (a)-[e:KNOWS {since: 2010}]->(b) RETURN a.name, e.since, b.name +MATCH (a)<-[e]-(b) RETURN a.name, b.name +MATCH (a)-[e]-(b) RETURN a.name, b.name +MATCH (a)-[r:KNOWS|WORKS_AT]->(b) RETURN a.name, type(r), b.name +MATCH (a {name: 'Alice'}) MATCH (a)-[:KNOWS]->(b) RETURN b.name +``` + +- `(n:Person:Employee)` matches nodes that have **both** labels. +- `(a)-[e]-(b)` (or `(a)--(b)`) ignores direction. +- `[:KNOWS|WORKS_AT]` matches either edge type. An edge has exactly one type, so each edge pattern takes a single type or a list of alternatives. +- `type(r)` returns the edge type as a string. +- Several `MATCH` clauses in a row are allowed, and a later one can reuse variables from an earlier one. +- Variables are only needed for entities you want to return or reuse. `MATCH (:Person)-[:KNOWS]->(m) RETURN m.name` is valid, and a node or edge may omit its variable when it has a label or type. + + + `RETURN n` returns the node's internal ID, not its properties. Return the properties you need (`RETURN n.name, n.age`). This is different from Neo4j, where `RETURN n` returns the whole node. + + +### Filtering inside the pattern + +A `WHERE` can sit inside a node or edge pattern. The variable must be named: + +```cypher +MATCH (a:Person WHERE a.age > 30)-[r:KNOWS WHERE r.since > 2012]->(b) RETURN b.name +``` + +### Ordering and limiting in `MATCH` + +As a TuringDB extension, `ORDER BY`, `SKIP` and `LIMIT` can also appear right after the pattern, before `RETURN`: + +```cypher +MATCH (n:Person) ORDER BY n.age LIMIT 2 RETURN n.name +``` + + + Unknown labels and property names do **not** raise an error. `MATCH (n:Nope) RETURN n` returns no rows and `RETURN n.misspelled` returns `null`. Use [`CALL db.labels()`, `CALL db.edgeTypes()` and `CALL db.propertyTypes()`](/query/procedures) to check what exists in a graph. + + +## OPTIONAL MATCH + +`OPTIONAL MATCH` keeps the row when the pattern has no match and returns `null` for its variables: + +```cypher +MATCH (a:Person) OPTIONAL MATCH (a)-[:WORKS_AT]->(c:Company) RETURN a.name, c.name +MATCH (a:Person) OPTIONAL MATCH (a)-[:KNOWS]->(b) RETURN a.name, count(b) +``` + +The second query returns `0` for people who know nobody. A `WHERE` attached to an `OPTIONAL MATCH` belongs to the optional part, so rows are kept, as in Neo4j. + + + An unmatched optional **node** returned bare (`RETURN c`) shows `18446744073709551615` (the largest unsigned 64-bit integer) instead of `null`. Properties (`c.name`) and `id(c)` are correctly `null`, so return a property or `id(c)`. + + +## WHERE and operators + +```cypher +MATCH (n) WHERE n:Person RETURN n.name +MATCH (n:Person) WHERE n.age >= 18 AND n.city = 'London' RETURN n.name +MATCH (n:Person) WHERE n.name STARTS WITH 'A' OR n.name ENDS WITH 'e' OR n.name CONTAINS 'ro' RETURN n.name +MATCH (n:Person) WHERE n.name IN ['Alice', 'Bob'] RETURN n.name +MATCH (n:Person) WHERE 'admin' IN n.tags RETURN n.name +MATCH (n:Person) WHERE n.tags IS NOT NULL RETURN n.name +MATCH (n:Person) WHERE n.active RETURN n.name +MATCH (n)-[e]->(m) WHERE e:KNOWS AND e.since > 2012 RETURN n.name, m.name +MATCH (p:Person) WHERE (p)-[:LIKES]->() RETURN p.name +MATCH (p:Person) WHERE NOT (p)-[:KNOWS]->() RETURN p.name +``` + +- **Label and type tests:** `n:Person` tests a node label, `e:KNOWS` tests an edge type. Combine them with `AND`, `OR` and `NOT`. +- **Pattern predicates:** `WHERE (p)-[:LIKES]->()` keeps the nodes for which the pattern exists. For richer conditions see [`EXISTS` subqueries](/query/subqueries). + +| Operators | | +| --------- | - | +| Comparison | `=`, `<>`, `!=`, `<`, `<=`, `>`, `>=`, and chained comparisons such as `1 < x < 3` | +| Null tests | `IS NULL`, `IS NOT NULL` | +| Boolean | `AND`, `OR`, `XOR`, `NOT`, with three-valued logic for `null` as in Neo4j | +| Membership and strings | `IN`, `STARTS WITH`, `ENDS WITH`, `CONTAINS` | +| Arithmetic | `+`, `-`, `*`, `/`, `%`, `^`. Integer `/` truncates and `^` returns a Double | +| Concatenation | `+` and `\|\|` join strings and lists | + +Property constraints in a pattern, `MATCH (n {name: 'Matt', age: 20})`, are exactly equivalent to the same equality tests in `WHERE`. + + + Regular expressions (`=~`) and the `exists(n.prop)` function are not supported. Use `n.prop IS NOT NULL` to test for a property. + + +### Numeric equality is strict + +Comparing an Integer with a Double using `=` is rejected when the query is analysed, and so is comparing two Doubles with `=`: + +```text +MATCH (n:Person) WHERE n.age = 30.0 RETURN n.name +ANALYZE_ERROR: Operands are not valid or compatible types +``` + +Use range comparisons for Doubles, or convert the value: + +```cypher +MATCH (n:Person) WHERE n.age > 29.5 AND n.age < 30.5 RETURN n.name +MATCH (n:Person) WHERE n.age = toInteger(30.0) RETURN n.name +``` + +Mixed-type `<`, `>` and arithmetic are fine. Integer division by zero raises an error, and Int64 overflow wraps silently. + +## Joins and Cartesian products + +Comma-separated patterns in a `MATCH` produce either a **join** or a **Cartesian product**, depending on whether they share a variable. + +```cypher +// Cartesian product: every Person with every Company (N x M rows) +MATCH (p:Person), (c:Company) RETURN p.name, c.name + +// Join: a shared variable links the patterns +MATCH (a:Person)-[:KNOWS]->(b), (b)-[:WORKS_AT]->(c) RETURN a.name, b.name, c.name + +// Join on a shared node: people who like the same thing +MATCH (a:Person)-->(i:Interest)<--(b:Person) WHERE a <> b RETURN a.name, b.name, i.name + +// Join on a value through WHERE (hash join) +MATCH (a:Person), (b:Person) WHERE a.city = b.city AND a <> b RETURN a.name, b.name +``` + + + A Cartesian product of N nodes by M nodes produces N × M rows. Always constrain it with `WHERE` or `LIMIT` before running it on a large graph. + + +Connected paths and independent patterns can be mixed in one query. Shared variables create joins and unrelated patterns create Cartesian products: + +```cypher +MATCH (a:Person)-->(i:Interest), (c:Company) WHERE c.name = 'Acme' RETURN a.name, i.name, c.name +``` + +## Look up nodes by native node ID + +Every node has a **native node ID**, the integer you get from `RETURN n` or `id(n)`. When you already know which nodes you want, **compare the node variable itself with the ID**, written as a literal. The planner turns this into a constant scan of exactly those nodes: + +```cypher +MATCH (n) WHERE n = 0 RETURN n.name +MATCH (n)-[:KNOWS]->(m) WHERE n = 0 RETURN m.name +UNWIND [0, 1, 2] AS x MATCH (n) WHERE n = x RETURN id(n), n.name +``` + +`EXPLAIN` shows whether it worked. The plan should contain `const_scan_nodes` rather than `scan_nodes`: + +```cypher +EXPLAIN MATCH (n) WHERE n = 0 RETURN n.name +``` + +On large graphs this is orders of magnitude faster than looking nodes up again through a key property such as `n.uuid`. The rules that keep it fast: + +- **Compare the variable, not the function.** `WHERE n = 123` is a direct seek. `WHERE id(n) = 123` scans every node. Use `id(n)` only to *return* the ID. +- **Use `UNWIND` for batches, not `IN`.** `WHERE n IN [...]` and `WHERE id(n) IN [...]` scan every node. The `UNWIND [] AS x MATCH (n) WHERE n = x` form is rewritten at plan time into one constant scan, which costs roughly a microsecond per node. +- **The batch form needs an exact shape:** + - the list is a **literal written in the query text** (build it in Python: `f"UNWIND {ids} AS x ..."`), not a variable, `collect(...)`, `range(...)` or a list property; + - its elements are **distinct** integers, so deduplicate first; + - `x` is used **only** in `n = x`. Return `id(n)` rather than `x`. +- **Fetch IDs once and reuse them.** Look nodes up with whatever filter you need (`MATCH (n:Person {name: 'Alice'}) RETURN id(n)`), keep the IDs in your application, and address the nodes by ID from then on. +- **Never compute IDs per row, and never match two nodes from one row.** Forms such as `WHERE a = p[0] AND b = p[1]` are not rewritten and run as a Cartesian product of two full node scans, which can exhaust memory on large graphs. To connect two known nodes, put both IDs in as literals, one pair per query: `MATCH (a) WHERE a = 5 MATCH (b) WHERE b = 7`. For bulk edges, see [Parquet import](/import_data/parquet). +- **Vector search results are already nodes.** `VECTOR SEARCH ... YIELD ids` binds `ids` as nodes, so use `ids.title` or `MATCH (ids)-[...]->(...)` directly. See [Vector search](/vector-search). + +**Stability.** The native IDs of existing nodes are stable across queries, change submits and server restarts, so it is safe to cache them. Two exceptions: + +- Nodes created in an open change get their final IDs at `COMMIT` or `CHANGE SUBMIT`. Read them after that. +- [`MERGE_DATAPARTS`](/concepts/dataparts) renumbers all nodes. Re-fetch any cached IDs afterwards. + +Keep a business key as a property as well, for durable identity outside TuringDB. There is no `elementId()`. For edges, `MATCH ()-[r]->() WHERE id(r) = 0 RETURN startNode(r), endNode(r)` works. diff --git a/query/paths.mdx b/query/paths.mdx new file mode 100644 index 0000000..12f6442 --- /dev/null +++ b/query/paths.mdx @@ -0,0 +1,98 @@ +--- +title: "Paths" +sidebarTitle: "Paths" +description: "Variable-length paths, quantified path patterns and named paths in TuringDB." +--- + +These examples use the [sample graph](/query/matching) from the matching page. + +## Variable-length paths + +TuringDB supports both its own postfix quantifiers and the Neo4j bracket form: + +| Syntax | Hops | +| ------ | ---- | +| `-[:KNOWS]->+` | 1 or more | +| `-[:KNOWS]->*` | **0** or more (includes the start node itself) | +| `-[:KNOWS]->{2,4}`, `{2}`, `{,3}`, `{2,}` | between, exactly, up to, at least | +| `-[:KNOWS*1..3]->`, `-[:KNOWS*2]->`, `-[*..3]-`, `-[*0..1]->` | Neo4j style | +| `-[:KNOWS*]->` | **1** or more (unlike the postfix `*`) | + +```cypher +MATCH (a:Person {name: 'Alice'})-[:KNOWS]->+(b) RETURN DISTINCT b.name +MATCH (a:Person {name: 'Alice'})-[:KNOWS*1..3]->(b) RETURN DISTINCT b.name +MATCH (a)-[:KNOWS|WORKS_AT]->{1,2}(b) RETURN a.name, b.name +MATCH (a)-[e:KNOWS WHERE e.since > 2012]->+(b) RETURN a.name, b.name +MATCH (a)-[:KNOWS*1..3 {since: 2010}]->(b) RETURN b.name +MATCH (a {name: 'Alice'})-[e:KNOWS]->+(b) RETURN b.name, size(e) +``` + +- **Edge type alternatives** (`[:KNOWS|WORKS_AT]`) work on variable-length hops. +- **A predicate or property map applies to every hop.** `-[e:KNOWS WHERE e.since > 2012]->+` only follows edges that satisfy it, and `-[:KNOWS*1..3 {since: 2010}]->` requires every edge on the path to have that property value. +- **Trail semantics.** An edge never repeats within one path, but nodes may. Each distinct path is its own row, so use `DISTINCT` or an aggregate to collapse them. +- **One quantifier per edge.** You cannot combine `[*1..3]` with `{1,3}` on the same edge. +- **The edge variable of a variable-length hop is a list of edge IDs.** `size(e)`, `e[0]`, `e[-1]` and `UNWIND e` work, but `e.since` does not. To read edge properties along a path, name the path and use `relationships(p)` (below). + + + Variable-length matching **enumerates** paths. On dense graphs the number of paths can grow very quickly, so always bound the length (`*1..6` or `{1,6}`). For weighted shortest paths on large graphs, use the [`shortestPath` statement](/shortest-path). + + +## Quantified path patterns + +A quantified path pattern repeats a one-hop sub-pattern, with its own `WHERE`, a number of times: + +```cypher +MATCH (a:Person {name: 'Alice'}) ((x)-[r:KNOWS]->(y) WHERE r.since > 2009){1,3} (b) RETURN b.name +MATCH (a:Person {name: 'Alice'}) ((x)-[:KNOWS]->(y) WHERE y.age > x.age)+ (b) RETURN b.name +``` + +The second query follows `KNOWS` edges only while ages strictly increase. Inside the parentheses only a **single hop** of the form `(node)-[edge]->(node) [WHERE ...]` is allowed, and the inner variables (`x`, `r`) become lists. + +## Named paths + +Give a path a name to inspect its nodes and edges: + +```cypher +MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) +RETURN [n IN nodes(p) | n.name] AS names, + [r IN relationships(p) | r.since] AS sinces, + length(p) AS hops +ORDER BY hops +``` + +- `nodes(p)` and `relationships(p)` return **lists of IDs**. Read properties through a [list comprehension](/query/aggregation#list-comprehensions). +- `length(p)` is the number of hops. +- A bare `RETURN p` comes back as a list of `{"type": "node" | "edge", "id": ...}` dictionaries. +- `count(p)` works. `collect(p)` does not. +- `OPTIONAL MATCH p = ... RETURN p IS NULL` works. + +## Finding routes + +For hop-count questions, undirected search, or when you need the properties along a route, combine variable-length paths with comprehensions: + +```cypher +// Fewest hops, returning the names along the route +MATCH p = (a:Person {name: 'Alice'})-[:KNOWS*1..6]->(b:Person {name: 'Dave'}) +RETURN [n IN nodes(p) | n.name] AS route, length(p) AS hops +ORDER BY hops LIMIT 1 + +// k-hop neighbourhood and reachability +MATCH (a:Person {name: 'Alice'})-[:KNOWS]-{1,3}(b) RETURN DISTINCT b.name +MATCH (a:Person {name: 'Alice'})-[:KNOWS]->+(b) RETURN count(DISTINCT b) +``` + +To sum a property along a route, build a list with a comprehension and aggregate over its elements: + +```cypher +MATCH p = (a:Person {name: 'Alice'})-[:KNOWS*1..6]->(b:Person {name: 'Dave'}) +WITH [n IN nodes(p) | n.name] AS route, [r IN relationships(p) | r.since] AS years +UNWIND years AS y +WITH route, sum(y) AS total +RETURN route, total ORDER BY total LIMIT 1 +``` + + + Summing directly over `UNWIND relationships(p) AS r` and then `sum(r.prop)` fails with an execution error. Compute the list with a comprehension first, as above. + + +For a true weighted shortest path, see [Shortest path](/shortest-path). diff --git a/query/procedures.mdx b/query/procedures.mdx new file mode 100644 index 0000000..934a070 --- /dev/null +++ b/query/procedures.mdx @@ -0,0 +1,140 @@ +--- +title: "Procedures and Commands" +sidebarTitle: "Procedures and Commands" +description: "Explore a graph with CALL db.* procedures, and manage graphs, changes, indexes and extensions with TuringDB engine commands." +--- + +Beyond queries over the data in a graph, TuringDB has **procedures**, which return information about the graph, and **commands**, which drive the engine itself. + +## Procedures + +Procedures use the `CALL` syntax. Use them to explore a graph before writing queries against it: + +```cypher +CALL db.labels() +CALL db.edgeTypes() +CALL db.propertyTypes() +CALL db.showIndexes() +CALL db.history() +CALL db.procedures() +``` + +| Procedure | Yields | +| --------- | ------ | +| `db.labels()` | `id`, `label` | +| `db.edgeTypes()` | `id`, `edgeType` | +| `db.propertyTypes()` | `id`, `propertyType`, `valueType` | +| `db.showIndexes()` | `name`, `size` | +| `db.history()` | `commit`, `nodeCount`, `edgeCount`, `partCount` | +| `db.describeCommit(commit)` | `nodeCount`, `edgeCount`, `partCount` | +| `db.procedures()` | `name`, `signature` (same as `SHOW PROCEDURES`) | +| `db.hierarchicalLabelCounts(labels)` | `label`, `nodeCount` | +| `db.listNodes(labels, properties, skip, limit)` | `id`, `labels`, `properties` | +| `db.getNodes(nodeIDs)` | `id`, `labels`, `inEdgeCount`, `outEdgeCount`, `properties` | +| `db.getEdges(edgeIDs)` | `id`, `src`, `tgt`, `edgeTypeID`, `properties` | +| `db.getNodeEdges(...)` | `id`, `outgoingEdges`, `incomingEdges`, `outEdgeCounts`, `inEdgeCounts` | +| `gnn.neighbourhoodSample(node, sampleSize, seed)` | `src`, `edge`, `edgeType`, `tgt` | +| `gnn.graphSAGE(seeds, fanouts, seed)` | `dst_nodes0..2`, `src_nodes0..2`, `tgt_nodes0..2` | + +`db.history()` lists commits newest first. Every `COMMIT` and every submitted change adds an entry, and `nodeCount` / `edgeCount` count what **that commit** added, not the graph totals. + +### Chaining procedures with YIELD + +A procedure's output columns can be renamed, filtered and used by the rest of the query with `YIELD`: + +```cypher +CALL db.propertyTypes() YIELD * +CALL db.labels() YIELD label AS l WHERE l STARTS WITH 'P' RETURN l +CALL db.labels() YIELD label MATCH (n) WHERE label IN labels(n) RETURN label, count(n) +CALL db.history() YIELD `commit`, nodeCount RETURN `commit`, nodeCount +``` + +- **Constant arguments only.** A procedure argument cannot read row values: `db.getNodes([id(n)])` fails with "must be constant". +- **`commit` is a keyword.** Yield it with backticks: ``YIELD `commit` ``. +- `valueType` has its own enum type, so `WHERE valueType = 'String'` is rejected. Filter the DataFrame in pandas instead. +- `OPTIONAL CALL` does not work on procedures. + +### Inspecting nodes and edges + +`RETURN n` gives only an internal ID, and there is no `properties(n)`. To see every property of a few nodes, use `db.getNodes`, which returns the properties as a JSON string: + +```cypher +CALL db.getNodes([0, 1, 2]) YIELD id, labels, properties RETURN id, labels, properties +CALL db.listNodes(['Person'], {name: 'ali'}, 0, 10) YIELD id, labels, properties +``` + +`db.listNodes(labels, properties, skip, limit)` does a case-insensitive substring search on property values. + +```python +import json + +df = client.query("CALL db.getNodes([0, 1, 2]) YIELD id, labels, properties") +df["properties"] = df["properties"].map(json.loads) +``` + +### Profiling a graph with plain Cypher + +```cypher +MATCH (n) RETURN labels(n) AS labels, count(*) AS n ORDER BY n DESC +MATCH (a)-[r]->(b) RETURN labels(a) AS src, type(r) AS rel, labels(b) AS dst, count(*) AS n ORDER BY n DESC +MATCH (n:Person) RETURN count(n.age) AS withAge, count(*) AS total +``` + +## Commands + +Outside of Cypher, a set of commands controls the engine. Send them through `client.query()` or from the interactive shell. + +### Graphs + +| Command | Description | +| ------- | ----------- | +| `CREATE GRAPH ` | Create an empty graph | +| `LOAD GRAPH ` | Load an on-disk graph. Its files must be in the TuringDB directory (`-turing-dir`, `graphs` subdirectory) | +| `LIST GRAPH` | List the graphs currently **loaded** in the server | +| `LIST AVAILABLE GRAPHS` | List the graphs on disk, with `isLoaded` and `isLoading` | +| `LOAD JSONL 'f.jsonl' AS ` | Build a graph from a [JSONL](/import_data/jsonl) file in the `data` subdirectory | +| `LOAD GML 'f.gml' AS ` | Build a graph from a [GML](/import_data/gml) file in the `data` subdirectory | +| `LOAD PARQUET '' AS ` | Build a graph from Parquet files, see [Parquet import](/import_data/parquet) | +| `LOAD CSV '' ... AS row` | Stream rows from a CSV file, see [CSV import](/import_data/csv) | +| `EXPLAIN ` | Show the compiled plan without running the query | + +There is no command to drop or unload a graph. + +### Changes and versions + +| Command | Description | +| ------- | ----------- | +| `CHANGE NEW` | Create a new change and return its ID | +| `CHANGE SUBMIT` | Commit pending writes and merge the current change into main | +| `CHANGE DELETE` | Discard the current change | +| `CHANGE LIST` | List the open changes on the current graph | +| `COMMIT` | Seal the pending writes inside a change. Needed before a later query can `MATCH` or `MERGE` newly created entities | +| `LOAD COMMIT ''` | Load a past commit for querying. Only needed when sending queries directly to the REST API. The CLI `checkout` and the Python SDK do it for you | +| `MERGE_DATAPARTS` | Compact all [DataParts](/concepts/dataparts) of the graph into one. Run it on main with no change open on any graph | + +`MERGE_DATAPARTS` **renumbers** all nodes. Re-fetch any cached node IDs afterwards, and rebuild vector indexes keyed by node IDs. + +### Indexes + +| Command | Description | +| ------- | ----------- | +| `CREATE INDEX FOR (n) ON n.prop` | Create a property index on a node property. A write, see [Property indexes](/query/writing#property-indexes) | +| `CREATE INDEX FOR [e] ON e.prop` | Create a property index on an edge property | +| `DROP INDEX ` | Drop a property or vector index | +| `CREATE VECTOR INDEX WITH DIMENSION METRIC ` | Create a vector index, see [Vector search](/vector-search) | +| `VECTOR SEARCH IN FOR () YIELD ids` | k-nearest-neighbour search | +| `SHOW VECTOR INDEXES` | List vector indexes | +| `DELETE VECTOR INDEX ` | Delete a vector index | + +## Extensions + +Extensions are native libraries that register extra procedure namespaces. Install one by name and then call its procedures: + +```cypher +INSTALL greeter +SHOW EXTENSIONS +SHOW PROCEDURES +CALL greeter.hello() YIELD message RETURN message +``` + +`SHOW PROCEDURES` lists every available procedure and its signature, including the ones from extensions. The `greeter` example ships with the Python wheel. Extensions are looked up in `/lib/turingdb/extensions/.so`, then in `/extensions/.so`. diff --git a/query/subqueries.mdx b/query/subqueries.mdx new file mode 100644 index 0000000..48974e4 --- /dev/null +++ b/query/subqueries.mdx @@ -0,0 +1,46 @@ +--- +title: "Subqueries" +sidebarTitle: "Subqueries" +description: "Use EXISTS, COUNT and CALL subqueries to filter, count and compute per-row results in TuringDB." +--- + +These examples use the [sample graph](/query/matching) from the matching page. + +## EXISTS + +`EXISTS { ... }` tests whether a pattern, or a full `MATCH` with its own `WHERE`, has at least one match for the current row. It works in `WHERE` and as a returned value: + +```cypher +MATCH (p:Person) WHERE EXISTS { (p)-[:WORKS_AT]->(:Company {name: 'Acme'}) } RETURN p.name +MATCH (p:Person) WHERE EXISTS { MATCH (p)-[:KNOWS]->(f) WHERE f.age > p.age } RETURN p.name +MATCH (p:Person) WHERE NOT EXISTS { (p)-[:KNOWS]->() } RETURN p.name +MATCH (p:Person) RETURN p.name, EXISTS { (p)-[:LIKES]->() } AS likesSomething +``` + +## COUNT + +`COUNT { ... }` returns the number of matches for each row: + +```cypher +MATCH (p:Person) RETURN p.name, COUNT { (p)-[:KNOWS]->() } AS outDegree ORDER BY outDegree DESC +MATCH (p:Person) WHERE COUNT { MATCH (p)-[:KNOWS]-(x) RETURN x } >= 2 RETURN p.name +``` + +## CALL subqueries + +A `CALL { ... }` subquery runs once per incoming row. It is the way to compute a per-row top-k or a per-row aggregate. Import the outer variables either in the `CALL (p)` header or with a `WITH p` as the first clause of the body: + +```cypher +MATCH (p:Person) CALL (p) { MATCH (p)-[:KNOWS]->(f) RETURN f.name AS friend ORDER BY f.name LIMIT 1 } RETURN p.name, friend +MATCH (p:Person) CALL { WITH p MATCH (p)-[:KNOWS]->(f) RETURN count(f) AS deg } RETURN p.name, deg +MATCH (p:Person) OPTIONAL CALL (p) { MATCH (p)-[:WORKS_AT]->(c) RETURN c.name AS co } RETURN p.name, co +CALL { MATCH (p:Person) RETURN p.name AS nm UNION MATCH (c:Company) RETURN c.name AS nm } RETURN nm ORDER BY nm +``` + +- **Dropped rows.** A non-optional `CALL` drops the outer rows for which its body returns nothing. Use `OPTIONAL CALL` to keep them with `null` values. +- **Imported variables.** A `CALL` body cannot return an imported variable under its own name. Alias it: `RETURN p AS me`. +- A `CALL` body can contain `UNION`, aggregation, `ORDER BY` and `LIMIT`. Write-only subqueries also work inside a [change](/query/writing). + + + Not supported: `COLLECT { ... }` (use `collect()` in a `CALL` body or a [pattern comprehension](/query/aggregation#list-comprehensions) instead), `CALL (*) { }`, `CALL { } IN TRANSACTIONS`, and `OPTIONAL CALL` on a *procedure*. + diff --git a/query/writing.mdx b/query/writing.mdx new file mode 100644 index 0000000..134678b --- /dev/null +++ b/query/writing.mdx @@ -0,0 +1,236 @@ +--- +title: "Writing Data" +sidebarTitle: "Writing Data" +description: "CREATE, MERGE, SET, REMOVE, DELETE and property indexes in TuringDB, and the change workflow that every write goes through." +--- + +All writes (`CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`, `CREATE INDEX`, `DROP INDEX`) must run **inside a change**. A change is TuringDB's equivalent of a git branch: you write in an isolated copy and then submit it to the main graph. A write sent outside a change fails with an explicit error such as `Cannot perform CREATE outside of a write transaction.` + +## The change workflow + +```python +client.new_change() # CHANGE NEW: the client is now on this change + +client.query("CREATE (:Person {name: 'Alice'})-[:KNOWS]->(:Person {name: 'Bob'})") +# ... more writes ... + +client.query("CHANGE SUBMIT") # merge the change into main +client.checkout() # return the client to main +``` + +- `new_change()` returns the change ID and puts the client on that change. +- After `CHANGE SUBMIT` the client still points at the submitted change. Call `client.checkout()` to return to main, otherwise the next query fails with `CHANGE_NOT_FOUND`. +- To abandon a change, run `CHANGE DELETE` and then `client.checkout()`. +- `new_change()` raises if the client is already on a change or on a past commit. Call `checkout()` first. + +See [Create a change](/graph_dev/create_change), [Submit a change](/graph_dev/submit_change) and [Time travelling](/graph_dev/time_travelling) for more on the versioning model. + +### When you need `COMMIT` inside a change + +`COMMIT` seals the pending writes of a change. It adds a commit to the history but does not publish anything to main. Whether a later query can see an earlier write depends on what the earlier write did: + +| An earlier query in the same change did... | Visible to later queries before `COMMIT`? | +| ------------------------------------------ | ----------------------------------------- | +| `CREATE` nodes or edges | **No.** A later `MATCH` finds nothing, a `MATCH ... CREATE` edge silently creates nothing, and `MERGE` creates duplicates | +| `SET`, `REMOVE` or `DELETE` on entities that already existed | Yes, immediately | +| anything, followed by `COMMIT` | Yes | +| writes earlier in the **same query** | Yes | + +**Rule of thumb: run `COMMIT` after any query that creates entities that a later query must `MATCH` or `MERGE`.** You can use several `COMMIT`s in one change. `CHANGE SUBMIT` commits whatever is still pending, so no `COMMIT` is needed right before it. + +```python +client.new_change() +client.query("CREATE (:Person {name: 'Alice'}), (:Person {name: 'Bob'})") +client.query("COMMIT") # make the new nodes visible to the next query +client.query("MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Bob'}) CREATE (a)-[:KNOWS]->(b)") +client.query("CHANGE SUBMIT") +client.checkout() +``` + +### Failed queries and conflicts + +- **A failed query can leave partial writes in the change.** After an execution error in the middle of a change, the safest recovery is `CHANGE DELETE` and redoing the work. +- **Changes are isolated.** Main does not see a change's writes, even committed ones, until the change is submitted. Each change is rebased onto main at submit time. +- **Disjoint edits submit cleanly.** If two changes edit the same entity, the second `CHANGE SUBMIT` fails with an error such as `This change attempted to update Node 0 (...) which has been modified on main.` Conflict detection is coarse: creating an edge to a node that was modified on main also conflicts. +- **A failed submit rejects the whole change but leaves it open.** You can fix it and submit again, or run `CHANGE DELETE`. +- **IDs of newly created nodes and edges are provisional until `COMMIT` or `CHANGE SUBMIT`.** Read them after committing. IDs of existing nodes are stable across submits and restarts (see [node ID lookup](/query/matching#look-up-nodes-by-native-node-id)). + +## CREATE + +```cypher +CREATE (:Person {name: 'Alice', age: 30}) +CREATE (a:Person:Employee {name: 'Bob'}) RETURN a.name, id(a) +CREATE (:Person {name: 'Mick'})-[:FRIEND_OF {since: 2020}]->(:Person {name: 'John'}) +CREATE (a:T {x: 1}), (b:T {x: 2}), (a)-[:R]->(b) +CREATE p = (:City {name: 'Paris'})-[:ROAD {km: 450}]->(:City {name: 'Lyon'}) RETURN p +CREATE (c:City {name: 'Li' + 'lle', pop: 2 * 1000, tags: ['a', 'b'], info: {zone: 1}, emb: (0.1, 0.2, 0.3)}) +CREATE (n:Item {sku: 'A1'}) SET n.price = 9.99 RETURN n.sku +CREATE (a:W {k: 'a'}) WITH a CREATE (b:W {k: 'b'})-[:NEXT]->(a) +``` + +To connect nodes that already exist, match them first. They must be committed, or created earlier **in the same query**: + +```cypher +MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Bob'}) +CREATE (a)-[:KNOWS {since: 2021}]->(b) +RETURN a.name, b.name +``` + +If you know their [native node IDs](/query/matching#look-up-nodes-by-native-node-id), address them directly. That is the fastest option, but the IDs must be **constants** written in the query, one pair per query: + +```cypher +MATCH (a) WHERE a = 0 MATCH (b) WHERE b = 1 CREATE (a)-[:KNOWS {since: 2021}]->(b) +``` + +**Rules:** + +- **Labels and edge types.** Every node needs at least one label, and every edge exactly one edge type. `CREATE (n)` is invalid, and so is `[:A:B]` on an edge. +- **Labels are fixed at creation.** `SET n:Label` and `REMOVE n:Label` are not supported. +- **Property types are global.** Each property name has **one value type across the whole graph** (see `CALL db.propertyTypes()`). If `age` is an Integer, `CREATE (:X {age: 'old'})` and `SET n.age = 2.5` both fail with a type incompatibility error. There is no Integer to Double widening. +- **Nulls.** `{name: null}` is rejected in `CREATE`. Leave the key out instead. +- **No variable-length edges in write patterns**, and you cannot reuse a matched edge variable in `CREATE`. +- **Parallel edges and self-loops are allowed.** An undirected `CREATE (a)-[:R]-(b)` is accepted and creates `a->b`. + + + Never take both endpoints of an edge from one `UNWIND` row (`a = p[0] AND b = p[1]`). It plans as a Cartesian product of two full node scans and can run the server out of memory. For thousands of edges use [`LOAD PARQUET`](/import_data/parquet). + + +## MERGE + +`MERGE` matches the whole pattern or creates it. It supports `ON CREATE SET` and `ON MATCH SET`: + +```cypher +MERGE (n:Person {name: 'Alice'}) + ON CREATE SET n.created = true + ON MATCH SET n.seen = true +RETURN n.name +``` + +```cypher +MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Carol'}) MERGE (a)-[e:KNOWS]->(b) ON CREATE SET e.since = 2024 +MERGE (a:Tag {name: 'x'})-[:LINKS]->(b:Tag {name: 'y'}) +MATCH (p:Person {name: 'Bob'}) MERGE (p)-[:OWNS]->(c:Car {model: 'Z'}) +UNWIND ['a', 'b', 'a'] AS k MERGE (t:Key {k: k}) RETURN t.k +``` + +`MERGE` deduplicates within one query, so the last example creates two `Key` nodes, not three. + + + `MERGE` **cannot see nodes created by an earlier uncommitted query in the same change**, so it will create duplicates. Run `COMMIT` first. There are no uniqueness constraints, so `MERGE` is the way to avoid duplicates. + + +## SET and REMOVE + +```cypher +MATCH (n:Person {name: 'Alice'}) SET n.age = 31 +MATCH (n:Person {name: 'Alice'}) SET n.score = n.age * 1.1, n.updated = true +MATCH (p:Product) SET p.discountPrice = p.price * (1 - 0.15) +MATCH (a:Person) SET a.band = CASE WHEN a.age > 30 THEN 'senior' ELSE 'junior' END +MATCH (a:Person) SET a.place = coalesce(a.city, 'unknown') +MATCH (n:Person {name: 'Alice'}), (m:Person {name: 'Bob'}) SET n.age = m.age +MATCH (n:Person {name: 'Alice'})-[e:KNOWS]->(m) SET e.weight = 0.5 +MATCH (n:Person) WHERE n.age < 18 SET n.isMinor = true +MATCH (n:Person {name: 'Alice'}) SET n.tags = ['a', 'b'], n.meta = {level: 3} +MATCH (n:Person {name: 'Alice'}) SET n.emb = (1.2, 2.0, 0.0, 12.0) +MATCH (n:Person {name: 'Alice'}) SET n.joined = datetime('2024-03-15T10:30:00Z') +MATCH (a:Person) WITH count(a) AS c MATCH (s:Stats {name: 'people'}) SET s.total = c +MATCH (n) WHERE n = 0 SET n.flag = true +UNWIND [0, 1, 2] AS x MATCH (n) WHERE n = x SET n.flag = true +MATCH (n:Person {name: 'Alice'}) SET n.age = null +MATCH (n:Person {name: 'Alice'}) REMOVE n.score, n.flag +``` + +- `SET` creates the property if it does not exist yet. Setting a property to `null` clears it, and `REMOVE n.prop` removes it. +- The value can be any expression, including a `CASE`, a function call or a value from another matched entity. +- Address nodes by [native node ID](/query/matching#look-up-nodes-by-native-node-id) for the fastest updates. Batch updates use the same literal `UNWIND` form as reads. +- For a one-off embedding, `SET n.emb = (...)` is fine. To write embeddings onto many existing nodes, use [`LOAD EMBEDDING`](/vector-search#embeddings-as-node-properties) instead of a `SET` loop. + + + **Not supported:** `SET n += {...}` ("SET cannot dynamically mutate properties yet"), `SET n = {...}`, `SET n:Label` and `REMOVE n:Label`, and aggregates directly inside `SET` (`SET n.c = count(*)`). Compute the aggregate in a `WITH` first, as in the `Stats` example above. + + +## DELETE and DETACH DELETE + +```cypher +MATCH (n:Person {name: 'Alice'}) DETACH DELETE n +MATCH (n:Person {name: 'Alice'})-[e:KNOWS]->(m) DELETE e +MATCH (n:Temp) WHERE n.v > 3 DELETE n +MATCH (n:Person {name: 'Remy'})-[e]->(m:Person) DELETE e, n +``` + +- `DETACH DELETE` removes a node together with all of its edges. +- A plain `DELETE` of a node that still has edges fails with `Cannot delete a node with relationships; use DETACH DELETE`. +- Deleting a path variable (`DELETE p`) is not supported. Delete its nodes or edges instead. + +## Bulk writes + + + **Beyond a few thousand entities, use [`LOAD PARQUET`](/import_data/parquet) rather than Cypher writes.** Above roughly 5,000 nodes or 1,000 edges, write your data to Parquet and load it into a new graph. It loads 100,000 nodes and 200,000 edges in about 0.2 seconds, while Cypher-driven loads that `MATCH` both endpoints of every edge take minutes at a few thousand edges. Use Cypher writes for small, incremental changes. + + +For small volumes, inline the data with `UNWIND`: + +```cypher +UNWIND range(1, 1000) AS i CREATE (:Num {v: i}) +UNWIND ['Brest', 'Rennes', 'Nantes'] AS nm MERGE (:City {name: nm}) +UNWIND [['Brest', 'FR-29'], ['Rennes', 'FR-35']] AS row CREATE (:City {name: row[0], code: row[1]}) +``` + +A `CREATE` cannot read an element of a mixed-type list row such as `['Brest', 139000]`. Use parallel lists instead: + +```cypher +WITH ['Brest', 'Rennes'] AS names, [139000, 220000] AS pops +UNWIND range(0, size(names) - 1) AS i CREATE (:City {name: names[i], pop: pops[i]}) +``` + +`LOAD CSV` can drive writes too. Create the nodes, `COMMIT`, then create the edges (see [CSV import](/import_data/csv)). Alternatively `MERGE` both endpoints in a single pass, which deduplicates across the rows of one query so no `COMMIT` is needed: + +```cypher +LOAD CSV 'knows.csv' WITH HEADERS AS row + MERGE (a:Person {pid: toInteger(row.src)}) + MERGE (b:Person {pid: toInteger(row.dst)}) + CREATE (a)-[:KNOWS {since: toInteger(row.since)}]->(b) +``` + +`FOREACH` is not supported. Use `UNWIND`. + +**Building the query text in Python.** Because there are no parameters, escape quotes yourself: + +```python +def lit(v): + if isinstance(v, str): + return "'" + v.replace("\\", "\\\\").replace("'", "\\'") + "'" + if isinstance(v, bool): + return "true" if v else "false" + return repr(v) + +names = ["Alice", "O'Brien"] +client.query(f"UNWIND [{', '.join(lit(n) for n in names)}] AS nm MERGE (:Person {{name: nm}})") +``` + +Keep each query to a few thousand rows at most. + +## Property indexes + +A property index speeds up equality lookups. The optimizer rewrites `WHERE n.prop = ` into an index lookup when a matching index exists. Creating and dropping an index are writes, so they run inside a change: + +```cypher +CREATE INDEX age_index FOR (n) ON n.age +CREATE INDEX since_index FOR [e] ON e.since +DROP INDEX age_index +``` + +```python +client.new_change() +client.query("CREATE INDEX age_index FOR (n) ON n.age") +client.query("CHANGE SUBMIT") +client.checkout() +``` + +- Node indexes use `FOR (n) ON n.prop`, edge indexes use square brackets: `FOR [e] ON e.prop`. +- The property must already exist in the graph, and the index name is required. +- Indexes are **not scoped** to a label or edge type. `FOR (n:Person)` and `FOR [e:KNOWS]` fail as not yet supported, and so do the Neo4j forms `CREATE INDEX FOR (n:Person) ON (n.name)` and `IF NOT EXISTS`. +- List indexes with `CALL db.showIndexes()`. A new index appears there after `COMMIT` or `CHANGE SUBMIT`. +- `CREATE CONSTRAINT` is not implemented. + +For vector indexes, see [Vector search](/vector-search). diff --git a/shortest-path.mdx b/shortest-path.mdx index 5f80d43..01501b8 100644 --- a/shortest-path.mdx +++ b/shortest-path.mdx @@ -1,10 +1,14 @@ --- title: "Shortest Path" sidebarTitle: "Shortest Path" -description: "Find the shortest path between nodes in a TuringDB graph using the Dijkstra processor" +description: "Find the weighted shortest path between nodes in a TuringDB graph with the shortestPath statement (Dijkstra)" --- -TuringDB's `shortestPath` processor runs a Dijkstra search from a source set of nodes to a target set of nodes, returning the shortest distance and the path taken based on a numeric edge weight property. +TuringDB's `shortestPath` statement runs a Dijkstra search from a source set of nodes to a target set of nodes, returning the shortest distance and the path taken based on a numeric edge weight property. + + + `shortestPath` is a TuringDB statement, not the openCypher function. The forms `shortestPath((a)-[*]-(b))`, `allShortestPaths(...)`, `ANY SHORTEST` and `SHORTEST k` are not supported. For hop-count questions or paths where you need the properties along the route, use [variable-length paths](/query/paths) instead. + ## Sample dataset @@ -44,9 +48,9 @@ shortestPath(sourceSet, targetSet, edgePropName, distanceReturnVar, pathReturnVa | ------------------- | ------------------------------------------------------------------- | | `sourceSet` | Variable bound to the source node(s) via a preceding `MATCH` clause | | `targetSet` | Variable bound to the target node(s) via a preceding `MATCH` clause | -| `edgePropName` | Name of the numeric edge property to use as weight | -| `distanceReturnVar` | Output variable name for the total shortest distance | -| `pathReturnVar` | Output variable name for the path string | +| `edgePropName` | Name of the numeric edge property to use as weight, written as a bare identifier (`distance`), **not** a quoted string. Edges where it is null are skipped | +| `distanceReturnVar` | Output variable name for the total shortest distance. A Double, or an Integer if the weight is an integer | +| `pathReturnVar` | Output variable name for the path: a flat list of alternating IDs `[node, edge, node, edge, ..., node]` | ## Basic usage @@ -71,13 +75,12 @@ shortestPath(sourceSet, targetSet, edgePropName, distanceReturnVar, pathReturnVa ``` -+-------+--------------+ -| dist | path | -+-------+--------------+ -| 11.29 | (0)-[1]->(6) | -+-------+--------------+ + dist path +0 11.29 [0, 1, 6] ``` +`path` alternates node and edge IDs: node `0`, edge `1`, node `6`. + ### Returning only the distance or only the path Both output variables are always computed — you can choose to return one or both: @@ -103,11 +106,8 @@ Both output variables are always computed — you can choose to return one or bo ``` -+-------+ -| dist | -+-------+ -| 11.29 | -+-------+ + dist +0 11.29 ``` ## Multiple sources or targets @@ -137,17 +137,16 @@ You can pass multiple nodes to the source or target set — `shortestPath` will ``` -+------+--------------+ -| dist | path | -+------+--------------+ -| 12.6 | (1)-[5]->(6) | -+------+--------------+ + dist path +0 12.6 [1, 5, 6] ``` +If a set matches several source and target nodes, only the single best pair is returned. + ## Limitations - Any variable on a pattern going into a `shortestPath` processor cannot be returned in the `RETURN` clause — they are consumed by the processor. The following query will produce an error: + Every variable on a pattern going into `shortestPath` is consumed by the statement, so it cannot be returned. Only `dist` and `path` can follow `RETURN`. The following query fails, and so does returning a separately matched copy of an endpoint: ```jsx MATCH (n{name:"Ashchurch"}), (m{name:"Worcestershire Parkway"}) @@ -156,24 +155,30 @@ You can pass multiple nodes to the source or target set — `shortestPath` will ``` ``` - PLAN_ERROR: Unexpected exception: projection variable n not found in output column + ANALYZE_ERROR: Cannot return n after SHORTESTPATH. ``` - - You can still `MATCH` additional nodes outside of the source and target sets and include them in `RETURN`. The result will be a cartesian product with the shortest path output: +- **Directed only.** Only outgoing edges are followed. When the target cannot be reached from the source, the query returns **zero rows**, not an error. +- **`RETURN` must follow directly.** `WITH` after `shortestPath` is a syntax error. `ORDER BY` and `LIMIT` on the `RETURN` are fine. +- **Do not call path functions on `path`.** `nodes(path)`, `length(path)` and comprehensions over it are not supported. +- **Where the other nodes come from.** A join can feed an endpoint: `MATCH (x {name: "Bromsgrove"})-[:LINK]->(a), (b {name: "Worcester Shrub Hill"}) shortestPath(a, b, distance, dist, path) RETURN dist, path`. - ```jsx - MATCH (n{name:"Ashchurch"}), (m{name:"Worcestershire Parkway"}), (p{name:"Cheltenham Spa"}) - shortestPath(n, m, distance, dist, path) - RETURN dist, path, p.name - ``` +## Naming the stations along the route - ``` - +-------+--------------+----------------+ - | dist | path | p.name | - +-------+--------------+----------------+ - | 11.29 | (0)-[1]->(6) | Cheltenham Spa | - +-------+--------------+----------------+ - ``` - \ No newline at end of file +To turn the IDs in `path` back into names, run a second query. The even positions of `path` are nodes: + +```python +row = client.query(""" +MATCH (a:Station {name: 'Bromsgrove'}), (b:Station {name: 'Worcester Shrub Hill'}) +shortestPath(a, b, distance, dist, path) +RETURN dist, path +""").iloc[0] + +node_ids = sorted({int(i) for i in row["path"][0::2]}) +names = client.query(f"UNWIND {node_ids} AS x MATCH (n) WHERE n = x RETURN id(n) AS id, n.name AS name") +``` + +The `UNWIND` list is a literal of distinct IDs and `x` is not returned, so this is a constant scan (see [lookup by node ID](/query/matching#look-up-nodes-by-native-node-id)). + +For large graphs prefer `shortestPath` over variable-length paths: variable-length matching enumerates every path. diff --git a/troubles/troubles.mdx b/troubles/troubles.mdx index 90c82f5..55e0aeb 100644 --- a/troubles/troubles.mdx +++ b/troubles/troubles.mdx @@ -27,6 +27,49 @@ turingdb -reset-default -turing-dir /path/to/your/turing/dir --- +## `Invalid response from the server` or `Proto header dataLen does not match chunk payload size` + +These two errors mean the client and the server disagree on the protocol. A TuringDB server speaks **either** JSON over HTTP **or** the binary protocol, chosen when it starts: + +| Error | Cause | Fix | +|---|---|---| +| `Invalid response from the server: Expecting value...` from `TuringDB(host=...)` | The server was started with `USE_TURING_PROTO=1` (binary mode) but you used the default JSON client | Connect with `TuringDB(type="native", host="localhost", port=6666)`, or restart the server without `USE_TURING_PROTO` | +| `Proto header dataLen does not match chunk payload size` from `TuringDB(type="native", ...)` | The server is a normal JSON server | Use the default `TuringDB(host="http://localhost:6666")`, or restart the server with `USE_TURING_PROTO=1` | + +See [client backends](/pythonsdk/reference#client-backends) for details. + +--- + +## `401 Unauthorized` + +The server was started with `-auth-on` and your client did not send the right token. Pass `token="..."` to `TuringDB(...)`, or set the `TURINGDB_AUTH_TOKEN` environment variable. The JSON client raises this as an `httpx.HTTPStatusError`. + +--- + +## `CHANGE_NOT_FOUND` after `CHANGE SUBMIT` + +After `CHANGE SUBMIT` the Python client still points at the submitted change. Call `client.checkout()` to return to main before running the next query. + +--- + +## `Cannot perform CREATE outside of a write transaction` + +All writes (`CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`, `CREATE INDEX`) must run inside a change. Call `client.new_change()` first, and submit with `CHANGE SUBMIT`. See [Writing data](/query/writing). + +--- + +## A `MATCH` after a `CREATE` finds nothing + +Entities created earlier in a change are not visible to later queries until you run `COMMIT`. Run `COMMIT` after creating nodes and before matching them to create edges or to `MERGE`. See [Writing data](/query/writing#when-you-need-commit-inside-a-change). + +--- + +## `Function 'X' does not exist` + +Function names are case-sensitive (`toInteger`, not `TOINTEGER`), and some functions are not available in TuringDB. `edgeType()` was replaced by `type()` in v3. See [Functions and data types](/query/functions). + +--- + ## Contact us If you have any feedback, bugs to report, or features requests please send us an email: [team@turingdb.ai](team@turingdb.ai) diff --git a/tutorials/create_graph.mdx b/tutorials/create_graph.mdx index e36246f..9250f9d 100644 --- a/tutorials/create_graph.mdx +++ b/tutorials/create_graph.mdx @@ -149,7 +149,7 @@ graph [ ## Key Points -1. **Single CREATE Query**: Due to TuringDB's current Cypher limitations, the entire graph must be created in a single CREATE statement. +1. **Single CREATE Query**: This tutorial builds the whole graph in one `CREATE` statement, which lets the edges refer to the nodes by variable. You can also create the nodes first, `COMMIT`, and then `MATCH` them to create the edges, see [Writing data](/query/writing). To import a GML file as-is, use [`LOAD GML`](/import_data/gml). 2. **Change Management**: Always use the change system when modifying data: - Create a new change with `CHANGE NEW` - Checkout into the change @@ -160,13 +160,13 @@ graph [ ## Troubleshooting -- **Large Graphs**: For very large GML files, you might hit query size limits. Consider splitting the import or contact support for guidance. +- **Large Graphs**: For very large GML files, you might hit query size limits. Consider [`LOAD GML`](/import_data/gml), or convert the graph to Parquet and use [`LOAD PARQUET`](/import_data/parquet), which loads hundreds of thousands of nodes and edges in well under a second. - **Invalid Characters**: The sanitization function handles common invalid characters, but you may need to extend it for specific use cases. - **Memory Usage**: Large graphs are loaded entirely into memory before conversion, so ensure you have sufficient RAM. ## Next Steps -After importing your GML file, you can query the data using TuringDB's supported Cypher subset: +After importing your GML file, you can query the data with [Cypher](/query/cypher_subset): ```python # Find all Person nodes diff --git a/vector-search.mdx b/vector-search.mdx index 304eb0b..48646ef 100644 --- a/vector-search.mdx +++ b/vector-search.mdx @@ -6,43 +6,54 @@ description: "Search through high-dimensional embeddings and connect the results TuringDB has a built-in vector index that lets you run k-nearest-neighbor searches over embedding vectors. You bring your own embeddings, from any model or provider, and TuringDB handles the indexing, storage, and fast retrieval. -Each vector is associated with a numerical ID. That ID can be a node property, an edge property, or a foreign key referencing data in an external system. This keeps the index lightweight and flexible: the vector store doesn't need to know what your data looks like. +Each vector is stored with a numerical ID. When that ID is the **native node ID** of the node the vector describes (`id(n)`), search results are the nodes themselves, so you can read their properties and expand from them with no extra lookup. - Vector indexes live at the TuringDB root level, independent of graphs and versioning. A single vector index can serve searches across multiple graphs and commits. + Vector indexes live at the TuringDB root level (`/vector/`), independent of graphs and versioning. A single vector index can serve searches across multiple graphs and commits, and it persists across restarts. + + Embedding literals use **parentheses**: `(1.2, 2.0, 0.0)`. Square brackets `[1.2, 2.0, 0.0]` make a list, which is a different type. An embedding literal needs at least 2 numbers. + + ## Create a vector index -A vector index is defined by a name, a dimension, and a distance metric. +A vector index is defined by a name, a dimension, a distance metric and, optionally, an index type. **Syntax:** ``` -CREATE VECTOR INDEX WITH DIMENSION METRIC +CREATE VECTOR INDEX WITH DIMENSION METRIC [TYPE ] ``` | Parameter | Description | | ---------- | --------------------------------------------------------------------------- | | `` | Identifier for the vector index | -| `` | Dimension of the embedding vectors (positive integer) | -| `` | Distance metric: `EUCLID` (Euclidean distance) or `COSINE` (cosine similarity) | +| `` | Dimension of the embedding vectors (positive integer, at most 8192) | +| `` | `COSINE` or `EUCLID` (uppercase) | +| `` | Optional. `FLAT` (exact, the default) or `HNSW` (approximate, for large collections) | ```jsx CREATE VECTOR INDEX doc_embeddings WITH DIMENSION 768 METRIC COSINE + CREATE VECTOR INDEX doc_hnsw WITH DIMENSION 768 METRIC EUCLID TYPE HNSW ``` ```python client.query("CREATE VECTOR INDEX doc_embeddings WITH DIMENSION 768 METRIC COSINE") + client.query("CREATE VECTOR INDEX doc_hnsw WITH DIMENSION 768 METRIC EUCLID TYPE HNSW") ``` + + The metrics return raw scores. `COSINE` scores are the **inner product** of the stored and query vectors, so L2-normalise both to get true cosine similarity. `EUCLID` scores are the **squared** L2 distance. + + ## Load embeddings -Once the index exists, load your pre-computed embeddings from a file. Each row in the file maps a numerical ID to a vector. +Once the index exists, load your pre-computed embeddings from a header-less CSV file. Each line is `id,v1,v2,...,vd`, where `id` is normally the native TuringDB node ID (`id(n)`). The file path is relative to your TuringDB data directory (`~/.turing/data` by default). @@ -73,36 +84,48 @@ LOAD VECTOR FROM "" IN +To build that file from your graph, export `id(n)` next to the text you want to embed, compute the embeddings in Python, and write `id,v1,...,vd` lines: + +```python +docs = client.query("MATCH (d:Document) RETURN id(d) AS id, d.text AS text") +vectors = model.encode(docs["text"].tolist()) # any embedding model + +with open("/home/me/.turing/data/document_vectors.csv", "w") as f: + for node_id, vec in zip(docs["id"], vectors): + f.write(f"{node_id}," + ",".join(f"{x:.6f}" for x in vec) + "\n") +``` + ## Search -`VECTOR SEARCH` finds the k nearest neighbors of a query vector and yields their IDs. It is a read statement, so you can chain it with `MATCH` to pull back the actual graph data. +`VECTOR SEARCH` finds the k nearest neighbors of a query vector and yields their IDs and scores. It is a read statement, so you can chain it with `MATCH`, `WHERE` and `RETURN` to pull back the actual graph data. **Syntax:** ``` -VECTOR SEARCH IN FOR () YIELD +VECTOR SEARCH IN FOR () YIELD ids [AS ] [, score [AS ]] ``` | Parameter | Description | | -------------- | ---------------------------------------------------------- | | `` | Name of the vector index to search | | `` | Number of nearest neighbors to return (positive integer) | -| `` | Query vector as a list literal of float values | -| `` | Variable name to hold the result IDs | +| `` | Query vector as an embedding literal, in parentheses. Its dimension must match the index | +| `ids` | The matched IDs, best first. Bound as nodes when the index is keyed by native node IDs | +| `score` | The score of each match | ### Standalone search ```jsx - VECTOR SEARCH IN doc_embeddings FOR 5 (0.12, 0.45, 0.78, 0.33) YIELD ids - RETURN ids + VECTOR SEARCH IN doc_embeddings FOR 5 (0.12, 0.45, 0.78, 0.33) YIELD ids, score + RETURN ids, score ``` ```python df = client.query(""" - VECTOR SEARCH IN doc_embeddings FOR 5 (0.12, 0.45, 0.78, 0.33) YIELD ids - RETURN ids + VECTOR SEARCH IN doc_embeddings FOR 5 (0.12, 0.45, 0.78, 0.33) YIELD ids, score + RETURN ids, score """) print(df) ``` @@ -111,29 +134,37 @@ VECTOR SEARCH IN FOR () YIELD ### Combining with MATCH -This is where it gets interesting. Chain `VECTOR SEARCH` with a `MATCH` clause to join the nearest-neighbor IDs back to your graph: +This is where it gets interesting. When the vector file is keyed by native node IDs, `ids` **is the node**. Read its properties directly, filter on `score`, or expand from it with a pattern: ```jsx - VECTOR SEARCH IN doc_embeddings FOR 10 (0.12, 0.45, 0.78, 0.33) YIELD ids - MATCH (d:Document) WHERE d.id = ids - RETURN d.title, d.summary + VECTOR SEARCH IN doc_embeddings FOR 10 (0.12, 0.45, 0.78, 0.33) YIELD ids AS doc, score + WHERE score > 0.8 + MATCH (doc)-[:ABOUT]->(t:Topic) + RETURN doc.title, t.name, score ``` ```python df = client.query(""" - VECTOR SEARCH IN doc_embeddings FOR 10 (0.12, 0.45, 0.78, 0.33) YIELD ids - MATCH (d:Document) WHERE d.id = ids - RETURN d.title, d.summary + VECTOR SEARCH IN doc_embeddings FOR 10 (0.12, 0.45, 0.78, 0.33) YIELD ids AS doc, score + WHERE score > 0.8 + MATCH (doc)-[:ABOUT]->(t:Topic) + RETURN doc.title, t.name, score """) print(df) ``` -The `ids` variable works exactly like a variable introduced by `CALL ... YIELD`, so any subsequent `MATCH` clause can reference it. + + **Use `ids` directly. Do not re-match it.** Writing `MATCH (d:Document) WHERE d.id = ids` (or `WHERE d = ids`) scans the whole label and cross-joins it with the results. Key the vector file by native node IDs, as above, so that no lookup is needed. If you must key the file by your own IDs, expect a label scan to join them back. + + + + Node IDs are renumbered by [`MERGE_DATAPARTS`](/concepts/dataparts). After compacting a graph, rebuild any vector index keyed by node IDs. + ## Manage indexes @@ -143,6 +174,8 @@ The `ids` variable works exactly like a variable introduced by `CALL ... YIELD`, SHOW VECTOR INDEXES ``` +This returns the name and dimension of each index. + ### Delete a vector index ```jsx @@ -159,13 +192,12 @@ This removes the index and frees the associated resources. // 1. Create a vector index for product embeddings CREATE VECTOR INDEX product_vectors WITH DIMENSION 384 METRIC COSINE - // 2. Load embeddings generated by your model + // 2. Load embeddings generated by your model (keyed by native node ID) LOAD VECTOR FROM "product_embeddings.csv" IN product_vectors // 3. Find the 10 most similar products to a query embedding - VECTOR SEARCH IN product_vectors FOR 10 (0.15, 0.82, 0.44, 0.91) YIELD ids - MATCH (p:Product) WHERE p.id = ids - RETURN p.name, p.price, p.category + VECTOR SEARCH IN product_vectors FOR 10 (0.15, 0.82, 0.44, 0.91) YIELD ids AS p, score + RETURN p.name, p.price, p.category, score // 4. Inspect existing indexes SHOW VECTOR INDEXES @@ -183,14 +215,13 @@ This removes the index and frees the associated resources. # 1. Create a vector index for product embeddings client.query("CREATE VECTOR INDEX product_vectors WITH DIMENSION 384 METRIC COSINE") - # 2. Load embeddings generated by your model + # 2. Load embeddings generated by your model (keyed by native node ID) client.query('LOAD VECTOR FROM "product_embeddings.csv" IN product_vectors') # 3. Find the 10 most similar products to a query embedding df = client.query(""" - VECTOR SEARCH IN product_vectors FOR 10 (0.15, 0.82, 0.44, 0.91) YIELD ids - MATCH (p:Product) WHERE p.id = ids - RETURN p.name, p.price, p.category + VECTOR SEARCH IN product_vectors FOR 10 (0.15, 0.82, 0.44, 0.91) YIELD ids AS p, score + RETURN p.name, p.price, p.category, score """) print(df) @@ -205,9 +236,7 @@ This removes the index and frees the associated resources. ## Embeddings as node properties -The vector index above is a standalone, root-level structure keyed by numeric IDs. Separately, you can store an embedding directly as a **node or edge property** (type `Embedding`) and compare embeddings inline with the `cosine_similarity` and `euclidean_distance` functions — no index required. - -Embedding literals use **parentheses** `(...)`, not square brackets (which are list literals): +The vector index above is a standalone, root-level structure keyed by numeric IDs. Separately, you can store an embedding directly as a **node or edge property** (type `Embedding`) and compare embeddings inline with the `cosine_similarity` and `euclidean_distance` functions, with no index required. ```jsx // Set an embedding property (inside a change) @@ -218,9 +247,11 @@ MATCH (n:Document) RETURN n.name, cosine_similarity(n.emb, (0.12, 0.45, 0.78, 0. MATCH (n:Document) RETURN n.name, euclidean_distance(n.emb, (0.12, 0.45, 0.78, 0.33)) ``` +`cosine_similarity` is true cosine similarity and `euclidean_distance` is true (not squared) L2 distance. Both accept Embeddings only, not lists, and work in `RETURN`, `WHERE` and `ORDER BY`. This is a brute-force scan, so for large collections use a vector index. + ### Bulk-loading embeddings from Parquet — `LOAD EMBEDDING FROM` -To attach embeddings to **existing** nodes in bulk, load them from a Parquet file. This is a write, so run it inside a change. +To attach embeddings to **existing** nodes in bulk, load them from a Parquet file. This is a write, so run it inside a change. Do not loop `SET n.emb = (...)` over many nodes: it is orders of magnitude slower (about 2.5 ms per node, against roughly 0.1 s for 20,000 embeddings of 384 dimensions with `LOAD EMBEDDING`). **Syntax:** ``` @@ -231,21 +262,36 @@ The Parquet file (relative to the TuringDB `data` directory) must have exactly t | Column | Type | Description | | ----------- | -------------------------------------- | -------------------------------------------------------- | -| `node_id` | `INT64` | Internal TuringDB node ID to attach the embedding to | +| `node_id` | `INT64` | Native TuringDB node ID (`id(n)`) to attach the embedding to | | `embedding` | `FIXED_LEN_BYTE_ARRAY` (`dim × 4` bytes) | Little-endian float32 vector; dimension inferred from width | ```python -change = client.new_change() -client.checkout(change=change) -client.query('LOAD EMBEDDING FROM "embeddings.parquet" AS emb') -client.query("COMMIT") +import numpy as np, pyarrow as pa, pyarrow.parquet as pq +from pathlib import Path + +docs = client.query("MATCH (d:Document) RETURN id(d) AS node_id, d.text AS text") +vecs = model.encode(docs["text"].tolist()).astype(" - When importing a graph from JSONL, you can mark embedding properties at load time instead — see [`LOAD JSONL ... WITH EMBEDDINGS`](/import_data/jsonl#loading-embedding-properties). + When importing a graph from JSONL, you can mark embedding properties at load time instead. See [`LOAD JSONL ... WITH EMBEDDINGS`](/import_data/jsonl#loading-embedding-properties). - From c5c96bc2feb953b82a32e522c6925a10572d638b Mon Sep 17 00:00:00 2001 From: dorukustay1 Date: Mon, 12 Oct 2026 16:26:02 +0100 Subject: [PATCH 2/5] Remove card grid from query language overview --- query/cypher_subset.mdx | 27 --------------------------- 1 file changed, 27 deletions(-) diff --git a/query/cypher_subset.mdx b/query/cypher_subset.mdx index d0e932d..f8fd8ef 100644 --- a/query/cypher_subset.mdx +++ b/query/cypher_subset.mdx @@ -8,33 +8,6 @@ TuringDB v3 (`turingdb` 3.0 on PyPI) runs every query on a new MLIR-based engine If you know Cypher from Neo4j, you can write the queries you are used to. This page covers the basics and the handful of places where TuringDB differs. The rest of the language is documented in the pages that follow. - - - `MATCH`, `OPTIONAL MATCH`, `WHERE`, joins and fast lookup by node ID - - - Variable-length paths, quantified path patterns and named paths - - - `WITH`, `ORDER BY`, `UNION`, aggregates, `UNWIND`, comprehensions, `CASE` - - - `EXISTS`, `COUNT` and `CALL` subqueries - - - Built-in functions, value types, dates and durations, maps - - - `CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`, changes and indexes - - - `CALL db.*`, graph and change commands, extensions - - - A compact reference of everything above - - - ## What is supported | Area | Supported | From 84221af1d53f70e624818ff16f044062a57fc0c2 Mon Sep 17 00:00:00 2001 From: dorukustay1 Date: Mon, 12 Oct 2026 18:23:24 +0100 Subject: [PATCH 3/5] Address review comments on the v3 docs - Reformat all query examples: one clause per line, continuation lines indented, blank lines between statements. - Aggregation: drop the WITH alias note and the list-literals point, explain filtering on an aggregate and mixing properties with aggregates (with examples), reword the parameters note. - Functions: positive "not yet available" wording, replace the arithmetic sentence with duration arithmetic examples. - Matching: remove the optional-node warning, reword the regex note. - Remove property index documentation for now. - Tutorial: point to LOAD PARQUET / LOAD JSONL instead of LOAD GML. --- claude-skill.mdx | 2 +- query/aggregation.mdx | 162 ++++++++++++++++----- query/cheatsheet.mdx | 288 +++++++++++++++++++++++++++---------- query/cypher_subset.mdx | 4 +- query/functions.mdx | 73 ++++++++-- query/matching.mdx | 160 ++++++++++++++++----- query/paths.mdx | 40 ++++-- query/procedures.mdx | 46 ++++-- query/subqueries.mdx | 69 +++++++-- query/writing.mdx | 169 ++++++++++++++-------- troubles/troubles.mdx | 2 +- tutorials/create_graph.mdx | 4 +- 12 files changed, 756 insertions(+), 263 deletions(-) diff --git a/claude-skill.mdx b/claude-skill.mdx index e76243f..a41c7fe 100644 --- a/claude-skill.mdx +++ b/claude-skill.mdx @@ -43,7 +43,7 @@ The skill is structured so Claude only loads the reference it needs: | --- | --- | | `startup` | Installation, starting the server, connecting (HTTP, authentication, embedded), loading a graph | | `querying` | `MATCH`, `OPTIONAL MATCH`, paths, `WITH`, aggregation, `UNWIND`, subqueries, built-in functions, `LOAD CSV` | -| `writing` | `CREATE`, `MERGE`, `SET`, `DELETE`, the change and commit workflow, indexes | +| `writing` | `CREATE`, `MERGE`, `SET`, `DELETE`, the change and commit workflow | | `importing` | CSV, Parquet, JSONL, GML and Neo4j imports | | `algorithms` | Shortest path, vector search, embeddings, GNN sampling | | `introspection` | Schema exploration, versioning and time travel, Python SDK reference | diff --git a/query/aggregation.mdx b/query/aggregation.mdx index 9e26a8d..bf73fe6 100644 --- a/query/aggregation.mdx +++ b/query/aggregation.mdx @@ -9,10 +9,18 @@ These examples use the [sample graph](/query/matching) from the matching page. ## RETURN, DISTINCT and expressions ```cypher -MATCH (n:Person) RETURN n.name, n.age + 5 AS adjusted -MATCH (n:Person) RETURN DISTINCT n.city -MATCH (n:Person) RETURN n.name AS nm ORDER BY size(nm) DESC, nm -MATCH (n:Person) RETURN * +MATCH (n:Person) +RETURN n.name, n.age + 5 AS adjusted + +MATCH (n:Person) +RETURN DISTINCT n.city + +MATCH (n:Person) +RETURN n.name AS nm +ORDER BY size(nm) DESC, nm + +MATCH (n:Person) +RETURN * ``` `RETURN *` returns every variable in scope (nodes as IDs). A column without an alias is named after its expression, for example `n.name`. @@ -20,10 +28,21 @@ MATCH (n:Person) RETURN * ## ORDER BY, SKIP and LIMIT ```cypher -MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age -MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age DESC -MATCH (n:Person) RETURN n.name, n.city, n.age ORDER BY n.city, n.age DESC -MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age DESC SKIP 1 LIMIT 2 +MATCH (n:Person) +RETURN n.name, n.age +ORDER BY n.age + +MATCH (n:Person) +RETURN n.name, n.age +ORDER BY n.age DESC + +MATCH (n:Person) +RETURN n.name, n.city, n.age +ORDER BY n.city, n.age DESC + +MATCH (n:Person) +RETURN n.name, n.age +ORDER BY n.age DESC SKIP 1 LIMIT 2 ``` - You can sort by several keys, by aliases and by expressions. Ascending is the default. @@ -37,21 +56,43 @@ MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age DESC SKIP 1 LIMIT 2 `WITH` pipes the result of one part of a query into the next. It is how you filter on aggregates and chain several stages: ```cypher -MATCH (n:Person) WITH n WHERE n.age > 28 RETURN n.name -MATCH (n:Person) WITH n.name AS name, n.age AS age ORDER BY age DESC LIMIT 2 RETURN name, age -MATCH (n:Person) WITH n WHERE n.age > 26 WITH n WHERE n.age < 36 RETURN n.name -MATCH (n:Person) WITH n.city AS city, count(*) AS c WHERE c > 1 RETURN city, c -``` +MATCH (n:Person) +WITH n +WHERE n.age > 28 +RETURN n.name - - Expressions in a `WITH` must be aliased: `WITH n.name AS name`. `RETURN` does not need aliases. - +MATCH (n:Person) +WITH n.name AS name, n.age AS age +ORDER BY age DESC LIMIT 2 +RETURN name, age + +MATCH (n:Person) +WITH n +WHERE n.age > 26 +WITH n +WHERE n.age < 36 +RETURN n.name + +MATCH (n:Person) +WITH n.city AS city, count(*) AS c +WHERE c > 1 +RETURN city, c +``` ## UNION ```cypher -MATCH (p:Person) RETURN p.name AS name UNION MATCH (c:Company) RETURN c.name AS name -MATCH (p:Person) RETURN p.name AS name UNION ALL MATCH (c:Company) RETURN c.name AS name +MATCH (p:Person) +RETURN p.name AS name +UNION +MATCH (c:Company) +RETURN c.name AS name + +MATCH (p:Person) +RETURN p.name AS name +UNION ALL +MATCH (c:Company) +RETURN c.name AS name ``` `UNION` removes duplicates and `UNION ALL` keeps them. All branches must return the same column names, and each column must keep one value type across the branches. @@ -67,13 +108,28 @@ ORDER BY employees DESC ``` ```cypher -MATCH (n:Person) RETURN count(*), count(n.age), sum(n.age), avg(n.age), min(n.age), max(n.age) -MATCH (a)-[w:WORKS_AT]->(c) RETURN w.role, count(DISTINCT c) AS companies -MATCH (n:Person) RETURN n.city, max(n.age) - min(n.age) AS spread -MATCH (n:Person) RETURN sum(CASE WHEN n.age > 30 THEN 1 ELSE 0 END) AS seniors -MATCH (n:Person) RETURN collect(n.name)[0..3] AS firstThree -MATCH ()-[r]->() RETURN type(r) AS t, count(*) AS c ORDER BY c DESC -MATCH (n) RETURN labels(n) AS l, count(*) AS c ORDER BY c DESC +MATCH (n:Person) +RETURN count(*), count(n.age), sum(n.age), avg(n.age), min(n.age), max(n.age) + +MATCH (a)-[w:WORKS_AT]->(c) +RETURN w.role, count(DISTINCT c) AS companies + +MATCH (n:Person) +RETURN n.city, max(n.age) - min(n.age) AS spread + +MATCH (n:Person) +RETURN sum(CASE WHEN n.age > 30 THEN 1 ELSE 0 END) AS seniors + +MATCH (n:Person) +RETURN collect(n.name)[0..3] AS firstThree + +MATCH ()-[r]->() +RETURN type(r) AS t, count(*) AS c +ORDER BY c DESC + +MATCH (n) +RETURN labels(n) AS l, count(*) AS c +ORDER BY c DESC ``` | Aggregate | Notes | @@ -88,39 +144,65 @@ There are no statistical aggregates such as `stdev` or `percentileCont`. Compute **Rules:** -- Aggregates are **not allowed in `WHERE`**. Filter after a `WITH` instead (see above). +- `WHERE` runs before aggregation, so to filter on an aggregate, compute it in a `WITH` first and filter on its alias (see the example below). - Aggregates cannot be nested: `count(count(*))` is an error. -- A non-aggregated expression next to an aggregate may only use the grouping keys. +- Aggregates can be combined with each other and with constants in one expression (`max(n.age) - min(n.age)`, `count(*) * 10`). To use a property in the same expression, return it as its own item so it is a grouping key: `RETURN n.age, n.age + count(*)` works. - With no grouping keys, an empty input still returns one row: `count` is `0`, `sum` is `0`, `avg` is `null` and `collect` is `[]`. With grouping keys, an empty input returns no rows. +### Filtering on an aggregate + +```cypher +MATCH (n:Person) +WITH n.city AS city, count(*) AS people +WHERE people > 1 +RETURN city, people +``` + ## UNWIND `UNWIND` expands a list into one row per element. It accepts any list expression, including lists you computed earlier in the query: ```cypher -UNWIND [1, 2, 3] AS x RETURN x * 10 -UNWIND range(1, 10, 3) AS x RETURN x -MATCH (n:Person) WITH collect(n.name) AS names UNWIND names AS nm RETURN nm -MATCH (n:Person) WITH collect(n) AS ns UNWIND ns AS p RETURN p.name -MATCH (n:Person) UNWIND n.tags AS t RETURN t, count(*) +UNWIND [1, 2, 3] AS x +RETURN x * 10 + +UNWIND range(1, 10, 3) AS x +RETURN x + +MATCH (n:Person) +WITH collect(n.name) AS names +UNWIND names AS nm +RETURN nm + +MATCH (n:Person) +WITH collect(n) AS ns +UNWIND ns AS p +RETURN p.name + +MATCH (n:Person) +UNWIND n.tags AS t +RETURN t, count(*) ``` `range(start, end[, step])` includes both bounds. Unwound nodes stay nodes, so `p.name` works after `UNWIND ns AS p`. You can also `UNWIND` a list of maps and read their keys: `UNWIND [{a: 1}, {a: 2}] AS m RETURN m.a`. - Query parameters (`UNWIND $list`) are not supported. Write the list into the query text instead, for example `f"UNWIND {ids} AS x ..."` in Python. + Query parameters (`UNWIND $list`) are not yet supported. Write the list into the query text instead, for example `f"UNWIND {ids} AS x ..."` in Python. ## Lists ```cypher RETURN [1, 2, 3, 4, 5][0], [1, 2, 3, 4, 5][-1], [1, 2, 3, 4, 5][1..3], [1, 2, 3][10] + RETURN size([1, 2]), head([1, 2]), last([1, 2]), tail([1, 2]), reverse([1, 2, 3]) -MATCH (n:Person {name: 'Alice'}) RETURN n.tags[0], size(n.tags) + +MATCH (n:Person {name: 'Alice'}) +RETURN n.tags[0], size(n.tags) + RETURN [1, 2] + [3], 'a' || 'b' ``` -- List literals can contain expressions, mixed types, nested lists and maps. - Indexing and slicing work on literals, list properties and function results (`labels(n)[0]`). Negative indexes count from the end, and an out-of-range index returns `null`. - Lists and maps compare by value with `=`. - `size(null)` and `head([])` return `null`. @@ -129,8 +211,14 @@ RETURN [1, 2] + [3], 'a' || 'b' ```cypher RETURN [x IN range(1, 6) WHERE x % 2 = 0 | x * x] -MATCH (a:Person) RETURN a.name, [(a)-[:KNOWS]->(b) | b.name] AS friends -MATCH (n:Person) WHERE any(t IN n.tags WHERE t = 'vip') RETURN n.name + +MATCH (a:Person) +RETURN a.name, [(a)-[:KNOWS]->(b) | b.name] AS friends + +MATCH (n:Person) +WHERE any(t IN n.tags WHERE t = 'vip') +RETURN n.name + RETURN reduce(total = 0, x IN [1, 2, 3] | total + x) ``` diff --git a/query/cheatsheet.mdx b/query/cheatsheet.mdx index 139b487..c45906e 100644 --- a/query/cheatsheet.mdx +++ b/query/cheatsheet.mdx @@ -11,14 +11,30 @@ Each section links to the page that explains it in full. [Matching patterns](/query/matching) ```cypher -MATCH (n) RETURN n -MATCH (n:Person {name: 'Alice'}) RETURN n.age -MATCH (a:Person)-->(b:Company) RETURN a.name, b.name -MATCH (a:Person)-[e:KNOWS {since: 2010}]->(b:Person) RETURN a.name, e.since, b.name -MATCH (a)-[r:KNOWS|WORKS_AT]->(b) RETURN a.name, type(r), b.name -MATCH (a:Person)-[e]-(b) RETURN a.name, b.name -MATCH (a:Person WHERE a.age > 30)-[r:KNOWS WHERE r.since > 2012]->(b) RETURN b.name -MATCH (n:Person) ORDER BY n.age LIMIT 2 RETURN n.name +MATCH (n) +RETURN n + +MATCH (n:Person {name: 'Alice'}) +RETURN n.age + +MATCH (a:Person)-->(b:Company) +RETURN a.name, b.name + +MATCH (a:Person)-[e:KNOWS {since: 2010}]->(b:Person) +RETURN a.name, e.since, b.name + +MATCH (a)-[r:KNOWS|WORKS_AT]->(b) +RETURN a.name, type(r), b.name + +MATCH (a:Person)-[e]-(b) +RETURN a.name, b.name + +MATCH (a:Person WHERE a.age > 30)-[r:KNOWS WHERE r.since > 2012]->(b) +RETURN b.name + +MATCH (n:Person) +ORDER BY n.age LIMIT 2 +RETURN n.name ``` - `()` is a node, `[]` is an edge, `-` ignores direction. @@ -30,21 +46,50 @@ MATCH (n:Person) ORDER BY n.age LIMIT 2 RETURN n.name ## OPTIONAL MATCH ```cypher -MATCH (a:Person) OPTIONAL MATCH (a)-[:WORKS_AT]->(c:Company) RETURN a.name, c.name -MATCH (a:Person) OPTIONAL MATCH (a)-[:KNOWS]->(b) RETURN a.name, count(b) +MATCH (a:Person) +OPTIONAL MATCH (a)-[:WORKS_AT]->(c:Company) +RETURN a.name, c.name + +MATCH (a:Person) +OPTIONAL MATCH (a)-[:KNOWS]->(b) +RETURN a.name, count(b) ``` ## WHERE: filter ```cypher -MATCH (n) WHERE n:Person RETURN n.name -MATCH (n:Person) WHERE n.age >= 18 AND n.city = 'London' RETURN n.name -MATCH (n:Person) WHERE n.name STARTS WITH 'A' OR n.name CONTAINS 'ro' RETURN n.name -MATCH (n:Person) WHERE n.name IN ['Alice', 'Bob'] RETURN n.name -MATCH (n:Person) WHERE 'admin' IN n.tags RETURN n.name -MATCH (n:Person) WHERE n.tags IS NOT NULL RETURN n.name -MATCH (n)-[e]->(m) WHERE e:KNOWS AND e.since > 2012 RETURN n.name, m.name -MATCH (p:Person) WHERE NOT (p)-[:KNOWS]->() RETURN p.name +MATCH (n) +WHERE n:Person +RETURN n.name + +MATCH (n:Person) +WHERE n.age >= 18 AND n.city = 'London' +RETURN n.name + +MATCH (n:Person) +WHERE n.name STARTS +WITH 'A' OR n.name CONTAINS 'ro' +RETURN n.name + +MATCH (n:Person) +WHERE n.name IN ['Alice', 'Bob'] +RETURN n.name + +MATCH (n:Person) +WHERE 'admin' IN n.tags +RETURN n.name + +MATCH (n:Person) +WHERE n.tags IS NOT NULL +RETURN n.name + +MATCH (n)-[e]->(m) +WHERE e:KNOWS AND e.since > 2012 +RETURN n.name, m.name + +MATCH (p:Person) +WHERE NOT (p)-[:KNOWS]->() +RETURN p.name ``` - Comparison: `= <> != < <= > >=`, null tests `IS NULL` / `IS NOT NULL`, boolean `AND OR XOR NOT`. @@ -57,10 +102,22 @@ MATCH (p:Person) WHERE NOT (p)-[:KNOWS]->() RETURN p.name [Details](/query/matching#look-up-nodes-by-native-node-id) ```cypher -MATCH (n) WHERE n = 0 RETURN n.name -MATCH (n)-[:KNOWS]->(m) WHERE n = 0 RETURN m.name -UNWIND [0, 1, 2] AS x MATCH (n) WHERE n = x RETURN id(n), n.name -EXPLAIN MATCH (n) WHERE n = 0 RETURN n.name +MATCH (n) +WHERE n = 0 +RETURN n.name + +MATCH (n)-[:KNOWS]->(m) +WHERE n = 0 +RETURN m.name + +UNWIND [0, 1, 2] AS x +MATCH (n) +WHERE n = x +RETURN id(n), n.name + +EXPLAIN MATCH (n) +WHERE n = 0 +RETURN n.name ``` - Compare the **variable** (`n = 0`), not `id(n) = 0`. The first is a direct seek and the second scans every node. @@ -69,10 +126,19 @@ EXPLAIN MATCH (n) WHERE n = 0 RETURN n.name ## Joins and Cartesian products ```cypher -MATCH (p:Person), (c:Company) RETURN p.name, c.name -MATCH (a:Person)-[:KNOWS]->(b), (b)-[:WORKS_AT]->(c) RETURN a.name, b.name, c.name -MATCH (a:Person)-->(i:Interest)<--(b:Person) WHERE a <> b RETURN a.name, b.name, i.name -MATCH (a:Person), (b:Person) WHERE a.city = b.city AND a <> b RETURN a.name, b.name +MATCH (p:Person), (c:Company) +RETURN p.name, c.name + +MATCH (a:Person)-[:KNOWS]->(b), (b)-[:WORKS_AT]->(c) +RETURN a.name, b.name, c.name + +MATCH (a:Person)-->(i:Interest)<--(b:Person) +WHERE a <> b +RETURN a.name, b.name, i.name + +MATCH (a:Person), (b:Person) +WHERE a.city = b.city AND a <> b +RETURN a.name, b.name ``` - A comma separates independent patterns. Shared variables create joins, unrelated patterns create Cartesian products (N × M rows). @@ -82,12 +148,23 @@ MATCH (a:Person), (b:Person) WHERE a.city = b.city AND a <> b RETURN a.name, b.n [Paths](/query/paths) ```cypher -MATCH (a:Person {name: 'Alice'})-[:KNOWS]->+(b) RETURN DISTINCT b.name -MATCH (a:Person {name: 'Alice'})-[:KNOWS]->*(b) RETURN DISTINCT b.name -MATCH (a:Person {name: 'Alice'})-[:KNOWS]->{1,2}(b) RETURN DISTINCT b.name -MATCH (a:Person {name: 'Alice'})-[:KNOWS*1..3]->(b) RETURN DISTINCT b.name -MATCH (a:Person {name: 'Alice'}) ((x)-[:KNOWS]->(y) WHERE y.age > x.age)+ (b) RETURN b.name -MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) RETURN [n IN nodes(p) | n.name], length(p) +MATCH (a:Person {name: 'Alice'})-[:KNOWS]->+(b) +RETURN DISTINCT b.name + +MATCH (a:Person {name: 'Alice'})-[:KNOWS]->*(b) +RETURN DISTINCT b.name + +MATCH (a:Person {name: 'Alice'})-[:KNOWS]->{1,2}(b) +RETURN DISTINCT b.name + +MATCH (a:Person {name: 'Alice'})-[:KNOWS*1..3]->(b) +RETURN DISTINCT b.name + +MATCH (a:Person {name: 'Alice'}) ((x)-[:KNOWS]->(y) WHERE y.age > x.age)+ (b) +RETURN b.name + +MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) +RETURN [n IN nodes(p) | n.name], length(p) ``` - Postfix `->*` is **0** or more hops. Bracket `-[:KNOWS*]->` is **1** or more. @@ -98,10 +175,23 @@ MATCH p = (a:Person {name: 'Alice'})-[:KNOWS]->+(b) RETURN [n IN nodes(p) | n.na [Results and aggregation](/query/aggregation) ```cypher -MATCH (n:Person) WITH n.city AS city, count(*) AS c WHERE c > 1 RETURN city, c -MATCH (n:Person) RETURN DISTINCT n.city -MATCH (n:Person) RETURN n.name, n.age ORDER BY n.age DESC SKIP 1 LIMIT 2 -MATCH (p:Person) RETURN p.name AS name UNION MATCH (c:Company) RETURN c.name AS name +MATCH (n:Person) +WITH n.city AS city, count(*) AS c +WHERE c > 1 +RETURN city, c + +MATCH (n:Person) +RETURN DISTINCT n.city + +MATCH (n:Person) +RETURN n.name, n.age +ORDER BY n.age DESC SKIP 1 LIMIT 2 + +MATCH (p:Person) +RETURN p.name AS name +UNION +MATCH (c:Company) +RETURN c.name AS name ``` - Expressions in `WITH` must be aliased. @@ -110,10 +200,18 @@ MATCH (p:Person) RETURN p.name AS name UNION MATCH (c:Company) RETURN c.name AS ## Aggregation ```cypher -MATCH (n:Person) RETURN count(*), count(n.age), sum(n.age), avg(n.age), min(n.age), max(n.age) -MATCH (a:Person)-[:WORKS_AT]->(c:Company) RETURN c.name, count(a) AS employees, collect(a.name) AS who -MATCH (n:Person) RETURN count(DISTINCT n.city) -MATCH ()-[r]->() RETURN type(r) AS t, count(*) AS c ORDER BY c DESC +MATCH (n:Person) +RETURN count(*), count(n.age), sum(n.age), avg(n.age), min(n.age), max(n.age) + +MATCH (a:Person)-[:WORKS_AT]->(c:Company) +RETURN c.name, count(a) AS employees, collect(a.name) AS who + +MATCH (n:Person) +RETURN count(DISTINCT n.city) + +MATCH ()-[r]->() +RETURN type(r) AS t, count(*) AS c +ORDER BY c DESC ``` - Grouping is implicit. Aggregates are not allowed in `WHERE`: filter after a `WITH`. @@ -122,14 +220,31 @@ MATCH ()-[r]->() RETURN type(r) AS t, count(*) AS c ORDER BY c DESC ## UNWIND, lists, CASE ```cypher -UNWIND [1, 2, 3] AS x RETURN x * 10 -MATCH (n:Person) WITH collect(n.name) AS names UNWIND names AS nm RETURN nm -MATCH (n:Person) UNWIND n.tags AS t RETURN t, count(*) +UNWIND [1, 2, 3] AS x +RETURN x * 10 + +MATCH (n:Person) +WITH collect(n.name) AS names +UNWIND names AS nm +RETURN nm + +MATCH (n:Person) +UNWIND n.tags AS t +RETURN t, count(*) + RETURN [x IN range(1, 6) WHERE x % 2 = 0 | x * x] -MATCH (a:Person) RETURN a.name, [(a)-[:KNOWS]->(b) | b.name] AS friends -MATCH (n:Person) WHERE any(t IN n.tags WHERE t = 'vip') RETURN n.name + +MATCH (a:Person) +RETURN a.name, [(a)-[:KNOWS]->(b) | b.name] AS friends + +MATCH (n:Person) +WHERE any(t IN n.tags WHERE t = 'vip') +RETURN n.name + RETURN reduce(total = 0, x IN [1, 2, 3] | total + x) -MATCH (n:Person) RETURN n.name, CASE WHEN n.age >= 35 THEN 'senior' ELSE 'junior' END AS band + +MATCH (n:Person) +RETURN n.name, CASE WHEN n.age >= 35 THEN 'senior' ELSE 'junior' END AS band ``` ## Subqueries @@ -137,10 +252,26 @@ MATCH (n:Person) RETURN n.name, CASE WHEN n.age >= 35 THEN 'senior' ELSE 'junior [Subqueries](/query/subqueries) ```cypher -MATCH (p:Person) WHERE EXISTS { (p)-[:WORKS_AT]->(:Company {name: 'Acme'}) } RETURN p.name -MATCH (p:Person) RETURN p.name, COUNT { (p)-[:KNOWS]->() } AS outDegree -MATCH (p:Person) CALL (p) { MATCH (p)-[:KNOWS]->(f) RETURN f.name AS friend LIMIT 1 } RETURN p.name, friend -MATCH (p:Person) OPTIONAL CALL (p) { MATCH (p)-[:WORKS_AT]->(c) RETURN c.name AS co } RETURN p.name, co +MATCH (p:Person) +WHERE EXISTS { (p)-[:WORKS_AT]->(:Company {name: 'Acme'}) } +RETURN p.name + +MATCH (p:Person) +RETURN p.name, COUNT { (p)-[:KNOWS]->() } AS outDegree + +MATCH (p:Person) +CALL (p) { + MATCH (p)-[:KNOWS]->(f) + RETURN f.name AS friend LIMIT 1 +} +RETURN p.name, friend + +MATCH (p:Person) +OPTIONAL CALL (p) { + MATCH (p)-[:WORKS_AT]->(c) + RETURN c.name AS co +} +RETURN p.name, co ``` ## Functions @@ -159,9 +290,13 @@ MATCH (p:Person) OPTIONAL CALL (p) { MATCH (p)-[:WORKS_AT]->(c) RETURN c.name AS | Aggregate | `count`, `collect`, `sum`, `avg`, `min`, `max` | ```cypher -MATCH (n)-[e]->(m) RETURN labels(n), type(e), labels(m) +MATCH (n)-[e]->(m) +RETURN labels(n), type(e), labels(m) + RETURN datetime('2024-03-15T10:30:00Z') + duration({days: 1}) -MATCH (n:Document) RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) + +MATCH (n:Document) +RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) ``` - Function names are case-sensitive, except `count` and `collect`. @@ -187,15 +322,32 @@ MATCH (n:Document) RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1) ```cypher CHANGE NEW + CREATE (:Person {name: 'Alice', age: 30}) + CREATE (:Person {name: 'Mick'})-[:FRIEND_OF {since: 2020}]->(:Person {name: 'John'}) + COMMIT -MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Mick'}) CREATE (a)-[:KNOWS]->(b) -MERGE (n:Person {name: 'Alice'}) ON CREATE SET n.created = true ON MATCH SET n.seen = true -MATCH (n:Person {name: 'Alice'}) SET n.age = 31, n.position = 'Developer' -MATCH (n:Person {name: 'Alice'}) REMOVE n.position -MATCH (n:Person {name: 'Alice'}) DETACH DELETE n -MATCH (n:Person)-[e:KNOWS]->(m) DELETE e + +MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Mick'}) +CREATE (a)-[:KNOWS]->(b) + +MERGE (n:Person {name: 'Alice'}) + ON CREATE SET n.created = true + ON MATCH SET n.seen = true + +MATCH (n:Person {name: 'Alice'}) +SET n.age = 31, n.position = 'Developer' + +MATCH (n:Person {name: 'Alice'}) +REMOVE n.position + +MATCH (n:Person {name: 'Alice'}) +DETACH DELETE n + +MATCH (n:Person)-[e:KNOWS]->(m) +DELETE e + CHANGE SUBMIT ``` @@ -215,7 +367,6 @@ CHANGE SUBMIT | `CALL db.propertyTypes()` | All property keys and their types | | `CALL db.history()` | Commit history | | `CALL db.describeCommit('')` | Counts for a single commit | -| `CALL db.showIndexes()` | Property indexes | | `CALL db.getNodes([0, 1])` | Labels and all properties of specific nodes | | `CALL db.procedures()` | All procedures and signatures | @@ -224,22 +375,15 @@ Outputs can be chained: `CALL db.labels() YIELD label MATCH (n) WHERE label IN l ## Graph algorithms and search ```cypher -MATCH (a:Station {name: 'Ashchurch'}), (b:Station {name: 'Pershore'}) shortestPath(a, b, distance, dist, path) RETURN dist, path -VECTOR SEARCH IN my_index FOR 5 (1.2, 0.5, 3.0, 0.1) YIELD ids, score RETURN ids, score -``` - -See [Shortest path](/shortest-path) and [Vector search](/vector-search). +MATCH (a:Station {name: 'Ashchurch'}), (b:Station {name: 'Pershore'}) +shortestPath(a, b, distance, dist, path) +RETURN dist, path -## Property indexes - -```cypher -CHANGE NEW -CREATE INDEX age_index FOR (n) ON n.age -CREATE INDEX since_index FOR [e] ON e.since -CHANGE SUBMIT +VECTOR SEARCH IN my_index FOR 5 (1.2, 0.5, 3.0, 0.1) YIELD ids, score +RETURN ids, score ``` -`DROP INDEX ` is also a write, so run it inside a change too. +See [Shortest path](/shortest-path) and [Vector search](/vector-search). ## Engine commands diff --git a/query/cypher_subset.mdx b/query/cypher_subset.mdx index f8fd8ef..86165f1 100644 --- a/query/cypher_subset.mdx +++ b/query/cypher_subset.mdx @@ -17,7 +17,7 @@ If you know Cypher from Neo4j, you can write the queries you are used to. This p | Expressions | Arithmetic, string and list concatenation, `CASE`, list and pattern comprehensions, `reduce`, `all` / `any` / `none` / `single`, list slicing, maps | | Aggregation | `count`, `sum`, `avg`, `min`, `max`, `collect`, `DISTINCT`, implicit grouping, aggregates inside expressions | | Subqueries | `EXISTS { }`, `COUNT { }`, `CALL { }` and `OPTIONAL CALL { }` | -| Writing | `CREATE`, `MERGE` with `ON CREATE` / `ON MATCH`, `SET`, `REMOVE`, `DELETE`, `DETACH DELETE`, property indexes | +| Writing | `CREATE`, `MERGE` with `ON CREATE` / `ON MATCH`, `SET`, `REMOVE`, `DELETE`, `DETACH DELETE` | | Types | String, Integer, Double, Boolean, List, Map, `DateTime`, `Duration`, Embedding | | TuringDB extras | Changes, commits and time travel, `shortestPath`, `VECTOR SEARCH`, `gnn.graphSAGE`, lookup by native node ID | @@ -61,7 +61,7 @@ df = client.query("MATCH (n:Person) RETURN n.name, n.age") # a pandas DataFram ``` - `EXPLAIN ` returns the compiled query plan without running the query. It is the easiest way to check that a lookup uses an index or a constant node scan. + `EXPLAIN ` returns the compiled query plan without running the query. It is the easiest way to check that a lookup uses a constant node scan. ## What to know if you come from Neo4j diff --git a/query/functions.mdx b/query/functions.mdx index 873e6af..834df71 100644 --- a/query/functions.mdx +++ b/query/functions.mdx @@ -22,29 +22,47 @@ Function names are **case-sensitive** (`toInteger`, not `TOINTEGER`), except `co | Aggregate | `count`, `collect`, `sum`, `avg`, `min`, `max` | See [Aggregation](/query/aggregation#aggregation) | ```cypher -MATCH (n)-[e]->(m) RETURN labels(n), type(e), labels(m) -MATCH (n:Person) RETURN toUpper(n.name), toString(n.age) -MATCH (n:Person) WHERE n.age > toInteger('29') RETURN n.name -MATCH (n:Person) RETURN n.name, n.age * toFloat('1.07') +MATCH (n)-[e]->(m) +RETURN labels(n), type(e), labels(m) + +MATCH (n:Person) +RETURN toUpper(n.name), toString(n.age) + +MATCH (n:Person) +WHERE n.age > toInteger('29') +RETURN n.name + +MATCH (n:Person) +RETURN n.name, n.age * toFloat('1.07') + RETURN toBoolean('true') + RETURN abs(-3), round(2.5), sqrt(16.0), pi() -MATCH (n:Person) RETURN n.name, coalesce(n.nickname, n.name) + +MATCH (n:Person) +RETURN n.name, coalesce(n.nickname, n.name) ``` **`edgeType()` was removed in v3.** Use `type(e)` to get an edge's type as a string. -Calling a function that does not exist fails with `Function 'X' does not exist`. Functions that are **not** available include `substring`, `split`, `replace`, `left`, `right`, `rand`, `date()`, `timestamp()`, `properties`, `keys` and `elementId`. Do that kind of post-processing in your application, and filter strings with `STARTS WITH`, `ENDS WITH` and `CONTAINS`. +The function library keeps growing. Some functions, such as `substring`, `split`, `rand()`, `keys` and `properties`, are not yet available. For now, filter strings with `STARTS WITH`, `ENDS WITH` and `CONTAINS`, and do any other post-processing in your application. ## Arithmetic in `RETURN` and `SET` Arithmetic expressions work anywhere an expression is allowed: ```cypher -MATCH (n:Person) RETURN n.age * 12 -MATCH (n:Person) RETURN n.name, n.age + 5 AS adjusted, n.age / 2 -MATCH ()-[r]->() WHERE r.since > 0 RETURN r.since / 10 * 100 +MATCH (n:Person) +RETURN n.age * 12 + +MATCH (n:Person) +RETURN n.name, n.age + 5 AS adjusted, n.age / 2 + +MATCH ()-[r]->() +WHERE r.since > 0 +RETURN r.since / 10 * 100 ``` `+ - * / % ^` are supported. Integer `/` truncates, and `^` returns a Double. `+` and `||` concatenate strings and lists, and `'x' + 1` gives `'x1'`. @@ -55,17 +73,36 @@ MATCH ()-[r]->() WHERE r.since > 0 RETURN r.since / 10 * 100 ```cypher RETURN datetime('2024-03-15T10:30:00Z') AS d + RETURN datetime('2024-03-15') AS midnight + RETURN datetime(1700000000) AS fromEpoch + RETURN datetime('2024-03-15T10:30:00Z') + duration({days: 1, hours: 2}) AS later + RETURN datetime('2024-03-16T00:00:00Z') - datetime('2024-03-15T00:00:00Z') AS elapsed + RETURN datetime('2024-03-15T10:30:00Z').year, datetime('2024-03-15T10:30:00Z').month + RETURN duration({days: 2, hours: 3}).hours ``` +Durations can be added, subtracted, scaled and compared with each other and with datetimes: + +```cypher +RETURN duration({hours: 36}) + duration({hours: 12}) AS total + +RETURN duration({days: 2, hours: 12}) - duration({days: 1}) AS remaining + +RETURN duration({days: 2}) * 3 AS threeTimes + +RETURN duration({hours: 36}) / 2 AS half + +RETURN duration({hours: 1}) > duration({minutes: 30}) AS longer +``` + - `datetime` values are stored as microseconds since the Unix epoch, **normalised to UTC**. A date-only string means midnight, and an offset (`+02:00`) is converted to the equivalent UTC instant. - `duration` is a signed count of microseconds. It has no month component, because a month has no fixed length. `duration()` reads the number as microseconds. -- Arithmetic works: `datetime ± duration`, `datetime − datetime`, `duration × number`. - Components are available as properties: `d.year`, `d.month`, `d.day`, `d.hour`, and so on. On a duration, `dur.days`, `dur.hours`, ... give the **total** in that unit, so `duration({days: 2, hours: 3}).hours` is `51`. - `<`, `>`, `min` and `max` work on temporal values. - When you [import JSONL](/import_data/jsonl) you can declare which properties are `DateTime` or `Duration` with `WITH DATETIMES [...]` and `WITH DURATIONS [...]`. @@ -76,8 +113,12 @@ Map literals can be built, returned, compared with `=`, and stored as node or ed ```cypher RETURN {name: 'Alice', tags: ['a', 'b']} AS m -WITH {a: 1, b: 'x'} AS m RETURN m.a, m.b -UNWIND [{a: 1}, {a: 2}] AS m RETURN m.a + +WITH {a: 1, b: 'x'} AS m +RETURN m.a, m.b + +UNWIND [{a: 1}, {a: 2}] AS m +RETURN m.a ``` You can read a key from a map **value** that is in scope, for example one bound by `WITH` or `UNWIND`, with `m.key`. @@ -114,8 +155,12 @@ You can read a key from a map **value** that is in scope, for example one bound Embeddings are stored as ordinary properties (type `Embedding`) and compared with `cosine_similarity` (true cosine) and `euclidean_distance`. Both work in `RETURN`, `WHERE` and `ORDER BY`: ```cypher -MATCH (n:Document) RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) AS sim ORDER BY sim DESC LIMIT 10 -MATCH (n:Document) RETURN n.title, euclidean_distance(n.emb, (0.4, 0.3, 0.8, 0.1)) +MATCH (n:Document) +RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) AS sim +ORDER BY sim DESC LIMIT 10 + +MATCH (n:Document) +RETURN n.title, euclidean_distance(n.emb, (0.4, 0.3, 0.8, 0.1)) ``` This is a brute-force scan over the matched nodes. For large collections, build a [vector index](/vector-search). diff --git a/query/matching.mdx b/query/matching.mdx index b12d581..b35f907 100644 --- a/query/matching.mdx +++ b/query/matching.mdx @@ -30,16 +30,36 @@ CREATE ## MATCH ```cypher -MATCH (n) RETURN count(n) -MATCH (n:Person) RETURN n.name, n.age -MATCH (n:Person {name: 'Alice', age: 30}) RETURN n.name -MATCH (n:Person:Employee) RETURN n.name -MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN a.name, b.name -MATCH (a)-[e:KNOWS {since: 2010}]->(b) RETURN a.name, e.since, b.name -MATCH (a)<-[e]-(b) RETURN a.name, b.name -MATCH (a)-[e]-(b) RETURN a.name, b.name -MATCH (a)-[r:KNOWS|WORKS_AT]->(b) RETURN a.name, type(r), b.name -MATCH (a {name: 'Alice'}) MATCH (a)-[:KNOWS]->(b) RETURN b.name +MATCH (n) +RETURN count(n) + +MATCH (n:Person) +RETURN n.name, n.age + +MATCH (n:Person {name: 'Alice', age: 30}) +RETURN n.name + +MATCH (n:Person:Employee) +RETURN n.name + +MATCH (a:Person)-[:KNOWS]->(b:Person) +RETURN a.name, b.name + +MATCH (a)-[e:KNOWS {since: 2010}]->(b) +RETURN a.name, e.since, b.name + +MATCH (a)<-[e]-(b) +RETURN a.name, b.name + +MATCH (a)-[e]-(b) +RETURN a.name, b.name + +MATCH (a)-[r:KNOWS|WORKS_AT]->(b) +RETURN a.name, type(r), b.name + +MATCH (a {name: 'Alice'}) +MATCH (a)-[:KNOWS]->(b) +RETURN b.name ``` - `(n:Person:Employee)` matches nodes that have **both** labels. @@ -58,7 +78,8 @@ MATCH (a {name: 'Alice'}) MATCH (a)-[:KNOWS]->(b) RETURN b.name A `WHERE` can sit inside a node or edge pattern. The variable must be named: ```cypher -MATCH (a:Person WHERE a.age > 30)-[r:KNOWS WHERE r.since > 2012]->(b) RETURN b.name +MATCH (a:Person WHERE a.age > 30)-[r:KNOWS WHERE r.since > 2012]->(b) +RETURN b.name ``` ### Ordering and limiting in `MATCH` @@ -66,7 +87,9 @@ MATCH (a:Person WHERE a.age > 30)-[r:KNOWS WHERE r.since > 2012]->(b) RETURN b.n As a TuringDB extension, `ORDER BY`, `SKIP` and `LIMIT` can also appear right after the pattern, before `RETURN`: ```cypher -MATCH (n:Person) ORDER BY n.age LIMIT 2 RETURN n.name +MATCH (n:Person) +ORDER BY n.age LIMIT 2 +RETURN n.name ``` @@ -78,29 +101,64 @@ MATCH (n:Person) ORDER BY n.age LIMIT 2 RETURN n.name `OPTIONAL MATCH` keeps the row when the pattern has no match and returns `null` for its variables: ```cypher -MATCH (a:Person) OPTIONAL MATCH (a)-[:WORKS_AT]->(c:Company) RETURN a.name, c.name -MATCH (a:Person) OPTIONAL MATCH (a)-[:KNOWS]->(b) RETURN a.name, count(b) +MATCH (a:Person) +OPTIONAL MATCH (a)-[:WORKS_AT]->(c:Company) +RETURN a.name, c.name + +MATCH (a:Person) +OPTIONAL MATCH (a)-[:KNOWS]->(b) +RETURN a.name, count(b) ``` The second query returns `0` for people who know nobody. A `WHERE` attached to an `OPTIONAL MATCH` belongs to the optional part, so rows are kept, as in Neo4j. - - An unmatched optional **node** returned bare (`RETURN c`) shows `18446744073709551615` (the largest unsigned 64-bit integer) instead of `null`. Properties (`c.name`) and `id(c)` are correctly `null`, so return a property or `id(c)`. - +Return a property (`c.name`) or `id(c)` from an optional node, rather than the bare variable. + ## WHERE and operators ```cypher -MATCH (n) WHERE n:Person RETURN n.name -MATCH (n:Person) WHERE n.age >= 18 AND n.city = 'London' RETURN n.name -MATCH (n:Person) WHERE n.name STARTS WITH 'A' OR n.name ENDS WITH 'e' OR n.name CONTAINS 'ro' RETURN n.name -MATCH (n:Person) WHERE n.name IN ['Alice', 'Bob'] RETURN n.name -MATCH (n:Person) WHERE 'admin' IN n.tags RETURN n.name -MATCH (n:Person) WHERE n.tags IS NOT NULL RETURN n.name -MATCH (n:Person) WHERE n.active RETURN n.name -MATCH (n)-[e]->(m) WHERE e:KNOWS AND e.since > 2012 RETURN n.name, m.name -MATCH (p:Person) WHERE (p)-[:LIKES]->() RETURN p.name -MATCH (p:Person) WHERE NOT (p)-[:KNOWS]->() RETURN p.name +MATCH (n) +WHERE n:Person +RETURN n.name + +MATCH (n:Person) +WHERE n.age >= 18 AND n.city = 'London' +RETURN n.name + +MATCH (n:Person) +WHERE n.name STARTS +WITH 'A' OR n.name ENDS +WITH 'e' OR n.name CONTAINS 'ro' +RETURN n.name + +MATCH (n:Person) +WHERE n.name IN ['Alice', 'Bob'] +RETURN n.name + +MATCH (n:Person) +WHERE 'admin' IN n.tags +RETURN n.name + +MATCH (n:Person) +WHERE n.tags IS NOT NULL +RETURN n.name + +MATCH (n:Person) +WHERE n.active +RETURN n.name + +MATCH (n)-[e]->(m) +WHERE e:KNOWS AND e.since > 2012 +RETURN n.name, m.name + +MATCH (p:Person) +WHERE (p)-[:LIKES]->() +RETURN p.name + +MATCH (p:Person) +WHERE NOT (p)-[:KNOWS]->() +RETURN p.name ``` - **Label and type tests:** `n:Person` tests a node label, `e:KNOWS` tests an edge type. Combine them with `AND`, `OR` and `NOT`. @@ -118,7 +176,7 @@ MATCH (p:Person) WHERE NOT (p)-[:KNOWS]->() RETURN p.name Property constraints in a pattern, `MATCH (n {name: 'Matt', age: 20})`, are exactly equivalent to the same equality tests in `WHERE`. - Regular expressions (`=~`) and the `exists(n.prop)` function are not supported. Use `n.prop IS NOT NULL` to test for a property. + Regular expressions (`=~`) and the `exists(n.prop)` function are not yet supported. Use `n.prop IS NOT NULL` to test for a property. ### Numeric equality is strict @@ -133,8 +191,13 @@ ANALYZE_ERROR: Operands are not valid or compatible types Use range comparisons for Doubles, or convert the value: ```cypher -MATCH (n:Person) WHERE n.age > 29.5 AND n.age < 30.5 RETURN n.name -MATCH (n:Person) WHERE n.age = toInteger(30.0) RETURN n.name +MATCH (n:Person) +WHERE n.age > 29.5 AND n.age < 30.5 +RETURN n.name + +MATCH (n:Person) +WHERE n.age = toInteger(30.0) +RETURN n.name ``` Mixed-type `<`, `>` and arithmetic are fine. Integer division by zero raises an error, and Int64 overflow wraps silently. @@ -145,16 +208,22 @@ Comma-separated patterns in a `MATCH` produce either a **join** or a **Cartesian ```cypher // Cartesian product: every Person with every Company (N x M rows) -MATCH (p:Person), (c:Company) RETURN p.name, c.name +MATCH (p:Person), (c:Company) +RETURN p.name, c.name // Join: a shared variable links the patterns -MATCH (a:Person)-[:KNOWS]->(b), (b)-[:WORKS_AT]->(c) RETURN a.name, b.name, c.name +MATCH (a:Person)-[:KNOWS]->(b), (b)-[:WORKS_AT]->(c) +RETURN a.name, b.name, c.name // Join on a shared node: people who like the same thing -MATCH (a:Person)-->(i:Interest)<--(b:Person) WHERE a <> b RETURN a.name, b.name, i.name +MATCH (a:Person)-->(i:Interest)<--(b:Person) +WHERE a <> b +RETURN a.name, b.name, i.name // Join on a value through WHERE (hash join) -MATCH (a:Person), (b:Person) WHERE a.city = b.city AND a <> b RETURN a.name, b.name +MATCH (a:Person), (b:Person) +WHERE a.city = b.city AND a <> b +RETURN a.name, b.name ``` @@ -164,7 +233,9 @@ MATCH (a:Person), (b:Person) WHERE a.city = b.city AND a <> b RETURN a.name, b.n Connected paths and independent patterns can be mixed in one query. Shared variables create joins and unrelated patterns create Cartesian products: ```cypher -MATCH (a:Person)-->(i:Interest), (c:Company) WHERE c.name = 'Acme' RETURN a.name, i.name, c.name +MATCH (a:Person)-->(i:Interest), (c:Company) +WHERE c.name = 'Acme' +RETURN a.name, i.name, c.name ``` ## Look up nodes by native node ID @@ -172,15 +243,26 @@ MATCH (a:Person)-->(i:Interest), (c:Company) WHERE c.name = 'Acme' RETURN a.name Every node has a **native node ID**, the integer you get from `RETURN n` or `id(n)`. When you already know which nodes you want, **compare the node variable itself with the ID**, written as a literal. The planner turns this into a constant scan of exactly those nodes: ```cypher -MATCH (n) WHERE n = 0 RETURN n.name -MATCH (n)-[:KNOWS]->(m) WHERE n = 0 RETURN m.name -UNWIND [0, 1, 2] AS x MATCH (n) WHERE n = x RETURN id(n), n.name +MATCH (n) +WHERE n = 0 +RETURN n.name + +MATCH (n)-[:KNOWS]->(m) +WHERE n = 0 +RETURN m.name + +UNWIND [0, 1, 2] AS x +MATCH (n) +WHERE n = x +RETURN id(n), n.name ``` `EXPLAIN` shows whether it worked. The plan should contain `const_scan_nodes` rather than `scan_nodes`: ```cypher -EXPLAIN MATCH (n) WHERE n = 0 RETURN n.name +EXPLAIN MATCH (n) +WHERE n = 0 +RETURN n.name ``` On large graphs this is orders of magnitude faster than looking nodes up again through a key property such as `n.uuid`. The rules that keep it fast: diff --git a/query/paths.mdx b/query/paths.mdx index 12f6442..767a94c 100644 --- a/query/paths.mdx +++ b/query/paths.mdx @@ -19,12 +19,23 @@ TuringDB supports both its own postfix quantifiers and the Neo4j bracket form: | `-[:KNOWS*]->` | **1** or more (unlike the postfix `*`) | ```cypher -MATCH (a:Person {name: 'Alice'})-[:KNOWS]->+(b) RETURN DISTINCT b.name -MATCH (a:Person {name: 'Alice'})-[:KNOWS*1..3]->(b) RETURN DISTINCT b.name -MATCH (a)-[:KNOWS|WORKS_AT]->{1,2}(b) RETURN a.name, b.name -MATCH (a)-[e:KNOWS WHERE e.since > 2012]->+(b) RETURN a.name, b.name -MATCH (a)-[:KNOWS*1..3 {since: 2010}]->(b) RETURN b.name -MATCH (a {name: 'Alice'})-[e:KNOWS]->+(b) RETURN b.name, size(e) +MATCH (a:Person {name: 'Alice'})-[:KNOWS]->+(b) +RETURN DISTINCT b.name + +MATCH (a:Person {name: 'Alice'})-[:KNOWS*1..3]->(b) +RETURN DISTINCT b.name + +MATCH (a)-[:KNOWS|WORKS_AT]->{1,2}(b) +RETURN a.name, b.name + +MATCH (a)-[e:KNOWS WHERE e.since > 2012]->+(b) +RETURN a.name, b.name + +MATCH (a)-[:KNOWS*1..3 {since: 2010}]->(b) +RETURN b.name + +MATCH (a {name: 'Alice'})-[e:KNOWS]->+(b) +RETURN b.name, size(e) ``` - **Edge type alternatives** (`[:KNOWS|WORKS_AT]`) work on variable-length hops. @@ -42,8 +53,11 @@ MATCH (a {name: 'Alice'})-[e:KNOWS]->+(b) RETURN b.name, size(e) A quantified path pattern repeats a one-hop sub-pattern, with its own `WHERE`, a number of times: ```cypher -MATCH (a:Person {name: 'Alice'}) ((x)-[r:KNOWS]->(y) WHERE r.since > 2009){1,3} (b) RETURN b.name -MATCH (a:Person {name: 'Alice'}) ((x)-[:KNOWS]->(y) WHERE y.age > x.age)+ (b) RETURN b.name +MATCH (a:Person {name: 'Alice'}) ((x)-[r:KNOWS]->(y) WHERE r.since > 2009){1,3} (b) +RETURN b.name + +MATCH (a:Person {name: 'Alice'}) ((x)-[:KNOWS]->(y) WHERE y.age > x.age)+ (b) +RETURN b.name ``` The second query follows `KNOWS` edges only while ages strictly increase. Inside the parentheses only a **single hop** of the form `(node)-[edge]->(node) [WHERE ...]` is allowed, and the inner variables (`x`, `r`) become lists. @@ -77,8 +91,11 @@ RETURN [n IN nodes(p) | n.name] AS route, length(p) AS hops ORDER BY hops LIMIT 1 // k-hop neighbourhood and reachability -MATCH (a:Person {name: 'Alice'})-[:KNOWS]-{1,3}(b) RETURN DISTINCT b.name -MATCH (a:Person {name: 'Alice'})-[:KNOWS]->+(b) RETURN count(DISTINCT b) +MATCH (a:Person {name: 'Alice'})-[:KNOWS]-{1,3}(b) +RETURN DISTINCT b.name + +MATCH (a:Person {name: 'Alice'})-[:KNOWS]->+(b) +RETURN count(DISTINCT b) ``` To sum a property along a route, build a list with a comprehension and aggregate over its elements: @@ -88,7 +105,8 @@ MATCH p = (a:Person {name: 'Alice'})-[:KNOWS*1..6]->(b:Person {name: 'Dave'}) WITH [n IN nodes(p) | n.name] AS route, [r IN relationships(p) | r.since] AS years UNWIND years AS y WITH route, sum(y) AS total -RETURN route, total ORDER BY total LIMIT 1 +RETURN route, total +ORDER BY total LIMIT 1 ``` diff --git a/query/procedures.mdx b/query/procedures.mdx index 934a070..5ad1963 100644 --- a/query/procedures.mdx +++ b/query/procedures.mdx @@ -1,7 +1,7 @@ --- title: "Procedures and Commands" sidebarTitle: "Procedures and Commands" -description: "Explore a graph with CALL db.* procedures, and manage graphs, changes, indexes and extensions with TuringDB engine commands." +description: "Explore a graph with CALL db.* procedures, and manage graphs, changes, vector indexes and extensions with TuringDB engine commands." --- Beyond queries over the data in a graph, TuringDB has **procedures**, which return information about the graph, and **commands**, which drive the engine itself. @@ -14,7 +14,6 @@ Procedures use the `CALL` syntax. Use them to explore a graph before writing que CALL db.labels() CALL db.edgeTypes() CALL db.propertyTypes() -CALL db.showIndexes() CALL db.history() CALL db.procedures() ``` @@ -24,7 +23,6 @@ CALL db.procedures() | `db.labels()` | `id`, `label` | | `db.edgeTypes()` | `id`, `edgeType` | | `db.propertyTypes()` | `id`, `propertyType`, `valueType` | -| `db.showIndexes()` | `name`, `size` | | `db.history()` | `commit`, `nodeCount`, `edgeCount`, `partCount` | | `db.describeCommit(commit)` | `nodeCount`, `edgeCount`, `partCount` | | `db.procedures()` | `name`, `signature` (same as `SHOW PROCEDURES`) | @@ -44,9 +42,19 @@ A procedure's output columns can be renamed, filtered and used by the rest of th ```cypher CALL db.propertyTypes() YIELD * -CALL db.labels() YIELD label AS l WHERE l STARTS WITH 'P' RETURN l -CALL db.labels() YIELD label MATCH (n) WHERE label IN labels(n) RETURN label, count(n) -CALL db.history() YIELD `commit`, nodeCount RETURN `commit`, nodeCount + +CALL db.labels() YIELD label AS l +WHERE l STARTS +WITH 'P' +RETURN l + +CALL db.labels() YIELD label +MATCH (n) +WHERE label IN labels(n) +RETURN label, count(n) + +CALL db.history() YIELD `commit`, nodeCount +RETURN `commit`, nodeCount ``` - **Constant arguments only.** A procedure argument cannot read row values: `db.getNodes([id(n)])` fails with "must be constant". @@ -59,7 +67,9 @@ CALL db.history() YIELD `commit`, nodeCount RETURN `commit`, nodeCount `RETURN n` gives only an internal ID, and there is no `properties(n)`. To see every property of a few nodes, use `db.getNodes`, which returns the properties as a JSON string: ```cypher -CALL db.getNodes([0, 1, 2]) YIELD id, labels, properties RETURN id, labels, properties +CALL db.getNodes([0, 1, 2]) YIELD id, labels, properties +RETURN id, labels, properties + CALL db.listNodes(['Person'], {name: 'ali'}, 0, 10) YIELD id, labels, properties ``` @@ -75,9 +85,16 @@ df["properties"] = df["properties"].map(json.loads) ### Profiling a graph with plain Cypher ```cypher -MATCH (n) RETURN labels(n) AS labels, count(*) AS n ORDER BY n DESC -MATCH (a)-[r]->(b) RETURN labels(a) AS src, type(r) AS rel, labels(b) AS dst, count(*) AS n ORDER BY n DESC -MATCH (n:Person) RETURN count(n.age) AS withAge, count(*) AS total +MATCH (n) +RETURN labels(n) AS labels, count(*) AS n +ORDER BY n DESC + +MATCH (a)-[r]->(b) +RETURN labels(a) AS src, type(r) AS rel, labels(b) AS dst, count(*) AS n +ORDER BY n DESC + +MATCH (n:Person) +RETURN count(n.age) AS withAge, count(*) AS total ``` ## Commands @@ -114,13 +131,10 @@ There is no command to drop or unload a graph. `MERGE_DATAPARTS` **renumbers** all nodes. Re-fetch any cached node IDs afterwards, and rebuild vector indexes keyed by node IDs. -### Indexes +### Vector indexes | Command | Description | | ------- | ----------- | -| `CREATE INDEX FOR (n) ON n.prop` | Create a property index on a node property. A write, see [Property indexes](/query/writing#property-indexes) | -| `CREATE INDEX FOR [e] ON e.prop` | Create a property index on an edge property | -| `DROP INDEX ` | Drop a property or vector index | | `CREATE VECTOR INDEX WITH DIMENSION METRIC ` | Create a vector index, see [Vector search](/vector-search) | | `VECTOR SEARCH IN FOR () YIELD ids` | k-nearest-neighbour search | | `SHOW VECTOR INDEXES` | List vector indexes | @@ -134,7 +148,9 @@ Extensions are native libraries that register extra procedure namespaces. Instal INSTALL greeter SHOW EXTENSIONS SHOW PROCEDURES -CALL greeter.hello() YIELD message RETURN message + +CALL greeter.hello() YIELD message +RETURN message ``` `SHOW PROCEDURES` lists every available procedure and its signature, including the ones from extensions. The `greeter` example ships with the Python wheel. Extensions are looked up in `/lib/turingdb/extensions/.so`, then in `/extensions/.so`. diff --git a/query/subqueries.mdx b/query/subqueries.mdx index 48974e4..ce62281 100644 --- a/query/subqueries.mdx +++ b/query/subqueries.mdx @@ -11,10 +11,23 @@ These examples use the [sample graph](/query/matching) from the matching page. `EXISTS { ... }` tests whether a pattern, or a full `MATCH` with its own `WHERE`, has at least one match for the current row. It works in `WHERE` and as a returned value: ```cypher -MATCH (p:Person) WHERE EXISTS { (p)-[:WORKS_AT]->(:Company {name: 'Acme'}) } RETURN p.name -MATCH (p:Person) WHERE EXISTS { MATCH (p)-[:KNOWS]->(f) WHERE f.age > p.age } RETURN p.name -MATCH (p:Person) WHERE NOT EXISTS { (p)-[:KNOWS]->() } RETURN p.name -MATCH (p:Person) RETURN p.name, EXISTS { (p)-[:LIKES]->() } AS likesSomething +MATCH (p:Person) +WHERE EXISTS { (p)-[:WORKS_AT]->(:Company {name: 'Acme'}) } +RETURN p.name + +MATCH (p:Person) +WHERE EXISTS { + MATCH (p)-[:KNOWS]->(f) + WHERE f.age > p.age +} +RETURN p.name + +MATCH (p:Person) +WHERE NOT EXISTS { (p)-[:KNOWS]->() } +RETURN p.name + +MATCH (p:Person) +RETURN p.name, EXISTS { (p)-[:LIKES]->() } AS likesSomething ``` ## COUNT @@ -22,8 +35,16 @@ MATCH (p:Person) RETURN p.name, EXISTS { (p)-[:LIKES]->() } AS likesSomething `COUNT { ... }` returns the number of matches for each row: ```cypher -MATCH (p:Person) RETURN p.name, COUNT { (p)-[:KNOWS]->() } AS outDegree ORDER BY outDegree DESC -MATCH (p:Person) WHERE COUNT { MATCH (p)-[:KNOWS]-(x) RETURN x } >= 2 RETURN p.name +MATCH (p:Person) +RETURN p.name, COUNT { (p)-[:KNOWS]->() } AS outDegree +ORDER BY outDegree DESC + +MATCH (p:Person) +WHERE COUNT { + MATCH (p)-[:KNOWS]-(x) + RETURN x +} >= 2 +RETURN p.name ``` ## CALL subqueries @@ -31,10 +52,38 @@ MATCH (p:Person) WHERE COUNT { MATCH (p)-[:KNOWS]-(x) RETURN x } >= 2 RETURN p.n A `CALL { ... }` subquery runs once per incoming row. It is the way to compute a per-row top-k or a per-row aggregate. Import the outer variables either in the `CALL (p)` header or with a `WITH p` as the first clause of the body: ```cypher -MATCH (p:Person) CALL (p) { MATCH (p)-[:KNOWS]->(f) RETURN f.name AS friend ORDER BY f.name LIMIT 1 } RETURN p.name, friend -MATCH (p:Person) CALL { WITH p MATCH (p)-[:KNOWS]->(f) RETURN count(f) AS deg } RETURN p.name, deg -MATCH (p:Person) OPTIONAL CALL (p) { MATCH (p)-[:WORKS_AT]->(c) RETURN c.name AS co } RETURN p.name, co -CALL { MATCH (p:Person) RETURN p.name AS nm UNION MATCH (c:Company) RETURN c.name AS nm } RETURN nm ORDER BY nm +MATCH (p:Person) +CALL (p) { + MATCH (p)-[:KNOWS]->(f) + RETURN f.name AS friend + ORDER BY f.name LIMIT 1 +} +RETURN p.name, friend + +MATCH (p:Person) +CALL { + WITH p + MATCH (p)-[:KNOWS]->(f) + RETURN count(f) AS deg +} +RETURN p.name, deg + +MATCH (p:Person) +OPTIONAL CALL (p) { + MATCH (p)-[:WORKS_AT]->(c) + RETURN c.name AS co +} +RETURN p.name, co + +CALL { + MATCH (p:Person) + RETURN p.name AS nm + UNION + MATCH (c:Company) + RETURN c.name AS nm +} +RETURN nm +ORDER BY nm ``` - **Dropped rows.** A non-optional `CALL` drops the outer rows for which its body returns nothing. Use `OPTIONAL CALL` to keep them with `null` values. diff --git a/query/writing.mdx b/query/writing.mdx index 134678b..18181b1 100644 --- a/query/writing.mdx +++ b/query/writing.mdx @@ -1,10 +1,10 @@ --- title: "Writing Data" sidebarTitle: "Writing Data" -description: "CREATE, MERGE, SET, REMOVE, DELETE and property indexes in TuringDB, and the change workflow that every write goes through." +description: "CREATE, MERGE, SET, REMOVE and DELETE in TuringDB, and the change workflow that every write goes through." --- -All writes (`CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`, `CREATE INDEX`, `DROP INDEX`) must run **inside a change**. A change is TuringDB's equivalent of a git branch: you write in an isolated copy and then submit it to the main graph. A write sent outside a change fails with an explicit error such as `Cannot perform CREATE outside of a write transaction.` +All writes (`CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`) must run **inside a change**. A change is TuringDB's equivalent of a git branch: you write in an isolated copy and then submit it to the main graph. A write sent outside a change fails with an explicit error such as `Cannot perform CREATE outside of a write transaction.` ## The change workflow @@ -59,13 +59,26 @@ client.checkout() ```cypher CREATE (:Person {name: 'Alice', age: 30}) -CREATE (a:Person:Employee {name: 'Bob'}) RETURN a.name, id(a) + +CREATE (a:Person:Employee {name: 'Bob'}) +RETURN a.name, id(a) + CREATE (:Person {name: 'Mick'})-[:FRIEND_OF {since: 2020}]->(:Person {name: 'John'}) + CREATE (a:T {x: 1}), (b:T {x: 2}), (a)-[:R]->(b) -CREATE p = (:City {name: 'Paris'})-[:ROAD {km: 450}]->(:City {name: 'Lyon'}) RETURN p + +CREATE p = (:City {name: 'Paris'})-[:ROAD {km: 450}]->(:City {name: 'Lyon'}) +RETURN p + CREATE (c:City {name: 'Li' + 'lle', pop: 2 * 1000, tags: ['a', 'b'], info: {zone: 1}, emb: (0.1, 0.2, 0.3)}) -CREATE (n:Item {sku: 'A1'}) SET n.price = 9.99 RETURN n.sku -CREATE (a:W {k: 'a'}) WITH a CREATE (b:W {k: 'b'})-[:NEXT]->(a) + +CREATE (n:Item {sku: 'A1'}) +SET n.price = 9.99 +RETURN n.sku + +CREATE (a:W {k: 'a'}) +WITH a +CREATE (b:W {k: 'b'})-[:NEXT]->(a) ``` To connect nodes that already exist, match them first. They must be committed, or created earlier **in the same query**: @@ -79,7 +92,11 @@ RETURN a.name, b.name If you know their [native node IDs](/query/matching#look-up-nodes-by-native-node-id), address them directly. That is the fastest option, but the IDs must be **constants** written in the query, one pair per query: ```cypher -MATCH (a) WHERE a = 0 MATCH (b) WHERE b = 1 CREATE (a)-[:KNOWS {since: 2021}]->(b) +MATCH (a) +WHERE a = 0 +MATCH (b) +WHERE b = 1 +CREATE (a)-[:KNOWS {since: 2021}]->(b) ``` **Rules:** @@ -107,10 +124,18 @@ RETURN n.name ``` ```cypher -MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Carol'}) MERGE (a)-[e:KNOWS]->(b) ON CREATE SET e.since = 2024 +MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Carol'}) +MERGE (a)-[e:KNOWS]->(b) + ON CREATE SET e.since = 2024 + MERGE (a:Tag {name: 'x'})-[:LINKS]->(b:Tag {name: 'y'}) -MATCH (p:Person {name: 'Bob'}) MERGE (p)-[:OWNS]->(c:Car {model: 'Z'}) -UNWIND ['a', 'b', 'a'] AS k MERGE (t:Key {k: k}) RETURN t.k + +MATCH (p:Person {name: 'Bob'}) +MERGE (p)-[:OWNS]->(c:Car {model: 'Z'}) + +UNWIND ['a', 'b', 'a'] AS k +MERGE (t:Key {k: k}) +RETURN t.k ``` `MERGE` deduplicates within one query, so the last example creates two `Key` nodes, not three. @@ -122,22 +147,59 @@ UNWIND ['a', 'b', 'a'] AS k MERGE (t:Key {k: k}) RETURN t.k ## SET and REMOVE ```cypher -MATCH (n:Person {name: 'Alice'}) SET n.age = 31 -MATCH (n:Person {name: 'Alice'}) SET n.score = n.age * 1.1, n.updated = true -MATCH (p:Product) SET p.discountPrice = p.price * (1 - 0.15) -MATCH (a:Person) SET a.band = CASE WHEN a.age > 30 THEN 'senior' ELSE 'junior' END -MATCH (a:Person) SET a.place = coalesce(a.city, 'unknown') -MATCH (n:Person {name: 'Alice'}), (m:Person {name: 'Bob'}) SET n.age = m.age -MATCH (n:Person {name: 'Alice'})-[e:KNOWS]->(m) SET e.weight = 0.5 -MATCH (n:Person) WHERE n.age < 18 SET n.isMinor = true -MATCH (n:Person {name: 'Alice'}) SET n.tags = ['a', 'b'], n.meta = {level: 3} -MATCH (n:Person {name: 'Alice'}) SET n.emb = (1.2, 2.0, 0.0, 12.0) -MATCH (n:Person {name: 'Alice'}) SET n.joined = datetime('2024-03-15T10:30:00Z') -MATCH (a:Person) WITH count(a) AS c MATCH (s:Stats {name: 'people'}) SET s.total = c -MATCH (n) WHERE n = 0 SET n.flag = true -UNWIND [0, 1, 2] AS x MATCH (n) WHERE n = x SET n.flag = true -MATCH (n:Person {name: 'Alice'}) SET n.age = null -MATCH (n:Person {name: 'Alice'}) REMOVE n.score, n.flag +MATCH (n:Person {name: 'Alice'}) +SET n.age = 31 + +MATCH (n:Person {name: 'Alice'}) +SET n.score = n.age * 1.1, n.updated = true + +MATCH (p:Product) +SET p.discountPrice = p.price * (1 - 0.15) + +MATCH (a:Person) +SET a.band = CASE WHEN a.age > 30 THEN 'senior' ELSE 'junior' END + +MATCH (a:Person) +SET a.place = coalesce(a.city, 'unknown') + +MATCH (n:Person {name: 'Alice'}), (m:Person {name: 'Bob'}) +SET n.age = m.age + +MATCH (n:Person {name: 'Alice'})-[e:KNOWS]->(m) +SET e.weight = 0.5 + +MATCH (n:Person) +WHERE n.age < 18 +SET n.isMinor = true + +MATCH (n:Person {name: 'Alice'}) +SET n.tags = ['a', 'b'], n.meta = {level: 3} + +MATCH (n:Person {name: 'Alice'}) +SET n.emb = (1.2, 2.0, 0.0, 12.0) + +MATCH (n:Person {name: 'Alice'}) +SET n.joined = datetime('2024-03-15T10:30:00Z') + +MATCH (a:Person) +WITH count(a) AS c +MATCH (s:Stats {name: 'people'}) +SET s.total = c + +MATCH (n) +WHERE n = 0 +SET n.flag = true + +UNWIND [0, 1, 2] AS x +MATCH (n) +WHERE n = x +SET n.flag = true + +MATCH (n:Person {name: 'Alice'}) +SET n.age = null + +MATCH (n:Person {name: 'Alice'}) +REMOVE n.score, n.flag ``` - `SET` creates the property if it does not exist yet. Setting a property to `null` clears it, and `REMOVE n.prop` removes it. @@ -152,10 +214,18 @@ MATCH (n:Person {name: 'Alice'}) REMOVE n.score, n.flag ## DELETE and DETACH DELETE ```cypher -MATCH (n:Person {name: 'Alice'}) DETACH DELETE n -MATCH (n:Person {name: 'Alice'})-[e:KNOWS]->(m) DELETE e -MATCH (n:Temp) WHERE n.v > 3 DELETE n -MATCH (n:Person {name: 'Remy'})-[e]->(m:Person) DELETE e, n +MATCH (n:Person {name: 'Alice'}) +DETACH DELETE n + +MATCH (n:Person {name: 'Alice'})-[e:KNOWS]->(m) +DELETE e + +MATCH (n:Temp) +WHERE n.v > 3 +DELETE n + +MATCH (n:Person {name: 'Remy'})-[e]->(m:Person) +DELETE e, n ``` - `DETACH DELETE` removes a node together with all of its edges. @@ -171,16 +241,22 @@ MATCH (n:Person {name: 'Remy'})-[e]->(m:Person) DELETE e, n For small volumes, inline the data with `UNWIND`: ```cypher -UNWIND range(1, 1000) AS i CREATE (:Num {v: i}) -UNWIND ['Brest', 'Rennes', 'Nantes'] AS nm MERGE (:City {name: nm}) -UNWIND [['Brest', 'FR-29'], ['Rennes', 'FR-35']] AS row CREATE (:City {name: row[0], code: row[1]}) +UNWIND range(1, 1000) AS i +CREATE (:Num {v: i}) + +UNWIND ['Brest', 'Rennes', 'Nantes'] AS nm +MERGE (:City {name: nm}) + +UNWIND [['Brest', 'FR-29'], ['Rennes', 'FR-35']] AS row +CREATE (:City {name: row[0], code: row[1]}) ``` A `CREATE` cannot read an element of a mixed-type list row such as `['Brest', 139000]`. Use parallel lists instead: ```cypher WITH ['Brest', 'Rennes'] AS names, [139000, 220000] AS pops -UNWIND range(0, size(names) - 1) AS i CREATE (:City {name: names[i], pop: pops[i]}) +UNWIND range(0, size(names) - 1) AS i +CREATE (:City {name: names[i], pop: pops[i]}) ``` `LOAD CSV` can drive writes too. Create the nodes, `COMMIT`, then create the edges (see [CSV import](/import_data/csv)). Alternatively `MERGE` both endpoints in a single pass, which deduplicates across the rows of one query so no `COMMIT` is needed: @@ -209,28 +285,3 @@ client.query(f"UNWIND [{', '.join(lit(n) for n in names)}] AS nm MERGE (:Person ``` Keep each query to a few thousand rows at most. - -## Property indexes - -A property index speeds up equality lookups. The optimizer rewrites `WHERE n.prop = ` into an index lookup when a matching index exists. Creating and dropping an index are writes, so they run inside a change: - -```cypher -CREATE INDEX age_index FOR (n) ON n.age -CREATE INDEX since_index FOR [e] ON e.since -DROP INDEX age_index -``` - -```python -client.new_change() -client.query("CREATE INDEX age_index FOR (n) ON n.age") -client.query("CHANGE SUBMIT") -client.checkout() -``` - -- Node indexes use `FOR (n) ON n.prop`, edge indexes use square brackets: `FOR [e] ON e.prop`. -- The property must already exist in the graph, and the index name is required. -- Indexes are **not scoped** to a label or edge type. `FOR (n:Person)` and `FOR [e:KNOWS]` fail as not yet supported, and so do the Neo4j forms `CREATE INDEX FOR (n:Person) ON (n.name)` and `IF NOT EXISTS`. -- List indexes with `CALL db.showIndexes()`. A new index appears there after `COMMIT` or `CHANGE SUBMIT`. -- `CREATE CONSTRAINT` is not implemented. - -For vector indexes, see [Vector search](/vector-search). diff --git a/troubles/troubles.mdx b/troubles/troubles.mdx index 55e0aeb..bdbdb97 100644 --- a/troubles/troubles.mdx +++ b/troubles/troubles.mdx @@ -54,7 +54,7 @@ After `CHANGE SUBMIT` the Python client still points at the submitted change. Ca ## `Cannot perform CREATE outside of a write transaction` -All writes (`CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`, `CREATE INDEX`) must run inside a change. Call `client.new_change()` first, and submit with `CHANGE SUBMIT`. See [Writing data](/query/writing). +All writes (`CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`) must run inside a change. Call `client.new_change()` first, and submit with `CHANGE SUBMIT`. See [Writing data](/query/writing). --- diff --git a/tutorials/create_graph.mdx b/tutorials/create_graph.mdx index 9250f9d..de9e6d2 100644 --- a/tutorials/create_graph.mdx +++ b/tutorials/create_graph.mdx @@ -149,7 +149,7 @@ graph [ ## Key Points -1. **Single CREATE Query**: This tutorial builds the whole graph in one `CREATE` statement, which lets the edges refer to the nodes by variable. You can also create the nodes first, `COMMIT`, and then `MATCH` them to create the edges, see [Writing data](/query/writing). To import a GML file as-is, use [`LOAD GML`](/import_data/gml). +1. **Single CREATE Query**: This tutorial builds the whole graph in one `CREATE` statement, which lets the edges refer to the nodes by variable. You can also create the nodes first, `COMMIT`, and then `MATCH` them to create the edges, see [Writing data](/query/writing). To import data in bulk, use [`LOAD PARQUET`](/import_data/parquet) or [`LOAD JSONL`](/import_data/jsonl). 2. **Change Management**: Always use the change system when modifying data: - Create a new change with `CHANGE NEW` - Checkout into the change @@ -160,7 +160,7 @@ graph [ ## Troubleshooting -- **Large Graphs**: For very large GML files, you might hit query size limits. Consider [`LOAD GML`](/import_data/gml), or convert the graph to Parquet and use [`LOAD PARQUET`](/import_data/parquet), which loads hundreds of thousands of nodes and edges in well under a second. +- **Large Graphs**: For very large GML files, you might hit query size limits. Convert the graph to Parquet and use [`LOAD PARQUET`](/import_data/parquet), which loads hundreds of thousands of nodes and edges in well under a second, or use [`LOAD JSONL`](/import_data/jsonl). - **Invalid Characters**: The sanitization function handles common invalid characters, but you may need to extend it for specific use cases. - **Memory Usage**: Large graphs are loaded entirely into memory before conversion, so ensure you have sufficient RAM. From a92aa3c88857a82ee9e72e6c895892ea9ebf5322 Mon Sep 17 00:00:00 2001 From: dorukustay1 Date: Tue, 13 Oct 2026 11:27:35 +0100 Subject: [PATCH 4/5] Address second round of review comments --- commands.mdx | 2 -- graph_dev/update_properties.mdx | 8 ++++++-- import_data/csv.mdx | 4 ++-- import_data/jsonl.mdx | 2 +- import_data/parquet.mdx | 4 ++-- 5 files changed, 11 insertions(+), 9 deletions(-) diff --git a/commands.mdx b/commands.mdx index b047ecc..814e996 100644 --- a/commands.mdx +++ b/commands.mdx @@ -28,8 +28,6 @@ turingdb start [options] | `-auth-on` | Require a bearer token on every request. The token is read from the `TURINGDB_AUTH_TOKEN` environment variable, and startup fails if it is not set | off | | `-start-timeout ` | Time to wait for daemon readiness | `500` | -Flags use a **single dash** (`-demon`, not `--demon`). - ### Examples Interactive mode (default): diff --git a/graph_dev/update_properties.mdx b/graph_dev/update_properties.mdx index 86c972a..0a9abb4 100644 --- a/graph_dev/update_properties.mdx +++ b/graph_dev/update_properties.mdx @@ -102,8 +102,10 @@ The `SET` clause is used to update property values on existing nodes and edges. ``` - Each property name has a single type across the whole graph, so `SET n.age = 'old'` fails if `age` is an Integer. Labels cannot be changed after creation: `SET n:Label` is not supported. `SET n += {...}` and `SET n = {...}` are not supported either. See [Writing data](/query/writing#set-and-remove) for the full rules. + Each property name has a single type across the whole graph, so `SET n.age = 'old'` fails if `age` is an Integer. See [Writing data](/query/writing#set-and-remove) for the full rules. + + Labels cannot be changed after creation. ```python @@ -204,7 +206,9 @@ The `SET` clause is used to update property values on existing nodes and edges. ``` - Each property name has a single type across the whole graph, so `SET n.age = 'old'` fails if `age` is an Integer. Labels cannot be changed after creation: `SET n:Label` is not supported. See [Writing data](/query/writing#set-and-remove) for the full rules. + Each property name has a single type across the whole graph, so `SET n.age = 'old'` fails if `age` is an Integer. See [Writing data](/query/writing#set-and-remove) for the full rules. + + Labels cannot be changed after creation. diff --git a/import_data/csv.mdx b/import_data/csv.mdx index e9a61ac..bdc2a8b 100644 --- a/import_data/csv.mdx +++ b/import_data/csv.mdx @@ -21,7 +21,7 @@ LOAD CSV "" [WITH HEADERS] [ON ERROR SKIP | ON ERROR FAIL] AS | `ON ERROR FAIL` | Abort on the first malformed row (the default) | | `AS ` | Variable bound to each row | -All values come back as **strings**. Wrap them in `toInteger(...)` / `toFloat(...)` / `datetime(...)` if you need other types. Files must be comma-separated: `FIELDTERMINATOR` and the Neo4j form `LOAD CSV WITH HEADERS FROM 'file:///...'` are not supported. +All values come back as **strings**. Wrap them in `toInteger(...)` / `toFloat(...)` / `datetime(...)` if you need other types. ## With headers @@ -38,7 +38,7 @@ RETURN row[0] AS name, row[1] AS age LIMIT 5 ``` -Without `WITH HEADERS`, the header line comes back as an ordinary data row. A row cannot be returned whole: read its fields as `row.col` or `row[]`. +Read a row's fields as `row.col` or `row[]`. Without `WITH HEADERS`, the header line comes back as an ordinary data row. ## Combining with the rest of a query diff --git a/import_data/jsonl.mdx b/import_data/jsonl.mdx index 8b1041a..a1d19dc 100644 --- a/import_data/jsonl.mdx +++ b/import_data/jsonl.mdx @@ -76,7 +76,7 @@ LOAD JSONL 'mydata.jsonl' AS mygraph WITH EMBEDDINGS [{"embedding", 384}, {"summ Each spec is `{"", }`. The dimension must be **greater than 1**, and every array under that property must have exactly that many numeric elements (otherwise the load fails). This works for both node and relationship properties. Array properties not listed in `WITH EMBEDDINGS` are stored as Lists. - When an import fails, the details are written to `/logs/turingdb.log`. + When an import fails, additional details are written to `/logs/turingdb.log`. diff --git a/import_data/parquet.mdx b/import_data/parquet.mdx index 2636acb..ff16b77 100644 --- a/import_data/parquet.mdx +++ b/import_data/parquet.mdx @@ -10,7 +10,7 @@ To load Parquet files containing a JSON string column, which is then converted i # `LOAD PARQUET` -`LOAD PARQUET` is the fastest way to get data into TuringDB, and the recommended path for anything beyond a few thousand entities. Above roughly 5,000 nodes or 1,000 edges, write your data to Parquet and load it with this command: 100,000 nodes and 200,000 edges load in about 0.2 seconds, whereas Cypher-driven loads (`LOAD CSV` or `UNWIND` with `MATCH` and `CREATE` for the edges) take minutes at a few thousand edges. +`LOAD PARQUET` is the fastest way to get data into TuringDB, and the recommended path for anything beyond a few thousand entities. ``` LOAD PARQUET '' [AS ] [WITH DURATIONS [, ...]] @@ -48,7 +48,7 @@ Every other column becomes a property of the node or edge on that row. The colum | timestamp | DateTime | | list of the types above | List | -`INT32` and `FLOAT` columns are rejected (for example `Node column was not of type INT64` or `Unsupported column type`), so cast them to `int64` / `float64` first. To load an `INT64` column of microseconds as a `Duration`, name it in `WITH DURATIONS []`. +`INT32` and `FLOAT` columns are rejected, so cast them to `int64` / `float64` first. To load an `INT64` column of microseconds as a `Duration`, name it in `WITH DURATIONS []`. ## From pandas From c6eaf5aa6e3995a5e836ca6db24791075df5587c Mon Sep 17 00:00:00 2001 From: dorukustay1 Date: Tue, 13 Oct 2026 11:35:48 +0100 Subject: [PATCH 5/5] Use more positive, consistent wording across the v3 docs --- commands.mdx | 8 +++--- graph_dev/create_nodes_edges.mdx | 2 +- graph_dev/submit_change.mdx | 4 +-- graph_dev/time_travelling.mdx | 4 +-- graph_dev/update_properties.mdx | 4 +-- import_data/csv.mdx | 6 ++--- import_data/jsonl.mdx | 4 +-- import_data/parquet.mdx | 6 ++--- pythonsdk/reference.mdx | 20 +++++++-------- query/aggregation.mdx | 10 ++++---- query/cheatsheet.mdx | 26 +++++++------------ query/cypher_subset.mdx | 38 +++++++++++++-------------- query/functions.mdx | 12 ++++----- query/matching.mdx | 20 +++++++-------- query/paths.mdx | 10 ++++---- query/procedures.mdx | 10 ++++---- query/subqueries.mdx | 8 +++--- query/writing.mdx | 44 ++++++++++++++++---------------- shortest-path.mdx | 12 ++++----- troubles/troubles.mdx | 6 ++--- vector-search.mdx | 8 +++--- 21 files changed, 127 insertions(+), 135 deletions(-) diff --git a/commands.mdx b/commands.mdx index 814e996..1a0cec1 100644 --- a/commands.mdx +++ b/commands.mdx @@ -25,7 +25,7 @@ turingdb start [options] | `-reset-default` | Delete the default graph before starting | off | | `-ui` | Launch the built-in visualizer proxy | off | | `-ui-port ` | Visualizer proxy port | `8080` | -| `-auth-on` | Require a bearer token on every request. The token is read from the `TURINGDB_AUTH_TOKEN` environment variable, and startup fails if it is not set | off | +| `-auth-on` | Require a bearer token on every request. The token is read from the `TURINGDB_AUTH_TOKEN` environment variable, and must be set when you use this flag | off | | `-start-timeout ` | Time to wait for daemon readiness | `500` | ### Examples @@ -67,7 +67,7 @@ USE_TURING_PROTO=1 turingdb start -demon ``` - Server logs are written to `/logs/turingdb.log`. When an import such as `LOAD JSONL` fails, the details are only in that file. + Server logs are written to `/logs/turingdb.log`. Additional details about an import, such as a `LOAD JSONL`, are written there. ### Interactive shell @@ -152,7 +152,7 @@ turingdb stop -turing-dir /data/myturing | Variable | Read by | Effect | |---|---|---| -| `USE_TURING_PROTO` | server | `1` makes the server speak TuringDB's **binary protocol** instead of JSON over HTTP. The server is then binary-only: use `TuringDB(type="native")` to connect, and `curl`, the JSON client and the visualizer stop working against it. See [client backends](/pythonsdk/reference#client-backends) | +| `USE_TURING_PROTO` | server | `1` makes the server speak TuringDB's **binary protocol** instead of JSON over HTTP. The server then serves the binary protocol only: connect with `TuringDB(type="native")`, and use a server in the default mode for `curl`, the JSON client and the visualizer. See [client backends](/pythonsdk/reference#client-backends) | | `TURINGDB_AUTH_TOKEN` | server and Python SDK | With `-auth-on`, the token the server requires. The SDK sends it as a bearer token when set | | `TURINGDB_TYPE` | Python SDK | Default client backend (`json`, `native` or `embedded`) when `type` is not passed | @@ -167,7 +167,7 @@ curl -s -X POST 'http://localhost:6666/query?graph=mygraph' -d "MATCH (n) RETURN ``` - The request body is the query text. The optional query-string parameters `change=` and `commit=` target a change or a past commit. -- With `-auth-on`, send `Authorization: Bearer `. A missing or wrong token returns HTTP `401` with an empty body. +- With `-auth-on`, send `Authorization: Bearer `. A missing or incorrect token returns HTTP `401` with an empty body. - A successful response is `{"header": {"column_names": [...], "column_types": [...]}, "data": [chunk, ...], "time": }`, where each chunk is a list of columns. - An error response is `{"error": "PARSE_ERROR", "error_details": "...", "time": }`. The code is one of `PARSE_ERROR`, `ANALYZE_ERROR`, `PLAN_ERROR`, `EXEC_ERROR`, `CHANGE_NOT_FOUND` or `COMMIT_NOT_FOUND`. - Queries on a non-HEAD commit need `LOAD COMMIT ''` first when you talk to the API directly. The CLI and the SDK do this for you. diff --git a/graph_dev/create_nodes_edges.mdx b/graph_dev/create_nodes_edges.mdx index 7370d33..8ffc3a9 100644 --- a/graph_dev/create_nodes_edges.mdx +++ b/graph_dev/create_nodes_edges.mdx @@ -73,7 +73,7 @@ description: "Once you have a valid change, you can start adding new nodes and e ## Merge, delete and bulk loading -- Use `MERGE` instead of `CREATE` to avoid duplicates: `MERGE (n:Person {name: 'Jane'})`. `MERGE` cannot see nodes created earlier in the same change until you `COMMIT`. +- Use `MERGE` instead of `CREATE` to avoid duplicates: `MERGE (n:Person {name: 'Jane'})`. Run `COMMIT` first when `MERGE` should find nodes created earlier in the same change. - Remove data with `DELETE` (edges, or nodes without edges) and `DETACH DELETE` (a node and all its edges). - Every node needs at least one label, and every edge exactly one edge type. - For more than a few thousand nodes or edges, build the graph with [`LOAD PARQUET`](/import_data/parquet) instead of Cypher writes. diff --git a/graph_dev/submit_change.mdx b/graph_dev/submit_change.mdx index 878d1ea..a410fa2 100644 --- a/graph_dev/submit_change.mdx +++ b/graph_dev/submit_change.mdx @@ -26,7 +26,7 @@ description: "Submit changes done to the database (just like a commit being subm - `CHANGE SUBMIT` commits any pending writes itself, so you do not need a `COMMIT` right before it. In the Python SDK, `client.checkout()` afterwards is **required**: the client still points at the submitted change, and the next query would fail with `CHANGE_NOT_FOUND`. + `CHANGE SUBMIT` commits any pending writes itself, so a `COMMIT` right before it is optional. In the Python SDK, call `client.checkout()` afterwards to return to main: the client points at the submitted change until you do. -If two changes edit the same node, the second submit fails with an error such as `This change attempted to update Node 0 (...) which has been modified on main.` The failed change stays open: fix it and submit again, or discard it with `CHANGE DELETE`. See [Writing data](/query/writing#failed-queries-and-conflicts). +If two changes edit the same node, the second submit reports a conflict, for example `This change attempted to update Node 0 (...) which has been modified on main.` The failed change stays open: fix it and submit again, or discard it with `CHANGE DELETE`. See [Writing data](/query/writing#failed-queries-and-conflicts). diff --git a/graph_dev/time_travelling.mdx b/graph_dev/time_travelling.mdx index 126c840..11a8e2b 100644 --- a/graph_dev/time_travelling.mdx +++ b/graph_dev/time_travelling.mdx @@ -44,8 +44,8 @@ description: "You can time travel in you graph databases and see the commit hist - Commits are listed newest first, and the latest one is marked with a literal `(HEAD)` suffix on its hash: strip it before passing the hash to `checkout`. - Every `COMMIT` and every submitted change adds an entry. - `nodeCount` and `edgeCount` count what **that commit** added, not the graph totals. -- Use hashes exactly as printed, since they are not zero-padded. An unknown hash passed to `client.checkout(commit=...)` raises `COMMIT_NOT_FOUND`, and `CALL db.describeCommit('')` returns zeros for an unknown hash. -- A past commit is read-only: writes are refused there. Call `client.checkout()` to return to main. +- Use hashes exactly as printed (they are not zero-padded). An unknown hash returns `COMMIT_NOT_FOUND` from `client.checkout(commit=...)`, and zeros from `CALL db.describeCommit('')`. +- A past commit is read-only, so make writes on main or in a change. Call `client.checkout()` to return to main. ## Lazy Commit Loading diff --git a/graph_dev/update_properties.mdx b/graph_dev/update_properties.mdx index 0a9abb4..038081b 100644 --- a/graph_dev/update_properties.mdx +++ b/graph_dev/update_properties.mdx @@ -102,7 +102,7 @@ The `SET` clause is used to update property values on existing nodes and edges. ``` - Each property name has a single type across the whole graph, so `SET n.age = 'old'` fails if `age` is an Integer. See [Writing data](/query/writing#set-and-remove) for the full rules. + Each property name has a single type across the whole graph, so `SET n.age = 'old'` is reported as a type incompatibility when `age` is an Integer. See [Writing data](/query/writing#set-and-remove) for the full rules. Labels cannot be changed after creation. @@ -206,7 +206,7 @@ The `SET` clause is used to update property values on existing nodes and edges. ``` - Each property name has a single type across the whole graph, so `SET n.age = 'old'` fails if `age` is an Integer. See [Writing data](/query/writing#set-and-remove) for the full rules. + Each property name has a single type across the whole graph, so `SET n.age = 'old'` is reported as a type incompatibility when `age` is an Integer. See [Writing data](/query/writing#set-and-remove) for the full rules. Labels cannot be changed after creation. diff --git a/import_data/csv.mdx b/import_data/csv.mdx index bdc2a8b..61df179 100644 --- a/import_data/csv.mdx +++ b/import_data/csv.mdx @@ -51,7 +51,7 @@ RETURN p.name, row.city ``` - An `OPTIONAL MATCH` directly after `LOAD CSV` that uses `row.x` inside its pattern hits an internal error. Project the field first: `... AS row WITH row.name AS nm OPTIONAL MATCH (p:Person {name: nm}) ...`. + To use `row.x` inside an `OPTIONAL MATCH` pattern that directly follows `LOAD CSV`, project the field first: `... AS row WITH row.name AS nm OPTIONAL MATCH (p:Person {name: nm}) ...`. ## Building a graph from CSV @@ -67,7 +67,7 @@ client.checkout() ``` - This is meant for small files and for adding to an existing graph. Creating edges with `LOAD CSV` has to `MATCH` both endpoints for every row, which becomes very slow beyond a few thousand edges. For bulk loads, convert the CSV to Parquet and use [`LOAD PARQUET`](/import_data/parquet), which loads 100,000 nodes and 200,000 edges in about 0.2 seconds. + This approach suits small files and additions to an existing graph. Creating edges with `LOAD CSV` matches both endpoints for every row, so it is best kept to a few thousand edges. For bulk loads, convert the CSV to Parquet and use [`LOAD PARQUET`](/import_data/parquet), which loads 100,000 nodes and 200,000 edges in about 0.2 seconds. -To skip malformed rows instead of failing, add `ON ERROR SKIP`. See [Writing data](/query/writing#bulk-writes) for a single-pass `MERGE` pattern that needs no `COMMIT`. +To skip malformed rows, add `ON ERROR SKIP` (the default, `ON ERROR FAIL`, stops at the first one). See [Writing data](/query/writing#bulk-writes) for a single-pass `MERGE` pattern that needs no `COMMIT`. diff --git a/import_data/jsonl.mdx b/import_data/jsonl.mdx index a1d19dc..5a7c721 100644 --- a/import_data/jsonl.mdx +++ b/import_data/jsonl.mdx @@ -48,7 +48,7 @@ Each line is a self-contained JSON object representing either a node or a relati - Nested objects are stored as **JSON strings**. - ISO date strings stay strings, and integer durations stay integers, unless you declare them (see below). -Each property name has a single type across the whole graph, so a property that is an integer on one node cannot be a string on another. +Each property name has a single type across the whole graph, so use the same type for a property on every node. ## Loading date, duration and embedding properties @@ -73,7 +73,7 @@ An array-valued property is stored as a List by default. To load specific array LOAD JSONL 'mydata.jsonl' AS mygraph WITH EMBEDDINGS [{"embedding", 384}, {"summary_vec", 768}] ``` -Each spec is `{"", }`. The dimension must be **greater than 1**, and every array under that property must have exactly that many numeric elements (otherwise the load fails). This works for both node and relationship properties. Array properties not listed in `WITH EMBEDDINGS` are stored as Lists. +Each spec is `{"", }`. The dimension must be **greater than 1**, and every array under that property must have exactly that many numeric elements. This works for both node and relationship properties. Array properties not listed in `WITH EMBEDDINGS` are stored as Lists. When an import fails, additional details are written to `/logs/turingdb.log`. diff --git a/import_data/parquet.mdx b/import_data/parquet.mdx index ff16b77..0974bc3 100644 --- a/import_data/parquet.mdx +++ b/import_data/parquet.mdx @@ -19,7 +19,7 @@ LOAD PARQUET '' [AS ] [WITH DURATIONS [, ...]] The graph name defaults to the name of the directory. The directory is resolved inside the `data` subdirectory of the TuringDB working directory (`-turing-dir`) and must contain `nodes.parquet` and `edges.parquet`. - `LOAD PARQUET` always creates a **new graph**. The target graph name must not already exist, and it cannot append to an existing graph. To add a large volume of data to an existing graph, write the existing data plus the new data to Parquet and load it under a new graph name. Use Cypher writes only for small, incremental changes. + `LOAD PARQUET` creates a **new graph**, so choose a graph name that does not exist yet. To add a large volume of data to an existing graph, write the existing data plus the new data to Parquet and load it under a new graph name. Use Cypher writes only for small, incremental changes. ## Required columns @@ -48,7 +48,7 @@ Every other column becomes a property of the node or edge on that row. The colum | timestamp | DateTime | | list of the types above | List | -`INT32` and `FLOAT` columns are rejected, so cast them to `int64` / `float64` first. To load an `INT64` column of microseconds as a `Duration`, name it in `WITH DURATIONS []`. +Use 64-bit columns: cast `INT32` and `FLOAT` columns to `int64` / `float64` first. To load an `INT64` column of microseconds as a `Duration`, name it in `WITH DURATIONS []`. ## From pandas @@ -67,7 +67,7 @@ nodes = pd.DataFrame({ "__labels": [["Person"]] * len(people), "email": people["email"], "name": people["name"], - "age": people["age"].astype("Int64"), # int64, never int32 + "age": people["age"].astype("Int64"), # int64 }) edges = pd.DataFrame({ "__source": knows["src"].map(ids).astype("int64"), diff --git a/pythonsdk/reference.mdx b/pythonsdk/reference.mdx index 99e47a6..32ba583 100644 --- a/pythonsdk/reference.mdx +++ b/pythonsdk/reference.mdx @@ -76,18 +76,18 @@ USE_TURING_PROTO=1 turingdb start -demon -p 6666 client = TuringDB(type="native", host="localhost", port=6666) ``` -A server speaks either JSON or the binary protocol, never both on the same port: +A server speaks either JSON or the binary protocol on a given port: | Server started | `type="json"` client | `type="native"` client | | -------------- | -------------------- | ---------------------- | -| normally (JSON, the default) | works | fails with `Proto header dataLen does not match chunk payload size` | -| with `USE_TURING_PROTO=1` | fails with `Invalid response from the server` | works | +| normally (JSON, the default) | works | returns `Proto header dataLen does not match chunk payload size` | +| with `USE_TURING_PROTO=1` | returns `Invalid response from the server` | works | -A server in binary mode is therefore **binary-only**: `curl`, the JSON client and the browser visualizer (`-ui`) stop working against it. If you need both, run two servers on different ports. +A server in binary mode serves the binary protocol only, so `curl`, the JSON client and the browser visualizer (`-ui`) connect to a server running in the default mode. To use both, run two servers on different ports. -- `host` accepts `"localhost"` together with `port=`, or a full URL such as `"http://localhost:6666"`. `"localhost:6666"` without a scheme is not understood. +- `host` accepts `"localhost"` together with `port=`, or a full URL such as `"http://localhost:6666"`. Use one of these two forms. - The query API is identical, and results come back as the same DataFrames. Embeddings are returned as numpy `float32` arrays instead of Python lists. -- `list_available_graphs()` is not available. Use the JSON client to enumerate the graphs on disk, or `list_loaded_graphs()`. +- To list the graphs on disk, use the JSON client. On the native backend, `list_loaded_graphs()` lists the loaded graphs. - `token`, `reconnect()` and the S3 helpers work as on the JSON backend. ### Authentication @@ -102,7 +102,7 @@ TURINGDB_AUTH_TOKEN=s3cret turingdb start -demon -auth-on client = TuringDB(host="http://localhost:6666", token="s3cret") ``` -A missing or wrong token gets an HTTP `401`, which the JSON client raises as an `httpx.HTTPStatusError` (not a `TuringDBException`). The embedded backend runs in-process and is unauthenticated: passing `token=` to it is an error. +A missing or incorrect token returns HTTP `401`, which the JSON client raises as an `httpx.HTTPStatusError`. The embedded backend runs in-process and needs no authentication, so it does not take a `token`. ### The embedded backend @@ -113,9 +113,9 @@ client = TuringDB(type="embedded", data_dir=os.path.expanduser("~/.turing")) ``` - `data_dir` is the TuringDB root directory (the one that holds `graphs/`, `data/` and `vector/`). Omit it to use `~/.turing`. -- **Give it an absolute path.** A leading `~` is not expanded, so use `os.path.expanduser("~/...")`. +- **Use an absolute path**, for example with `os.path.expanduser("~/...")`, since a leading `~` is not expanded. - Writes still persist to disk when you submit a change, so your work survives the process ending. -- The embedded backend does **not** support the browser visualizer, `list_available_graphs()`, concurrent access from several processes, the S3 helpers, or `token=`. Use a server for those. +- The embedded backend is designed for a single process. For the browser visualizer, `list_available_graphs()`, concurrent access from several processes, the S3 helpers and `token=`, use a server. `list_available_graphs()` is only supported on the default JSON backend. The S3 helpers (`s3_connect`, `transfer`) work on the JSON and native backends. @@ -192,7 +192,7 @@ change = client.new_change() `new_change()` raises if the client is already on a change or on a past commit. Call `client.checkout()` first. - There is no `submit()` method. Submit a change by running `client.query("CHANGE SUBMIT")`, then call `client.checkout()` to return to main. Until you do, the client still points at the submitted change and the next query fails with `CHANGE_NOT_FOUND`. + Submit a change by running `client.query("CHANGE SUBMIT")`, then call `client.checkout()` to return to main. The client points at the submitted change until you do, so call `checkout()` before the next query. ## Version Control Helpers diff --git a/query/aggregation.mdx b/query/aggregation.mdx index bf73fe6..0eb4121 100644 --- a/query/aggregation.mdx +++ b/query/aggregation.mdx @@ -140,12 +140,12 @@ ORDER BY c DESC | `avg(x)` | Always a Double | | `min(x)`, `max(x)` | Numbers, strings, booleans, `DateTime` and `Duration` | -There are no statistical aggregates such as `stdev` or `percentileCont`. Compute those in your application, for example in pandas. +Statistical aggregates such as `stdev` and `percentileCont` are not yet available. Compute them in your application, for example in pandas. **Rules:** - `WHERE` runs before aggregation, so to filter on an aggregate, compute it in a `WITH` first and filter on its alias (see the example below). -- Aggregates cannot be nested: `count(count(*))` is an error. +- To aggregate an aggregate, compute the first one in a `WITH` and aggregate the result in the next stage. - Aggregates can be combined with each other and with constants in one expression (`max(n.age) - min(n.age)`, `count(*) * 10`). To use a property in the same expression, return it as its own item so it is a grouping key: `RETURN n.age, n.age + count(*)` works. - With no grouping keys, an empty input still returns one row: `count` is `0`, `sum` is `0`, `avg` is `null` and `collect` is `[]`. With grouping keys, an empty input returns no rows. @@ -186,9 +186,9 @@ RETURN t, count(*) `range(start, end[, step])` includes both bounds. Unwound nodes stay nodes, so `p.name` works after `UNWIND ns AS p`. You can also `UNWIND` a list of maps and read their keys: `UNWIND [{a: 1}, {a: 2}] AS m RETURN m.a`. - - Query parameters (`UNWIND $list`) are not yet supported. Write the list into the query text instead, for example `f"UNWIND {ids} AS x ..."` in Python. - + + Write the list into the query text, for example `f"UNWIND {ids} AS x ..."` in Python. Query parameters such as `UNWIND $list` are not yet supported. + ## Lists diff --git a/query/cheatsheet.mdx b/query/cheatsheet.mdx index c45906e..134d321 100644 --- a/query/cheatsheet.mdx +++ b/query/cheatsheet.mdx @@ -41,7 +41,7 @@ RETURN n.name - Node labels use `:Label` and can be stacked: `(n:Person:Employee)`. - Edge types use `:EDGE_TYPE`: `-[:KNOWS]->`. An edge has exactly one type, and `[:A|B]` matches either. - `RETURN n` gives the internal node ID. Return properties explicitly: `RETURN n.name`. -- Unknown labels and properties do not error. They match nothing or read as `null`. +- Unknown labels and properties are forgiving: they match nothing or read as `null`. ## OPTIONAL MATCH @@ -94,8 +94,8 @@ RETURN p.name - Comparison: `= <> != < <= > >=`, null tests `IS NULL` / `IS NOT NULL`, boolean `AND OR XOR NOT`. - Strings and lists: `IN`, `STARTS WITH`, `ENDS WITH`, `CONTAINS`, concatenation with `+` or `||`. -- No `=~` regex and no `exists(n.prop)` (use `IS NOT NULL`). -- `Integer = Double` and `Double = Double` comparisons are rejected. Use ranges or `toInteger`. +- Use `IS NOT NULL` to test for a property (`=~` and `exists(n.prop)` are not yet supported). +- For Doubles, use ranges (`n.score > 1.99 AND n.score < 2.01`) or convert with `toInteger`. ## Lookup by native node ID @@ -121,7 +121,7 @@ RETURN n.name ``` - Compare the **variable** (`n = 0`), not `id(n) = 0`. The first is a direct seek and the second scans every node. -- For batches, `UNWIND` a literal list of distinct IDs and do not return `x`. +- For batches, `UNWIND` a literal list of distinct IDs and return `id(n)` rather than `x`. ## Joins and Cartesian products @@ -168,7 +168,7 @@ RETURN [n IN nodes(p) | n.name], length(p) ``` - Postfix `->*` is **0** or more hops. Bracket `-[:KNOWS*]->` is **1** or more. -- Edges never repeat within a path. Bound the length on large graphs. +- Each edge is used at most once per path. Bound the length on large graphs. ## WITH, ORDER BY, SKIP, LIMIT, UNION @@ -214,8 +214,8 @@ RETURN type(r) AS t, count(*) AS c ORDER BY c DESC ``` -- Grouping is implicit. Aggregates are not allowed in `WHERE`: filter after a `WITH`. -- No `stdev` or `percentile*` aggregates. +- Grouping is implicit. To filter on an aggregate, compute it in a `WITH` and filter afterwards. +- `stdev` and `percentile*` aggregates are not yet available. ## UNWIND, lists, CASE @@ -300,7 +300,7 @@ RETURN n.title, cosine_similarity(n.emb, (0.4, 0.3, 0.8, 0.1)) ``` - Function names are case-sensitive, except `count` and `collect`. -- `edgeType()` was removed: use `type(e)`. +- Use `type(e)` for an edge's type (it replaces `edgeType()`). ## Data types @@ -351,7 +351,7 @@ DELETE e CHANGE SUBMIT ``` -- Every node needs at least one label and every edge exactly one type. Labels cannot be changed later. +- Every node needs at least one label and every edge exactly one type. Labels are set at creation. - Run `COMMIT` after creating entities that a later query must match. `CHANGE SUBMIT` commits for you. - In the Python SDK, call `client.checkout()` after `CHANGE SUBMIT`. - For more than a few thousand entities, use [`LOAD PARQUET`](/import_data/parquet). @@ -404,11 +404,3 @@ See [Shortest path](/shortest-path) and [Vector search](/vector-search). | `SHOW VECTOR INDEXES` / `DELETE VECTOR INDEX ` | List / delete vector indexes | | `INSTALL ` / `SHOW EXTENSIONS` / `SHOW PROCEDURES` | Load a procedure extension / list extensions / list procedures | | `EXPLAIN ` | Show the query plan without running it | - -## Not supported - -- Query parameters (`$name`), more than one statement per request, `--` and `/* */` comments. -- `=~` regular expressions, `FOREACH`, `COLLECT { }`, `CREATE CONSTRAINT`. -- Label expressions `(n:A|B)` and `(n:!A)`. Use `WHERE n:A OR n:B`. -- `SET n:Label`, `REMOVE n:Label`, `SET n += {...}`, `SET n = {...}`. -- `properties()`, `keys()`, `elementId()`, map projection `n {.name}`. diff --git a/query/cypher_subset.mdx b/query/cypher_subset.mdx index 86165f1..7f57456 100644 --- a/query/cypher_subset.mdx +++ b/query/cypher_subset.mdx @@ -45,9 +45,9 @@ Keywords are case-insensitive (`MATCH` and `match` are the same). Labels, edge t MATCH (n:Person) RETURN n.name // also fine at the end of a line ``` - - SQL-style `-- comments` and `/* block comments */` are **not** supported. `--` is parsed as part of the query. - + + Use `//` for comments. SQL-style `--` and `/* */` comments are not yet supported. + ### Running a query @@ -64,21 +64,21 @@ df = client.query("MATCH (n:Person) RETURN n.name, n.age") # a pandas DataFram `EXPLAIN ` returns the compiled query plan without running the query. It is the easiest way to check that a lookup uses a constant node scan. -## What to know if you come from Neo4j - -TuringDB follows openCypher closely. These are the differences you are most likely to hit: - -- **`RETURN n` returns the node's internal ID**, an integer, not a node object with all its properties. Return the properties you need: `RETURN n.name, n.age`. There is no `properties(n)` or `keys(n)`; use [`db.getNodes`](/query/procedures#inspecting-nodes-and-edges) to see every property of a few nodes. -- **No query parameters.** `$name` is a parse error and `client.query()` only takes a string. Put literal values in the query text and escape quotes yourself. -- **One statement per request.** Scripts separated by `;` are rejected (a single trailing `;` is fine). -- **Labels are fixed at creation.** `SET n:Label` and `REMOVE n:Label` are not supported. Every node has at least one label, and every edge has exactly one edge type. -- **Property types are global.** A property name has a single value type across the whole graph. If `age` is an Integer, you cannot store a String `age` on another node. -- **Strict numeric equality.** Comparing an Integer with a Double using `=` (for example `n.age = 30.0`) is rejected. Use a range (`n.score > 1.99 AND n.score < 2.01`) or convert with `toInteger`. `<`, `>` and arithmetic across numeric types are fine. -- **Unknown labels and properties do not raise an error.** A label that does not exist matches nothing, and a misspelled property reads as `null`. Check names with [`CALL db.labels()` and `CALL db.propertyTypes()`](/query/procedures). -- **Label expressions such as `(n:A|B)` or `(n:!A)` are not supported.** Write `WHERE n:A OR n:B` or `WHERE NOT n:A` instead. Edge type alternatives (`[:A|B]`) are supported. -- **No regular expressions.** `=~` is a parse error. Use `STARTS WITH`, `ENDS WITH` and `CONTAINS`. -- **No `FOREACH` and no constraints.** Use `UNWIND` for loops and `MERGE` to avoid duplicates. -- **Shortest paths use a TuringDB statement**, not `shortestPath((a)-[*]-(b))`. See [Shortest path](/shortest-path). +## Coming from Neo4j + +TuringDB follows openCypher closely, so the queries you know carry over. These are the TuringDB specifics worth knowing: + +- **`RETURN n` returns the node's internal ID**, an integer. Return the properties you need with `RETURN n.name, n.age`, or use [`db.getNodes`](/query/procedures#inspecting-nodes-and-edges) to see every property of a few nodes. +- **Values go in the query text.** Queries are plain strings, so put literal values directly in the text and escape quotes in your application. Query parameters such as `$name` are not yet supported. +- **One statement per request.** Send each statement as its own request (a single trailing `;` is fine). +- **Labels are set at creation.** Every node has at least one label and every edge exactly one edge type, chosen when it is created. +- **Property types are global.** A property name has a single value type across the whole graph: if `age` is an Integer, it is an Integer on every node. +- **Typed numeric equality.** Compare Integers with Integers, and match Doubles with a range (`n.score > 1.99 AND n.score < 2.01`) or convert with `toInteger`. `<`, `>` and arithmetic work across numeric types. +- **Forgiving names.** An unknown label matches nothing and a misspelled property reads as `null`. Check names with [`CALL db.labels()` and `CALL db.propertyTypes()`](/query/procedures). +- **Label alternatives.** Write `WHERE n:A OR n:B` or `WHERE NOT n:A`. Edge type alternatives (`[:A|B]`) work directly in patterns. +- **String matching.** Use `STARTS WITH`, `ENDS WITH` and `CONTAINS`. Regular expressions (`=~`) are not yet supported. +- **Loops and uniqueness.** Use `UNWIND` for loops and `MERGE` to avoid duplicates. +- **Shortest paths** use a TuringDB statement, see [Shortest path](/shortest-path). - **Function names are case-sensitive** (`toInteger`, not `TOINTEGER`), except `count` and `collect`. @@ -109,4 +109,4 @@ More detail on types, conversion and results is in [Functions and data types](/q ## Roadmap -We keep extending Cypher coverage and growing the library of procedures available through `CALL`. If you need a query form that is not supported yet, please [tell us](/troubles/troubles#contact-us). +We keep extending Cypher coverage and growing the library of procedures available through `CALL`. If there is a query form you would like to see, please [tell us](/troubles/troubles#contact-us). diff --git a/query/functions.mdx b/query/functions.mdx index 834df71..cec5adf 100644 --- a/query/functions.mdx +++ b/query/functions.mdx @@ -43,9 +43,9 @@ MATCH (n:Person) RETURN n.name, coalesce(n.nickname, n.name) ``` - - **`edgeType()` was removed in v3.** Use `type(e)` to get an edge's type as a string. - + + In v3, `type(e)` returns an edge's type as a string. It replaces `edgeType()` from earlier versions. + The function library keeps growing. Some functions, such as `substring`, `split`, `rand()`, `keys` and `properties`, are not yet available. For now, filter strings with `STARTS WITH`, `ENDS WITH` and `CONTAINS`, and do any other post-processing in your application. @@ -123,9 +123,9 @@ RETURN m.a You can read a key from a map **value** that is in scope, for example one bound by `WITH` or `UNWIND`, with `m.key`. - - A map **stored as a property** cannot be dereferenced inside a query: `n.meta.level` and `m['key']` are errors, and map projection (`n {.name, .age}`) is not supported. Return the property as a whole, or store the individual values as separate properties. - + + To read individual values from a map stored as a property, return the property as a whole or store the values as separate properties. Dereferencing a stored map inside a query (`n.meta.level`, `m['key']`) and map projection (`n {.name, .age}`) are not yet supported. + ## How results come back diff --git a/query/matching.mdx b/query/matching.mdx index b35f907..6caaf37 100644 --- a/query/matching.mdx +++ b/query/matching.mdx @@ -4,7 +4,7 @@ sidebarTitle: "Matching" description: "Read data with MATCH, OPTIONAL MATCH and WHERE, join patterns, and look up nodes quickly by native node ID." --- -Read queries run directly against the current graph. There is no change or transaction to open first. +Read queries run directly against the current graph, with no change to open first. The examples on this page use a small graph of people, companies and interests. Create it once in a change (see [Writing data](/query/writing)) if you want to follow along: @@ -93,7 +93,7 @@ RETURN n.name ``` - Unknown labels and property names do **not** raise an error. `MATCH (n:Nope) RETURN n` returns no rows and `RETURN n.misspelled` returns `null`. Use [`CALL db.labels()`, `CALL db.edgeTypes()` and `CALL db.propertyTypes()`](/query/procedures) to check what exists in a graph. + Unknown labels and property names are forgiving: `MATCH (n:Nope) RETURN n` returns no rows and `RETURN n.misspelled` returns `null`. Use [`CALL db.labels()`, `CALL db.edgeTypes()` and `CALL db.propertyTypes()`](/query/procedures) to check what exists in a graph. ## OPTIONAL MATCH @@ -175,13 +175,13 @@ RETURN p.name Property constraints in a pattern, `MATCH (n {name: 'Matt', age: 20})`, are exactly equivalent to the same equality tests in `WHERE`. - - Regular expressions (`=~`) and the `exists(n.prop)` function are not yet supported. Use `n.prop IS NOT NULL` to test for a property. - + + Use `n.prop IS NOT NULL` to test for a property, and `STARTS WITH`, `ENDS WITH` and `CONTAINS` to match strings. Regular expressions (`=~`) and `exists(n.prop)` are not yet supported. + -### Numeric equality is strict +### Comparing numbers -Comparing an Integer with a Double using `=` is rejected when the query is analysed, and so is comparing two Doubles with `=`: +Equality compares numbers of the same kind. An Integer compared with a Double using `=`, or two Doubles compared with `=`, is reported when the query is analysed: ```text MATCH (n:Person) WHERE n.age = 30.0 RETURN n.name @@ -200,7 +200,7 @@ WHERE n.age = toInteger(30.0) RETURN n.name ``` -Mixed-type `<`, `>` and arithmetic are fine. Integer division by zero raises an error, and Int64 overflow wraps silently. +Mixed-type `<`, `>` and arithmetic work as expected. Division by zero returns an execution error, and Int64 arithmetic wraps around on overflow. ## Joins and Cartesian products @@ -274,7 +274,7 @@ On large graphs this is orders of magnitude faster than looking nodes up again t - its elements are **distinct** integers, so deduplicate first; - `x` is used **only** in `n = x`. Return `id(n)` rather than `x`. - **Fetch IDs once and reuse them.** Look nodes up with whatever filter you need (`MATCH (n:Person {name: 'Alice'}) RETURN id(n)`), keep the IDs in your application, and address the nodes by ID from then on. -- **Never compute IDs per row, and never match two nodes from one row.** Forms such as `WHERE a = p[0] AND b = p[1]` are not rewritten and run as a Cartesian product of two full node scans, which can exhaust memory on large graphs. To connect two known nodes, put both IDs in as literals, one pair per query: `MATCH (a) WHERE a = 5 MATCH (b) WHERE b = 7`. For bulk edges, see [Parquet import](/import_data/parquet). +- **Keep IDs constant.** Use literal IDs rather than IDs computed per row, and match one node per `MATCH`. Forms such as `WHERE a = p[0] AND b = p[1]` are not rewritten into a constant scan, so they run as a Cartesian product of two full node scans, which is expensive on large graphs. To connect two known nodes, put both IDs in as literals, one pair per query: `MATCH (a) WHERE a = 5 MATCH (b) WHERE b = 7`. For bulk edges, see [Parquet import](/import_data/parquet). - **Vector search results are already nodes.** `VECTOR SEARCH ... YIELD ids` binds `ids` as nodes, so use `ids.title` or `MATCH (ids)-[...]->(...)` directly. See [Vector search](/vector-search). **Stability.** The native IDs of existing nodes are stable across queries, change submits and server restarts, so it is safe to cache them. Two exceptions: @@ -282,4 +282,4 @@ On large graphs this is orders of magnitude faster than looking nodes up again t - Nodes created in an open change get their final IDs at `COMMIT` or `CHANGE SUBMIT`. Read them after that. - [`MERGE_DATAPARTS`](/concepts/dataparts) renumbers all nodes. Re-fetch any cached IDs afterwards. -Keep a business key as a property as well, for durable identity outside TuringDB. There is no `elementId()`. For edges, `MATCH ()-[r]->() WHERE id(r) = 0 RETURN startNode(r), endNode(r)` works. +Keep a business key as a property as well, for durable identity outside TuringDB. Use `id(n)` for node IDs. For edges, `MATCH ()-[r]->() WHERE id(r) = 0 RETURN startNode(r), endNode(r)` works. diff --git a/query/paths.mdx b/query/paths.mdx index 767a94c..6505a3d 100644 --- a/query/paths.mdx +++ b/query/paths.mdx @@ -40,9 +40,9 @@ RETURN b.name, size(e) - **Edge type alternatives** (`[:KNOWS|WORKS_AT]`) work on variable-length hops. - **A predicate or property map applies to every hop.** `-[e:KNOWS WHERE e.since > 2012]->+` only follows edges that satisfy it, and `-[:KNOWS*1..3 {since: 2010}]->` requires every edge on the path to have that property value. -- **Trail semantics.** An edge never repeats within one path, but nodes may. Each distinct path is its own row, so use `DISTINCT` or an aggregate to collapse them. -- **One quantifier per edge.** You cannot combine `[*1..3]` with `{1,3}` on the same edge. -- **The edge variable of a variable-length hop is a list of edge IDs.** `size(e)`, `e[0]`, `e[-1]` and `UNWIND e` work, but `e.since` does not. To read edge properties along a path, name the path and use `relationships(p)` (below). +- **Trail semantics.** Each edge is used at most once within a path, while nodes may repeat. Each distinct path is its own row, so use `DISTINCT` or an aggregate to collapse them. +- **One quantifier per edge.** Choose either `[*1..3]` or `{1,3}` for an edge. +- **The edge variable of a variable-length hop is a list of edge IDs.** `size(e)`, `e[0]`, `e[-1]` and `UNWIND e` work on it. To read edge properties along a path, name the path and use `relationships(p)` (below). Variable-length matching **enumerates** paths. On dense graphs the number of paths can grow very quickly, so always bound the length (`*1..6` or `{1,6}`). For weighted shortest paths on large graphs, use the [`shortestPath` statement](/shortest-path). @@ -77,7 +77,7 @@ ORDER BY hops - `nodes(p)` and `relationships(p)` return **lists of IDs**. Read properties through a [list comprehension](/query/aggregation#list-comprehensions). - `length(p)` is the number of hops. - A bare `RETURN p` comes back as a list of `{"type": "node" | "edge", "id": ...}` dictionaries. -- `count(p)` works. `collect(p)` does not. +- `count(p)` counts paths. - `OPTIONAL MATCH p = ... RETURN p IS NULL` works. ## Finding routes @@ -110,7 +110,7 @@ ORDER BY total LIMIT 1 ``` - Summing directly over `UNWIND relationships(p) AS r` and then `sum(r.prop)` fails with an execution error. Compute the list with a comprehension first, as above. + To sum a property along a route, compute the list with a comprehension first, as above. For a true weighted shortest path, see [Shortest path](/shortest-path). diff --git a/query/procedures.mdx b/query/procedures.mdx index 5ad1963..095b5ee 100644 --- a/query/procedures.mdx +++ b/query/procedures.mdx @@ -57,14 +57,14 @@ CALL db.history() YIELD `commit`, nodeCount RETURN `commit`, nodeCount ``` -- **Constant arguments only.** A procedure argument cannot read row values: `db.getNodes([id(n)])` fails with "must be constant". +- **Constant arguments.** Procedure arguments are constants, so pass literal values (for example `db.getNodes([0, 1, 2])`) rather than values read from a row. - **`commit` is a keyword.** Yield it with backticks: ``YIELD `commit` ``. -- `valueType` has its own enum type, so `WHERE valueType = 'String'` is rejected. Filter the DataFrame in pandas instead. -- `OPTIONAL CALL` does not work on procedures. +- To filter on `valueType`, filter the resulting DataFrame in pandas. +- `OPTIONAL CALL` is for subqueries. It is not yet available for procedures. ### Inspecting nodes and edges -`RETURN n` gives only an internal ID, and there is no `properties(n)`. To see every property of a few nodes, use `db.getNodes`, which returns the properties as a JSON string: +`RETURN n` gives a node's internal ID. To see every property of a few nodes, use `db.getNodes`, which returns the properties as a JSON string: ```cypher CALL db.getNodes([0, 1, 2]) YIELD id, labels, properties @@ -115,7 +115,7 @@ Outside of Cypher, a set of commands controls the engine. Send them through `cli | `LOAD CSV '' ... AS row` | Stream rows from a CSV file, see [CSV import](/import_data/csv) | | `EXPLAIN ` | Show the compiled plan without running the query | -There is no command to drop or unload a graph. +Dropping and unloading graphs from a running server is not yet available. ### Changes and versions diff --git a/query/subqueries.mdx b/query/subqueries.mdx index ce62281..998daf0 100644 --- a/query/subqueries.mdx +++ b/query/subqueries.mdx @@ -87,9 +87,9 @@ ORDER BY nm ``` - **Dropped rows.** A non-optional `CALL` drops the outer rows for which its body returns nothing. Use `OPTIONAL CALL` to keep them with `null` values. -- **Imported variables.** A `CALL` body cannot return an imported variable under its own name. Alias it: `RETURN p AS me`. +- **Imported variables.** To return an imported variable from a `CALL` body, give it an alias: `RETURN p AS me`. - A `CALL` body can contain `UNION`, aggregation, `ORDER BY` and `LIMIT`. Write-only subqueries also work inside a [change](/query/writing). - - Not supported: `COLLECT { ... }` (use `collect()` in a `CALL` body or a [pattern comprehension](/query/aggregation#list-comprehensions) instead), `CALL (*) { }`, `CALL { } IN TRANSACTIONS`, and `OPTIONAL CALL` on a *procedure*. - + + `COLLECT { ... }`, `CALL (*) { }`, `CALL { } IN TRANSACTIONS` and `OPTIONAL CALL` on a procedure are not yet available. To collect values, use `collect()` in a `CALL` body or a [pattern comprehension](/query/aggregation#list-comprehensions). + diff --git a/query/writing.mdx b/query/writing.mdx index 18181b1..842eef9 100644 --- a/query/writing.mdx +++ b/query/writing.mdx @@ -4,7 +4,7 @@ sidebarTitle: "Writing Data" description: "CREATE, MERGE, SET, REMOVE and DELETE in TuringDB, and the change workflow that every write goes through." --- -All writes (`CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`) must run **inside a change**. A change is TuringDB's equivalent of a git branch: you write in an isolated copy and then submit it to the main graph. A write sent outside a change fails with an explicit error such as `Cannot perform CREATE outside of a write transaction.` +All writes (`CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`) must run **inside a change**. A change is TuringDB's equivalent of a git branch: you write in an isolated copy and then submit it to the main graph. A write sent outside a change returns a clear message such as `Cannot perform CREATE outside of a write transaction.` ## The change workflow @@ -19,7 +19,7 @@ client.checkout() # return the client to main ``` - `new_change()` returns the change ID and puts the client on that change. -- After `CHANGE SUBMIT` the client still points at the submitted change. Call `client.checkout()` to return to main, otherwise the next query fails with `CHANGE_NOT_FOUND`. +- After `CHANGE SUBMIT` the client still points at the submitted change. Call `client.checkout()` to return to main before running the next query. - To abandon a change, run `CHANGE DELETE` and then `client.checkout()`. - `new_change()` raises if the client is already on a change or on a past commit. Call `checkout()` first. @@ -31,7 +31,7 @@ See [Create a change](/graph_dev/create_change), [Submit a change](/graph_dev/su | An earlier query in the same change did... | Visible to later queries before `COMMIT`? | | ------------------------------------------ | ----------------------------------------- | -| `CREATE` nodes or edges | **No.** A later `MATCH` finds nothing, a `MATCH ... CREATE` edge silently creates nothing, and `MERGE` creates duplicates | +| `CREATE` nodes or edges | **After `COMMIT`.** Run `COMMIT` so that a later `MATCH`, `MATCH ... CREATE` or `MERGE` sees the new entities | | `SET`, `REMOVE` or `DELETE` on entities that already existed | Yes, immediately | | anything, followed by `COMMIT` | Yes | | writes earlier in the **same query** | Yes | @@ -49,9 +49,9 @@ client.checkout() ### Failed queries and conflicts -- **A failed query can leave partial writes in the change.** After an execution error in the middle of a change, the safest recovery is `CHANGE DELETE` and redoing the work. +- **Recovering from a failed query.** If a query stops with an execution error in the middle of a change, the writes it already made stay in the change. The safest recovery is `CHANGE DELETE`, then redoing the work. - **Changes are isolated.** Main does not see a change's writes, even committed ones, until the change is submitted. Each change is rebased onto main at submit time. -- **Disjoint edits submit cleanly.** If two changes edit the same entity, the second `CHANGE SUBMIT` fails with an error such as `This change attempted to update Node 0 (...) which has been modified on main.` Conflict detection is coarse: creating an edge to a node that was modified on main also conflicts. +- **Disjoint edits submit cleanly.** When two changes edit the same entity, the second `CHANGE SUBMIT` reports a conflict, for example `This change attempted to update Node 0 (...) which has been modified on main.` Conflict detection is coarse: creating an edge to a node that was modified on main also conflicts. - **A failed submit rejects the whole change but leaves it open.** You can fix it and submit again, or run `CHANGE DELETE`. - **IDs of newly created nodes and edges are provisional until `COMMIT` or `CHANGE SUBMIT`.** Read them after committing. IDs of existing nodes are stable across submits and restarts (see [node ID lookup](/query/matching#look-up-nodes-by-native-node-id)). @@ -101,15 +101,15 @@ CREATE (a)-[:KNOWS {since: 2021}]->(b) **Rules:** -- **Labels and edge types.** Every node needs at least one label, and every edge exactly one edge type. `CREATE (n)` is invalid, and so is `[:A:B]` on an edge. -- **Labels are fixed at creation.** `SET n:Label` and `REMOVE n:Label` are not supported. -- **Property types are global.** Each property name has **one value type across the whole graph** (see `CALL db.propertyTypes()`). If `age` is an Integer, `CREATE (:X {age: 'old'})` and `SET n.age = 2.5` both fail with a type incompatibility error. There is no Integer to Double widening. -- **Nulls.** `{name: null}` is rejected in `CREATE`. Leave the key out instead. -- **No variable-length edges in write patterns**, and you cannot reuse a matched edge variable in `CREATE`. +- **Labels and edge types.** Create nodes with a label, `CREATE (n:Label)`, and give each edge a single type, `[:TYPE]`. +- **Labels are set at creation** and cannot be changed afterwards. +- **Property types are global.** Each property name has **one value type across the whole graph** (see `CALL db.propertyTypes()`). If `age` is an Integer, `CREATE (:X {age: 'old'})` and `SET n.age = 2.5` are both reported as type incompatibilities, and values are not widened from Integer to Double automatically. +- **Nulls.** Leave a key out of `CREATE` when it has no value, rather than writing `{name: null}`. +- **Write patterns use single hops.** Variable-length edges are for reading, and a matched edge variable is not reused in `CREATE`. - **Parallel edges and self-loops are allowed.** An undirected `CREATE (a)-[:R]-(b)` is accepted and creates `a->b`. - Never take both endpoints of an edge from one `UNWIND` row (`a = p[0] AND b = p[1]`). It plans as a Cartesian product of two full node scans and can run the server out of memory. For thousands of edges use [`LOAD PARQUET`](/import_data/parquet). + Match the two endpoints of an edge with separate `MATCH` clauses and constant IDs. Taking both from one `UNWIND` row (`a = p[0] AND b = p[1]`) plans as a Cartesian product of two full node scans, which is expensive on large graphs. For thousands of edges, use [`LOAD PARQUET`](/import_data/parquet). ## MERGE @@ -140,9 +140,9 @@ RETURN t.k `MERGE` deduplicates within one query, so the last example creates two `Key` nodes, not three. - - `MERGE` **cannot see nodes created by an earlier uncommitted query in the same change**, so it will create duplicates. Run `COMMIT` first. There are no uniqueness constraints, so `MERGE` is the way to avoid duplicates. - + + Run `COMMIT` before a `MERGE` that should find nodes created by an earlier query in the same change, otherwise `MERGE` creates them again. `MERGE` is the way to avoid duplicates. + ## SET and REMOVE @@ -207,9 +207,9 @@ REMOVE n.score, n.flag - Address nodes by [native node ID](/query/matching#look-up-nodes-by-native-node-id) for the fastest updates. Batch updates use the same literal `UNWIND` form as reads. - For a one-off embedding, `SET n.emb = (...)` is fine. To write embeddings onto many existing nodes, use [`LOAD EMBEDDING`](/vector-search#embeddings-as-node-properties) instead of a `SET` loop. - - **Not supported:** `SET n += {...}` ("SET cannot dynamically mutate properties yet"), `SET n = {...}`, `SET n:Label` and `REMOVE n:Label`, and aggregates directly inside `SET` (`SET n.c = count(*)`). Compute the aggregate in a `WITH` first, as in the `Stats` example above. - + + To update several properties at once, list the assignments: `SET n.a = 1, n.b = 2`. Map forms (`SET n += {...}`, `SET n = {...}`), label changes (`SET n:Label`) and aggregates directly inside `SET` (`SET n.c = count(*)`) are not yet supported. Compute aggregates in a `WITH` first, as in the `Stats` example above. + ## DELETE and DETACH DELETE @@ -229,8 +229,8 @@ DELETE e, n ``` - `DETACH DELETE` removes a node together with all of its edges. -- A plain `DELETE` of a node that still has edges fails with `Cannot delete a node with relationships; use DETACH DELETE`. -- Deleting a path variable (`DELETE p`) is not supported. Delete its nodes or edges instead. +- Use `DETACH DELETE` for a node that still has edges. A plain `DELETE` reports `Cannot delete a node with relationships; use DETACH DELETE`. +- To delete a path, delete its nodes or edges. ## Bulk writes @@ -251,7 +251,7 @@ UNWIND [['Brest', 'FR-29'], ['Rennes', 'FR-35']] AS row CREATE (:City {name: row[0], code: row[1]}) ``` -A `CREATE` cannot read an element of a mixed-type list row such as `['Brest', 139000]`. Use parallel lists instead: +For rows with mixed types, such as `['Brest', 139000]`, use parallel lists: ```cypher WITH ['Brest', 'Rennes'] AS names, [139000, 220000] AS pops @@ -268,9 +268,9 @@ LOAD CSV 'knows.csv' WITH HEADERS AS row CREATE (a)-[:KNOWS {since: toInteger(row.since)}]->(b) ``` -`FOREACH` is not supported. Use `UNWIND`. +Use `UNWIND` to loop over a list. -**Building the query text in Python.** Because there are no parameters, escape quotes yourself: +**Building the query text in Python.** Queries are plain text, so escape quotes in your application: ```python def lit(v): diff --git a/shortest-path.mdx b/shortest-path.mdx index 01501b8..c47037f 100644 --- a/shortest-path.mdx +++ b/shortest-path.mdx @@ -7,7 +7,7 @@ description: "Find the weighted shortest path between nodes in a TuringDB graph TuringDB's `shortestPath` statement runs a Dijkstra search from a source set of nodes to a target set of nodes, returning the shortest distance and the path taken based on a numeric edge weight property. - `shortestPath` is a TuringDB statement, not the openCypher function. The forms `shortestPath((a)-[*]-(b))`, `allShortestPaths(...)`, `ANY SHORTEST` and `SHORTEST k` are not supported. For hop-count questions or paths where you need the properties along the route, use [variable-length paths](/query/paths) instead. + `shortestPath` is a TuringDB statement for weighted shortest paths. For hop-count questions, or when you need the properties along the route, use [variable-length paths](/query/paths). ## Sample dataset @@ -143,10 +143,10 @@ You can pass multiple nodes to the source or target set — `shortestPath` will If a set matches several source and target nodes, only the single best pair is returned. -## Limitations +## Usage notes - Every variable on a pattern going into `shortestPath` is consumed by the statement, so it cannot be returned. Only `dist` and `path` can follow `RETURN`. The following query fails, and so does returning a separately matched copy of an endpoint: + The variables in the pattern going into `shortestPath` are consumed by the statement, so `RETURN` takes `dist` and `path`. Asking for a pattern variable such as `n` is reported as an error: ```jsx MATCH (n{name:"Ashchurch"}), (m{name:"Worcestershire Parkway"}) @@ -159,9 +159,9 @@ If a set matches several source and target nodes, only the single best pair is r ``` -- **Directed only.** Only outgoing edges are followed. When the target cannot be reached from the source, the query returns **zero rows**, not an error. -- **`RETURN` must follow directly.** `WITH` after `shortestPath` is a syntax error. `ORDER BY` and `LIMIT` on the `RETURN` are fine. -- **Do not call path functions on `path`.** `nodes(path)`, `length(path)` and comprehensions over it are not supported. +- **Directed.** Outgoing edges are followed. When the target cannot be reached from the source, the query returns zero rows. +- **`RETURN` follows directly.** Put `ORDER BY` and `LIMIT` on that `RETURN`. +- **Reading `path`.** `path` is a flat list of IDs. To get names and properties, map the IDs back with a second query (see below). - **Where the other nodes come from.** A join can feed an endpoint: `MATCH (x {name: "Bromsgrove"})-[:LINK]->(a), (b {name: "Worcester Shrub Hill"}) shortestPath(a, b, distance, dist, path) RETURN dist, path`. ## Naming the stations along the route diff --git a/troubles/troubles.mdx b/troubles/troubles.mdx index bdbdb97..3a2cf45 100644 --- a/troubles/troubles.mdx +++ b/troubles/troubles.mdx @@ -42,7 +42,7 @@ See [client backends](/pythonsdk/reference#client-backends) for details. ## `401 Unauthorized` -The server was started with `-auth-on` and your client did not send the right token. Pass `token="..."` to `TuringDB(...)`, or set the `TURINGDB_AUTH_TOKEN` environment variable. The JSON client raises this as an `httpx.HTTPStatusError`. +The server was started with `-auth-on` and your client did not send a valid token. Pass `token="..."` to `TuringDB(...)`, or set the `TURINGDB_AUTH_TOKEN` environment variable. The JSON client raises this as an `httpx.HTTPStatusError`. --- @@ -58,7 +58,7 @@ All writes (`CREATE`, `MERGE`, `SET`, `REMOVE`, `DELETE`) must run inside a chan --- -## A `MATCH` after a `CREATE` finds nothing +## A `MATCH` right after a `CREATE` returns no rows Entities created earlier in a change are not visible to later queries until you run `COMMIT`. Run `COMMIT` after creating nodes and before matching them to create edges or to `MERGE`. See [Writing data](/query/writing#when-you-need-commit-inside-a-change). @@ -66,7 +66,7 @@ Entities created earlier in a change are not visible to later queries until you ## `Function 'X' does not exist` -Function names are case-sensitive (`toInteger`, not `TOINTEGER`), and some functions are not available in TuringDB. `edgeType()` was replaced by `type()` in v3. See [Functions and data types](/query/functions). +Function names are case-sensitive (`toInteger`, not `TOINTEGER`), and some functions are not yet available in TuringDB. In v3, `type()` replaces `edgeType()`. See [Functions and data types](/query/functions). --- diff --git a/vector-search.mdx b/vector-search.mdx index 48646ef..b276125 100644 --- a/vector-search.mdx +++ b/vector-search.mdx @@ -159,7 +159,7 @@ This is where it gets interesting. When the vector file is keyed by native node - **Use `ids` directly. Do not re-match it.** Writing `MATCH (d:Document) WHERE d.id = ids` (or `WHERE d = ids`) scans the whole label and cross-joins it with the results. Key the vector file by native node IDs, as above, so that no lookup is needed. If you must key the file by your own IDs, expect a label scan to join them back. + **Use `ids` directly.** `ids` is already the node, so read properties from it (`ids.title`) or expand from it (`MATCH (ids)-[...]->(...)`). Re-matching it with `MATCH (d:Document) WHERE d.id = ids` scans the whole label and cross-joins it with the results, so key the vector file by native node IDs, as above. If you key the file by your own IDs, joining them back needs a label scan. @@ -251,7 +251,7 @@ MATCH (n:Document) RETURN n.name, euclidean_distance(n.emb, (0.12, 0.45, 0.78, 0 ### Bulk-loading embeddings from Parquet — `LOAD EMBEDDING FROM` -To attach embeddings to **existing** nodes in bulk, load them from a Parquet file. This is a write, so run it inside a change. Do not loop `SET n.emb = (...)` over many nodes: it is orders of magnitude slower (about 2.5 ms per node, against roughly 0.1 s for 20,000 embeddings of 384 dimensions with `LOAD EMBEDDING`). +To attach embeddings to **existing** nodes in bulk, load them from a Parquet file. This is a write, so run it inside a change. `LOAD EMBEDDING` is the fastest way to write many embeddings: about 0.1 s for 20,000 embeddings of 384 dimensions, against about 2.5 ms per node for a loop of `SET n.emb = (...)`. **Syntax:** ``` @@ -287,10 +287,10 @@ client.query("CHANGE SUBMIT") client.checkout() ``` -- The embedding column must be a **fixed-size binary** column. A list or fixed-size-list column is rejected. +- The embedding column is a **fixed-size binary** column (`pa.binary(dim * 4)` above), rather than a list or fixed-size list. - The file can cover any subset of nodes. Only those nodes get the property. - Running it again with the same property name overwrites the stored vectors, which is how you refresh embeddings after a model change. -- The load **fails** if any `node_id` is missing from the graph, or if the property name already exists with a non-embedding type. +- Every `node_id` must exist in the graph, and the property name must be new or already an embedding property. When importing a graph from JSONL, you can mark embedding properties at load time instead. See [`LOAD JSONL ... WITH EMBEDDINGS`](/import_data/jsonl#loading-embedding-properties).