Crystallography on TuringDB

The intermolecular contact network of 84,805 crystal structures, stored as a graph and queryable in milliseconds.

COD and the CSD store atoms and coordinates. Hydrogen bonds, halogen bonds and molecular connectivity are recomputed geometrically every time somebody asks a question. This demo computes them once, at ingest, with the generating symmetry operation attached to every edge — so packing questions become pattern matches.

Built from the Crystallography Open Database (CC0): CrystEngComm, Crystal Growth & Design and Acta Cryst. E, the three journals closest to CCDC's small-molecule remit.

• TUTORIAL.md — the graph model, the queries, the UI, and how the data gets in. Start there.

---

Quick start

./run.sh --ingest     # first run: acquire COD and build the graphs
./run.sh              # thereafter: start TuringDB + the UI on the built data

The CIFs are not in this repository — they are 2.8 GB and belong to COD — so the first run has to be --ingest. It takes roughly 25 minutes.

Then open http://localhost:8087 for the browser studio, or query from a shell:

crystal facts                                       # corpus summary
crystal run dimer                                   # a named query
crystal query "MATCH (s:Structure) RETURN count(s)" # your own Cypher
crystal at 2009 "MATCH (s:Structure) RETURN count(s)"   # as the corpus stood then

Requires uv, Node 20+ and rsync. The TuringDB server binary ships inside the turingdb Python wheel, so there is nothing else to install. --ingest needs network access to crystallography.net.

---

The corpus

Structures	84,805
Atomic sites	4,694,045
Components (molecules & ions)	74,939
Covalent bonds	5,024,842
Intermolecular contacts	687,048
Space groups	214

The contact layer is three populations, never blended, because a query has to be able to refuse the weaker evidence:

	count	geometry
Strong H-bond, hydrogen located	152,433	mean H···A 2.048 Å at 160.4°
Inferred — no H refined, heavy-atom D···A fallback	114,456	42.9% of the H-bond layer
Weak C–H···A, hydrogen located	391,877	mean H···A 2.674 Å at 145.0°
Halogen bond	4,122	C–X···A > 150°, X···A under ΣvdW
close_contact — excluded from every H-bond statistic	24,160	implausible inferred donor

82.1% of structures are in a centrosymmetric space group.

Periodic-net dimensionality is scored for 40,320 structures (47.5%) — the rest have no located-hydrogen strong hydrogen bond and carry net_dim = -1. Any dimensionality percentage quoted here is over that scored subset, not the whole corpus.

Every figure on this page is re-derived from the live graph by python tests/corpus_facts.py.

---

What TuringDB is doing here that a relational store would not

The contact network is persisted, not derived. 687,048 edges, each carrying the symmetry operation that generated the neighbour in CIF form (2_565). The periodic network is therefore traversable without materialising a supercell, and a motif search is a pattern match rather than a geometry job over every candidate structure.

Molecular identity is cross-structure. A Component node is deduplicated corpus-wide by InChIKey, so a molecule appearing in forty structures is one node with forty edges rather than forty rows joined on a string. Polymorph, solvate and coformer questions are traversals from that node. Benzoic acid is one node; so is water, shared by thousands of structures.

The motifs are cycles and paths. An R²₂(8) acid dimer is a closed loop. Periodic-net dimensionality is the rank of the lattice-translation subgroup generated by cycles. Neither has a natural relational form; both are one query here.

The corpus is versioned, not snapshotted. The graph is ingested chronologically, one commit per publication year. cod@2009 is the literature as it stood at the end of 2009 — the structures published later do not exist in it. Reproducing a statistic published then is a checkout, not a restore from an archived dump.

Measured on this corpus (server execution time, median of three):

Count structures	9 ms
Space-group filter	59 ms
Packing subgraph, one structure	65 ms
Three-hop synthon join	118 ms
R²₂(8) dimer over 4.7 M atoms	161 ms
LOAD JSONL, 21.2 M elements	54.7 s (388 k elements/s)
Graph on disk	2.85 GB — 32.8 kB per structure

---

Chemistry: the criteria

Every cutoff lives in ingest/chemistry.py with its literature source, and the SMARTS patterns in ingest/fragments.py, so both can be reviewed in one place.

• Covalent bonds — d < r_cov(A) + r_cov(B) + 0.40 Å, radii from Cordero et al., Dalton Trans. 2008, 2832. Sites below 0.5 occupancy are excluded.
• Hydrogen bonds — H···A < 2.50 Å with D–H···A > 120°, donors and acceptors from {N, O, F, S, Cl}. Chlorine appears as an acceptor almost entirely as chloride: of 60 sampled Cl acceptor sites, none was covalently bonded to carbon.
• Inferred hydrogen bonds — where no hydrogen was refined, a heavy-atom D···A < 3.50 Å fallback applies and the edge is flagged h_inferred. This is a distance-only criterion: there is no angular test, and 3.50 Å is loose for O···O against the ~3.2 Å usually used. Only N, O and S are admitted as inferred donors. The fallback fires only when the donor's whole molecule carries no hydrogen — the "this entry has no H positions" case — rather than per atom, which is what stops a carbonyl oxygen being treated as a donor in a structure that is otherwise fully refined. 42.8% of the hydrogen-bond layer is inferred; treat it as a separate, weaker population and filter it out of any geometry claim.
• Weak hydrogen bonds — C–H···A with a located H, H···A ≤ 2.90 Å and C–H···A ≥ 120°, after Taylor & Kennard, JACS 104 (1982) 5063 and Steiner, Angew. Chem. Int. Ed. 41 (2002) 48. Stored as kind='hbond_weak'; there is no inferred variant.
• Halogen bonds — C–X···A with X ∈ {Cl, Br, I}, X···A below the sum of van der Waals radii (Bondi 1964, extended by Mantina 2009) and C–X···A > 150°.
• Components — connected components of the covalent graph after symmetry expansion, so they are real molecules and ions rather than whatever the CIF deposited. Identity is an InChIKey where RDKit perception succeeds and a formula+connectivity digest otherwise, stored in a separate property so it is never mistaken for an InChIKey.

Verification

python tests/validate_symmetry.py 200     # symop round-trip
python tests/validate_chemistry.py 400    # valence plausibility
python tests/validate_components.py 10    # component perception
python tests/corpus_facts.py              # re-derive every published figure
python tests/check_cli_queries.py         # every CLI query runs
python tests/check_frontend_queries.py --negative-control   # every UI query runs

validate_symmetry re-derives the operation stored on each edge and re-applies it: over 28,409 neighbour pairs, 100% resolve and reproduce the neighbour's position, median residual 1e-15 Å.

validate_chemistry reports valence distributions. In purely organic environments: H 98.2%, C 97.4%, N 99.6%, O 97.4%, S 97.5% plausible. Metal-coordinated atoms are reported separately rather than scored against organic valences, because a μ-carboxylate oxygen bridging two nickels genuinely has three bonds.

---

Known limits

• π-stacking is not implemented. It needs ring perception, and a centroid-distance criterion with no slippage constraint admits badly offset pairs that most crystal engineers would not call stacking.
• Hydrogen positions are not normalised. X-ray H sits at ~0.85 Å (O–H) against the neutron value of 0.983 Å, so the 2.50 Å H···A ceiling is effectively ~2.35–2.40 Å normalised, and the measured H···A histogram is still rising at that wall. D, H and A are all stored, so it can be renormalised without re-running the geometry.
• Disorder is an occupancy threshold, not disorder handling. Sites below 0.5 occupancy are excluded, but the commonest disorder ratio is exactly 50:50, so both alternates survive. _atom_site_disorder_group is not read.
• InChIKey perception reaches ~36% of components. Metal complexes fail by construction — InChI has no well-defined representation for them.
• Halogen bonds are dominated by chlorine (1,980 of 4,119 donors), and an unactivated C–Cl is a very weak σ-hole donor. The 150° angle floor is loose against the 155–165° convention — 458 edges, 11%, fall in that gap. Acceptors are restricted to {N, O, F, S, Cl}, so X···π is undetectable; Cl···Cl is detected and accounts for 1,184 edges.
• pyridine_nitrogen misses the diazines. The SMARTS requires the other five ring atoms to be aromatic carbon, so pyrimidine, pyrazine, pyridazine and triazine acceptors are excluded. carboxylic_acid requires an explicit hydroxyl hydrogen, so acids in hydrogen-free refinements never match.
• Z′ is COD metadata, not computed here.
• The SUPERSEDES layer is thin — 684 edges, all asserted by COD's own duplicateof column. Curated related-structure relationships would populate it properly.
• Variable-length traversal cannot be restricted to an edge type. The engine rejects -[e:CONTACT]->{1,8}, so the contact network is also materialised as its own graph, cod_contacts_v2, where an untyped quantifier is a hydrogen-bond traversal.

---

Provenance

CIFs come from the live rsync tree at rsync://www.crystallography.net/cif/; metadata from COD's public read-only MySQL mirror. COD's robots.txt is Disallow: /, so nothing is scraped over HTTP.

--ingest writes data/manifest/slice_manifest.json recording the release revision, the metadata SVN revision, the exact journal-name strings matched, the size cap applied, and the sorted COD ID list with a checksum; skipped IDs go to slice_skipped_ids.txt. Those are build outputs, not repository contents — data/ is 2.8 GB and gitignored — so they describe your build, and two builds from different COD revisions will differ. Files above 512 kB are skipped, which keeps 95.0% of the slice.

COD data is dedicated to the public domain under CC0. If you use COD in published work, cite Gražulis et al., J. Appl. Cryst. 42 (2009) 726 and Nucleic Acids Res. 40 (2012) D420.

---

Layout

crystal.py         the command-line client
ingest/            acquisition and the ingest pipeline
  download.py        COD acquisition (rsync + MySQL) with an auditable manifest
  chemistry.py       radii tables and geometric criteria, each cited
  fragments.py       the SMARTS table
  periodic.py        symmetry expansion and neighbour search
  symmetry.py        symop encoding in CIF form
  structure.py       bonds and periodic connected components
  contacts.py        hydrogen, weak and halogen bond detection
  netdim.py          periodic-net dimensionality; the involution test
  perception.py      RDKit identity and fragment matching
  rdkit_guard.py     killable worker for RDKit's bond-order search
  solvents.py        solvent / counter-ion classification
  build_graph.py     emits LOAD JSONL-shaped output
  project_contacts.py  the atoms+H-bonds projection
  load_turingdb.py   load and verify
  build_versioned.py one commit per publication year, plus the tag ledger
queries/           analytical scripts
tests/             validation and the two query gates
visualizer/        the browser studio

visualizer/ is a trimmed fork of turing-db/turingdb-visualizer. The WebGL canvas, node inspector and design system are theirs; this repo adds src/components/viewer/crystal/.

Licensed MIT — see LICENSE.