diff --git a/README.md b/README.md index 14e6816..0cb4504 100644 --- a/README.md +++ b/README.md @@ -77,7 +77,9 @@ For custom protocols, declare typed sample metadata with `protocol.add_sample(sa `lab.provenance` provides immutable SBOL3 designs, material implementations, activities, qualified usages and associations, agents, and plans. Author a `Document`, explicitly add its objects, and call `freeze()` to validate references and obtain a reproducible snapshot. Import and export local Turtle with `Document.read()` and `snapshot.write()`, or exchange a detached pySBOL3 document with `from_sbol3()` and `to_sbol3()`. -See the [provenance guide](docs/provenance.md) and run `uv run --no-sync python -m examples.provenance` for an authoring and round-trip example. +The [provenance guide](docs/provenance.md) covers the Python API and evidence states for supplied material records and planned work. Run `uv run --no-sync python -m examples.provenance` for a complete authoring and round-trip example. The [experiment walkthrough](docs/cloning-provenance.md) covers inventory-aware assembly, transformation, plating, external preparation, Addgene records, and compilation to SBOL/LabOP/robot bundles. LabOP export describes the protocol plan, including its operations, parameters, and material flows. Run `uv run python -m examples.cloning_workflow --target manual` for the complete synthetic software example. See the [implementation status](docs/provenance-implementation.md) for remaining integrations. + +The [worked notebook](examples/cloning_provenance/cloning_provenance.ipynb) follows a received vector lot through a reporter-reference project. It reads the [example input files](examples/cloning_provenance/data/reporter_reference/README.md), explains actual Turtle excerpts from SBOL and LabOP, and traces a planned operation into robot code. Optional cells compile Flex and STAR as well as OT-2 and verify that each target shares the same LabOP protocol. Saved outputs let you read the story without running it. Launch it with `uv run --extra opentrons --extra star --with jupyterlab jupyter lab examples/cloning_provenance/cloning_provenance.ipynb`. ## Describe a deck @@ -174,7 +176,7 @@ uv run --no-sync python -m examples.deck_layouts --target ot2 uv run --no-sync python -m examples.deck_layouts --target flex ``` -Use `manual` or `ot2` for the cloning example. Its assembly recipe includes transfers below the current Flex preset's supported pipetting range. The deck layouts example supports `ot2`, `flex`, and `star`. +The provenance assembly and complete cloning workflow examples support `manual`, `ot2`, `flex`, and `star`. Flex selects the 50 µL pipette for small transfers. STAR thermal operations require explicit runtime callbacks. The deck layouts example supports `ot2`, `flex`, and `star`. The cloning example writes separate `assembly`, `transformation`, and `plating` bundles under `build/cloning/`, including for the manual target. The deck layouts example writes to `build/decks/`. Both accept `--out` to choose a different output directory. Compilation never connects to hardware. @@ -213,6 +215,7 @@ src/lab/ protocol.py # Protocol builder, Plate, Well model.py # Recorded operations, protocol snapshots, target plans part.py # SBOL part identity + provenance/ # SBOL3 designs, provenance, immutable document snapshots samples.py # Sample, Location, SamplePlacement, OutputManifest labware.py # Logical labware specifications equipment.py # Liquid handlers and equipment identifiers diff --git a/docs/cloning-provenance.md b/docs/cloning-provenance.md new file mode 100644 index 0000000..7509ec2 --- /dev/null +++ b/docs/cloning-provenance.md @@ -0,0 +1,138 @@ +# Planning and compiling an experiment + +The pipeline is `SBOL designs + inventory + recipes + resolved methods → BuildPlan → ExperimentPlan → ExperimentCompilation`. Each arrow creates a new immutable snapshot. Target selection happens after the experiment has been frozen. SBOL, LabOP, methods, and robot programs are projections of those snapshots. + +## Python API + +```python +from lab import compile +from lab.experiments import cloning +from lab.inventory import Inventory +from lab.provenance import Document +from lab.targets import LiquidHandler + +designs = Document.read("designs.ttl").freeze() +inventory = Inventory.read("inventory.json", document=designs) +system = cloning.CloningSystem.read("system.json") +methods = cloning.CloningMethods.read("methods.json") + +build = cloning.plan( + request, + document=designs, + inventory=inventory, + system=system, + methods=methods, +) +build.write("review/build") +build.require_ready() + +experiment = cloning.build(build) +compile(experiment, LiquidHandler.OT2).write("review/ot2") +compile(experiment, LiquidHandler.FLEX).write("review/flex") +compile(experiment, LiquidHandler.STAR).write("review/star") +``` + +`BuildRequest.targets` accepts `BuildTarget(design=component.ref, volume_ul=Decimal(...), form=...)` for liquids and `CountTarget(design=component.ref, count=..., form=...)` for whole material units. A `CountTarget` defaults to `MaterialForm.PLATED_SAMPLE`. Method and system files are typed JSON snapshots written by `CloningMethods.write()` and `CloningSystem.write()`. Recipe parameter values cannot be recovered from SBOL usage edges: they must be specified separately. + +The complete runnable [example](../examples/provenance_build.py) authors every input, using short synthetic DNA strings and arbitrary parameters as a software fixture. Run `uv run python -m examples.provenance_build --target manual`; choose `ot2`, `flex`, or `star` with the corresponding SDK extra installed. + +## Designs, quantities, and alternative routes + +`Stock` requires an explicit `Ref[Implementation]`, `Ref[Component]`, `MaterialForm`, and measured available volume. Pint inputs are normalized to immutable Decimal quantities. Inventory contains recorded implementations; catalog listings, planned products, and simulated material are not available physical inventory. An inventory design assertion does not establish that its sequence was verified. `CountedStock` records assessed whole-unit counts for bacterial stabs and plated samples. A supplier package count does not automatically establish an available material count. Both stock classes require recorded provenance; simulated material cannot satisfy physical inventory. + +An `AssemblyRecipe` specifies an enzyme, intended product, method identity, and ordered `FragmentSelection` objects. Each fragment identifies its source component and zero-based Watson cut positions; `None` denotes a linear end. A circular single-cut linearization has equal left and right cuts. Reversal is explicit. The sequence calculator tracks both strands and sticky ends, checks ligation compatibility, and compares a specified target sequence modulo circular rotation. Linear products with unpaired ends require explicit end repair. No fragment is selected by size. + +`AssemblyMethod` supplies all reagent and DNA volumes, diluent, thermal holds, cycle counts, mixing parameters, and planned usable output. The planner accounts for available quantities across the entire request, including repeated reaction batches and shared intermediates. It explores permitted recipe combinations and prefers existing stock, then fewer outstanding design/acquisition/preparation requirements, fewer dependent stages, and fewer reactions. `PlanningPolicy.max_states` bounds exploration explicitly; exceeding it fails rather than silently switching algorithms. + +Calculated designs have recorded **computational** activities and content-dependent identities. Planned physical implementations point to them without asserting `Implementation.built`. `propose_edits()` examines only positions the caller marks editable. Applying a selected proposal creates a new design; it does not change inventory, establish biological function, or automatically modify a recipe. + +The planner schedules assembly, transformation, plating, and explicitly defined external preparation. It matches both the component identity and material form; a design in culture is not available DNA. Missing material, an unspecified conversion procedure, or an invalid design remains an explicit requirement and blocks executable generation. Each reaction has its own output implementation. Allocations consume one shared quantity ledger across all requested targets; planning leaves the input inventory snapshot unchanged. + +## Transformation, plating, and preparation + +A `CloningSystem` contains an immutable tuple of typed recipes. Recipes choose biological inputs and intended products; `CloningMethods` supplies the full procedure parameters. For example, with authored components and methods: + +```python +from lab.inventory import MaterialForm + +system = cloning.CloningSystem( + identity="https://example.org/system", + recipes=( + assembly_recipe, + cloning.TransformationRecipe( + identity="https://example.org/transform", + product=strain.ref, + chassis=cells.ref, + plasmids=(plasmid.ref,), + method=transformation_method.identity, + ), + cloning.PlatingRecipe( + identity="https://example.org/plate", + product=strain.ref, + method=plating_method.identity, + ), + cloning.ExternalPreparationRecipe( + identity="https://example.org/prepare_dna", + source=vector.ref, + source_form=MaterialForm.BACTERIAL_STAB, + product=vector.ref, + output_form=MaterialForm.DNA, + method=preparation_method.identity, + ), + ), +) +request = cloning.BuildRequest( + identity="https://example.org/build", + targets=(cloning.CountTarget(design=strain.ref, count=2),), +) +``` + +`TransformationMethod` requires cell and DNA volumes, a recovery-medium `Reagent`, a treatment `profile`, a `recovery_profile`, mixing volumes/cycles, and a planned usable output volume. `initial_celsius` is optional. The generated protocol adds and mixes the cells, mixes and adds each allocated DNA source, applies the treatment profile, adds the recovery medium, and applies the recovery profile. Thermal block volumes follow the actual additions. The output has form `CULTURE` and represents the intended recovery mixture; it does not assert transformation success, clonality, or sequence verification. + +`PlatingMethod` requires a substrate component, diluent component, transfer volume, ordered dilution factors, mixing volume/cycles, spot volume, and `spot_height_mm` above the destination well bottom. Each recipe invocation produces one `PLATED_SAMPLE` at the final dilution. Repeated count targets generate independent dilution series and spots. No colony count, incubation, or selection is inferred. The substrate is a supplied plate precondition, recorded in the method and planned SBOL usage; it is a consumable like tips and labware, not an automatically reserved inventory aliquot. A plated sample cannot be pipetted as liquid. Spotting height is carried into LabOP, Opentrons positioning, and the STAR dispense command. + +`ExternalPreparationMethod` requires a procedure IRI and explicit instructions, plus exactly one source quantity (`source_volume_ul` or `source_count`) and one expected output quantity (`output_volume_ul` or `output_count`). Optional `reagents` name additional consumed liquids. The matching recipe specifies source and output forms; quantity kinds must agree with those forms. Instructions are preserved in SBOL, LabOP, methods, and the input snapshot. Supply complete instructions and a versioned procedure identifier; Lab does not fetch or invent the external procedure. + +External preparation becomes a `ProtocolStage(external=True)` containing an `ExternalPreparation` operation. Its input consumption and prospective output are checked at the operation boundary, so an output is not falsely declared present at the start of the stage. `compile(experiment, LiquidHandler.OT2)` emits an operator document for this stage and robot code for the automated stages. Explicit per-stage target mappings must assign external stages to `Manual()`. The preparation output remains a planned implementation with an expected quantity. + +The runnable [complete workflow](../examples/cloning_workflow.py) demonstrates counted input → external preparation → assembly → transformation → two plated samples using synthetic test data. Run `uv run python -m examples.cloning_workflow --target ot2` with the Opentrons extra, or choose `manual`, `flex`, or `star`. + +## Supplier evidence + +`AddgeneClient(token=...).plasmid(id)` explicitly retrieves one catalog item through Addgene's [read-only API](https://developers.addgene.org/access-options/). Parsing preserves depositor versus Addgene sequences, full versus partial sequence assertions, retrieval time, and original metadata. `parse_plasmid()` accepts saved responses without network access. + +`CatalogEntry(item=..., design=component.ref, form=MaterialForm.BACTERIAL_STAB)` is a caller-reviewed mapping. Pass a frozen `Catalog` to `cloning.plan(catalog=...)`. Acquisition requests contain matching candidates, with Addgene first. A candidate is not an order or usable stock. + +`QuoteRecord` and `OrderReference` record transactions performed outside Lab. `Receipt.record(document)` adds an observed arrival and material implementation. A liquid receipt enters `Inventory` only through an explicitly measured `receipt.stock(...)`. `receipt.counted_stock(identity=..., count=...)` records an explicitly assessed count for a counted material form. The declared preparation route can then consume that stock; the receipt alone does not establish a quantity or a usable liquid yield. Addgene authentication and response shape are contract-tested with synthetic fixtures; no authenticated live retrieval is claimed. + +## Frozen experiments and hardware + +`ExperimentPlan` contains a provenance snapshot and ordered `ProtocolStage` objects. Each stage holds a `RecordedProtocol`, logical deck requirements, dependencies, and material handoffs. `cloning.build()` retains full design and implementation IRIs, carries forward remaining liquid volumes and whole-unit counts after previous consumers, and separates usable-yield planning from the physical volume ledger. + +Default automated stages use PCR plates for reagents and products, with separate dilution and substrate plates for plating. A caller may supply a different `reagent_container`; `preparation_container` controls the logical output container of external procedures. Counted external inputs occupy abstract storage positions that do not prescribe robot labware. Loading is an initial condition; the compiler does not invent aliquoting, stock preparation, or an additional liquid transfer during a plate handoff. A loading amount that exceeds a container's capacity fails validation. + +Every operation has an identity. Supplying `Protocol(..., identity="https://.../protocol")` gives a stable parent for generated step identities. Without an explicit identity, the snapshot derives one from its contents. Semantic digests exclude developer source paths and hardware; target compilation digests include physical configuration and generated source. Source maps identify inclusive generated line spans for each semantic step. + +`compile(experiment, handler)` resolves each stage's deck. A mapping from stage identities to concrete targets or decks supports custom equipment. Flex supports the 50 µL pipette's documented [volume modes](https://docs.opentrons.com/python-api/pipettes/volume-modes/), configured before picking up a tip. STAR thermal stages require supplied asynchronous runtime callbacks: `thermocycle` returns the plate after the profile, while `set_temperature` must maintain the hold with the plate accessible for subsequent pipetting. Generated STAR scripts use a software preview backend when invoked directly. SDK simulations and previews do not qualify physical equipment. + +## LabOP and digital methods + +`lab.labop.export(experiment)` emits protocol RDF from the frozen experiment, including its stage protocols, operations, parameters, and material flows. This is the scope of the LabOP integration; execution recording is outside the compiler. The packaged UML ontology, LabOP ontology, and primitive libraries are pinned to upstream commit `2e2bd88150c71a440771fd369f40295dfd622324`; their hashes and license are included in `lab/labop/resources`. + +| Lab semantics | LabOP representation | +| --- | --- | +| Experiment and stage | Protocols with ordered subprotocol calls | +| Initial plate contents | Input SampleArray defaults and explicit loading conditions | +| Well selection | PlateCoordinates and object flows | +| Transfer | liquid_handling/Transfer; TransferAtHeight when a destination height is specified | +| External preparation | Named extension primitive with source/destination collections, procedure, instructions, and expected amounts | +| Counted material | Sample assertions and amount measures in OM one, separate from microlitres | +| Mixing | liquid_handling/PipetteMix, including volume and cycle count | +| Wait, persistent temperature, thermal profile, distribution, operator pause | Named Lab extension primitives with explicit typed parameters | +| Material handoff | Subprotocol output-to-input object flow | + +The extension primitives are defined inside each artifact. Thermal profiles contain ordered holds with unit-bearing temperatures and durations. Distribution uses an ordered collection of sample aliases and preserves its air-gap and tip-reuse semantics. Lab extensions also retain material identities, initial volumes, planned sample assertions, and the semantic step payload. Initial loads never become fictitious Provision actions. The exporter supports whole-plate continuity across stage handoffs; arbitrary inter-stage well remapping requires an explicit material operation. + +The bundle includes the Lab, LabOP, and UML schemas. The independent `scripts/check_labop.py` check loads them through SBOLFactory 1.1.2 and pySBOL3, validates the document, and verifies that thermal-profile parameters deserialize into typed objects. Run it in an isolated interpreter because SBOLFactory registers process-wide builders. This check does not execute the protocol. + +Bundles contain `experiment.json`, `build.json` where applicable, `provenance.ttl`, `protocol.labop.ttl`, `methods.md`, input snapshots, schemas, per-stage protocol/target plans, manifests, source maps, and generated programs. Inputs include the build-plan provenance before its protocol links were resolved, inventory, catalog mappings, recipes, and methods. `bundle.json` records SHA-256 checksums of the other artifacts. Methods describe the planned procedure and its expected outputs. diff --git a/docs/provenance-implementation.md b/docs/provenance-implementation.md new file mode 100644 index 0000000..b88de04 --- /dev/null +++ b/docs/provenance-implementation.md @@ -0,0 +1,20 @@ +# Provenance implementation status + +Lab owns immutable design, inventory, planning, and protocol models. SBOL3 describes designs, supplied materials, and planned work; LabOP describes the protocol's operations and flows. Target programs are compiled from the same frozen semantics. + +## Implemented path + +1. **SBOL3 foundation.** `provenance/types.py`, `_schema.py`, `document.py`, `validation.py`, `sbol3.py`, and `resources/lab.ttl` implement the taxonomy, explicit references, authoring, validation, deterministic snapshots, and pySBOL3 interchange. Ordinary imports keep annotations resolvable at runtime. +2. **Cloning planning.** `inventory.py` and cloning `systems.py`, `methods.py`, `_dna.py`, `sequences.py`, `domestication.py`, `routes.py`, and `planning.py` handle measured stock snapshots, explicit fragments, sequence calculations, caller-selected edit proposals, alternative routes, shared quantities, repeated batches, and multi-level dependencies. Typed transformation, counted plating, and explicit external preparation share the allocation and dependency model. Unspecified preparation and acquisition remain requirements. Sequence calculations use Biopython 1.84 enzyme definitions; pydna is not a runtime dependency. +3. **Supplier records.** `suppliers/types.py` and `addgene.py` provide read-only catalog retrieval, preserved sequence evidence, reviewed design mappings, acquisition requests, external quote/order references, and counted receipts. No purchasing action or available stock is inferred from a listing. +4. **Shared experiment compilation.** `experiment.py`, `operations.py`, `target.py`, `model.py`, and cloning `build.py` freeze stage protocols and material handoffs. Samples retain design and implementation references. The compiler accepts an experiment, supports per-stage target selection, and writes source maps with stable semantic step identities. Shared target values live in `target.py` so importing validation does not initialize the backend package. +5. **LabOP and methods.** `labop/protocol.py`, `primitives.py`, pinned resources, `artifacts.py`, and `methods.py` produce static protocol RDF, typed parameter and sample flows, methods, and artifact checksums. Exact upstream primitives are used where their semantics match. Named extensions specify the remaining operations. Compilation never invokes an execution engine. +6. **Robot targets.** Flex supports 50 µL pipettes and explicit volume-mode changes. STAR separates persistent-temperature and thermal-profile callbacks. The assembly, transformation, and plating paths have software simulation/preview coverage for OT-2, Flex, and STAR; physical qualification is outside those checks. + +The developer walkthrough is [Planning and compiling an experiment](cloning-provenance.md). [The complete runnable example](../examples/cloning_workflow.py) supplies all inputs using a synthetic software fixture. + +## Boundaries and remaining integrations + +External procedures remain operator stages; their expected yields are prospective until observed. Plating plans one deposited sample per reaction and does not infer colony formation or selection. The substrate, tips, and labware are supplied consumables, not stock reservations. The existing standalone transformation and plating builders also remain available for direct protocol authoring. + +The LabOP integration ends at protocol export; post-execution recording is outside its scope. Durable inventory reservation/depletion, automatic procurement, arbitrary inter-stage well remapping, and physical instrument qualification are not implemented. The Addgene adapter is contract-tested without authenticated live credentials. LabOP validation checks pinned ontology cardinalities and primitive contracts; it does not claim that every third-party interpreter implements the Lab extension primitives. diff --git a/docs/provenance.md b/docs/provenance.md index c0cd42e..6db6c42 100644 --- a/docs/provenance.md +++ b/docs/provenance.md @@ -118,6 +118,6 @@ The small Lab extension vocabulary is packaged at `lab/provenance/resources/lab. uv run --no-sync python -m examples.provenance --out build/provenance.ttl ``` -[The example](../examples/provenance.py) builds a design, an inventory assertion, a prospective activity, and a planned output; checks the SBOL3 export; writes it; and verifies its round trip. It specifies no executable cloning method. +[The example](../examples/provenance.py) builds a design, an inventory assertion, a prospective activity, and a planned output; checks the SBOL3 export; writes it; and verifies its round trip. It specifies no executable cloning method. The [experiment walkthrough](cloning-provenance.md) covers inventory planning, compiler integration, and LabOP protocol export; the [implementation status](provenance-implementation.md) lists the remaining integrations. -The mappings follow the [SBOL 3.1 specification](https://sbolstandard.org/docs/SBOL3.1.0.pdf) and [pySBOL3's provenance model](https://raw.githubusercontent.com/SynBioDex/pySBOL3/main/sbol3/provenance.py). +The mappings follow the [SBOL 3.1 specification](https://sbolstandard.org/docs/SBOL3.1.0.pdf) and [pySBOL3's provenance model](https://raw.githubusercontent.com/SynBioDex/pySBOL3/main/sbol3/provenance.py). Protocol interchange uses [LabOP](https://bioprotocols.github.io/labop/). diff --git a/examples/cloning_provenance/README.md b/examples/cloning_provenance/README.md new file mode 100644 index 0000000..89d313f --- /dev/null +++ b/examples/cloning_provenance/README.md @@ -0,0 +1,11 @@ +# Cloning provenance walkthrough + +Open [cloning_provenance.ipynb](cloning_provenance.ipynb) for the worked reporter-reference example, from supplied material records through planning, readable SBOL and LabOP excerpts, and robot compilation. Saved outputs are included. + +From the repository root, launch it with: + +```sh +uv run --extra opentrons --extra star --with jupyterlab jupyter lab examples/cloning_provenance/cloning_provenance.ipynb +``` + +The notebook's [presentation helper](notebook_views.py) and [input files](data/reporter_reference/README.md) belong to this example. Generated bundles go under the repository's ignored `build/cloning-notebook/` directory. diff --git a/examples/cloning_provenance/cloning_provenance.ipynb b/examples/cloning_provenance/cloning_provenance.ipynb new file mode 100644 index 0000000..6dda04b --- /dev/null +++ b/examples/cloning_provenance/cloning_provenance.ipynb @@ -0,0 +1,1006 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "reporter-00", + "metadata": {}, + "source": [ + "# A reporter reference, from receipt to a protocol plan\n", + "\n", + "A lab is preparing a **reporter reference for a later fluorescence study**. The intended construct is *pReference-Green*, and the intended recipient is *host H1*. The immediate request is two deposited samples from the resulting culture; fluorescence measurement and clone verification will happen later.\n", + "\n", + "The project starts with an awkward detail: the vector arrived as **receipt R-042, lot V-17, in a bacterial stab**. The insert is already available as DNA. Someone must prepare vector DNA before a robot can use it.\n", + "\n", + "We will follow that one received material through the files: **what arrived → what is planned → what the protocol specifies → what the robot would do**. At each point, we will open the relevant Turtle rather than treat it as an opaque export.\n", + "\n", + "The study and records are illustrative. The dataset uses clearly marked dummy sequences and software-test method parameters, not a validated laboratory protocol. No robot is run in this notebook.\n" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-01", + "metadata": {}, + "source": [ + "
\n", + "Run this notebook\n", + "

From the repository root, with Python 3.11 or 3.12:

\n", + "
uv run --extra opentrons --extra star --with jupyterlab jupyter lab examples/cloning_provenance/cloning_provenance.ipynb
\n", + "

Use the environment's Python kernel and run the cells in order. New artifacts go under the ignored build/cloning-notebook/ directory. Saved outputs can be read without running anything.

\n", + "
\n", + "\n", + "The [four input files](data/reporter_reference/README.md) are part of this example. Table formatting and Turtle selection live in [notebook_views.py](notebook_views.py), keeping the compiler calls visible here.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "reporter-02", + "metadata": { + "jupyter": { + "source_hidden": true + } + }, + "outputs": [], + "source": [ + "import notebook_views as view\n", + "from notebook_views import EX, LAB, LABOP, PROV, SBOL\n", + "from rdflib import RDF, Graph\n", + "\n", + "import lab\n", + "from lab.experiments import cloning\n", + "from lab.inventory import Inventory\n", + "from lab.operations import Transfer\n", + "from lab.provenance import Component, Document, EvidenceState\n", + "from lab.targets import LiquidHandler\n", + "\n", + "output = view.fresh_output()" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-03", + "metadata": {}, + "source": [ + "## 1. What did the lab receive?\n", + "\n", + "Start with the designs and the material records. The vector **design** and the received **lot** are different objects. The inventory then says where that lot is and in what form it is available. Storage positions are distinct from the robot deck addresses assigned later.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "reporter-04", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
Material recordIntended designFormAvailableLocation
Host aliquot H1-03Host H1competent_cells20 µLCell box H1 / A1
Insert aliquot I-07Green reporter cassettedna10 µLDNA rack D1 / C2
Receipt R-042 / lot V-17pReference backbonebacterial_stab1 unitReceipt box / B3
" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "input_ttl = view.CASE / \"designs-and-materials.ttl\"\n", + "document = Document.read(input_ttl, namespace=str(EX).rstrip(\"/\")).freeze()\n", + "inventory = Inventory.read(view.CASE / \"inventory.json\", document=document)\n", + "\n", + "view.inventory(inventory, document)" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-05", + "metadata": {}, + "source": [ + "**Read the corresponding Turtle.** A `.ttl` file is a text representation of an RDF graph: statements with a subject, a relationship, and an object. Prefixes abbreviate full identifiers. `ex:received_stab` is one persistent identity, not a Python variable or a file path.\n", + "\n", + "The next cell selects real statements about the vector and its received material. It omits housekeeping fields and uses prefixes for readability; it does not change their meaning.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "reporter-06", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
@prefix prov: <http://www.w3.org/ns/prov#> .\n",
+       "@prefix ex: <https://example.org/reporter/> .\n",
+       "@prefix sbol: <http://sbols.org/v3#> .\n",
+       "@prefix lab: <https://the-lab-compiler.github.io/lab-py/ns#> .\n",
+       "\n",
+       "ex:received_stab a sbol:Implementation ;\n",
+       "    sbol:name \"Receipt R-042 / lot V-17\" ;\n",
+       "    prov:wasDerivedFrom ex:vector ;\n",
+       "    lab:evidenceState lab:recorded .\n",
+       "\n",
+       "ex:vector a sbol:Component ;\n",
+       "    sbol:name \"pReference backbone\" .\n",
+       "
\n" + ], + "text/latex": [ + "\\begin{Verbatim}[commandchars=\\\\\\{\\}]\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{prov:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}http://www.w3.org/ns/prov\\PYZsh{}\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{ex:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}https://example.org/reporter/\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{sbol:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}http://sbols.org/v3\\PYZsh{}\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{lab:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}https://the\\PYZhy{}lab\\PYZhy{}compiler.github.io/lab\\PYZhy{}py/ns\\PYZsh{}\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\n", + "\\PY{n+nn}{ex}\\PY{p}{:}\\PY{n+nt}{received\\PYZus{}stab} \\PY{k+kt}{a} \\PY{n+nn}{sbol}\\PY{p}{:}\\PY{n+nt}{Implementation} \\PY{p}{;}\n", + " \\PY{n+nn}{sbol}\\PY{p}{:}\\PY{n+nt}{name} \\PY{l+s}{\\PYZdq{}}\\PY{l+s}{Receipt R\\PYZhy{}042 / lot V\\PYZhy{}17}\\PY{l+s}{\\PYZdq{}} \\PY{p}{;}\n", + " \\PY{n+nn}{prov}\\PY{p}{:}\\PY{n+nt}{wasDerivedFrom} \\PY{n+nn}{ex}\\PY{p}{:}\\PY{n+nt}{vector} \\PY{p}{;}\n", + " \\PY{n+nn}{lab}\\PY{p}{:}\\PY{n+nt}{evidenceState} \\PY{n+nn}{lab}\\PY{p}{:}\\PY{n+nt}{recorded} \\PY{p}{.}\n", + "\n", + "\\PY{n+nn}{ex}\\PY{p}{:}\\PY{n+nt}{vector} \\PY{k+kt}{a} \\PY{n+nn}{sbol}\\PY{p}{:}\\PY{n+nt}{Component} \\PY{p}{;}\n", + " \\PY{n+nn}{sbol}\\PY{p}{:}\\PY{n+nt}{name} \\PY{l+s}{\\PYZdq{}}\\PY{l+s}{pReference backbone}\\PY{l+s}{\\PYZdq{}} \\PY{p}{.}\n", + "\\end{Verbatim}\n" + ], + "text/plain": [ + "@prefix prov: .\n", + "@prefix ex: .\n", + "@prefix sbol: .\n", + "@prefix lab: .\n", + "\n", + "ex:received_stab a sbol:Implementation ;\n", + " sbol:name \"Receipt R-042 / lot V-17\" ;\n", + " prov:wasDerivedFrom ex:vector ;\n", + " lab:evidenceState lab:recorded .\n", + "\n", + "ex:vector a sbol:Component ;\n", + " sbol:name \"pReference backbone\" ." + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "view.turtle(\n", + " input_ttl,\n", + " [str(EX.vector), str(EX.received_stab)],\n", + " predicates=(RDF.type, SBOL.name, PROV.wasDerivedFrom, LAB.evidenceState),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-07", + "metadata": {}, + "source": [ + "Read this as: **“Receipt R-042 is a material implementation associated with the pReference vector design.”**\n", + "\n", + "- `a sbol:Component` declares a design; `a sbol:Implementation` declares a material realization or intended realization.\n", + "- `prov:wasDerivedFrom` links the material record to its intended design. It does not establish that the sequence was verified.\n", + "- `lab:evidenceState lab:recorded` says this is a supplied inventory assertion in our example.\n", + "\n", + "A semicolon continues statements about the same subject; a period finishes that group. The storage position and the counted `bacterial_stab` form live in `inventory.json`. The TTL and inventory share the same implementation IRI.\n" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-08", + "metadata": {}, + "source": [ + "## 2. What needs to happen before there are two samples?\n", + "\n", + "We request the intended H1/reference material. Two samples here means **two deposits from one upstream culture**, not two independent clones.\n", + "\n", + "The project supplies [routes](data/reporter_reference/system.json) and [method parameters](data/reporter_reference/methods.json) as separate inputs. One route explicitly converts a vector stab into vector DNA using external procedure **P01**. The planner may use that route; it may not silently treat the stab as pipettable DNA.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "reporter-09", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
OrderTaskExpected materialResponsible system
1Prepare vector DNAdna: 10 µLExternal procedure P01
2Assemble reporter designdna: 5 µLLiquid handler
3Introduce it into host H1culture: 5 µLLiquid handler
4Deposit a sampleplated_sample: 1 sampleLiquid handler
5Deposit a sampleplated_sample: 1 sampleLiquid handler
" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/svg+xml": [ + "1. Preparationdna2. Assemblydna3. Transformationculture4. Platingplated_sample5. Platingplated_sample" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "strain = document.get(str(EX.strain), Component)\n", + "request = cloning.BuildRequest(\n", + " identity=str(EX.build),\n", + " targets=(cloning.CountTarget(design=strain.ref, count=2),),\n", + ")\n", + "plan = cloning.plan(\n", + " request,\n", + " document=document,\n", + " inventory=inventory,\n", + " system=cloning.CloningSystem.read(view.CASE / \"system.json\"),\n", + " methods=cloning.CloningMethods.read(view.CASE / \"methods.json\"),\n", + ")\n", + "plan.require_ready()\n", + "view.plan(plan)" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-10", + "metadata": {}, + "source": [ + "The first stage is manual preparation; the remaining stages can be compiled for a liquid handler. Both deposits share the culture produced by the transformation stage. These are material dependencies; the protocol specifies an ordered stage list.\n", + "\n", + "**What did planning add to the SBOL graph?** Follow the new vector-DNA output back to its preparation activity, then through the activity's `Usage` to receipt R-042.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "reporter-11", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
@prefix prov: <http://www.w3.org/ns/prov#> .\n",
+       "@prefix ex: <https://example.org/reporter/> .\n",
+       "@prefix sbol: <http://sbols.org/v3#> .\n",
+       "@prefix lab: <https://the-lab-compiler.github.io/lab-py/ns#> .\n",
+       "@prefix build: <https://example.org/reporter/build/> .\n",
+       "@prefix prep: <https://example.org/reporter/build/preparation_1_6a0c3b428b5f/> .\n",
+       "\n",
+       "build:preparation_1_6a0c3b428b5f a prov:Activity ;\n",
+       "    prov:qualifiedUsage prep:Usage1 ;\n",
+       "    lab:evidenceState lab:planned .\n",
+       "\n",
+       "prep:Usage1 a prov:Usage ;\n",
+       "    prov:entity ex:received_stab .\n",
+       "\n",
+       "prep:output a sbol:Implementation ;\n",
+       "    prov:wasDerivedFrom ex:vector ;\n",
+       "    prov:wasGeneratedBy build:preparation_1_6a0c3b428b5f ;\n",
+       "    lab:evidenceState lab:planned .\n",
+       "
\n" + ], + "text/latex": [ + "\\begin{Verbatim}[commandchars=\\\\\\{\\}]\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{prov:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}http://www.w3.org/ns/prov\\PYZsh{}\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{ex:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}https://example.org/reporter/\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{sbol:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}http://sbols.org/v3\\PYZsh{}\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{lab:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}https://the\\PYZhy{}lab\\PYZhy{}compiler.github.io/lab\\PYZhy{}py/ns\\PYZsh{}\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{build:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}https://example.org/reporter/build/\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{prep:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}https://example.org/reporter/build/preparation\\PYZus{}1\\PYZus{}6a0c3b428b5f/\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\n", + "\\PY{n+nn}{build}\\PY{p}{:}\\PY{n+nt}{preparation\\PYZus{}1\\PYZus{}6a0c3b428b5f} \\PY{k+kt}{a} \\PY{n+nn}{prov}\\PY{p}{:}\\PY{n+nt}{Activity} \\PY{p}{;}\n", + " \\PY{n+nn}{prov}\\PY{p}{:}\\PY{n+nt}{qualifiedUsage} \\PY{n+nn}{prep}\\PY{p}{:}\\PY{n+nt}{Usage1} \\PY{p}{;}\n", + " \\PY{n+nn}{lab}\\PY{p}{:}\\PY{n+nt}{evidenceState} \\PY{n+nn}{lab}\\PY{p}{:}\\PY{n+nt}{planned} \\PY{p}{.}\n", + "\n", + "\\PY{n+nn}{prep}\\PY{p}{:}\\PY{n+nt}{Usage1} \\PY{k+kt}{a} \\PY{n+nn}{prov}\\PY{p}{:}\\PY{n+nt}{Usage} \\PY{p}{;}\n", + " \\PY{n+nn}{prov}\\PY{p}{:}\\PY{n+nt}{entity} \\PY{n+nn}{ex}\\PY{p}{:}\\PY{n+nt}{received\\PYZus{}stab} \\PY{p}{.}\n", + "\n", + "\\PY{n+nn}{prep}\\PY{p}{:}\\PY{n+nt}{output} \\PY{k+kt}{a} \\PY{n+nn}{sbol}\\PY{p}{:}\\PY{n+nt}{Implementation} \\PY{p}{;}\n", + " \\PY{n+nn}{prov}\\PY{p}{:}\\PY{n+nt}{wasDerivedFrom} \\PY{n+nn}{ex}\\PY{p}{:}\\PY{n+nt}{vector} \\PY{p}{;}\n", + " \\PY{n+nn}{prov}\\PY{p}{:}\\PY{n+nt}{wasGeneratedBy} \\PY{n+nn}{build}\\PY{p}{:}\\PY{n+nt}{preparation\\PYZus{}1\\PYZus{}6a0c3b428b5f} \\PY{p}{;}\n", + " \\PY{n+nn}{lab}\\PY{p}{:}\\PY{n+nt}{evidenceState} \\PY{n+nn}{lab}\\PY{p}{:}\\PY{n+nt}{planned} \\PY{p}{.}\n", + "\\end{Verbatim}\n" + ], + "text/plain": [ + "@prefix prov: .\n", + "@prefix ex: .\n", + "@prefix sbol: .\n", + "@prefix lab: .\n", + "@prefix build: .\n", + "@prefix prep: .\n", + "\n", + "build:preparation_1_6a0c3b428b5f a prov:Activity ;\n", + " prov:qualifiedUsage prep:Usage1 ;\n", + " lab:evidenceState lab:planned .\n", + "\n", + "prep:Usage1 a prov:Usage ;\n", + " prov:entity ex:received_stab .\n", + "\n", + "prep:output a sbol:Implementation ;\n", + " prov:wasDerivedFrom ex:vector ;\n", + " prov:wasGeneratedBy build:preparation_1_6a0c3b428b5f ;\n", + " lab:evidenceState lab:planned ." + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "preparation = plan.tasks[0]\n", + "planning_ttl = plan.document.write(output / \"build-provenance.ttl\")\n", + "view.turtle(\n", + " planning_ttl,\n", + " [preparation.output.identity, preparation.identity],\n", + " predicates=(\n", + " RDF.type,\n", + " PROV.wasDerivedFrom,\n", + " PROV.wasGeneratedBy,\n", + " PROV.qualifiedUsage,\n", + " PROV.entity,\n", + " LAB.evidenceState,\n", + " ),\n", + " follow=(PROV.qualifiedUsage,),\n", + " prefixes={\"build\": request.identity + \"/\", \"prep\": preparation.identity + \"/\"},\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-12", + "metadata": {}, + "source": [ + "`prov:wasGeneratedBy` identifies the preparation activity that would produce the new implementation. `prov:qualifiedUsage` points to a usage record whose `prov:entity` is the received stab. The output still refers to the same vector design, but it has **a new material identity**.\n", + "\n", + "Here `lab:planned` matters: these edges describe intended work. They do not say P01 has happened. There is no `sbol:built` assertion because no resulting structure has been verified. `require_ready()` established that the route and required inputs were resolved, not that the bench work was complete.\n" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-13", + "metadata": {}, + "source": [ + "## 3. What would the robot do with the prepared DNA?\n", + "\n", + "Build the experiment and compile it for an OT-2. Preparation stays a manual stage. We will inspect one operation in the next stage: the transfer from the prepared vector material into the assembly mixture.\n", + "\n", + "The arbitrary quantities below belong to the software fixture. Our focus is the link between a material, a logical operation, and its emitted code.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "reporter-14", + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "`protocol.py`, lines **24–30**" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
    # lab:step https://example.org/reporter/build/assembly_1_90ae1a318f82/protocol/step_2\n",
+       "    context.comment('Transfer 1 µL from upstream_1:A1 to products:A1.')\n",
+       "    pipette.pick_up_tip(labware_4.wells_by_name()['B1'])\n",
+       "    pipette.aspirate(1, labware_3.wells_by_name()['A1'])\n",
+       "    pipette.dispense(1, labware_2.wells_by_name()['A1'])\n",
+       "    pipette.drop_tip()\n",
+       "    # lab:end https://example.org/reporter/build/assembly_1_90ae1a318f82/protocol/step_2\n",
+       "
\n" + ], + "text/latex": [ + "\\begin{Verbatim}[commandchars=\\\\\\{\\}]\n", + " \\PY{c+c1}{\\PYZsh{} lab:step https://example.org/reporter/build/assembly\\PYZus{}1\\PYZus{}90ae1a318f82/protocol/step\\PYZus{}2}\n", + " \\PY{n}{context}\\PY{o}{.}\\PY{n}{comment}\\PY{p}{(}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{Transfer 1 µL from upstream\\PYZus{}1:A1 to products:A1.}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{)}\n", + " \\PY{n}{pipette}\\PY{o}{.}\\PY{n}{pick\\PYZus{}up\\PYZus{}tip}\\PY{p}{(}\\PY{n}{labware\\PYZus{}4}\\PY{o}{.}\\PY{n}{wells\\PYZus{}by\\PYZus{}name}\\PY{p}{(}\\PY{p}{)}\\PY{p}{[}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{B1}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{]}\\PY{p}{)}\n", + " \\PY{n}{pipette}\\PY{o}{.}\\PY{n}{aspirate}\\PY{p}{(}\\PY{l+m+mi}{1}\\PY{p}{,} \\PY{n}{labware\\PYZus{}3}\\PY{o}{.}\\PY{n}{wells\\PYZus{}by\\PYZus{}name}\\PY{p}{(}\\PY{p}{)}\\PY{p}{[}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{A1}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{]}\\PY{p}{)}\n", + " \\PY{n}{pipette}\\PY{o}{.}\\PY{n}{dispense}\\PY{p}{(}\\PY{l+m+mi}{1}\\PY{p}{,} \\PY{n}{labware\\PYZus{}2}\\PY{o}{.}\\PY{n}{wells\\PYZus{}by\\PYZus{}name}\\PY{p}{(}\\PY{p}{)}\\PY{p}{[}\\PY{l+s+s1}{\\PYZsq{}}\\PY{l+s+s1}{A1}\\PY{l+s+s1}{\\PYZsq{}}\\PY{p}{]}\\PY{p}{)}\n", + " \\PY{n}{pipette}\\PY{o}{.}\\PY{n}{drop\\PYZus{}tip}\\PY{p}{(}\\PY{p}{)}\n", + " \\PY{c+c1}{\\PYZsh{} lab:end https://example.org/reporter/build/assembly\\PYZus{}1\\PYZus{}90ae1a318f82/protocol/step\\PYZus{}2}\n", + "\\end{Verbatim}\n" + ], + "text/plain": [ + " # lab:step https://example.org/reporter/build/assembly_1_90ae1a318f82/protocol/step_2\n", + " context.comment('Transfer 1 µL from upstream_1:A1 to products:A1.')\n", + " pipette.pick_up_tip(labware_4.wells_by_name()['B1'])\n", + " pipette.aspirate(1, labware_3.wells_by_name()['A1'])\n", + " pipette.dispense(1, labware_2.wells_by_name()['A1'])\n", + " pipette.drop_tip()\n", + " # lab:end https://example.org/reporter/build/assembly_1_90ae1a318f82/protocol/step_2" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "experiment = cloning.build(plan)\n", + "compiled = lab.compile(experiment, LiquidHandler.OT2)\n", + "compiled.write(output / \"ot2\")\n", + "\n", + "assembly = compiled.stages[1]\n", + "transfer = next(\n", + " step\n", + " for step in assembly.protocol.steps\n", + " if isinstance(step, Transfer) and step.source.resource == \"upstream_1\"\n", + ")\n", + "view.source(assembly, transfer.identity)" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-15", + "metadata": {}, + "source": [ + "The `lab:step` comment is a stable operation identity. The source map connects that identity to these exact lines. Well bindings and robot API calls belong to the target compilation.\n", + "\n", + "**The same operation in `protocol.labop.ttl`** is a behavior call. This excerpt follows just its amount parameter; source and destination sample selections and their object-flow edges remain in the full file.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "reporter-16", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .\n",
+       "@prefix sbol: <http://sbols.org/v3#> .\n",
+       "@prefix uml: <http://bioprotocols.org/uml#> .\n",
+       "@prefix om: <http://www.ontology-of-units-of-measure.org/resource/om-2/> .\n",
+       "@prefix stage: <https://example.org/reporter/build/assembly_1_90ae1a318f82/protocol/> .\n",
+       "@prefix step: <https://example.org/reporter/build/assembly_1_90ae1a318f82/protocol/step_2/> .\n",
+       "@prefix pin: <https://example.org/reporter/build/assembly_1_90ae1a318f82/protocol/step_2/input_amount/> .\n",
+       "@prefix amount: <https://example.org/reporter/build/assembly_1_90ae1a318f82/protocol/step_2/input_amount/value/> .\n",
+       "@prefix liquid: <https://bioprotocols.org/labop/primitives/liquid_handling/> .\n",
+       "\n",
+       "stage:step_2 a uml:CallBehaviorAction ;\n",
+       "    uml:behavior liquid:Transfer ;\n",
+       "    uml:input step:input_amount .\n",
+       "\n",
+       "step:input_amount a uml:ValuePin ;\n",
+       "    uml:value pin:value ;\n",
+       "    sbol:name \"amount\" .\n",
+       "\n",
+       "pin:value a uml:LiteralIdentified ;\n",
+       "    uml:identifiedValue amount:measure .\n",
+       "\n",
+       "amount:measure a om:Measure ;\n",
+       "    om:hasNumericalValue \"1\"^^xsd:decimal ;\n",
+       "    om:hasUnit om:microlitre .\n",
+       "
\n" + ], + "text/latex": [ + "\\begin{Verbatim}[commandchars=\\\\\\{\\}]\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{xsd:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}http://www.w3.org/2001/XMLSchema\\PYZsh{}\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{sbol:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}http://sbols.org/v3\\PYZsh{}\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{uml:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}http://bioprotocols.org/uml\\PYZsh{}\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{om:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}http://www.ontology\\PYZhy{}of\\PYZhy{}units\\PYZhy{}of\\PYZhy{}measure.org/resource/om\\PYZhy{}2/\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{stage:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}https://example.org/reporter/build/assembly\\PYZus{}1\\PYZus{}90ae1a318f82/protocol/\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{step:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}https://example.org/reporter/build/assembly\\PYZus{}1\\PYZus{}90ae1a318f82/protocol/step\\PYZus{}2/\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{pin:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}https://example.org/reporter/build/assembly\\PYZus{}1\\PYZus{}90ae1a318f82/protocol/step\\PYZus{}2/input\\PYZus{}amount/\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{amount:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}https://example.org/reporter/build/assembly\\PYZus{}1\\PYZus{}90ae1a318f82/protocol/step\\PYZus{}2/input\\PYZus{}amount/value/\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\\PY{k}{@prefix}\\PY{+w}{ }\\PY{n+nn}{liquid:}\\PY{+w}{ }\\PY{n+nv}{\\PYZlt{}https://bioprotocols.org/labop/primitives/liquid\\PYZus{}handling/\\PYZgt{}}\\PY{+w}{ }\\PY{p}{.}\n", + "\n", + "\\PY{n+nn}{stage}\\PY{p}{:}\\PY{n+nt}{step\\PYZus{}2} \\PY{k+kt}{a} \\PY{n+nn}{uml}\\PY{p}{:}\\PY{n+nt}{CallBehaviorAction} \\PY{p}{;}\n", + " \\PY{n+nn}{uml}\\PY{p}{:}\\PY{n+nt}{behavior} \\PY{n+nn}{liquid}\\PY{p}{:}\\PY{n+nt}{Transfer} \\PY{p}{;}\n", + " \\PY{n+nn}{uml}\\PY{p}{:}\\PY{n+nt}{input} \\PY{n+nn}{step}\\PY{p}{:}\\PY{n+nt}{input\\PYZus{}amount} \\PY{p}{.}\n", + "\n", + "\\PY{n+nn}{step}\\PY{p}{:}\\PY{n+nt}{input\\PYZus{}amount} \\PY{k+kt}{a} \\PY{n+nn}{uml}\\PY{p}{:}\\PY{n+nt}{ValuePin} \\PY{p}{;}\n", + " \\PY{n+nn}{uml}\\PY{p}{:}\\PY{n+nt}{value} \\PY{n+nn}{pin}\\PY{p}{:}\\PY{n+nt}{value} \\PY{p}{;}\n", + " \\PY{n+nn}{sbol}\\PY{p}{:}\\PY{n+nt}{name} \\PY{l+s}{\\PYZdq{}}\\PY{l+s}{amount}\\PY{l+s}{\\PYZdq{}} \\PY{p}{.}\n", + "\n", + "\\PY{n+nn}{pin}\\PY{p}{:}\\PY{n+nt}{value} \\PY{k+kt}{a} \\PY{n+nn}{uml}\\PY{p}{:}\\PY{n+nt}{LiteralIdentified} \\PY{p}{;}\n", + " \\PY{n+nn}{uml}\\PY{p}{:}\\PY{n+nt}{identifiedValue} \\PY{n+nn}{amount}\\PY{p}{:}\\PY{n+nt}{measure} \\PY{p}{.}\n", + "\n", + "\\PY{n+nn}{amount}\\PY{p}{:}\\PY{n+nt}{measure} \\PY{k+kt}{a} \\PY{n+nn}{om}\\PY{p}{:}\\PY{n+nt}{Measure} \\PY{p}{;}\n", + " \\PY{n+nn}{om}\\PY{p}{:}\\PY{n+nt}{hasNumericalValue} \\PY{l+s}{\\PYZdq{}}\\PY{l+s}{1}\\PY{l+s}{\\PYZdq{}}\\PY{p}{\\PYZca{}}\\PY{p}{\\PYZca{}}\\PY{n+nn}{xsd}\\PY{p}{:}\\PY{n+nt}{decimal} \\PY{p}{;}\n", + " \\PY{n+nn}{om}\\PY{p}{:}\\PY{n+nt}{hasUnit} \\PY{n+nn}{om}\\PY{p}{:}\\PY{n+nt}{microlitre} \\PY{p}{.}\n", + "\\end{Verbatim}\n" + ], + "text/plain": [ + "@prefix xsd: .\n", + "@prefix sbol: .\n", + "@prefix uml: .\n", + "@prefix om: .\n", + "@prefix stage: .\n", + "@prefix step: .\n", + "@prefix pin: .\n", + "@prefix amount: .\n", + "@prefix liquid: .\n", + "\n", + "stage:step_2 a uml:CallBehaviorAction ;\n", + " uml:behavior liquid:Transfer ;\n", + " uml:input step:input_amount .\n", + "\n", + "step:input_amount a uml:ValuePin ;\n", + " uml:value pin:value ;\n", + " sbol:name \"amount\" .\n", + "\n", + "pin:value a uml:LiteralIdentified ;\n", + " uml:identifiedValue amount:measure .\n", + "\n", + "amount:measure a om:Measure ;\n", + " om:hasNumericalValue \"1\"^^xsd:decimal ;\n", + " om:hasUnit om:microlitre ." + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "view.labop_amount(output / \"ot2/protocol.labop.ttl\", transfer.identity)" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-17", + "metadata": {}, + "source": [ + "Follow the chain in the excerpt:\n", + "\n", + "1. `uml:behavior liquid:Transfer` names the operation's meaning.\n", + "2. `uml:input` links the action to a parameter pin named `amount`.\n", + "3. `uml:value` and `uml:identifiedValue` lead to a measure.\n", + "4. `om:hasNumericalValue` and `om:hasUnit` carry its quantity and unit explicitly. `\"1\"^^xsd:decimal` is a typed numeric value.\n", + "\n", + "This is why the LabOP file is more than a prose description. Another implementation can interpret the operation and its parameters independently of OT-2 deck slots. The full document also includes stage calls, sample collections, and control flow. Operations without an exact upstream primitive use explicit Lab extensions, whose schemas travel with the bundle.\n", + "\n", + "**SBOL answers “which materials, designs, and provenance?” LabOP answers “which operations and flows?”** Both refer to this same experiment. Compilation does not invoke a LabOP execution engine.\n" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-26", + "metadata": {}, + "source": [ + "## 4. What is in the compilation bundle?\n", + "\n", + "The bundle preserves the planned procedure and connects its representations:\n", + "\n", + "| File | Question it answers |\n", + "|---|---|\n", + "| `ot2/provenance.ttl` | Which designs and materials does the planned experiment connect? |\n", + "| `ot2/protocol.labop.ttl` | What procedure is specified, including operations, parameters, and material and control flow? |\n", + "| `ot2/methods.md` | How can a reader inspect the planned procedure in prose? |\n", + "| `ot2/stages//protocol.py` | What program implements an automated stage for the selected robot? |\n", + "\n", + "These artifacts specify the intended preparation, assembly, transformation, and plating. Expected yields and products remain part of the plan; compilation does not establish that the reporter was built, colonies were verified, or fluorescence was measured.\n", + "\n", + "The generated artifacts below preserve the complete graphs. The notebook's excerpts are selected triples from those files, reserialized with prefixes; the compiler's canonical files use full IRIs for deterministic output.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "reporter-27", + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "- [Planned SBOL](../../build/cloning-notebook/reporter-675dh7jn/ot2/provenance.ttl)\n", + "- [LabOP procedure](../../build/cloning-notebook/reporter-675dh7jn/ot2/protocol.labop.ttl)\n", + "- [Planned methods](../../build/cloning-notebook/reporter-675dh7jn/ot2/methods.md)" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "view.links(\n", + " output,\n", + " {\n", + " \"Planned SBOL\": \"ot2/provenance.ttl\",\n", + " \"LabOP procedure\": \"ot2/protocol.labop.ttl\",\n", + " \"Planned methods\": \"ot2/methods.md\",\n", + " },\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-28", + "metadata": {}, + "source": [ + "## Optional: other robots and integrity checks\n", + "\n", + "The same `experiment` can be compiled for Flex or STAR. The target-specific source and deck configuration change; the semantic protocols and LabOP remain shared. The cell below also verifies checksums and SBOL validation. These checks establish software-artifact consistency, not experimental success or physical instrument qualification.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "reporter-29", + "metadata": { + "jupyter": { + "source_hidden": true + } + }, + "outputs": [ + { + "data": { + "text/html": [ + "
BundleVerified file checksums
ot244
flex44
star44
" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "SBOL validation passed; the planned LabOP protocol is identical across all three targets.\n" + ] + } + ], + "source": [ + "for hardware in (LiquidHandler.FLEX, LiquidHandler.STAR):\n", + " alternative = lab.compile(experiment, hardware)\n", + " alternative.write(output / hardware.value)\n", + " assert alternative.files[\"protocol.labop.ttl\"] == compiled.files[\"protocol.labop.ttl\"]\n", + "\n", + "view.table(\n", + " (\"Bundle\", \"Verified file checksums\"),\n", + " [(name, view.check_bundle(output / name)) for name in (\"ot2\", \"flex\", \"star\")],\n", + ")\n", + "assert not experiment.provenance.to_sbol3().validate().errors\n", + "prepared = experiment.provenance.resolve(preparation.output)\n", + "assert prepared.built is None and prepared.evidence_state is EvidenceState.PLANNED\n", + "protocol_graph = Graph().parse(output / \"ot2/protocol.labop.ttl\", format=\"turtle\")\n", + "assert not tuple(protocol_graph.subjects(RDF.type, LABOP.ProtocolExecution))\n", + "print(\"SBOL validation passed; the planned LabOP protocol is identical across all three targets.\")" + ] + }, + { + "cell_type": "markdown", + "id": "reporter-30", + "metadata": {}, + "source": [ + "The [case README](data/reporter_reference/README.md) explains every input file and the use of dummy sequences. The [provenance guide](../../docs/provenance.md) and [cloning guide](../../docs/cloning-provenance.md) provide the full Python APIs.\n", + "\n", + "When opening this notebook in another checkout, run it to regenerate the artifact links locally. Input files and saved cell outputs are included in the repository.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/cloning_provenance/data/reporter_reference/README.md b/examples/cloning_provenance/data/reporter_reference/README.md new file mode 100644 index 0000000..be85105 --- /dev/null +++ b/examples/cloning_provenance/data/reporter_reference/README.md @@ -0,0 +1,18 @@ +# Reporter-reference case + +An illustrative lab wants to prepare two deposited samples of a reporter reference for a later fluorescence study. Its starting records include vector receipt R-042 (lot V-17), insert aliquot I-07, and host aliquot H1-03. Procedure P01 is an external preparation boundary: the vector arrives as a counted stab, not an available DNA aliquot. + +The names, receipt, lots, locations, and study are fictional. This case supplies realistic record-keeping relationships, not a reproduced experiment. The short DNA strings are compiler surrogates, not the sequences of an actual reporter or plasmid; quantities, cut selections, and method parameters are the arbitrary software fixture from `examples/cloning_workflow.py`. No sequence or fluorescence verification is asserted. + +## Input files + +| File | What it supplies | +|---|---| +| `designs-and-materials.ttl` | SBOL3 designs, named material implementations, and their asserted relationships. Readable Turtle with namespace prefixes. | +| `inventory.json` | Available material forms, quantities, and storage locations, referencing the SBOL identities. | +| `system.json` | Explicit preparation, assembly, transformation, and plating routes. | +| `methods.json` | The method identity and parameters selected by each route. | + +The notebook reads these files directly using `Document.read`, `Inventory.read`, `CloningSystem.read`, and `CloningMethods.read`. It does not substitute objects held inside a display helper. The presentation helpers in [notebook_views.py](../../notebook_views.py) inspect the resulting objects and RDF; their Turtle excerpts are checked to contain only triples from the named source file. + +The walkthrough follows receipt R-042 through the planned preparation and downstream protocol. Its generated SBOL, LabOP, methods, and robot programs describe the intended work and expected outputs. diff --git a/examples/cloning_provenance/data/reporter_reference/designs-and-materials.ttl b/examples/cloning_provenance/data/reporter_reference/designs-and-materials.ttl new file mode 100644 index 0000000..7534e43 --- /dev/null +++ b/examples/cloning_provenance/data/reporter_reference/designs-and-materials.ttl @@ -0,0 +1,136 @@ +@prefix edam: . +@prefix ex: . +@prefix lab: . +@prefix prov: . +@prefix sbo: . +@prefix sbol: . +@prefix so: . + + a sbol:Implementation ; + sbol:description "Illustrative reagent stock." ; + sbol:displayId "material" ; + sbol:hasNamespace ; + sbol:name "broth/material" ; + prov:wasDerivedFrom ex:broth ; + lab:evidenceState lab:recorded . + + a sbol:Implementation ; + sbol:description "Illustrative reagent stock." ; + sbol:displayId "implementation" ; + sbol:hasNamespace ; + sbol:name "buffer/implementation" ; + prov:wasDerivedFrom ex:buffer ; + lab:evidenceState lab:recorded . + + a sbol:Implementation ; + sbol:description "Illustrative aliquot in cell box H1, position A1." ; + sbol:displayId "material" ; + sbol:hasNamespace ; + sbol:name "Host aliquot H1-03" ; + prov:wasDerivedFrom ex:cells ; + lab:evidenceState lab:recorded . + + a sbol:Implementation ; + sbol:description "Illustrative aliquot in DNA rack D1, position C2." ; + sbol:displayId "implementation" ; + sbol:hasNamespace ; + sbol:name "Insert aliquot I-07" ; + prov:wasDerivedFrom ex:insert ; + lab:evidenceState lab:recorded . + +ex:received_stab a sbol:Implementation ; + sbol:description "Illustrative received vector stab; logged in receipt box B3." ; + sbol:displayId "received_stab" ; + sbol:hasNamespace ; + sbol:name "Receipt R-042 / lot V-17" ; + prov:wasDerivedFrom ex:vector ; + lab:evidenceState lab:recorded . + +ex:strain a sbol:Component ; + sbol:description "Intended reporter reference material; no fluorescence or sequence verification is asserted." ; + sbol:displayId "strain" ; + sbol:hasNamespace ; + sbol:name "H1 / pReference-Green" ; + sbol:type . + +ex:substrate a sbol:Component ; + sbol:description "Supplied consumable; automatic consumable reservation is outside this example." ; + sbol:displayId "substrate" ; + sbol:hasNamespace ; + sbol:name "Deposition substrate" ; + sbol:type sbo:0000247 . + +ex:target a sbol:Component ; + sbol:description "Intended reporter reference construct for the illustrative study." ; + sbol:displayId "target" ; + sbol:hasNamespace ; + sbol:name "pReference-Green" ; + sbol:type sbo:0000251, + so:0000988 . + + a sbol:Implementation ; + sbol:description "Illustrative reagent stock." ; + sbol:displayId "implementation" ; + sbol:hasNamespace ; + sbol:name "water/implementation" ; + prov:wasDerivedFrom ex:water ; + lab:evidenceState lab:recorded . + +ex:broth a sbol:Component ; + sbol:description "Supplied reagent in the illustrative inventory." ; + sbol:displayId "broth" ; + sbol:hasNamespace ; + sbol:name "Recovery and dilution medium" ; + sbol:type sbo:0000247 . + +ex:buffer a sbol:Component ; + sbol:description "Supplied reagent in the illustrative inventory." ; + sbol:displayId "buffer" ; + sbol:hasNamespace ; + sbol:name "Assembly buffer" ; + sbol:type sbo:0000247 . + +ex:cells a sbol:Component ; + sbol:description "A named recipient stock in the illustrative laboratory record." ; + sbol:displayId "cells" ; + sbol:hasNamespace ; + sbol:name "Host H1" ; + sbol:type . + +ex:insert a sbol:Component ; + sbol:description "Illustrative cassette design; the sequence is a compiler surrogate, not a reporter coding sequence." ; + sbol:displayId "insert" ; + sbol:hasNamespace ; + sbol:hasSequence ex:insert_sequence ; + sbol:name "Green reporter cassette" ; + sbol:type sbo:0000251, + so:0000987 . + +ex:insert_sequence a sbol:Sequence ; + sbol:displayId "insert_sequence" ; + sbol:elements "GGGAATTCCCCGAATTCGG" ; + sbol:encoding edam:format_1207 ; + sbol:hasNamespace . + +ex:vector a sbol:Component ; + sbol:description "Illustrative reporter-study design; its short sequence is a compiler surrogate, not a usable plasmid." ; + sbol:displayId "vector" ; + sbol:hasNamespace ; + sbol:hasSequence ex:vector_sequence ; + sbol:name "pReference backbone" ; + sbol:type sbo:0000251, + so:0000988 . + +ex:vector_sequence a sbol:Sequence ; + sbol:displayId "vector_sequence" ; + sbol:elements "AAAAGAATTCTTTT" ; + sbol:encoding edam:format_1207 ; + sbol:hasNamespace . + +ex:water a sbol:Component ; + sbol:description "Supplied reagent in the illustrative inventory." ; + sbol:displayId "water" ; + sbol:hasNamespace ; + sbol:name "Diluent" ; + sbol:type sbo:0000247 . + diff --git a/examples/cloning_provenance/data/reporter_reference/inventory.json b/examples/cloning_provenance/data/reporter_reference/inventory.json new file mode 100644 index 0000000..49153fa --- /dev/null +++ b/examples/cloning_provenance/data/reporter_reference/inventory.json @@ -0,0 +1,109 @@ +{ + "format": "lab.inventory.v1", + "inventory": { + "identity": "https://example.org/reporter/integrated_inventory", + "stocks": [ + { + "concentration_ng_ul": null, + "design": { + "identity": "https://example.org/reporter/broth" + }, + "form": "reagent", + "identity": "https://example.org/reporter/broth/stock", + "implementation": { + "identity": "https://example.org/reporter/broth/material" + }, + "location": { + "container": "Reagent rack", + "position": "broth" + }, + "quantity_ul": "20", + "supplier_item": null + }, + { + "concentration_ng_ul": null, + "design": { + "identity": "https://example.org/reporter/buffer" + }, + "form": "reagent", + "identity": "https://example.org/reporter/buffer/stock", + "implementation": { + "identity": "https://example.org/reporter/buffer/implementation" + }, + "location": { + "container": "Reagent rack", + "position": "buffer" + }, + "quantity_ul": "10", + "supplier_item": null + }, + { + "concentration_ng_ul": null, + "design": { + "identity": "https://example.org/reporter/cells" + }, + "form": "competent_cells", + "identity": "https://example.org/reporter/cells/stock", + "implementation": { + "identity": "https://example.org/reporter/cells/material" + }, + "location": { + "container": "Cell box H1", + "position": "A1" + }, + "quantity_ul": "20", + "supplier_item": null + }, + { + "concentration_ng_ul": null, + "design": { + "identity": "https://example.org/reporter/insert" + }, + "form": "dna", + "identity": "https://example.org/reporter/insert/stock", + "implementation": { + "identity": "https://example.org/reporter/insert/implementation" + }, + "location": { + "container": "DNA rack D1", + "position": "C2" + }, + "quantity_ul": "10", + "supplier_item": null + }, + { + "count": 1, + "design": { + "identity": "https://example.org/reporter/vector" + }, + "form": "bacterial_stab", + "identity": "https://example.org/reporter/stab_stock", + "implementation": { + "identity": "https://example.org/reporter/received_stab" + }, + "location": { + "container": "Receipt box", + "position": "B3" + }, + "supplier_item": null + }, + { + "concentration_ng_ul": null, + "design": { + "identity": "https://example.org/reporter/water" + }, + "form": "reagent", + "identity": "https://example.org/reporter/water/stock", + "implementation": { + "identity": "https://example.org/reporter/water/implementation" + }, + "location": { + "container": "Reagent rack", + "position": "water" + }, + "quantity_ul": "10", + "supplier_item": null + } + ] + } +} diff --git a/examples/cloning_provenance/data/reporter_reference/methods.json b/examples/cloning_provenance/data/reporter_reference/methods.json new file mode 100644 index 0000000..663cfcf --- /dev/null +++ b/examples/cloning_provenance/data/reporter_reference/methods.json @@ -0,0 +1,97 @@ +{ + "format": "lab.cloning-methods.v1", + "methods": { + "assemblies": [ + { + "cycles": 1, + "diluent": { + "identity": "https://example.org/reporter/water" + }, + "dna_volume_ul": "1", + "enzyme": "EcoRI", + "identity": "https://example.org/reporter/method", + "lid_celsius": null, + "mix_cycles": 1, + "mix_volume_ul": "2", + "output_volume_ul": "5", + "profile": [ + { + "celsius": "25", + "seconds": "1" + } + ], + "reaction_volume_ul": "5", + "reagents": [ + { + "component": { + "identity": "https://example.org/reporter/buffer" + }, + "volume_ul": "1" + } + ] + } + ], + "platings": [ + { + "diluent": { + "identity": "https://example.org/reporter/broth" + }, + "dilution_factors": [ + "2", + "3" + ], + "identity": "https://example.org/reporter/plating_method", + "mix_cycles": 1, + "mix_volume_ul": "1", + "spot_height_mm": "2", + "spot_volume_ul": "1", + "substrate": { + "identity": "https://example.org/reporter/substrate" + }, + "transfer_volume_ul": "1" + } + ], + "preparations": [ + { + "identity": "https://example.org/reporter/preparation_method", + "instructions": "Follow the supplied preparation procedure P01; record the recovered material identity and measured quantity.", + "output_count": 0, + "output_volume_ul": "10", + "procedure": "https://example.org/reporter/SOP_P01", + "reagents": [], + "source_count": 1, + "source_volume_ul": "0" + } + ], + "transformations": [ + { + "cell_mix_cycles": 1, + "cell_mix_volume_ul": "1", + "cell_volume_ul": "2", + "dna_mix_cycles": 1, + "dna_volume_ul": "1", + "identity": "https://example.org/reporter/transformation_method", + "initial_celsius": "25", + "output_volume_ul": "5", + "profile": [ + { + "celsius": "25", + "seconds": "1" + } + ], + "recovery": { + "component": { + "identity": "https://example.org/reporter/broth" + }, + "volume_ul": "2" + }, + "recovery_profile": [ + { + "celsius": "25", + "seconds": "1" + } + ] + } + ] + } +} diff --git a/examples/cloning_provenance/data/reporter_reference/system.json b/examples/cloning_provenance/data/reporter_reference/system.json new file mode 100644 index 0000000..fd50d47 --- /dev/null +++ b/examples/cloning_provenance/data/reporter_reference/system.json @@ -0,0 +1,73 @@ +{ + "format": "lab.cloning-system.v1", + "system": { + "identity": "https://example.org/reporter/complete_system", + "recipes": [ + { + "identity": "https://example.org/reporter/plating_recipe", + "kind": "plating", + "method": "https://example.org/reporter/plating_method", + "product": { + "identity": "https://example.org/reporter/strain" + } + }, + { + "identity": "https://example.org/reporter/preparation_recipe", + "kind": "preparation", + "method": "https://example.org/reporter/preparation_method", + "output_form": "dna", + "product": { + "identity": "https://example.org/reporter/vector" + }, + "source": { + "identity": "https://example.org/reporter/vector" + }, + "source_form": "bacterial_stab" + }, + { + "circular": true, + "enzyme": "EcoRI", + "fragments": [ + { + "component": { + "identity": "https://example.org/reporter/vector" + }, + "left_cut": 5, + "reverse_complement": false, + "right_cut": 5 + }, + { + "component": { + "identity": "https://example.org/reporter/insert" + }, + "left_cut": 3, + "reverse_complement": false, + "right_cut": 12 + } + ], + "identity": "https://example.org/reporter/recipe", + "kind": "assembly", + "method": "https://example.org/reporter/method", + "product": { + "identity": "https://example.org/reporter/target" + } + }, + { + "chassis": { + "identity": "https://example.org/reporter/cells" + }, + "identity": "https://example.org/reporter/transformation_recipe", + "kind": "transformation", + "method": "https://example.org/reporter/transformation_method", + "plasmids": [ + { + "identity": "https://example.org/reporter/target" + } + ], + "product": { + "identity": "https://example.org/reporter/strain" + } + } + ] + } +} diff --git a/examples/cloning_provenance/notebook_views.py b/examples/cloning_provenance/notebook_views.py new file mode 100644 index 0000000..70babde --- /dev/null +++ b/examples/cloning_provenance/notebook_views.py @@ -0,0 +1,279 @@ +"""Presentation helpers for the reporter-reference notebook. + +These functions present ordinary Lab objects and emitted files as tables, +diagrams, source excerpts, and selected RDF statements. +""" + +import hashlib +import json +from collections.abc import Iterable, Mapping +from html import escape +from os.path import relpath +from pathlib import Path +from tempfile import mkdtemp + +from IPython.display import HTML, SVG, Code, Markdown, display +from rdflib import RDF, Graph, Namespace, URIRef +from rdflib.term import Node + +from lab.compiler import Compilation +from lab.experiments.cloning.planning import BuildPlan +from lab.inventory import Inventory +from lab.provenance import DocumentSnapshot + +EXAMPLE = Path(__file__).resolve().parent +ROOT = EXAMPLE.parents[1] +CASE = EXAMPLE / "data/reporter_reference" +EX = Namespace("https://example.org/reporter/") +SBOL = Namespace("http://sbols.org/v3#") +PROV = Namespace("http://www.w3.org/ns/prov#") +LAB = Namespace("https://the-lab-compiler.github.io/lab-py/ns#") +UML = Namespace("http://bioprotocols.org/uml#") +LABOP = Namespace("http://bioprotocols.org/labop#") +OM = Namespace("http://www.ontology-of-units-of-measure.org/resource/om-2/") + + +def fresh_output() -> Path: + parent = ROOT / "build/cloning-notebook" + parent.mkdir(parents=True, exist_ok=True) + return Path(mkdtemp(prefix="reporter-", dir=parent)) + + +def table(headers: Iterable[object], rows: Iterable[Iterable[object]]) -> None: + head = "".join(f"{escape(str(value))}" for value in headers) + body = "".join( + "" + + "".join( + f'{escape(str(value))}' + for value in row + ) + + "" + for row in rows + ) + display(HTML(f"{head}{body}
")) + + +def inventory(inventory: Inventory, document: DocumentSnapshot) -> None: + table( + ("Material record", "Intended design", "Form", "Available", "Location"), + ( + ( + document.resolve(stock.implementation).name, + document.resolve(stock.design).name, + stock.form.value, + f"{stock.amount} {'unit' if stock.form.counted else 'µL'}", + f"{stock.location.container} / {stock.location.position}" + if stock.location + else "not assigned", + ) + for stock in inventory.stocks + if stock.form.value in {"bacterial_stab", "dna", "competent_cells"} + ), + ) + + +def plan(plan: BuildPlan) -> None: + labels = { + "preparation": "Prepare vector DNA", + "assembly": "Assemble reporter design", + "transformation": "Introduce it into host H1", + "plating": "Deposit a sample", + } + table( + ("Order", "Task", "Expected material", "Responsible system"), + ( + ( + index, + labels[task.kind.value], + f"{task.output_form.value}: " + + ( + f"{task.output_count} sample" + if task.output_form.counted + else f"{task.output_volume_ul} µL" + ), + "External procedure P01" if task.kind.value == "preparation" else "Liquid handler", + ) + for index, task in enumerate(plan.tasks, 1) + ), + ) + positions = [(16, 66), (254, 66), (492, 66), (730, 8), (730, 126)] + if len(plan.tasks) != len(positions): + return + numbers = {task.identity: index for index, task in enumerate(plan.tasks)} + parts = [ + '', + '', + '', + ] + for index, task in enumerate(plan.tasks): + x, y = positions[index] + for parent in task.depends_on: + px, py = positions[numbers[parent]] + parts.append( + f'' + ) + parts.extend( + [ + f'', + f'{index + 1}. ' + f"{escape(task.kind.value.title())}", + f'{escape(task.output_form.value)}', + ] + ) + parts.append("") + display(SVG("".join(parts))) + + +def _excerpt(source: Graph, selected: Graph, prefixes: Mapping[str, str]) -> str: + for name, namespace in { + "ex": str(EX), + "sbol": str(SBOL), + "prov": str(PROV), + "lab": str(LAB), + "uml": str(UML), + "labop": str(LABOP), + "om": str(OM), + "build": str(EX.build) + "/", + **prefixes, + }.items(): + selected.bind(name, Namespace(namespace), replace=True) + # Keep typed numeric literals explicit. RDFLib's compact Turtle writer can + # change a decimal's lexical spelling (for example "1" to 1.0). + used = set() + + def term(node: Node) -> str: + rendered = node.n3(namespace_manager=selected.namespace_manager) + for prefix, _ in selected.namespaces(): + if rendered.startswith(prefix + ":") or "^^" + prefix + ":" in rendered: + used.add(prefix) + return rendered + + blocks = [] + for subject in sorted(set(selected.subjects()), key=str): + statements = [] + for predicate in sorted( + set(selected.predicates(subject)), key=lambda value: (value != RDF.type, str(value)) + ): + objects = ", ".join( + term(obj) for obj in sorted(selected.objects(subject, predicate), key=str) + ) + statements.append(f"{'a' if predicate == RDF.type else term(predicate)} {objects}") + blocks.append(term(subject) + " " + " ;\n ".join(statements) + " .") + declarations = [ + f"@prefix {prefix}: <{namespace}> ." + for prefix, namespace in selected.namespaces() + if prefix in used + ] + text = "\n".join(declarations) + "\n\n" + "\n\n".join(blocks) + "\n" + # Excerpts must be real triples from the source, with only notation changed. + assert len(selected) and set(selected) <= set(source) + assert set(Graph().parse(data=text, format="turtle")) == set(selected) + return text + + +def turtle( + path: Path, + subjects: Iterable[str], + *, + predicates: Iterable[str] | None = None, + follow: Iterable[str] = (), + prefixes: Mapping[str, str] | None = None, +) -> None: + """Show an actual RDF subset using compact Turtle notation. + + Housekeeping identifiers, redundant SBOL base classes, and embedded semantic + JSON are omitted. Explicit subject and predicate selection keeps each view + small; the original artifact is never modified. + """ + source = Graph().parse(path, format="turtle") + selected = Graph() + allowed = None if predicates is None else {URIRef(value) for value in predicates} + follow_edges = {URIRef(value) for value in follow} + pending = [URIRef(value) for value in subjects] + visited = set() + excluded = {SBOL.displayId, SBOL.hasNamespace, LAB.semanticStep, LAB.planDigest} + while pending: + subject = pending.pop() + if subject in visited: + continue + visited.add(subject) + for triple in source.triples((subject, None, None)): + _, predicate, obj = triple + if predicate in excluded or (allowed is not None and predicate not in allowed): + continue + if predicate == RDF.type and obj in {SBOL.TopLevel, SBOL.Identified}: + continue + selected.add(triple) + if predicate in follow_edges: + pending.append(obj) + display(Code(_excerpt(source, selected, prefixes or {}), language="turtle")) + + +def labop_amount(path: Path, step: str) -> None: + """Follow an action's amount pin into its measured value in the real RDF.""" + source = Graph().parse(path, format="turtle") + action = URIRef(step) + pin = URIRef(step + "/input_amount") + value = source.value(pin, UML.value, any=False) + if value is None: + raise ValueError("Select an action with an explicit amount parameter") + measure = source.value(value, UML.identifiedValue, any=False) + if measure is None: + raise ValueError("Select an action with an explicit amount parameter") + selected = Graph() + for subject, predicates in ( + (action, (RDF.type, UML.behavior)), + (pin, (RDF.type, SBOL.name, UML.value)), + (value, (RDF.type, UML.identifiedValue)), + (measure, (RDF.type, OM.hasNumericalValue, OM.hasUnit)), + ): + for predicate in predicates: + for triple in source.triples((subject, predicate, None)): + if predicate != RDF.type or triple[2] != SBOL.Identified: + selected.add(triple) + selected.add((action, UML.input, pin)) + protocol = step.rsplit("/", 1)[0] + display( + Code( + _excerpt( + source, + selected, + { + "stage": protocol + "/", + "step": step + "/", + "pin": str(pin) + "/", + "amount": str(value) + "/", + "liquid": "https://bioprotocols.org/labop/primitives/liquid_handling/", + }, + ), + language="turtle", + ) + ) + + +def source(stage: Compilation, step: str) -> None: + span = next(row for row in stage.source_map if row["step"] == step) + if stage.target.source is None: + raise ValueError("Select an automated stage") + lines = stage.target.source.splitlines() + display(Markdown(f"`protocol.py`, lines **{span['start_line']}–{span['end_line']}**")) + display(Code("\n".join(lines[span["start_line"] - 1 : span["end_line"]]), language="python")) + + +def links(directory: Path, paths: Mapping[str, str]) -> None: + relative = Path(relpath(directory, start=EXAMPLE)).as_posix() + display(Markdown("\n".join(f"- [{label}]({relative}/{path})" for label, path in paths.items()))) + + +def check_bundle(directory: Path) -> int: + checksums = json.loads((directory / "bundle.json").read_text())["sha256"] + for name, expected in checksums.items(): + assert hashlib.sha256((directory / name).read_bytes()).hexdigest() == expected + return len(checksums)