docs: address issue #1690 (fixes #1690)

samples/README.md

@@ -0,0 +1,11 @@
+# Dozer samples
+
+Sample applications showing how to use Dozer together with other tools.
+
+## LLM vector LangChain sample
+
+The [`llm-vector-langchain`](llm-vector-langchain/README.md) sample implements
+the use case from the [Dozer + LLM article](https://getdozer.io/blog/llm-chatbot):
+Dozer sources customer profiles, transactions, and credit-card products, the
+data is exposed through the Dozer API, and a LangChain application indexes it in
+Chroma to answer personalized credit-card recommendation questions.

samples/llm-vector-langchain/README.md

@@ -0,0 +1,99 @@
+# Dozer + LangChain credit-card recommendation sample
+
+This sample is a working implementation of the use case described in the
+[Dozer + LLM chatbot article](https://getdozer.io/blog/llm-chatbot): a bank
+builds an LLM-powered assistant that uses Dozer to assemble a unified customer
+picture (profile, transactions, products) and passes it to an LLM as context to
+hyper-personalize credit-card recommendations.
+
+It shows three pieces working together:
+
+- **Dozer** is configured to source from three datasets at once: customer
+  profiles, transactions, and credit-card products.
+- The datasets are exposed through the Dozer API.
+- A **LangChain** application reads the Dozer API, indexes the data in a
+  **vector database** (Chroma), and answers a personalized product
+  recommendation question with an LLM.
+
+## Repository layout
+
+```
+llm-vector-langchain/
+├── dozer-config.yaml              # Dozer app configuration
+├── data/
+│   ├── customer_profiles/         # customer_profiles.csv
+│   ├── transactions/              # transactions.csv
+│   └── credit_card_products/      # credit_card_products.csv
+├── app.py                         # LangChain entry point
+├── dozer_client.py                # reads datasets from the Dozer API
+├── recommender.py                 # context building + offline ranking helpers
+├── requirements.txt
+└── tests/
+    ├── test_recommender.py        # offline tests for recommender.py
+    └── test_dozer_config.py       # checks dozer-config.yaml + CSV data
+```
+
+## How it works
+
+`dozer-config.yaml` declares a single `LocalStorage` connection that reads three
+CSV datasets, three sources that map those tables into Dozer, and a `Dummy`
+sink for each one so every dataset is materialized and available through the
+Dozer API:
+
+| Endpoint                   | Contents                                            |
+| -------------------------- | --------------------------------------------------- |
+| `/customer_profiles`       | customer segment, income band, and stated goals     |
+| `/transactions`            | per-customer transaction history                    |
+| `/credit_card_products`    | available card products and eligibility metadata    |
+
+`app.py` fetches the three datasets from the Dozer API, builds one document per
+customer profile and one per product, indexes them in Chroma, and answers a
+question such as *"Which credit card should this customer be offered and why?"*.
+
+If `OPENAI_API_KEY` is set, embeddings and the final answer use OpenAI through
+LangChain. Without a key the sample uses a deterministic local embedding and a
+local answer, so it stays runnable offline and in CI.
+
+If the Dozer API is not reachable, the application falls back to reading the
+same CSV files under `data/`, so the recommendation flow can be exercised even
+without a running Dozer instance.
+
+## Run the sample
+
+Prerequisites: a Rust toolchain for Dozer and Python 3.10+ for the application.
+
+Start Dozer from this directory:
+
+```bash
+dozer run
+```
+
+In another terminal, install dependencies and run the app:
+
+```bash
+python -m venv .venv
+. .venv/bin/activate
+pip install -r requirements.txt
+python app.py --customer-id cust_001
+```
+
+By default the app reads the Dozer API at `http://localhost:8080`. Point it at a
+different host with `DOZER_BASE_URL`:
+
+```bash
+DOZER_BASE_URL=http://localhost:8080 python app.py --customer-id cust_003
+```
+
+Ask for a different customer or a different question:
+
+```bash
+python app.py --customer-id cust_004 --question "Which card fits a frequent traveler?"
+```
+
+## Run the tests
+
+The tests do not need Dozer, a vector database, or an LLM:
+
+```bash
+python -m unittest discover -s tests
+```

samples/llm-vector-langchain/app.py

@@ -0,0 +1,174 @@
+from __future__ import annotations
+
+import argparse
+import hashlib
+import os
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+from langchain_chroma import Chroma
+from langchain_core.documents import Document
+from langchain_core.prompts import ChatPromptTemplate
+from langchain_openai import ChatOpenAI, OpenAIEmbeddings
+from langchain_text_splitters import RecursiveCharacterTextSplitter
+
+from dozer_client import DEFAULT_BASE_URL, load_datasets
+from recommender import build_customer_context, build_product_context, rank_products
+
+
+@dataclass(frozen=True)
+class RetrievalBundle:
+    documents: list[Document]
+    customer_context: str
+    ranked_products: list[dict[str, Any]]
+
+
+class HashEmbedding:
+    """Deterministic local embedding used when no OpenAI key is configured.
+
+    Good enough for the sample's retrieval flow and keeps the sample runnable
+    offline and in CI.
+    """
+
+    def __init__(self, dimensions: int = 64) -> None:
+        self.dimensions = dimensions
+
+    def embed_documents(self, texts: list[str]) -> list[list[float]]:
+        return [self._embed(text) for text in texts]
+
+    def embed_query(self, text: str) -> list[float]:
+        return self._embed(text)
+
+    def _embed(self, text: str) -> list[float]:
+        vector = [0.0] * self.dimensions
+        for token in text.lower().split():
+            digest = hashlib.sha256(token.encode("utf-8")).digest()
+            index = int.from_bytes(digest[:4], byteorder="big") % self.dimensions
+            vector[index] += 1.0
+        magnitude = sum(value * value for value in vector) ** 0.5 or 1.0
+        return [value / magnitude for value in vector]
+
+
+def load_bundle(datasets: dict[str, list[dict[str, Any]]], customer_id: str) -> RetrievalBundle:
+    customers = datasets["customer_profiles"]
+    transactions = datasets["transactions"]
+    products = datasets["credit_card_products"]
+
+    customer = next(
+        (row for row in customers if row.get("customer_id") == customer_id),
+        None,
+    )
+    if customer is None:
+        raise ValueError(f"Customer {customer_id!r} was not found in the datasets")
+
+    customer_transactions = [
+        row for row in transactions if row.get("customer_id") == customer_id
+    ]
+    customer_context = build_customer_context(customer, customer_transactions)
+    documents = [
+        Document(
+            page_content=customer_context,
+            metadata={"kind": "customer", "customer_id": customer_id},
+        )
+    ]
+    documents.extend(
+        Document(
+            page_content=build_product_context(product),
+            metadata={"kind": "product", "product_id": product["product_id"]},
+        )
+        for product in products
+    )
+
+    return RetrievalBundle(
+        documents=documents,
+        customer_context=customer_context,
+        ranked_products=rank_products(customer, customer_transactions, products),
+    )
+
+
+def build_vector_store(documents: list[Document]) -> Chroma:
+    splitter = RecursiveCharacterTextSplitter(chunk_size=700, chunk_overlap=80)
+    chunks = splitter.split_documents(documents)
+    embeddings = (
+        OpenAIEmbeddings()
+        if os.getenv("OPENAI_API_KEY")
+        else HashEmbedding()
+    )
+    return Chroma.from_documents(chunks, embedding=embeddings)
+
+
+def answer(bundle: RetrievalBundle, question: str) -> str:
+    vector_store = build_vector_store(bundle.documents)
+    retrieved = vector_store.as_retriever(search_kwargs={"k": 4}).invoke(question)
+    retrieved_context = "\n\n".join(doc.page_content for doc in retrieved)
+    ranked_context = "\n".join(
+        f"- {product['name']}: score={product['score']}, reason={product['reason']}"
+        for product in bundle.ranked_products
+    )
+
+    if not os.getenv("OPENAI_API_KEY"):
+        best = bundle.ranked_products[0]
+        return (
+            f"Recommended card: {best['name']}.\n"
+            f"Reason: {best['reason']}.\n\n"
+            f"Retrieved context:\n{retrieved_context}"
+        )
+
+    prompt = ChatPromptTemplate.from_messages(
+        [
+            (
+                "system",
+                "You recommend credit-card products using only the supplied "
+                "Dozer context. Mention the card, why it fits, and one caveat.",
+            ),
+            (
+                "human",
+                "Customer context:\n{customer_context}\n\n"
+                "Ranked products:\n{ranked_context}\n\n"
+                "Retrieved context:\n{retrieved_context}\n\n"
+                "Question: {question}",
+            ),
+        ]
+    )
+    model = ChatOpenAI(model="gpt-4o-mini", temperature=0)
+    message = (prompt | model).invoke(
+        {
+            "customer_context": bundle.customer_context,
+            "ranked_context": ranked_context,
+            "retrieved_context": retrieved_context,
+            "question": question,
+        }
+    )
+    return str(message.content)
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Personalized credit-card recommendation using Dozer + LangChain."
+    )
+    parser.add_argument("--customer-id", default="cust_001")
+    parser.add_argument(
+        "--question",
+        default="Which credit card should this customer be offered and why?",
+    )
+    parser.add_argument(
+        "--base-url",
+        default=os.getenv("DOZER_BASE_URL", DEFAULT_BASE_URL),
+        help="Base URL of the Dozer REST API.",
+    )
+    parser.add_argument(
+        "--data-dir",
+        default=Path(__file__).resolve().parent / "data",
+        type=Path,
+        help="Local CSV datasets used when the Dozer API is not reachable.",
+    )
+    args = parser.parse_args()
+
+    datasets = load_datasets(args.base_url, args.data_dir)
+    bundle = load_bundle(datasets, args.customer_id)
+    print(answer(bundle, args.question))
+
+
+if __name__ == "__main__":
+    main()

samples/llm-vector-langchain/data/credit_card_products/credit_card_products.csv

@@ -0,0 +1,5 @@
+product_id,name,annual_fee_usd,rewards_focus,intro_offer,eligibility,caveat
+card_001,Aurora Travel Signature,395,"travel dining lounge","75000 bonus points after qualifying spend","premium income and excellent credit","High annual fee only pays off for frequent travelers"
+card_002,Everyday Cashback Plus,0,"groceries streaming cashback","$200 statement credit after qualifying spend","starter or mass market income","Lower earn rate on travel purchases"
+card_003,Family Balance Saver,95,"groceries utilities balance transfer","0% intro APR for 15 months","mass affluent income and good credit","Balance transfer fee applies"
+card_004,Nomad Rewards Flex,150,"travel dining subscriptions","50000 bonus points after qualifying spend","premium income and good credit","Subscription credits require activation"

samples/llm-vector-langchain/data/customer_profiles/customer_profiles.csv

@@ -0,0 +1,5 @@
+customer_id,age,income_band,segment,goals,risk_tolerance
+cust_001,34,premium,frequent_traveler,"travel rewards, lounge access, low foreign transaction fees",medium
+cust_002,27,starter,young_professional,"cashback on everyday spend, no annual fee",low
+cust_003,42,mass_affluent,family_planner,"groceries, utilities, balance transfer",low
+cust_004,31,premium,digital_nomad,"travel rewards, dining, subscription credits",medium

samples/llm-vector-langchain/data/transactions/transactions.csv

@@ -0,0 +1,13 @@
+transaction_id,customer_id,merchant_category,amount_usd,transaction_date
+txn_001,cust_001,travel,820.50,2026-01-12
+txn_002,cust_001,dining,188.10,2026-01-15
+txn_003,cust_001,travel,1320.00,2026-02-03
+txn_004,cust_002,groceries,122.23,2026-01-17
+txn_005,cust_002,streaming,18.99,2026-01-19
+txn_006,cust_002,groceries,98.11,2026-02-02
+txn_007,cust_003,groceries,240.42,2026-01-07
+txn_008,cust_003,utilities,210.00,2026-01-10
+txn_009,cust_003,education,510.00,2026-02-05
+txn_010,cust_004,dining,305.45,2026-01-21
+txn_011,cust_004,travel,940.00,2026-02-08
+txn_012,cust_004,subscriptions,64.99,2026-02-11

samples/llm-vector-langchain/dozer-config.yaml

@@ -0,0 +1,57 @@
+app_name: llm-vector-langchain-sample
+version: 1
+
+api:
+  rest:
+    port: 8080
+    host: "0.0.0.0"
+    cors: true
+  grpc:
+    port: 50051
+    host: "0.0.0.0"
+    cors: true
+    web: true
+
+connections:
+  - name: local_customer_data
+    config: !LocalStorage
+      details:
+        path: ./data
+      tables:
+        - !Table
+          name: customer_profiles
+          config: !CSV
+            path: customer_profiles
+            extension: .csv
+        - !Table
+          name: transactions
+          config: !CSV
+            path: transactions
+            extension: .csv
+        - !Table
+          name: credit_card_products
+          config: !CSV
+            path: credit_card_products
+            extension: .csv
+
+sources:
+  - name: customer_profiles
+    table_name: customer_profiles
+    connection: local_customer_data
+  - name: transactions
+    table_name: transactions
+    connection: local_customer_data
+  - name: credit_card_products
+    table_name: credit_card_products
+    connection: local_customer_data
+
+sinks:
+  - name: customer_profiles
+    config: !Dummy
+      table_name: customer_profiles
+  - name: transactions
+    config: !Dummy
+      table_name: transactions
+  - name: credit_card_products
+    config: !Dummy
+      table_name: credit_card_products