Skip to content

Getting Started with Development

Jump to bottom
Eric Fitzgerald edited this page Oct 4, 2026 · 12 revisions

Getting Started with Development

This guide walks you through setting up your local development environment for both the TMI server (Go) and the TMI web application (Angular/TypeScript).

Overview

TMI consists of two main components:

  • TMI Server - Go-based RESTful API server with WebSocket support
  • TMI-UX - Angular-based web application for threat modeling

Prerequisites

Required Tools

For TMI Server (Go)

  • Go 1.26+ - Download
  • Docker - Builds the images (on Docker Desktop it also hosts the cluster node)
  • kubectl - The dev environment deploys into a local Kubernetes cluster
  • A local cluster - Docker Desktop's built-in Kubernetes (CLUSTER=docker-desktop) or a dedicated k3s cluster (CLUSTER=k3s)
  • uv - The make dev-* targets run scripts/devenv.py through uv run
  • Make - Build automation (pre-installed on macOS/Linux)

For TMI-UX (Angular)

  • Node.js 22.x (>=22.0.0 <23.0.0) - Download
  • pnpm - Fast, disk space efficient package manager - Install

Optional Tools

  • Newman - API testing -- pnpm install -g newman
  • Grype - Container vulnerability scanning -- brew install grype
  • golangci-lint - Go code linting

Quick Start

⚠️ Breaking change: make dev-up now deploys into a local Kubernetes cluster — it no longer runs the server and workers as host Go processes. See Local Development Cluster for the full cluster onramp (prerequisites, Docker Desktop/k3s setup, database configuration, and DB=oracle). The summary below is the short version.

Option 1: Server Only (Go Backend)

# Clone the repository
git clone https://github.com/ericfitz/tmi.git
cd tmi

# One-time: create your dev config
cp config-example.yml config-development.yml   # then edit database.url, OAuth, JWT secret

# Build images, deploy the platform, and port-forward to :8080
# CLUSTER is required: docker-desktop or k3s (DB=postgres|oracle, default postgres)
make dev-up CLUSTER=docker-desktop

# Server will be running on http://localhost:8080

The make dev-up command:

  1. Builds the server + controller + worker images and delivers them to the cluster (imported into the Docker Desktop node, or pushed to the in-cluster registry on k3s)
  2. Deploys the server, Redis (TLS + password), NATS (mTLS), KEDA, the component controller, and the workers into the tmi-platform namespace, installing cert-manager and Reloader as needed
  3. With DB=postgres (the default), deploys an in-cluster Postgres StatefulSet
  4. Delivers your config-development.yml into the cluster as a ConfigMap; a postgres://user:pass@localhost… database.url is rewritten to the in-cluster postgres Service
  5. Waits for the rollout and starts a managed kubectl port-forward to http://localhost:8080

go is also required: dev-up/dev-down/dev-reset/dev-nuke build bin/tmi-dbtool to snapshot and restore operational settings. See Local Development Cluster → Database configuration for cloud Postgres and the Oracle (DB=oracle, Docker Desktop only) path.

Option 2: Full Stack (Server + Web App)

Terminal 1 - Start Server:

cd tmi
make dev-up CLUSTER=docker-desktop

Terminal 2 - Start Web Application:

cd tmi-ux
pnpm install
pnpm run dev

The web application will be available at http://localhost:4200

TMI Server Setup (Detailed)

1. Environment Configuration

TMI uses YAML configuration files with environment variable overrides. On first setup, copy config-example.yml to config-development.yml:

# Copy example config and customize
cp config-example.yml config-development.yml

Environment variables use the TMI_ prefix and override any YAML value. Key variables:

  • TMI_DATABASE_URL - Database connection URL (required), e.g., postgres://user:pass@localhost:5432/tmi?sslmode=disable
  • TMI_SERVER_PORT - Server port (default: 8080)
  • TMI_JWT_SECRET - JWT signing secret

The configuration file covers:

  • Database connection via URL (PostgreSQL, MySQL, SQLite, SQL Server, Oracle)
  • Redis connection (caching and sessions)
  • OAuth provider credentials
  • Server settings (ports, timeouts, TLS)
  • Cookie-based authentication settings

2. Database Setup

With DB=postgres (the default), make dev-up deploys Postgres inside the cluster (StatefulSet + PVC) and the server connects to it. Keep database.url pointing at localhost in config-development.yml (as host-side tools expect); when dev-up renders the ConfigMap it rewrites a postgres://user:pass@localhost… (or @127.0.0.1) URL to the in-cluster postgres Service. The schema is built automatically by GORM AutoMigrate() on the server's first successful connect; there is no separate manual migration step in the dev loop.

# config-development.yml
database:
  url: "postgres://tmi_dev:dev123@localhost:5432/tmi_dev?sslmode=disable"

To reach the in-cluster Postgres from the host: kubectl port-forward -n tmi-platform svc/postgres 5432:5432. The standalone make start-database container still exists but dev-up does not use it. A remote database.url is left as written (works, untested), and DB=oracle targets an external OCI ADB (Docker Desktop only).

For cloud Postgres or OCI ADB (DB=oracle + wallet), see Local Development Cluster → Database configuration.

3. OAuth Provider Configuration

To enable authentication, configure OAuth providers in config-development.yml. TMI supports Google, GitHub, and Microsoft providers.

Google OAuth Setup

  1. Go to Google Cloud Console
  2. Create OAuth 2.0 credentials
  3. Add redirect URI: http://localhost:8080/oauth2/callback
  4. Set environment variables or update config-development.yml:
# Via environment variables (recommended):
export OAUTH_PROVIDERS_GOOGLE_CLIENT_ID=YOUR_CLIENT_ID.apps.googleusercontent.com
export OAUTH_PROVIDERS_GOOGLE_CLIENT_SECRET=YOUR_CLIENT_SECRET
# Or via config-development.yml under auth.oauth.providers.google:
auth:
  oauth:
    callback_url: "http://localhost:8080/oauth2/callback"
    providers:
      google:
        id: "google"
        name: "Google"
        enabled: true
        client_id: "YOUR_CLIENT_ID.apps.googleusercontent.com"
        client_secret: "YOUR_CLIENT_SECRET"
        authorization_url: "https://accounts.google.com/o/oauth2/auth"
        token_url: "https://oauth2.googleapis.com/token"
        userinfo:
          - url: "https://www.googleapis.com/oauth2/v3/userinfo"
            claims: {}
        scopes:
          - "openid"
          - "profile"
          - "email"

TMI Test OAuth Provider

For development, you can use the built-in TMI provider (no configuration needed):

# Access TMI OAuth provider (creates random user)
curl "http://localhost:8080/oauth2/authorize?idp=tmi"

The TMI provider creates users like: testuser-12345678@tmi.local

For predictable test users, use login hints:

# Create specific test user 'alice@tmi.local'
curl "http://localhost:8080/oauth2/authorize?idp=tmi&login_hint=alice"

4. Available Make Commands

# Development environment (CLUSTER=docker-desktop|k3s is REQUIRED on every dev-* target; DB=postgres|oracle, default postgres)
make dev-cluster-up CLUSTER=<c>   # Only switches the kube context; creates nothing
make dev-cluster-down CLUSTER=<c> # No-op (TMI does not own the cluster)
make dev-up CLUSTER=<c>           # Build images + deploy the platform into the cluster
make dev-restart CLUSTER=<c>      # Rebuild all four images, re-apply the platform, roll the pods
make dev-down CLUSTER=<c>         # Tear down what dev-up deployed; keeps DB data and the cluster
make dev-reset CLUSTER=<c>        # dev-down + dev-up with fresh images; keeps DB data
make dev-nuke CLUSTER=<c>         # Delete the tmi-platform namespace (including DB data) and rebuild

# Building
make build-server       # Build server binary
make build-migrate      # Build migration tool
make clean-build        # Clean build artifacts
make generate-api       # Generate API code from OpenAPI spec

# Database (optional local Postgres container)
make start-database     # Start a local PostgreSQL container
make stop-database      # Stop the PostgreSQL container
make clean-database     # Remove container and data
make reset-database     # Drop and recreate database (DESTRUCTIVE)

# Testing
make test-unit          # Run Go unit tests
make test-integration   # Run integration tests (PostgreSQL)
make test-integration-pg  # Run integration tests (PostgreSQL, explicit)
make test-api           # Run Newman API tests
make test-coverage      # Generate coverage reports

The schema is built by GORM AutoMigrate() on first connect — there is no separate make migrate-database step in the cluster dev loop. See Local Development Cluster for the full command reference.

5. Verify Server Installation

# Check server status
curl http://localhost:8080/

# Expected response
{
  "api": {
    "version": "1.0",
    "build": "0.9.0"
  },
  "service": {
    "name": "TMI API Server",
    "build": "0.9.0-abc123"
  }
}

TMI-UX Setup (Detailed)

1. Install Dependencies

cd tmi-ux
pnpm install

2. Environment Configuration

TMI-UX supports multiple environment configurations:

# Default development (uses environment.ts)
pnpm run dev

# Test environment
pnpm run dev:test

# Production environment (local)
pnpm run dev:prod

Creating Custom Environment

  1. Copy example: cp src/environments/environment.example.ts src/environments/environment.local.ts
  2. Configure environment.local.ts:
import { Environment } from './environment.interface';

export const environment: Environment = {
  production: false,
  logLevel: 'DEBUG',
  apiUrl: 'http://localhost:8080',
  authTokenExpiryMinutes: 1440, // 24 hours for dev
  operatorName: 'TMI Development',
  operatorContact: 'dev@example.com',
  operatorJurisdiction: '',
  suppressAboutLink: false,
  suppressPrivacyTosLinks: false,
  serverPort: 4200,
  serverInterface: 'localhost',
  enableTLS: false,
  tlsKeyPath: undefined,
  tlsCertPath: undefined,
  tlsSubjectName: undefined,
  enableConfidentialThreatModels: false,
  defaultThreatModelFramework: 'STRIDE',
};
  1. Update angular.json to add configuration:
"configurations": {
  "local": {
    "fileReplacements": [
      {
        "replace": "src/environments/environment.ts",
        "with": "src/environments/environment.local.ts"
      }
    ]
  }
}

3. Available pnpm Scripts

# Development
pnpm run dev              # Start dev server on localhost:4200 (opens browser)
pnpm run dev:test         # Start with test config
pnpm run dev:prod         # Start with production config
pnpm run shannon          # Start on 0.0.0.0 (accessible from network)

# Building
pnpm run build            # Development build
pnpm run build:prod       # Production build
pnpm run build:test       # Test build

# Testing (Vitest)
pnpm run test             # Run Vitest unit tests
pnpm run test:watch       # Run tests in watch mode
pnpm run test:ui          # Run tests with Vitest UI
pnpm run test:coverage    # Generate coverage report

# E2E Testing (Playwright)
pnpm run test:e2e         # Run Playwright E2E tests
pnpm run test:e2e:ui      # Open Playwright UI
pnpm run test:e2e:debug   # Run E2E tests in debug mode
pnpm run test:e2e:headed  # Run E2E tests in headed browser

# Code Quality
pnpm run lint             # Lint TypeScript/HTML files
pnpm run lint:scss        # Lint SCSS files
pnpm run lint:all         # Lint everything
pnpm run format           # Format code with Prettier
pnpm run format:check     # Check formatting
pnpm run check            # Run format:check + lint:all

# Validation
pnpm run validate-json        # Validate JSON files
pnpm run validate-json:i18n   # Validate i18n files
pnpm run validate-jsonc       # Validate JSONC files
pnpm run validate-all         # Run all validation checks

4. Verify Web Application

# Start dev server
pnpm run dev

# Open browser to http://localhost:4200
# You should see the TMI login page

Development Workflow

1. Server Development

# Terminal 1: Start services
cd tmi
make dev-up CLUSTER=docker-desktop

# Terminal 2: Run tests on code changes
make test-unit

# Terminal 3: Watch server logs (from the pod)
kubectl -n tmi-platform logs deploy/tmi-server -f

To pick up server code changes, run make dev-restart CLUSTER=<c> (rebuilds all four images and rolls the pods).

2. Web Application Development

# Terminal 1: Start server
cd tmi
make dev-up CLUSTER=docker-desktop

# Terminal 2: Start web app
cd tmi-ux
pnpm run dev

# Hot reload is enabled - changes reflect immediately

3. Full Stack Testing

# Terminal 1: Start server
cd tmi
make dev-up CLUSTER=docker-desktop

# Terminal 2: Start web app
cd tmi-ux
pnpm run dev

# Terminal 3: Run E2E tests
cd tmi-ux
pnpm run test:e2e

Project Structure

TMI Server (Go)

tmi/
├── api/                  # API handlers and types
├── api-schema/           # OpenAPI spec (tmi-openapi.json), AsyncAPI, Arazzo
├── auth/                 # Authentication services
├── cmd/
│   ├── server/          # Server entry point
│   ├── migrate/         # Database migration tool
│   ├── dbtool/          # Database administration tool (seeding, CATS test data, etc.)
│   └── dedup-group-members/  # Group member deduplication
├── config/              # Configuration loading
├── internal/            # Internal packages
├── docs/
│   └── developer/       # Developer documentation
├── scripts/             # Build and utility scripts
├── test/
│   └── postman/         # Newman API test collections
├── terraform/           # Infrastructure as code
├── config-example.yml      # Example configuration
├── config-development.yml  # Dev configuration (create from example)
└── Makefile            # Build automation

TMI-UX (Angular)

tmi-ux/
├── src/
│   ├── app/
│   │   ├── auth/        # Authentication components
│   │   ├── core/        # Core services, guards
│   │   ├── generated/   # Generated API types
│   │   ├── i18n/        # Internationalization setup
│   │   ├── pages/       # Page components
│   │   ├── shared/      # Shared components
│   │   └── types/       # TypeScript type definitions
│   ├── assets/          # Static assets, i18n JSON files
│   └── environments/    # Environment configs
├── e2e/                 # Playwright E2E tests
├── docs/
│   ├── mockups/         # UI mockups
│   └── reference/       # Architecture docs
├── scripts/             # Build and utility scripts
├── playwright.config.ts # Playwright configuration
└── package.json         # Dependencies and scripts

File Naming Conventions

This section outlines the file naming conventions for the TMI-UX project.

File Naming Standards

Type Pattern Example Usage
Components *.component.ts user-preferences-dialog.component.ts All Angular components
Services *.service.ts threat-model.service.ts All @Injectable() classes that provide business logic
Guards *.guard.ts auth.guard.ts All route guards
Interceptors *.interceptor.ts jwt.interceptor.ts All HTTP interceptors
Resolvers *.resolver.ts threat-model.resolver.ts All route resolvers
Models *.model.ts threat-model.model.ts Data models and interface definitions
Interfaces *.interface.ts auth.interface.ts Interface definitions (alternative to .model.ts)
Types *.types.ts websocket-message.types.ts Type definitions and type aliases
Constants *-constants.ts styling-constants.ts Constant values and configuration objects
Utilities *.util.ts or *.utils.ts cell-normalization.util.ts Utility functions and helper methods
Adapters *.adapter.ts infra-x6-graph.adapter.ts Infrastructure adapters for external libraries
Tests *.spec.ts auth.service.spec.ts Unit tests
Routes *.routes.ts tm.routes.ts Route configuration files
State *.state.ts dfd.state.ts State management files
Barrel Exports index.ts src/app/shared/index.ts Re-exporting multiple items from a directory

DFD Module Structure (Domain-Driven Design)

The DFD (Data Flow Diagram) module uses a layered DDD architecture:

src/app/pages/dfd/
├── application/            # Application services (orchestration, coordinators)
│   ├── executors/         # Operation executors
│   ├── services/          # app-*.service.ts files
│   └── validators/        # Operation validators
├── domain/                # Domain models and events
│   ├── entities/          # Domain entities
│   ├── events/            # Domain events
│   └── value-objects/     # Value objects
├── infrastructure/        # External library adapters
│   ├── adapters/          # infra-*.adapter.ts files
│   ├── constants/         # Infrastructure configuration
│   ├── interfaces/        # Adapter interfaces
│   └── services/          # infra-*.service.ts files
├── presentation/          # UI components and services
│   ├── components/        # UI components
│   └── services/          # ui-*.service.ts files
├── constants/             # Shared constants
├── interfaces/            # Shared interfaces
├── state/                 # State management
├── types/                 # Type definitions
└── utils/                 # Utility functions

Naming Rules

  1. Use kebab-case for file names

    • Correct: threat-model.service.ts
    • Incorrect: ThreatModelService.ts or threat_model_service.ts

  2. Be descriptive in naming

    • Correct: threat-model-authorization.service.ts
    • Incorrect: tm-auth.service.ts

  3. Group related files by feature

    • Keep related components, services, and models together
    • Use barrel exports (index.ts) for clean imports

  4. Match class names to file names

    • File: threat-model.service.ts
    • Class: ThreatModelService

  5. Use layer prefixes in DFD module

    • Application layer: app-*.service.ts
    • Infrastructure layer: infra-*.service.ts, infra-*.adapter.ts
    • Presentation layer: ui-*.service.ts

Migration Notes

When renaming files:

  1. Update all imports in the codebase
  2. Update any barrel exports (index.ts files)
  3. Check for impacts on tests
  4. Run build and tests to verify

Common Development Tasks

Adding a New API Endpoint (Server)

  1. Define endpoint in OpenAPI spec: api-schema/tmi-openapi.json
  2. Generate API code: make generate-api
  3. Implement handler in api/
  4. Add database operations if needed
  5. Write unit tests
  6. Write integration tests in api/*_integration_test.go
  7. Add Newman tests in test/postman/

Adding a New UI Feature (Web App)

  1. Generate component: pnpm exec ng generate component pages/my-feature
  2. Add to routing: src/app/app.routes.ts
  3. Create service if needed: pnpm exec ng generate service core/services/my-service
  4. Add i18n keys: src/assets/i18n/en-US.json
  5. Write unit tests: *.spec.ts
  6. Write E2E tests: e2e/tests/*.spec.ts (Playwright)

Troubleshooting

Server Won't Start

# Check the rollout / pod status
kubectl -n tmi-platform get pods
kubectl -n tmi-platform rollout status deploy/tmi-server

# Read the server logs from the pod
kubectl -n tmi-platform logs deploy/tmi-server -f

# With DB=postgres, a postgres://user:pass@localhost... database.url is rewritten
# to the in-cluster Postgres; other URLs must be reachable from inside a pod.

# Re-deliver config + roll the pods after editing config-development.yml
make dev-restart CLUSTER=docker-desktop

See Local Development Cluster → Troubleshooting for ImagePullBackOff, context-guard, and port-forward issues.

Database Issues

The schema is built by GORM AutoMigrate() on the server's first successful connect — there is no manual migration step. Since server 1.18.0, the server retries the database and Redis connections at startup with exponential backoff (0.5s, doubling to at most 5s) for up to 30 seconds of wall-clock time. It does not retry database errors a retry cannot fix, such as bad or locked credentials, an unknown TNS alias, wallet or certificate failures, a missing Oracle client, or a missing Postgres database. Redis connection errors are always retried until the 30 seconds run out. If it still can't connect, it exits, the pod logs the connection error, and Kubernetes restarts it. A common cause in the cluster is a database.url host that is unreachable from a pod (localhost is only rewritten for postgres://user:pass@localhost… URLs with DB=postgres).

# Reset a local Postgres container (DESTRUCTIVE - deletes all data)
make reset-database

Web App Won't Connect to API

  1. Check server is running: curl http://localhost:8080/
  2. Verify apiUrl in environment file
  3. Check browser console for CORS errors
  4. Ensure you're authenticated (valid JWT token)

OAuth Not Working

  1. Check OAuth config in config-development.yml
  2. Verify redirect URI matches OAuth provider
  3. Use TMI provider for development: ?idp=tmi
  4. Check server logs: tail -f logs/server.log

Next Steps

Getting Help

Home

Releases

Getting Started

Deployment

Operation

Troubleshooting

Development

Integrations

Tools

API Reference

Reference

Clone this wiki locally