Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ load("@gazelle//:def.bzl", "gazelle")
# gazelle:exclude .claude

# Resolve protobuf import ambiguities - use the actual protopb packages, not the proto aliases
# gazelle:resolve go github.com/uber/submitqueue/submitqueue/gateway/protopb //submitqueue/gateway/protopb
# gazelle:resolve go github.com/uber/submitqueue/submitqueue/orchestrator/protopb //submitqueue/orchestrator/protopb
# gazelle:resolve go github.com/uber/submitqueue/stovepipe/gateway/protopb //stovepipe/gateway/protopb
# gazelle:resolve go github.com/uber/submitqueue/stovepipe/orchestrator/protopb //stovepipe/orchestrator/protopb
# gazelle:resolve go github.com/uber/submitqueue/api/submitqueue/gateway/protopb //api/submitqueue/gateway/protopb
# gazelle:resolve go github.com/uber/submitqueue/api/submitqueue/orchestrator/protopb //api/submitqueue/orchestrator/protopb
# gazelle:resolve go github.com/uber/submitqueue/api/stovepipe/gateway/protopb //api/stovepipe/gateway/protopb
# gazelle:resolve go github.com/uber/submitqueue/api/stovepipe/orchestrator/protopb //api/stovepipe/orchestrator/protopb

# Export marker files for test data dependencies (used by FindRepoRoot in tests)
exports_files(
Expand Down
31 changes: 19 additions & 12 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ request.Version = newVersion

```
submitqueue/ # repo root (Go module github.com/uber/submitqueue)
├── api/ # Wire contracts (proto) by domain/service
│ ├── submitqueue/{gateway,orchestrator}/{proto,protopb}/
│ └── stovepipe/{gateway,orchestrator}/{proto,protopb}/
├── platform/ # SHARED cross-domain packages — no domain deps
│ ├── errs/, metrics/, consumer/, http/
│ ├── base/ # SHARED entities (change/, messagequeue/, …)
Expand Down Expand Up @@ -65,6 +68,8 @@ submitqueue/ # repo root (Go module github.com/uber/submi

The `platform/` tree holds code reused across domains (infrastructure, shared entities, shared extension contracts). Each **domain** (`submitqueue/`, `stovepipe/`, …) keeps the same internal layout (`gateway/`, `orchestrator/`, `entity/`, `extension/`, `core/`); a domain's own `core/` (e.g. `submitqueue/core/`) holds infra shared only between that domain's services.

The `api/` tree holds all wire contracts (proto definitions and their committed generated stubs), organized by `domain/service`: `api/{domain}/{service}/proto/` for `.proto` sources and `api/{domain}/{service}/protopb/` for generated Go. A service package may hold multiple `.proto` files — its RPC contract (`{service}.proto`) alongside messagequeue contracts (queue payload schemas) — all generating into the same `protopb/`.

### Platform notes

- Import path `github.com/uber/submitqueue/platform/http` uses Go package name `http` and aliases the standard library as `nethttp` inside the package. Source files that also import `net/http` should import the platform package with a distinct alias (for example `phttp "github.com/uber/submitqueue/platform/http"`) and call `phttp.NewClient`, `phttp.BaseURLTransport`, etc.
Expand All @@ -76,16 +81,16 @@ Each service follows the same layout:

```
<service>/
├── controller/ # Business logic (pure, transport-agnostic)
│ ├── {method}.go # RPC controllers (e.g., land.go, ping.go)
│ ├── {method}_test.go
│ └── {step}/ # Queue message controllers (e.g., request/)
│ ├── {step}.go
│ └── {step}_test.go
├── proto/ # Proto definitions (.proto files)
└── protopb/ # Generated proto code (committed to repo)
└── controller/ # Business logic (pure, transport-agnostic)
├── {method}.go # RPC controllers (e.g., land.go, ping.go)
├── {method}_test.go
└── {step}/ # Queue message controllers (e.g., request/)
├── {step}.go
└── {step}_test.go
```

Wire contracts for a service live separately under `api/{domain}/{service}/` (see Project Layout): `proto/` holds `.proto` sources and `protopb/` holds the committed generated stubs.

### Controllers

Two types, both containing pure business logic independent of infrastructure:
Expand Down Expand Up @@ -150,7 +155,7 @@ Paths follow the directory layout: shared packages live under `platform/` at the

- RPC Controllers: `github.com/uber/submitqueue/{domain}/{service}/controller` (e.g. `.../submitqueue/gateway/controller`)
- Queue Controllers: `github.com/uber/submitqueue/{domain}/{service}/controller/{step}`
- Proto (generated): `github.com/uber/submitqueue/{domain}/{service}/protopb`
- Proto (generated): `github.com/uber/submitqueue/api/{domain}/{service}/protopb`
- Domain entities: `github.com/uber/submitqueue/{domain}/entity` (e.g. `.../submitqueue/entity`)
- Domain extensions: `github.com/uber/submitqueue/{domain}/extension/{ext}[/{impl}]` (e.g. `.../submitqueue/extension/storage/mysql`)
- Cross-domain consumer framework: `github.com/uber/submitqueue/platform/consumer`; domain pipeline topic keys: `github.com/uber/submitqueue/{domain}/core/topickey`
Expand All @@ -173,10 +178,12 @@ Bazel with Bzlmod (NOT WORKSPACE).
### Proto Generation

Generated proto files are committed. When modifying `.proto` files:
1. Edit in `{domain}/{service}/proto/` (e.g. `submitqueue/gateway/proto/`)
2. `make proto` (generates `*.pb.go`, `*_grpc.pb.go`, `*.pb.yarpc.go`)
1. Edit in `api/{domain}/{service}/proto/` (e.g. `api/submitqueue/gateway/proto/`)
2. `make proto` (generates `*.pb.go`, `*_grpc.pb.go`, `*.pb.yarpc.go` into `api/{domain}/{service}/protopb/`)
3. Commit all generated files

To add a new `.proto` to a service (e.g. messagequeue contracts), drop it in the service's `api/{domain}/{service}/proto/` dir, add it to that package's `srcs` in `api/{domain}/{service}/proto/BUILD.bazel` and its `exports_files`, then `make proto && make gazelle`. The codegen and `make proto` copy loop already handle multiple `.proto` files per package.

### Naming Conventions

- **Directories**: singular (`mock/`, `entity/`, not `mocks/`, `entities/`)
Expand Down Expand Up @@ -220,7 +227,7 @@ make clean # Clean Bazel cache
### Common Workflows

**Add new RPC method:**
1. Edit `{domain}/{service}/proto/*.proto` → `make proto`
1. Edit `api/{domain}/{service}/proto/*.proto` → `make proto`
2. Add controller in `{domain}/{service}/controller/`
3. Wire up in `example/{domain}/{service}/server/main.go`

Expand Down
16 changes: 9 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,12 @@ YAMLFMT_VERSION ?= v0.16.0
# goimports version for Go formatting + import fixing
GOIMPORTS_VERSION ?= v0.33.0

# Proto packages: <domain>/<service> dirs whose protopb/ holds the generated
# stubs. Each is generated by Bazel into bazel-bin/tool/proto/<domain>_<service>/
# (the out_dir convention in tool/proto/BUILD.bazel) and copied back here.
PROTO_PACKAGES = submitqueue/gateway submitqueue/orchestrator stovepipe/gateway stovepipe/orchestrator
# Proto packages: api/<domain>/<service> dirs whose protopb/ holds the generated
# stubs. Each is generated by Bazel into bazel-bin/tool/proto/api_<domain>_<service>/
# (the out_dir convention in tool/proto/BUILD.bazel) and copied back here. A
# package may hold multiple .proto files (e.g. an RPC contract plus messagequeue
# contracts); all generated stubs land in the same protopb/ dir.
PROTO_PACKAGES = api/submitqueue/gateway api/submitqueue/orchestrator api/stovepipe/gateway api/stovepipe/orchestrator

# Set REPO_ROOT for docker-compose
export REPO_ROOT := $(shell pwd)
Expand Down Expand Up @@ -343,9 +345,9 @@ proto: ## Generate protobuf files from .proto definitions
@echo "Generating protobuf files with Bazel..."
@$(BAZEL) build //tool/proto:generated
@set -e; for pkg in $(PROTO_PACKAGES); do \
out=$$(echo $$pkg | tr / _); base=$$(basename $$pkg); \
for f in $$base.pb.go $$base.pb.yarpc.go $${base}_grpc.pb.go; do \
cp -f bazel-bin/tool/proto/$$out/$$f $$pkg/protopb/$$f; \
out=$$(echo $$pkg | tr / _); \
for f in bazel-bin/tool/proto/$$out/*.go; do \
cp -f $$f $$pkg/protopb/$$(basename $$f); \
done; \
done
@$(BAZEL) run @rules_go//go -- run golang.org/x/tools/cmd/goimports@$(GOIMPORTS_VERSION) -w $(addsuffix /protopb,$(PROTO_PACKAGES))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,21 +20,21 @@ go_proto_library(
"@rules_go//proto:go_proto",
"@rules_go//proto:go_grpc_v2",
],
importpath = "github.com/uber/submitqueue/stovepipe/gateway/proto",
importpath = "github.com/uber/submitqueue/api/stovepipe/gateway/proto",
proto = ":gatewaypb_proto",
visibility = ["//visibility:public"],
)

go_library(
name = "proto",
embed = [":gatewaypb_go_proto"],
importpath = "github.com/uber/submitqueue/stovepipe/gateway/proto",
importpath = "github.com/uber/submitqueue/api/stovepipe/gateway/proto",
visibility = ["//visibility:public"],
)

go_library(
name = "protopb",
embed = [":gatewaypb_go_proto"],
importpath = "github.com/uber/submitqueue/stovepipe/gateway/protopb",
importpath = "github.com/uber/submitqueue/api/stovepipe/gateway/protopb",
visibility = ["//visibility:public"],
)
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ syntax = "proto3";

package uber.submitqueue.stovepipe;

option go_package = "github.com/uber/submitqueue/stovepipe/gateway/protopb";
option go_package = "github.com/uber/submitqueue/api/stovepipe/gateway/protopb";
option java_multiple_files = true;
option java_outer_classname = "GatewayProto";
option java_package = "com.uber.submitqueue.stovepipe";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ go_library(
"gateway.pb.yarpc.go",
"gateway_grpc.pb.go",
],
importpath = "github.com/uber/submitqueue/stovepipe/gateway/protopb",
importpath = "github.com/uber/submitqueue/api/stovepipe/gateway/protopb",
visibility = ["//visibility:public"],
deps = [
"@org_golang_google_grpc//:grpc",
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -20,21 +20,21 @@ go_proto_library(
"@rules_go//proto:go_proto",
"@rules_go//proto:go_grpc_v2",
],
importpath = "github.com/uber/submitqueue/submitqueue/orchestrator/proto",
importpath = "github.com/uber/submitqueue/api/stovepipe/orchestrator/proto",
proto = ":orchestratorpb_proto",
visibility = ["//visibility:public"],
)

go_library(
name = "proto",
embed = [":orchestratorpb_go_proto"],
importpath = "github.com/uber/submitqueue/submitqueue/orchestrator/proto",
importpath = "github.com/uber/submitqueue/api/stovepipe/orchestrator/proto",
visibility = ["//visibility:public"],
)

go_library(
name = "protopb",
embed = [":orchestratorpb_go_proto"],
importpath = "github.com/uber/submitqueue/submitqueue/orchestrator/protopb",
importpath = "github.com/uber/submitqueue/api/stovepipe/orchestrator/protopb",
visibility = ["//visibility:public"],
)
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ syntax = "proto3";

package uber.submitqueue.stovepipe.orchestrator;

option go_package = "github.com/uber/submitqueue/stovepipe/orchestrator/protopb";
option go_package = "github.com/uber/submitqueue/api/stovepipe/orchestrator/protopb";
option java_multiple_files = true;
option java_outer_classname = "OrchestratorProto";
option java_package = "com.uber.submitqueue.stovepipe.orchestrator";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ go_library(
"orchestrator.pb.yarpc.go",
"orchestrator_grpc.pb.go",
],
importpath = "github.com/uber/submitqueue/submitqueue/orchestrator/protopb",
importpath = "github.com/uber/submitqueue/api/stovepipe/orchestrator/protopb",
visibility = ["//visibility:public"],
deps = [
"@org_golang_google_grpc//:grpc",
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading