feat(netprobe): HTTP/DoH probes v2 — success criteria, TLS cert expiry, headers/proxy/mTLS by leoparente · Pull Request #793 · netboxlabs/pktvisor

leoparente · 2026-07-19T00:51:55Z

Summary

v2 of the netprobe HTTP/DoH probes (#792 follow-up): benchmark parity with cloudprober's HTTP probe and a superset of the OTel httpcheckreceiver's feature set.

New capabilities (http test type unless noted):

Success criteria — expected_status / failure_status (exact codes, Nxx classes, A-B ranges; failure list evaluated first, cloudprober semantics) and response-body checks (expected_body substring AND expected_body_regex), with a new content_failures counter distinct from http_status_failures.
TLS cert expiry metric — tls_cert_expiry_epoch_sec per target (unix timestamp of the chain's earliest notAfter, blackbox_exporter convention; http + doh) via CURLOPT_CERTINFO on the existing transfers. A probe-side cache stamps the last-known expiry onto every sample so the metric doesn't flap with keep-alive connection reuse (CERTINFO only populates on fresh handshakes).
Per-target request headers (e.g. Authorization) with strict redaction: header values, proxy value, and request body never appear in info_json, the taps API, logs, or error messages — only header names are surfaced.
Request body (body, POST/PUT/PATCH only) and response_size_bytes quantiles.
Proxy (proxy, http+doh) and TLS options (tls.verify / tls.ca_file / tls.cert_file+tls.key_file for mTLS).
Default User-Agent: pktvisor/<version> (overridable via headers).

Design notes

Check evaluation happens probe-side (config lives on the input); results travel in a single HttpSample POD so future fields don't churn the signal signatures.
Status/body grammar and cert-date parsing live in a new pure HttpCheck unit with direct tests; cert dates parse via curl_getdate() and file checks use std::filesystem (MSVC-portable — no POSIX-only APIs).
Security: because v2 introduces the first secret-bearing input config keys, the raw tap-config echo (GET /api/v1/taps, Policy::info_json) became a disclosure path — fixed via a new InputModulePlugin::redact_config_json() hook that Tap serialization invokes, backed by the same scrub used by the input's own info_json, with dump-and-grep tests proving no secret literal survives either path.
topn_count/topn_percentile_threshold apply to the netprobe TopN metrics; keep-alive comes free from the per-stream curl_multi connection pool (documented).

Out of scope (future)

json_path/size validations, OAuth token refresh, header validators, requests_per_probe, resolve_first, ip_version forcing, DoH EDNS/DNSSEC. A pre-existing scrape-time pattern (emit-vehicle counters mutated under a shared lock, same as v1 response_min_us/max_us) is noted for a separate cleanup.

Testing

unit-tests-visor-http-check (32) — status grammar, body checks, cert-date parsing (incl. leading-space CERTINFO format).
unit-tests-visor-http-client (51) — proxy-through-forward-proxy (absolute-form proof against a TEST-NET target), UA echo, response size, TLS option wiring, cert==0 on plain http, plus all v1 cases.
unit-tests-input-netprobe (148) — config grammar/scoping/consistency errors (secret-free messages), both redaction paths (module info_json + tap-shaped scrub), and e2e: per-target auth headers (both outcomes), expected_status 401-flip, failure_status precedence, body match/mismatch → content_failures, POST body echo.
unit-tests-handler-netprobe (100) — classification matrix, cert-expiry latest-wins/merge-max, response_size_bytes gating, merge coverage.

All suites pass locally (Release, 3× stable on the e2e suite).

🤖 Generated with Claude Code

…rser

…onse size/cert expiry results

… cert_expiry passthrough

…get headers, proxy, tls, UA

…_size_bytes metrics

… body, redaction; docs

…T /api/v1/taps echoed raw headers/proxy/body)

github-actions · 2026-07-19T01:14:57Z

LCOV of commit `139232d` during Debug Builds #209

  lines......: 84.2% (16368 of 19443 lines)
  functions..: 75.4% (1614 of 2142 functions)
  branches...: no data found

Files changed coverage rate: n/a

Full coverage report

leoparente · 2026-07-19T01:29:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2a43a61c78

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…; configurable body_check_max_bytes

leoparente · 2026-07-19T02:59:52Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f7b8d411da

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…ul wrap could accept them as an in-range code)

leoparente · 2026-07-19T10:48:10Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e439a84aac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

… (max suppressed alerts after cert downgrade)

leoparente · 2026-07-19T11:02:04Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ce21bf9701

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…msg (probes log it on transport failure)

leoparente · 2026-07-19T11:41:41Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c0ec1b0f57

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…ders (libcurl re-sends them cross-host, leaking secret headers)

leoparente · 2026-07-19T12:10:46Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3c6976e4d8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…8 resends the payload to the redirect target)

leoparente · 2026-07-19T12:20:50Z

@codex review

chatgpt-codex-connector · 2026-07-19T12:28:11Z

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 139232db46

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

leoparente added 7 commits July 18, 2026 19:22

feat(http): HttpCheck — status-set grammar, body checks, cert-date pa…

977bdcc

…rser

feat(http): transport options (proxy, TLS files, UA, CERTINFO) + resp…

fde76fa

…onse size/cert expiry results

refactor(netprobe): carry http probe results as HttpSample; doh gains…

284de35

… cert_expiry passthrough

feat(netprobe): v2 config — status/body checks, request body, per-tar…

fd0af34

…get headers, proxy, tls, UA

feat(netprobe): content_failures, tls_cert_expiry_epoch_sec, response…

43911a6

…_size_bytes metrics

test(netprobe): v2 e2e — headers/auth, status sets, body checks, POST…

bb49c3d

… body, redaction; docs

fix(netprobe): redact tap-config secrets via an input-plugin hook (GE…

2a43a61

…T /api/v1/taps echoed raw headers/proxy/body)

chatgpt-codex-connector Bot reviewed Jul 19, 2026

View reviewed changes

Comment thread src/inputs/netprobe/HttpProbe.cpp Outdated

fix(netprobe): don't classify content checks against a truncated body…

f7b8d41

…; configurable body_check_max_bytes

chatgpt-codex-connector Bot reviewed Jul 19, 2026

View reviewed changes

Comment thread libs/visor_http_client/HttpCheck.cpp Outdated

fix(http): reject oversized status-code entries before narrowing (sto…

e439a84

…ul wrap could accept them as an in-range code)

chatgpt-codex-connector Bot reviewed Jul 19, 2026

View reviewed changes

Comment thread src/handlers/netprobe/NetProbeStreamHandler.cpp Outdated

leoparente added 2 commits July 19, 2026 08:01

fix(netprobe): keep newest window's tls_cert_expiry on merge, not max…

5ee96c8

… (max suppressed alerts after cert downgrade)

build: bump libcurl 8.20.0->8.21.0, libnghttp2 1.61.0->1.68.1

ce21bf9

leoparente force-pushed the feat/netprobe-v2 branch from b283ae1 to ce21bf9 Compare July 19, 2026 11:01

chatgpt-codex-connector Bot reviewed Jul 19, 2026

View reviewed changes

Comment thread src/inputs/netprobe/NetProbeInputStream.cpp

fix(http): redact proxy value + embedded credentials from curl error_…

c0ec1b0

…msg (probes log it on transport failure)

chatgpt-codex-connector Bot reviewed Jul 19, 2026

View reviewed changes

Comment thread src/inputs/netprobe/HttpProbe.cpp

fix(netprobe): don't follow redirects when a probe carries custom hea…

3c6976e

…ders (libcurl re-sends them cross-host, leaking secret headers)

leoparente force-pushed the feat/netprobe-v2 branch from b3b4b46 to 3c6976e Compare July 19, 2026 12:10

chatgpt-codex-connector Bot reviewed Jul 19, 2026

View reviewed changes

Comment thread src/inputs/netprobe/HttpProbe.cpp Outdated

fix(netprobe): also disable redirects for body-bearing probes (307/30…

139232d

…8 resends the payload to the redirect target)

leoparente self-assigned this Jul 19, 2026

Uh oh!

Conversation

leoparente commented Jul 19, 2026

Summary

Design notes

Out of scope (future)

Testing

Uh oh!

github-actions Bot commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LCOV of commit 139232d during Debug Builds #209

Uh oh!

leoparente commented Jul 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

leoparente commented Jul 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

leoparente commented Jul 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

leoparente commented Jul 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

leoparente commented Jul 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

leoparente commented Jul 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

leoparente commented Jul 19, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jul 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions Bot commented Jul 19, 2026 •

edited

Loading

LCOV of commit `139232d` during Debug Builds #209