Survive the unpaginated get_conversion_rules on rule-heavy accounts - #10
Open
amitl-levi wants to merge 8 commits into
Open
Survive the unpaginated get_conversion_rules on rule-heavy accounts#10amitl-levi wants to merge 8 commits into
amitl-levi wants to merge 8 commits into
Conversation
Users run the plugin in their own terminal, so Taboola Professional Services has no visibility into these conversations — a report that "the plugin gave me the wrong number" arrived with no evidence attached. /realize-plugin:support packages the session into one local Markdown file the user emails to Support@taboola.com. Everything above the transcript is extracted mechanically by build-bundle.js, never model-authored: a plugin that misread the user must not carry that same misreading into its own bug report. The model's judgment is used for exactly one field, the case title. Design constraints enforced in the script, not merely documented: - No upload path. Writes one local file and prints it; transmitting is the user's decision. - Refuses to write inside a git work tree. The bundle holds customer campaign data and this repo is public. - Credentials redacted (bearer/JWT, access_token, api_key, password, client_secret) in both JSON and key=value form, and before any truncation. account_id / campaign_id / item_id are preserved on purpose — PS cannot reproduce an issue without them. - Complaint and title are passed by file, never as shell arguments: a quoted $12.40 arrives as 2.40 and $(…) executes. - Session identified via CLAUDE_CODE_SESSION_ID; the newest-file fallback surfaces confidence: guessed. - Realize tool output gets a larger truncation budget than other output, because a report CSV usually is the case. os/guardrails.md needs an explicit carve-out to offer the path at all: the existing bans on skill names, @taboola.com addresses, and local file paths each block a required part of the escalation message. Scenario 18 is the regression test — without the carve-out the feature quietly never fires, which is invisible rather than an error. 36 dependency-free unit tests cover redaction, table-cell safety, truncation, and destination guards, wired into validate.yml. Read scenarios 17 and 18 added. Also fixes the local-dev instructions in INSTALL.md and README.md, which showed --plugin-dir ./realize-claude-plugin without a cd and so resolved to a doubled path that loads nothing and prints no error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This repo is public, so the gap between "untracked" and "ignored" is one `git add -A`. Three things in a normal working tree here are disclosures rather than mess: - .claude/transcript-uploader/ holds a fetched service-config cache with internal Taboola hostnames and datasource credentials. - /inputs/ holds eval and hypothesis-verdict files carrying real customer campaign data — account and campaign IDs, spend, performance. - Support bundles are the same customer data by construction. build-bundle.js already refuses to write inside a git work tree; this is defense in depth behind that check, not a replacement for it. Also widens the draft patterns. /pr*-comment-*.txt and /reply-to-*.txt each required a trailing segment after the keyword, so real drafts named pr2-readme-comment.txt and reply-mcp-prioritization-feedback.txt fell through and showed up as untracked. Verified with git check-ignore against the actual files, and that no currently tracked file is matched by any new rule. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Requested by Professional Services after reviewing the bundle. Case intake copies an email's subject into the case Subject and its body into the case Description, so the bundle now renders a copy-ready subject line and a `## 1. Summary` meant to be pasted as the body, opening with the requested prolog "This case has been created by the Realize Plugin…". The subject is now the user's own complaint rather than a title written about it. That removes the last model-authored field in the bundle: the user reported the problem, so their sentence is the most faithful description of it available. The Summary reports which Realize tools ran, which skills were invoked, and which knowledge files were read. This answers a question the bundle previously could not — whether the plugin read the right guidance and still got it wrong, or never read it at all. Those are different bugs with different fixes, and from outside they looked identical. PS's request could have been met with a model-written narrative of the session. It deliberately was not. The bundle exists for cases where the plugin misread the request, and a narrative composed by that same plugin reproduces the misreading in the one document meant to expose it. §1 carries counts, tool names and file paths only, and says so in the file. Review of this change found three real defects, all fixed here: - Subject truncation deleted the complaint. account_id is an opaque API string with no length bound; with a long one the slice bound went negative and sliced from the end, replacing the user's entire sentence with a bare "…" in the subject describing their own problem. A MIN_SUBJECT_TEXT_CHARS floor now drops the account suffix instead. - The complaint was never redacted. It stays the user's own wording, but users paste error output they never read, and that text now reaches an email subject. redact() is safe on prose by construction — its flat rule is `=`-only with a length floor — so sentences survive and `Bearer …` does not. - knowledgeRef matched anywhere in a path, so a user's own ~/Documents/os/notes.md was reported as plugin guidance. A false entry is worse than none when the section exists to answer "did it read the guidance". Now anchored to the plugin root. Tests 36 → 77. The no-narrative check is structural — every summary line must be blank, a heading, a bullet, a bold label, the prolog, or the italic disclaimer — so a future edit cannot slip prose past it with wording a keyword blacklist didn't anticipate. Sections renumbered; scenario 17's stale "Section 4" references followed. --title-file is now the fallback for a run with no user description. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The knowledge base is 12 topic files; advertisers ask hundreds of reasonable "how do I..." questions that land just outside them. Those got a thin answer from model priors or a shrug. Closes the P0 "Help Center connection" gap from the 2026-07-06 capability audit. The new web-fallback skill searches Taboola's public advertiser help documentation and answers with a "here's what I found online" framing. It is a fallback, never a supplement: it fires only on a real miss, judged per question rather than per topic, and when a web source contradicts a knowledge file the knowledge file wins silently. Scoped to realize.com/help/ only. A domain allowlist alone is not enough - /marketing-hub/ is promotional copy on the same host and supplied 3 of 10 results in one live test search and 7 of 10 in another, carrying the guaranteed-outcome framing brand-check bans. Search results are treated as a link index, never as an answer: the tool synthesizes prose across every hit before the URL list is visible, and a correctly domain-restricted query returned a summary that recommended a competitor's browser extension. The absolute ban on a "Sources:" footer is rescoped to plugin internals, with an explicit exception for a public URL the user asked for. Left absolute, it either blocked the citation-on-request behaviour or left the model resolving a contradiction at runtime, which is how a capability quietly stops firing. Taboola's developer documentation is deliberately excluded: its API reference covers the API the MCP already wraps and the plugin does not emit endpoint paths or client code, half its sections are publisher-side, and its one advertiser-relevant area is better covered by the help center for a non-developer reader. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…evel writes Upstream shipped create_conversion_rule, update_conversion_rule, and get_conversion_rules. Tool counts go from 19 read + 6 write to 20 read + 8 write. Users can now create a conversion rule, change its attribution windows, add revenue value, stop it counting toward Total Conversions, and retire it - all through the existing preview-then- confirm gate. Per the sync policy in CLAUDE.md the writes route exclusively through manage-campaigns, never the agent. No one-line confirm tier for these. Every other write scopes to one campaign or item; a conversion rule feeds attribution and, when include_in_total_conversions is set, the account's Total Conversions, which Target CPA and Maximize Conversions bid against. One boolean moves reported performance and live bidding across every campaign on the account. The duplicate-event guard is the load-bearing rule. Only one ACTIVE rule may hold an event, and the backend accepts a second once the incumbent is DISABLED - which makes "disable the old one and retry" look like a legitimate recovery from a rejected create. It stops conversion reporting for every campaign using the incumbent. Both the skill and the guardrails forbid taking it unilaterally. Merge semantics are inverted from campaign targeting: these tools partial-merge everything including condition and effects, so the read-and-merge reflex the rest of the skill trains overwrites fields nobody asked to change. The read payload is also not a valid update payload - explicit nulls fail validation and seven fields have no parameter on the update tool. Documented from live QA against a real account: ownership runs both directions (a child account returned 62 rules, all owned by the parent network), the read returns ENGAGEMENT rules the write schema cannot express, and some accounts are write-blocked server-side in a way that looks exactly like a validation error. search_conversion_rules is deprecated upstream and removed after 2026-11-01; migrated everywhere. Stale UI-only claims corrected - the triage table, tool-existence boundary, README, and guardrails all still sent users to the UI for work the plugin can now do. W10-W14 remain unrun: both internal test accounts tried are on the server-side write blocklist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…pi install name
Installing the plugin still reported "installed realize-ads-api@realize" on
machines that registered a pre-rename marketplace catalog: the install
identifier comes from the marketplace entry, not the plugin manifest, and a
static catalog never picks up the July rename to realize-plugin. Meanwhile
the repo itself could not be added as a marketplace at all ("Marketplace
file not found"), which is what pushed installs onto those ad-hoc catalogs.
- Add .claude-plugin/marketplace.json (marketplace "realize", single entry
"realize-plugin" sourced from the repo root). Verified end-to-end:
/plugin marketplace add taboola/realize-claude-plugin +
/plugin install realize-plugin@realize now reports
"Successfully installed plugin: realize-plugin@realize".
- INSTALL.md / README.md: replace the non-working community-marketplace
commands with the repo-marketplace path, and document the one-time
cleanup for machines still carrying the pre-rename registration.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback from PR taboola#9: the capability-baseline section narrated when conversion-rule tools arrived instead of just stating what exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Field finding after the 20260806.1 MCP release: the read has no pagination and no status filter, so a rule-heavy account (observed: 278 rules / ~270 KB, only 102 ACTIVE) overflows the tool-result cap and arrives as an error plus a dumped-file path. rule_id narrowing is circular - the listing is the only way to learn the IDs. Interim guidance until upstream ships pagination/status filtering: - Recover from the dumped file (read in slices); never re-call unmodified, never abandon, never treat the overflow as "no rules". - The mandatory pre-write read inherits the overflow: event/name collision and ownership checks run against the dumped file, across all statuses (a DISABLED rule still holds its display_name). - User-facing listings default to ACTIVE rules with a mandatory one-line disclosure of skipped counts; non-ACTIVE only on explicit ask. Disclosure phrasing carries no file paths or tool names. - Read Scenario 21 (overflow recovery), new pass criteria on 20A and W10, README troubleshooting entry, CHANGELOG and CLAUDE.md notes marking this guidance stale-capability-claims class once upstream pagination ships. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Field finding after the 20260806.1 MCP release (reported by Tzafrir):
get_conversion_ruleshas no pagination and no status filter. On a rule-heavy account — observed live at 278 rules / ~270 KB, only 102 ACTIVE — the response overflows the tool-result cap and arrives as an error plus a path to a dumped result file instead of inline rules.rule_idnarrowing can't rescue it: the listing is the only way to learn the IDs. Upstream has a pagination fix planned; until it ships, the plugin needs to survive.What this adds
display_name). An overflowed pre-read is never license to skip the read-before-write gate, and never a reason to refuse the write.Where
Agent tool reference + error handling,
discoveryandmanage-campaignsskills,mcp-write-surfacefailure-modes table, guardrails' pre-read gate, README troubleshooting, CHANGELOG, CLAUDE.md maintainer note. Tests: new read Scenario 21 (overflow recovery), new pass criteria on Scenario 20A and W10.Interim by design
Every touchpoint carries an "interim until upstream adds pagination / status filtering" marker. When upstream ships, this is stale-capability-claims class — grep for
overflow/no status filterand rewrite rather than layering on top (noted in CLAUDE.md).🤖 Generated with Claude Code