Repository navigation
docs(skills): replace "Scrape single URL" guidance with Web Fetch Standby - #91
Merged
Merged
Conversation
…ndby The integration development skill told agents to build the single-URL action as a content-crawler run. Point it at the apify/web-fetch Actor in Standby mode instead, and state where the run rules (cost cap, polling, retry) do not apply. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Keep the shared contract once in SKILL.md, link the Actor docs for the rest, and leave only category-specific points in the references. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
drobnikj
approved these changes
Oct 5, 2026
drobnikj
left a comment
drobnikj
left a comment
Member
There was a problem hiding this comment.
two notes, but pre aprroving
| - API key resolution order: plugin config field -> `APIFY_API_KEY` (or `APIFY_TOKEN`) env var. Normalize pasted input - strip line/paragraph separators and trim whitespace (defends against copy-paste artifacts). | ||
| - The key is **never** included in tool output, **never** logged, only passed to the client constructor. | ||
| - Validate `baseUrl` against an allowlist prefix (`https://api.apify.com`) to prevent SSRF - a misconfigured plugin must not point at an arbitrary host. | ||
| - Validate `baseUrl` against an allowlist prefix (`https://api.apify.com`, plus `https://web-fetch.apify.actor` if the plugin ships a Web Fetch tool) to prevent SSRF - a misconfigured plugin must not point at an arbitrary host. |
Member
There was a problem hiding this comment.
Note: ai-framework-package.md gains the rule that headers must never be exposed to the model (a fetched page can talk the agent into sending credentials to a host of its choosing), but a harness plugin is just as LLM-driven and only gets the baseUrl allowlist. That rule belongs in this file's security bullets too.
Contributor
Author
There was a problem hiding this comment.
@drobnikj I added a new bullet point. Could you please check if thats what you meant?
|
|
||
| When extracting page content from crawling Actors, prefer `markdown` over `text`, with a trailing `or ''` to guarantee a string even when a key is present but null. Follow a fixed fallback order for the source URL: nested `metadata.url` -> `crawledUrl` -> top-level `url`. Tolerate a `metadata` field that is missing or not a dict (some Actor responses surface `null`). Actor output shapes are inconsistent across versions and configurations; centralize one canonical fallback order so the retriever, loaders, and tools all agree on what "the content", "the source URL", and "the title" mean. | ||
|
|
||
| **Single known URL -> Web Fetch.** For the "fetch this URL" tool, do not run a crawler with a page limit of 1 - call the Web Fetch Standby endpoint described in `SKILL.md`. It is the one client method that is a plain HTTPS request instead of `client.actor(id).call(...)`, so add the Bearer token and the attribution header by hand, and set `run` to `null` in the envelope. Take the content from `markdown`, the source URL from `fetch.loadedUrl`, and the title from `metadata.title`. For the LLM-facing tool: pin `formats` to `["markdown"]`, do not expose `headers` to the model (a fetched page could talk the agent into sending credentials to a host of its choosing), and fetch one URL per call with the content truncated to a developer-controlled cap. |
Member
There was a problem hiding this comment.
Note: The naming will change soon, from Stanby Actor to Server Actor. Maybe use Server Actor instead of Stanby right away to be future proof 😄 see more in #product-server-actors slack channel.
protoss70
approved these changes
Oct 6, 2026
… in harness plugins Address review: say "Server Actor" instead of "Standby", noting the API and docs may still use the old name, and add the rule that a harness plugin's Web Fetch tool must not expose headers to the model. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes apify/integrations-team#144
The integration skill told agents to build "Scrape single URL" as a crawler run. It now tells them to build a Web Fetch action that calls
apify/web-fetchin Standby mode, as agreed in apify/integrations-team#114.Changes
SKILL.md: the convenience action is now Web Fetch. The cost and HTTP rules say that a Standby call is not a run.workflow-automation.md: section 13 is rewritten for Web Fetch, with a note on how to migrate an old "Scrape single URL" action. Also fixes a wrong section reference.ai-framework-package.md: the "scrape URL" tool becomes "web fetch", with rules for LLM use. Also fixes a broken table.ai-harness-plugin.md: two small exceptions for the Web Fetch host.