Skip to content

feat: add optional you.com search integration - #51

Open
brainsparker wants to merge 1 commit into
saigontechnology:mainfrom
brainsparker:feat/youcom-search
Open

brainsparker wants to merge 1 commit into
saigontechnology:mainfrom
brainsparker:feat/youcom-search

Conversation

@brainsparker

Copy link
Copy Markdown

I work at You.com, so treat the recommendation accordingly.

Right now search_web only exists when TAVILY_API_KEY is set. Without it, agents start with no web search at all and setup prints a hint to go get a key.

This PR keeps Tavily exactly as it is and adds You.com as the fallback for that case. You.com's free endpoint (https://api.you.com/v1/agents/search) needs no API key or account, so search_web works on first run.

What changes

  • New AgentCrew/modules/web_search/you_service.py (YouSearchService). Stdlib urllib only, no new dependency.
    • No key: GET https://api.you.com/v1/agents/search (free tier, 100 searches a day, counted per IP).
    • YDC_API_KEY set: POST https://ydc-index.io/v1/search with X-API-Key, higher limits.
    • Returns results in the same {results: [{title, url, content}]} shape as Tavily, so the existing search_web handler and formatter work unchanged.
    • 401/402/403/429 come back as readable tool errors. A short Retry-After on 429 is honored once.
    • Sends User-Agent: youdotcom-integration/saigontechnology-agentcrew on You.com requests only.
  • setup.py: if TavilySearchService() raises the missing-key ValueError, use YouSearchService instead and print one line saying so. Any other error still disables web search as before.
  • web_search/tool.py: when the service is You.com, only search_web is registered. fetch_webpage and crawl_website still need Tavily.
  • YDC_API_KEY added to the config key list, the GUI API key settings (marked optional), and CONFIGURATION.md.
  • tests/test_you_search.py: mocked tests for the keyless GET, the keyed POST, error mapping and handler output. No network.

What doesn't change

  • With TAVILY_API_KEY set, behavior is identical: Tavily handles search, extract and crawl.
  • No new packages in pyproject.toml.

Testing

  • pytest tests/test_you_search.py: 4 passed.
  • Full pytest tests: same pass/fail set before and after this change (the existing failures are environment-related, e.g. PortAudio and GUI tests on a headless box).
  • black and ruff clean on the changed files.
  • Live check with no key: YouSearchService().search("AgentCrew multi-agent framework", max_results=3) returned 3 results, top hit agentcrew.dev.

I saw #49 adds Parallel as an MCP example in the docs. This is a different shape (built-in fallback, no config needed), so they don't conflict.

If you'd rather have this as an explicit opt-in (for example a search_provider setting) instead of a fallback, say so and I'll change it.

Fall back to You.com search when TAVILY_API_KEY is not set. Works without a key on the free tier; YDC_API_KEY raises limits. Search only; extract and crawl still use Tavily.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant