Build a Web Research Agent With Google ADK and Zenrows

This article was originally published on the Zenrows blog. Your research agent might be reading fake pages. When a target sits behind Cloudflare, a plain requests call returns a 403 and a "Just a moment..." challenge page in place of the content. Unless the tool checks for that, the agent can treat the challenge markup as the answer and report on a page it never saw. In this tutorial, you'll see how that happens and how to fix it in a Google Agent Development Kit (ADK) agent. You'll give the agent a web tool that routes each fetch through Zenrows Fetch and returns the page as Markdown, then connect a second agent to the hosted Zenrows MCP server with no function code. You'll need Python 3.10 or later, a Google AI Studio API key, and a Zenrows API key. The free plan includes 5,000 credits a month. The complete code is on GitHub. What You'll Build You'll register a function tool, fetch_protected_page, on a Gemini-powered ADK agent. You'll write a comparison script that fetches one URL with plain HTTP and with Zenrows. You'll add a second agent that loads the Zenrows MCP tools through ADK's MCPToolset. 1. Set Up the Project python -m venv .venv source .venv/bin/activate python -m pip install -r requirements.txt The pinned requirements.txt installs google-adk 2.9.2 with its MCP extra. google-adk[mcp]==2.9.2 mcp==1.30.0 python-dotenv==1.2.3 requests==2.34.2 pytest==9.1.1 ruff==0.16.8 -e . Copy the environment template and add your keys. The .gitignore excludes .env. cp .env.example .env GOOGLE_API_KEY=your_google_api_key ZENROWS_API_KEY=your_zenrows_api_key Here's the project layout. google-adk-zenrows-research-agent/ ├── scripts/ │ ├── compare_fetch.py │ └── list_mcp_tools.py ├── tests/ │ ├── test_agent.py │ ├── test_mcp_agent.py │ └── test_web.py ├── zenrows_mcp_agent/ │ ├── __init__.py │ └── agent.py ├── zenrows_research_agent/ │ ├── __init__.py │ ├── agent.py │ └── web.py ├── .env.example ├── .gitignore ├── LICENSE ├── pyproject.toml ├── README.md └── requirements.txt 2. Write the Fetch Function Create zenrows_research_agent/web.py. """Web retrieval tools used by the Google ADK agent.""" import os import requests ZENROWS_ENDPOINT = "https://api.zenrows.com/v1/" DEFAULT_TIMEOUT = 180 MAX_TOOL_CONTENT = 12_000 def fetch_with_http(url: str) -> dict: """Fetch a public URL with a standard HTTP request for comparison.""" try: # send the request without a protected-site access service. response = requests.get( url, headers={"User-Agent": "google-adk-research-agent/0.1"}, timeout=20, ) except requests.RequestException as exc: return {"status": "error", "url": url, "error": str(exc)} return { "status": "success" if response.ok else "blocked", "url": url, "status_code": response.status_code, "content": response.text[:2_000], } def fetch_protected_page(url: str) -> dict: """Fetch a protected or JavaScript-rendered page through Zenrows. Args: url: The complete public URL the research agent needs to read. Returns: A dictionary containing the request status, source URL, and page content. """ api_key = os.getenv("ZENROWS_API_KEY") if not api_key: return { "status": "error", "url": url, "error": "ZENROWS_API_KEY is not configured.", } # let Adaptive Stealth Mode select the request configuration. params = { "apikey": api_key, "url": url, "mode": "auto", "response_type": "markdown", "original_status": "true", } try: # give Adaptive Stealth Mode enough time to retrieve the page. response = requests.get( ZENROWS_ENDPOINT, params=params, timeout=DEFAULT_TIMEOUT, ) response.raise_for_status() except requests.RequestException as exc: # remove the API key from the error text before it reaches the model. error = str(exc).replace(api_key, "[REDACTED]") return {"status": "error", "url": url, "error": error} # do not return a successful result without page content. if not response.text.strip(): return { "status": "error", "url": url, "error": "Zenrows returned an empty response.", } return { "status": "success", "url": url, "content": response.text[:MAX_TOOL_CONTENT], } fetch_protected_page sends the URL to Zenrows with mode=auto, which turns on Adaptive Stealth Mode. It starts with the lightest configuration likely to work and escalates only when the target needs it. A few safeguards keep the agent honest. response_type=markdown returns readable text in place of raw HTML. original_status=true passes through the target's status code, so an error page from the target counts as an error. The 180-second timeout gives mode=auto room to finish on protected targets. Content is capped at 12,000 characters, and an empty response is reported as an error. The API key is redacted from any error text before it reaches the model. fetch_with_http is a plain request used only for the comparison. ## 3. Register the Function as an ADK Tool Create zenrows_research_agent/agent.py. """Google ADK agent configuration.""" from pathlib import Path from dotenv import load_dotenv from google.adk.agents import Agent from .web import fetch_protected_page # load the API keys stored in the project environment file. load_dotenv(Path(__file__).resolve().parents[1] / ".env") # register the Zenrows retrieval function as an ADK tool. root_agent = Agent( model="gemini-3.6-flash", name="web_research_agent", description="Researches public web pages that ordinary HTTP tools cannot read.", instruction=( "You are a web research agent. Use fetch_protected_page whenever a task " "requires content from a supplied URL. Base the answer only on returned " "page content, mention the source URL, and state clearly when retrieval " "fails or the page does not contain the requested information." ), tools=[fetch_protected_page], ) ADK builds the tool schema from the function's name, type hints, and docstring, so the model only has to supply a URL. 4. Compare Plain HTTP With Zenrows Create scripts/compare_fetch.py. """Compare standard HTTP access with Zenrows access.""" import json from dotenv import load_dotenv from zenrows_research_agent.web import fetch_protected_page, fetch_with_http TARGET_URL = "https://www.scrapingcourse.com/cloudflare-challenge" def summarize(result: dict) -> dict: """Reduce a fetch result to fields that are safe to print.""" return { "status": result["status"], "url": result["url"], "status_code": result.get("status_code"), "content_preview": result.get("content", "")[:300], "error": result.get("error"), } if __name__ == "__main__": # load the Zenrows API key from the project environment. load_dotenv() # retrieve the same URL through both access methods. comparison = { "standard_http": summarize(fetch_with_http(TARGET_URL)), "zenrows": summarize(fetch_protected_page(TARGET_URL)), } # print the comparison without exposing the API key. print(json.dumps(comparison, indent=2)) python scripts/compare_fetch.py { "standard_http": { "status": "blocked", "url": "https://www.scrapingcourse.com/cloudflare-challenge", "status_code": 403, "content_preview": "Just a moment... None: # ask the hosted Zenrows MCP server which tools it exposes. tools = await zenrows_toolset.get_tools() await zenrows_toolset.close() names = {tool.name for tool in tools} # confirm the tools a research agent relies on are loaded. print(f"Loaded {len(names)} Zenrows tools") for name in RESEARCH_TOOLS: print(f"{name}: {'available' if name in names else 'not loaded'}") if __name__ == "__main__": asyncio.run(main()) python scripts/list_mcp_tools.py adk run zenrows_mcp_agent Loaded 62 Zenrows tools scrape: available extract: available crawl_create: available batch_create: available browser_navigate: available ADK prints experimental-feature warnings before that output. Function Tool or MCP Server Function tool Zenrows MCP What the agent calls fetch_protected_page, one typed function scrape, extract, Crawl, Batch, and browser tools Request settings Fixed in code (mode=auto, Markdown) Chosen by the MCP server per task Code you maintain The function and its tests One MCPToolset block Pick it when You want one fixed, typed request You want the agent to choose among scraping, crawling, and browser actions Wrapping Up Your ADK agent now reads protected pages as Markdown through a typed function, and a second agent reaches the full Zenrows toolset over MCP. Change the URL and the prompt to point it at your own research tasks. What's Next Use Extract with extract=auto when you need structured JSON fields. Discover linked pages with Crawl or run large URL lists as one job with Batch. Build the same idea with Agno and Zenrows or as a multi-agent system with AG2.