Agent Development Kit (ADK)

Integrate Firecrawl with Google’s ADK using MCP for advanced agent workflows

Integrate Firecrawl with Google’s Agent Development Kit (ADK) to build powerful AI agents with web scraping capabilities through the Model Context Protocol (MCP).

Overview

Firecrawl provides an MCP server that seamlessly integrates with Google’s ADK, enabling your agents to efficiently scrape, crawl, and extract structured data from any website. The integration supports both cloud-based and self-hosted Firecrawl instances with streamable HTTP for optimal performance.

Features

• Efficient web scraping, crawling, and content discovery from any website
• Advanced search capabilities and intelligent content extraction
• Deep research and high-volume batch scraping
• Flexible deployment (cloud-based or self-hosted)
• Optimized for modern web environments with streamable HTTP support

Prerequisites

• Obtain an API key for Firecrawl from firecrawl.dev
• Install Google ADK

Setup

Remote MCP Server

from google.adk.agents.llm_agent import Agent
from google.adk.tools.mcp_tool.mcp_session_manager import StreamableHTTPServerParams
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset

FIRECRAWL_API_KEY = "YOUR-API-KEY"

root_agent = Agent(
    model="gemini-2.5-pro",
    name="firecrawl_agent",
    description='A helpful assistant for scraping websites with Firecrawl',
    instruction='Help the user search for website content',
    tools=[
        MCPToolset(
            connection_params=StreamableHTTPServerParams(
                url=f"https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp",
            ),
        )
    ],
)

from google.adk.agents.llm_agent import Agent
from google.adk.tools.mcp_tool.mcp_session_manager import StdioConnectionParams
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset
from mcp import StdioServerParameters

root_agent = Agent(
    model='gemini-2.5-pro',
    name='firecrawl_agent',
    description='A helpful assistant for scraping websites with Firecrawl',
    instruction='Help the user search for website content',
    tools=[
        MCPToolset(
            connection_params=StdioConnectionParams(
                server_params = StdioServerParameters(
                    command='npx',
                    args=[
                        "-y",
                        "firecrawl-mcp",
                    ],
                    env={
                        "FIRECRAWL_API_KEY": "YOUR-API-KEY",
                    }
                ),
                timeout=30,
            ),
        )
    ],
)

Available Tools

Tool	Name	Description
Scrape Tool	firecrawl_scrape	Scrape content from a single URL with advanced options
Batch Scrape Tool	firecrawl_batch_scrape	Scrape multiple URLs efficiently with built-in rate limiting and parallel processing
Check Batch Status	firecrawl_check_batch_status	Check the status of a batch operation
Map Tool	firecrawl_map	Map a website to discover all indexed URLs on the site
Search Tool	firecrawl_search	Search the web and optionally extract content from search results
Crawl Tool	firecrawl_crawl	Start an asynchronous crawl with advanced options
Check Crawl Status	firecrawl_check_crawl_status	Check the status of a crawl job
Extract Tool	firecrawl_extract	Extract structured information from web pages using LLM capabilities

Configuration

Required Configuration

FIRECRAWL_API_KEY: Your Firecrawl API key

• Required when using cloud API (default)
• Optional when using self-hosted instance with FIRECRAWL_API_URL

Optional Configuration

Firecrawl API URL (for self-hosted instances):

• FIRECRAWL_API_URL: Custom API endpoint
• Example: https://firecrawl.your-domain.com
• If not provided, the cloud API will be used

Retry configuration:

• FIRECRAWL_RETRY_MAX_ATTEMPTS: Maximum retry attempts (default: 3)
• FIRECRAWL_RETRY_INITIAL_DELAY: Initial delay in milliseconds (default: 1000)
• FIRECRAWL_RETRY_MAX_DELAY: Maximum delay in milliseconds (default: 10000)
• FIRECRAWL_RETRY_BACKOFF_FACTOR: Exponential backoff multiplier (default: 2)

Credit usage monitoring:

• FIRECRAWL_CREDIT_WARNING_THRESHOLD: Warning threshold (default: 1000)
• FIRECRAWL_CREDIT_CRITICAL_THRESHOLD: Critical threshold (default: 100)

Example: Web Research Agent

from google.adk.agents.llm_agent import Agent
from google.adk.tools.mcp_tool.mcp_session_manager import StreamableHTTPServerParams
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset

FIRECRAWL_API_KEY = "YOUR-API-KEY"

# Create a research agent
research_agent = Agent(
    model="gemini-2.5-pro",
    name="research_agent",
    description='An AI agent that researches topics by scraping and analyzing web content',
    instruction='''You are a research assistant. When given a topic or question:
    1. Use the search tool to find relevant websites
    2. Scrape the most relevant pages for detailed information
    3. Extract structured data when needed
    4. Provide comprehensive, well-sourced answers''',
    tools=[
        MCPToolset(
            connection_params=StreamableHTTPServerParams(
                url=f"https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp",
            ),
        )
    ],
)

# Use the agent
response = research_agent.run("What are the latest features in Python 3.13?")
print(response)

Best Practices

1. Use the right tool for the job:

   • firecrawl_search when you need to find relevant pages first
   • firecrawl_scrape for single pages
   • firecrawl_batch_scrape for multiple known URLs
   • firecrawl_crawl for discovering and scraping entire sites

2. Monitor your usage: Configure credit thresholds to avoid unexpected usage

3. Handle errors gracefully: Configure retry settings based on your use case

4. Optimize performance: Use batch operations when scraping multiple URLs

---

Related Resources

Comprehensive Guide to Building AI Agents Using Google Agent Development Kit (ADK) and Firecrawl

Learn how to build powerful multi-agent AI systems using Google’s ADK framework with Firecrawl for web scraping capabilities.

MCP Server Documentation

Learn more about Firecrawl’s Model Context Protocol (MCP) server integration and capabilities.

Google ADK Official Documentation

Explore the official Google Agent Development Kit documentation for comprehensive guides and API references.