> ## Documentation Index
> Fetch the complete documentation index at: https://docs.context.dev/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Setting up Context.dev for a user? Follow https://docs.context.dev/agent-quickstart.md. MCP clients sign in with OAuth and need no API key; application code reads CONTEXT_DEV_API_KEY.

# Context.dev: the web data API

> Fast web scraping with browsers, proxies, retries, batches, and monitors built in.

Get data from the internet, fast. Context.dev handles the scraping infrastructure so you can turn web pages into Markdown, structured JSON, screenshots, and more through one API.

Start with one URL, scrape thousands of pages in a batch, or monitor sites for changes.

**Quality comes standard.** JavaScript rendering and managed proxies are always on for web scraping, with no opt-out. We keep both enabled by design to maximize data quality and reliability, so you can focus on building.

<CardGroup cols={3}>
  <Card title="Quickstart" icon="terminal" href="/quickstart">
    Get an API key and make your first request.
  </Card>

  <Card title="Agent quickstart" icon="robot" href="/agent-quickstart">
    Paste one prompt into your coding agent to set up Context.dev.
  </Card>

  <Card title="API reference" icon="code" href="/api-reference/web-scraping/scrape">
    Check each endpoint's parameters and response fields.
  </Card>
</CardGroup>

## Make a request

This request scrapes `example.com` and returns its main content as Markdown. The examples read your API key from `CONTEXT_DEV_API_KEY`; the [Quickstart](/quickstart) covers getting a key and installing an SDK.

<CodeGroup>
  ```typescript TypeScript theme={null}
  import ContextDev from "context.dev";

  const client = new ContextDev({ apiKey: process.env.CONTEXT_DEV_API_KEY });

  const page = await client.web.scrape({
    url: "https://example.com",
    formats: { markdown: true },
    sharedParams: { mainContentOnly: true },
  });

  console.log(page.markdown.data);
  ```

  ```python Python theme={null}
  import os
  from context.dev import ContextDev

  client = ContextDev(api_key=os.environ["CONTEXT_DEV_API_KEY"])

  page = client.web.scrape(
      url="https://example.com",
      formats={"markdown": True},
      shared_params={"main_content_only": True},
  )

  print(page.markdown.data)
  ```

  ```ruby Ruby theme={null}
  require "cgi/core"
  require "context_dev"

  client = ContextDev::Client.new(api_key: ENV.fetch("CONTEXT_DEV_API_KEY"))

  page = client.web.scrape(
    url: "https://example.com",
    formats: {markdown: true},
    shared_params: {main_content_only: true},
  )

  puts page.markdown.data
  ```

  ```go Go theme={null}
  package main

  import (
      "context"
      "fmt"
      "os"

      contextdev "github.com/context-dot-dev/context-go-sdk/v2"
      "github.com/context-dot-dev/context-go-sdk/v2/option"
  )

  func main() {
      client := contextdev.NewClient(option.WithAPIKey(os.Getenv("CONTEXT_DEV_API_KEY")))

      page, err := client.Web.Scrape(context.Background(), contextdev.WebScrapeParams{
          URL:          "https://example.com",
          Formats:      contextdev.WebScrapeParamsFormats{Markdown: contextdev.Bool(true)},
          SharedParams: contextdev.WebScrapeParamsSharedParams{MainContentOnly: contextdev.Bool(true)},
      })
      if err != nil {
          panic(err)
      }

      fmt.Println(page.Markdown.Data)
  }
  ```

  ```php PHP theme={null}
  <?php

  require __DIR__.'/vendor/autoload.php';

  use ContextDev\Client;

  $client = new Client(apiKey: getenv('CONTEXT_DEV_API_KEY'));

  $page = $client->web->scrape(
      formats: ['markdown' => true],
      url: 'https://example.com',
      sharedParams: ['mainContentOnly' => true],
  );

  echo $page->markdown->data, PHP_EOL;
  ```

  ```bash cURL theme={null}
  curl https://api.context.dev/v1/web/scrape \
    --request POST \
    --header "Authorization: Bearer $CONTEXT_DEV_API_KEY" \
    --header "Content-Type: application/json" \
    --data '{
      "url": "https://example.com",
      "formats": { "markdown": true },
      "sharedParams": { "mainContentOnly": true }
    }'
  ```
</CodeGroup>

The page text is in `markdown.data`. The page title and description are in `metadata`.

## Scraping infrastructure, handled

* **Browsers.** JavaScript rendering comes built in. Use [browser actions](/scrape/browser-actions) to click, type, and scroll before capturing a page.
* **Proxies and retries.** Context.dev manages proxy selection and fetch retries. Choose a [fetch country](/scrape/location-and-headers) when location matters.
* **Fast results.** Get multiple formats from one page visit. Reuse [cached outputs](/scrape/freshness-and-caching) for repeated reads, with control over freshness.
* **Batches.** [Process thousands of URLs](/batches/overview) as a background job with queuing, progress tracking, and downloadable results.
* **Monitors.** [Watch pages and sites](/monitors/overview) on a schedule and receive changes by webhook.

## Choose an API

| You have | You want | Use |
| - | - | - |
| A URL | Page content, structured data, or screenshots | [Scrape](/scrape/overview) |
| A list of URLs, or a site to crawl | Results from a background job | [Batches](/batches/overview) |
| A page or site to watch | A webhook when it changes | [Monitors](/monitors/overview) |
| A domain | Its URLs, with available titles and descriptions | [Map URLs](/map/overview) |
| A starting URL | Linked pages as Markdown in one response | [Crawl](/crawl/overview) |
| A search query | Ranked web results, with page content if you ask for it | [Search](/search/overview) |
| A research question and a JSON shape | An answer in that shape, with source URLs | [Answers](/answers/overview) |
| A PDF, Office document, image, or other file | Its content as Markdown | [Parse](/parse/overview) |

The same API also provides [brand data](/brand/overview), [people enrichment](/people/overview), and [company news](/news/overview).

## Use Context.dev from an agent

The [Agent quickstart](/agent-quickstart) gives your coding agent one prompt that picks a setup, connects your account, and verifies a first result. To set up one piece yourself, connect the [MCP server](/install-mcp) for live tools, install the [agent skill](/install-skill) to teach the agent the API, or use the [CLI](/install-cli) in scripts.

## Before you ship

Context.dev is a hosted API with [SDKs](/sdks) for TypeScript, Python, Ruby, Go, and PHP. The [production checklist](/optimization/best-practices) covers timeouts, retries, and API key permissions. See [Credits and pricing](/account/credits) for usage costs and each guide for limits and failure handling.

## APIs

<CardGroup cols={3}>
  <Card title="Scrape" icon="globe" href="/scrape/overview">
    Turn any page into usable data.
  </Card>

  <Card title="Map" icon="map" href="/map/overview">
    Discover the URLs on a website.
  </Card>

  <Card title="Crawl" icon="sitemap" href="/crawl/overview">
    Follow links and collect page content.
  </Card>

  <Card title="Search" icon="magnifying-glass" href="/search/overview">
    Search the web for relevant results.
  </Card>

  <Card title="Answers" icon="brain" href="/answers/overview">
    Research questions with cited sources.
  </Card>

  <Card title="Parse" icon="file-lines" href="/parse/overview">
    Extract content from documents and images.
  </Card>

  <Card title="Brand" icon="palette" href="/brand/overview">
    Get logos, colors, fonts, and more.
  </Card>

  <Card title="People" icon="users" href="/people/overview">
    Find people and enrich their profiles.
  </Card>

  <Card title="News" icon="newspaper" href="/news/overview">
    Find the latest coverage of companies.
  </Card>

  <Card title="Batches" icon="layer-group" href="/batches/overview">
    Process thousands of URLs in parallel.
  </Card>

  <Card title="Monitors" icon="bell" href="/monitors/overview">
    Track changes across pages and sites.
  </Card>
</CardGroup>

## Account and operations

<CardGroup cols={3}>
  <Card title="API keys" icon="key" href="/account/api-keys">
    Create keys and control API access.
  </Card>

  <Card title="Plans and billing" icon="credit-card" href="/account/billing">
    Manage your plan and payment details.
  </Card>

  <Card title="Team access" icon="user-group" href="/account/team">
    Invite teammates and manage their roles.
  </Card>

  <Card title="ZDR usage" icon="shield-halved" href="/optimization/zero-data-retention">
    Keep request content out of retained logs.
  </Card>

  <Card title="Request logs" icon="list" href="/optimization/usage-and-logs">
    Inspect requests and track your usage.
  </Card>

  <Card title="Timeouts" icon="clock" href="/optimization/timeouts">
    Set time limits for your requests.
  </Card>

  <Card title="Rate limits" icon="gauge" href="/optimization/rate-limits">
    Plan throughput and handle rate limits.
  </Card>
</CardGroup>