Provided by DataForSEO
OnPage site crawl
Crawl a website with DataForSEO OnPage, bounded by max_crawl_pages (at most 100), and return the crawl summary: domain info, crawl status and site-wide page metrics. The task id in the response feeds the free OnPage report Endpoints (pages, links, resources, duplicate content, keyword density, microdata, raw HTML and more); enable the matching crawl option here first. Completion means this bounded crawl ended, not that the whole site was crawled.
Contract snapshot: 2026-09-29. Inspect in Glasser for current terms.
{
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"target",
"max_crawl_pages"
],
"properties": {
"target": {
"type": "string",
"pattern": "^(?!https?:\\/\\/)(?!www\\.)[a-z0-9-]+(?:\\.[a-z0-9-]+)+$",
"maxLength": 253,
"minLength": 1
},
"start_url": {
"type": "string",
"pattern": "^https?:\\/\\/",
"maxLength": 2048,
"minLength": 8
},
"load_resources": {
"type": "boolean"
},
"store_raw_html": {
"type": "boolean"
},
"max_crawl_pages": {
"type": "integer",
"maximum": 100,
"minimum": 1
},
"validate_micromarkup": {
"type": "boolean"
},
"enable_content_parsing": {
"type": "boolean"
},
"calculate_keyword_density": {
"type": "boolean"
}
},
"additionalProperties": false
}{
"target": "example.com",
"max_crawl_pages": 10,
"calculate_keyword_density": true
}This Endpoint publishes no output schema. You get the Provider's own response body, unchanged — we do not warrant a shape we cannot guarantee.
How to run it
Two fields identify it: the Provider and the endpoint key. The key looks like a path because it usually mirrors the Provider's own, but it is an opaque identifier — not a URL you can open. The same provider:endpoint pair is what your Policy lists.
Once: paste this into your agent's chat. The Skill installs the CLI and logs in by itself. For an agent that cannot use a shell, connect it over MCP instead.
set up https://glasser.ai/SKILL.mdThen ask for this Endpoint:
Run the DataForSEO Endpoint /v3/on_page/task_post on Glasser with input {"target":"example.com","max_crawl_pages":10,"calculate_keyword_density":true}glasser run -p dataforseo -e '/v3/on_page/task_post' -i '{"target":"example.com","max_crawl_pages":10,"calculate_keyword_density":true}' --endpoint-version 1Pinned to version 1, the one this page shows: if a newer version is published first, the Run is refused instead of charged under new terms. Install and log in first on Get Started.
Once per Run:
IDEMPOTENCY_KEY=$(uuidgen)Run it, and copy this same block to send it again:
curl -X POST https://api.glasser.ai/v1/runs \
-H "Authorization: Bearer gl_..." \
-H "Idempotency-Key: $IDEMPOTENCY_KEY" \
-H "Content-Type: application/json" \
-d '{"provider":"dataforseo","endpoint":"/v3/on_page/task_post","input":{"target":"example.com","max_crawl_pages":10,"calculate_keyword_density":true},"endpoint_version":1}'Reuse the same Idempotency-Key to send it again: the second send is a read of the first Run. A new key is a new Run and a new Charge. Pinned to version 1 for the same reason as the CLI.
What each outcome charges
| No result | The Provider answered: nothing matched. | $0.00 |
|---|---|---|
| Provider error | The Provider failed to answer. | $0.00 |
| Timed out | The deadline won. | $0.00 |
| Internal failure | Our defect. | $0.00 |