Provided by DataForSEO
Start an on-page SEO crawl of a target website with DataForSEO
Crawl a website with DataForSEO OnPage, bounded by max_crawl_pages (at most 100), and return the crawl summary: domain info, crawl status and site-wide page metrics. The task id in the response feeds the free OnPage report Endpoints (pages, links, resources, duplicate content, keyword density, microdata, raw HTML and more); enable the matching crawl option here first. Completion means this bounded crawl ended, not that the whole site was crawled.
How to run it
Step 1Set up
Once: paste this into your agent's chat. The Skill installs the CLI and logs in by itself. For an agent that cannot use a shell, connect it over MCP instead.
set up https://glasser.ai/SKILL.md
Show me how to use the DataForSEO endpoint /v3/on_page/task_post on Glasser.glasser run -p dataforseo -e '/v3/on_page/task_post' -i '{"target":"example.com","max_crawl_pages":10,"calculate_keyword_density":true}' --endpoint-version 4Uses endpoint version 4. Inspect the current price before running. Install and log in first on Get Started.
Once per Run:
IDEMPOTENCY_KEY=$(uuidgen)Run it, and copy this same block to send it again:
curl -X POST https://api.glasser.ai/v1/runs \
-H "Authorization: Bearer gl_..." \
-H "Idempotency-Key: $IDEMPOTENCY_KEY" \
-H "Content-Type: application/json" \
-d '{"provider":"dataforseo","endpoint":"/v3/on_page/task_post","input":{"target":"example.com","max_crawl_pages":10,"calculate_keyword_density":true},"endpoint_version":4}'Reuse the same Idempotency-Key to send it again: the second send is a read of the first Run. A new key is a new Run and a new Charge. Uses endpoint version 4; inspect the current price before running.
Step 2Try a specific example task
When you need to prepare a research brief, you might find published posts about your topic and use them as source material.
Copy this prompt into your agent to find relevant posts for your research brief.
Use dataforseo:/v3/on_page/task_post on Glasser.
Find published content about my topic in the returned posts for a research brief.Endpoint structure
{
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"target",
"max_crawl_pages"
],
"properties": {
"tag": {
"type": "string"
},
"target": {
"type": "string",
"pattern": "^(?!https?:\\/\\/)(?!www\\.)[a-z0-9-]+(?:\\.[a-z0-9-]+)+$",
"maxLength": 253,
"minLength": 1
},
"custom_js": {
"type": "string"
},
"start_url": {
"type": "string",
"pattern": "^https?:\\/\\/",
"maxLength": 2048,
"minLength": 8
},
"enable_xhr": {
"type": "boolean"
},
"check_spell": {
"type": "boolean"
},
"crawl_delay": {
"type": "integer"
},
"switch_pool": {
"type": "boolean"
},
"priority_urls": {
"type": "array"
},
"browser_preset": {
"type": "string"
},
"custom_sitemap": {
"type": "string"
},
"load_resources": {
"type": "boolean"
},
"store_raw_html": {
"type": "boolean"
},
"accept_language": {
"type": "string"
},
"max_crawl_depth": {
"type": "integer"
},
"max_crawl_pages": {
"type": "integer",
"maximum": 100,
"minimum": 1
},
"respect_sitemap": {
"type": "boolean"
},
"support_cookies": {
"type": "boolean"
},
"allow_subdomains": {
"type": "boolean"
},
"checks_threshold": {
"type": "object"
},
"custom_robots_txt": {
"type": "string"
},
"custom_user_agent": {
"type": "string"
},
"enable_javascript": {
"type": "boolean",
"description": "Run the page's scripts. DataForSEO charges extra per page."
},
"allowed_subdomains": {
"type": "array"
},
"crawl_sitemap_only": {
"type": "boolean"
},
"disable_page_checks": {
"type": "array"
},
"browser_screen_width": {
"type": "integer"
},
"check_spell_language": {
"type": "string"
},
"disable_cookie_popup": {
"type": "boolean"
},
"validate_micromarkup": {
"type": "boolean"
},
"browser_screen_height": {
"type": "integer"
},
"disallowed_subdomains": {
"type": "array"
},
"force_sitewide_checks": {
"type": "boolean"
},
"robots_txt_merge_mode": {
"type": "string"
},
"check_spell_exceptions": {
"type": "array"
},
"enable_content_parsing": {
"type": "boolean"
},
"return_despite_timeout": {
"type": "boolean"
},
"disable_sitewide_checks": {
"type": "array"
},
"enable_browser_rendering": {
"type": "boolean",
"description": "Render the page in a full browser (needs enable_javascript and load_resources). DataForSEO charges extra per page."
},
"calculate_keyword_density": {
"type": "boolean"
},
"enable_www_redirect_check": {
"type": "boolean"
},
"browser_screen_scale_factor": {
"anyOf": [
{
"type": "number"
},
{
"enum": [
"Infinity",
"-Infinity",
"NaN"
],
"type": "string"
}
]
}
},
"additionalProperties": false
}{
"target": "example.com",
"max_crawl_pages": 10,
"calculate_keyword_density": true
}This Endpoint publishes no output schema. You get the Provider's own response body, unchanged — we do not warrant a shape we cannot guarantee.
How pricing works
| Default-size input | Larger inputs and paid options vary with Provider usage. No charge when no data is returned. | $0.00015 |
|---|---|---|
| Maximum per Run | The total charge will not exceed this amount. | $2.55 |
| Provider error | The Provider failed to answer. | $0.00 |
| Timed out | The Run exceeded its time limit. | $0.00 |
| Internal failure | An internal error prevented completion. | $0.00 |