Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI
Free account. Installing gives you the manifest plus copy-paste snippets.
See reviewsImported from the official MCP registry - entry
io.github.jmshinhwa/robots-txt-ai-crawler-auditv0.1.1. Ratings and installs below are from this marketplace.
Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI
@readystack/robots-txt-ai-crawler-audit @ 0.1.1Source: https://github.com/jmshinhwa/readystack-themes
Registry entry: io.github.jmshinhwa/robots-txt-ai-crawler-audit
Copy one of these into your project. Installing also returns the manifest and these snippets.
# after Install: the listing is in your workspace's routing pool - nothing else to configure
curl -s -X POST https://api.opensmartroute.ai/api/v1/route -H 'Authorization: Bearer $OSR_API_KEY' -H 'Content-Type: application/json' -d '{"text": "...", "plan": true}'
# or pin it on the OpenAI-compatible endpoint: {"model": "mcp-io-github-jmshinhwa-robots-txt-ai-crawler-audit", ...}An Open Capability Manifest: the router reads it to know what this does, what it costs and when to pick it.
{
"ocm": "1",
"id": "mcp-io-github-jmshinhwa-robots-txt-ai-crawler-audit",
"kind": "tool",
"name": "robots-txt-ai-crawler-audit",
"description": "Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI",
"publisher": "jmshinhwa",
"version": "0.1.1",
"capabilities": {
"domains": [
"general"
],
"tags": [
"mcp",
"mcp-server",
"npm",
"mcp-registry"
],
"languages": [
"en"
]
},
"quality_prior": 0.6,
"examples": [
"Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI"
],
"primary": false,
"metadata": {
"source": {
"provider": "mcp-registry",
"repository": "https://github.com/jmshinhwa/readystack-themes",
"path": "server.json",
"ref": "0.1.1",
"url": "https://github.com/jmshinhwa/readystack-themes",
"key": "io.github.jmshinhwa/robots-txt-ai-crawler-audit",
"registry": "https://registry.modelcontextprotocol.io"
}
}
}Fetch it by URL: GET /api/v1/registry/mcp-io-github-jmshinhwa-robots-txt-ai-crawler-audit/manifest?version=0.1.1
Star ratings from people who tried it. One review per account; edit yours any time.
No reviews yet. Install it, try it, and be the first to rate it.