About

Imported from bighardperson/computer-science-skills-collection (skills/web-scraper-jina/SKILL.md). Install upstream with npx skills add bighardperson/computer-science-skills-collection --skill web-scraper-jina. Copyright stays with the author.

Web Scraper using r.jina.ai

Name: Web Scraper (r.jina.ai)
Author: bighardperson

Bypass Cloudflare and scrape any website using free r.jina.ai API.

Features

Bypass Cloudflare, Turnstile, and other protections
Works on Truth Social, Bitget, and other protected sites
Returns clean Markdown content
Free to use

Usage

Simply prepend https://r.jina.ai/ to any URL:

https://r.jina.ai/https://truthsocial.com/@realDonaldTrump
https://r.jina.ai/https://bitget.com/events/poolx

Examples

Get Trump Truth Social posts:

curl -s "https://r.jina.ai/https://truthsocial.com/@realDonaldTrump"

Get any protected page:

curl -s "https://r.jina.ai/https://example.com"

In Code

import requests

def scrape(url):
    return requests.get(f"https://r.jina.ai/{url}").text

Use Cases

Scrape Truth Social, Gab, Gettr
Bypass Cloudflare protected sites
Extract content from news articles
Monitor competitors

Use it

Copy one of these into your project. Installing also returns the manifest and these snippets.

yaml

targets:
  - https://api.opensmartroute.ai/api/v1/registry/bighardperson-computer-science-skills-collection-web-scr-13375a/manifest   # or paste the manifest below

Manifest

An Open Capability Manifest: the router reads it to know what this does, what it costs and when to pick it.

bighardperson-computer-science-skills-collection-web-scr-13375a.ocm.jsonjson

{
  "ocm": "1",
  "id": "bighardperson-computer-science-skills-collection-web-scr-13375a",
  "kind": "skill",
  "name": "Web Scraper (r.jina.ai)",
  "description": "Bypass Cloudflare and scrape any website using r.jina.ai API. Works on sites with strong protection like Truth Social, Cloudflare Turnstile, etc.",
  "publisher": "bighardperson",
  "version": "1.0.0",
  "capabilities": {
    "domains": [
      "coding"
    ],
    "tags": [
      "skill-md",
      "web",
      "scraper",
      "cloudflare",
      "bypass",
      "jina",
      "skills-sh"
    ],
    "languages": [
      "en"
    ]
  },
  "quality_prior": 0.6,
  "examples": [
    "Bypass Cloudflare and scrape any website using r.jina.ai API. Works on sites with strong protection like Truth Social, Cloudflare Turnstile, etc."
  ],
  "primary": false,
  "metadata": {
    "source": {
      "provider": "skills.sh",
      "repository": "https://github.com/bighardperson/computer-science-skills-collection",
      "path": "skills/web-scraper-jina/SKILL.md",
      "ref": "HEAD",
      "url": "https://github.com/bighardperson/computer-science-skills-collection/blob/HEAD/skills/web-scraper-jina/SKILL.md",
      "key": "bighardperson/computer-science-skills-collection/skills/web-scraper-jina/SKILL.md"
    }
  },
  "instructions": "# Web Scraper using r.jina.ai\n\nBypass Cloudflare and scrape any website using free r.jina.ai API.\n\n## Features\n\n- Bypass Cloudflare, Turnstile, and other protections\n- Works on Truth Social, Bitget, and other protected sites\n- Returns clean Markdown content\n- Free to use\n\n## Usage\n\nSimply prepend `https://r.jina.ai/` to any URL:\n\n```\nhttps://r.jina.ai/https://truthsocial.com/@realDonaldTrump\nhttps://r.jina.ai/https://bitget.com/events/poolx\n```\n\n## Examples\n\n### Get Trump Truth Social posts:\n```\ncurl -s \"https://r.jina.ai/https://truthsocial.com/@realDonaldTrump\"\n```\n\n### Get any protected pag",
  "cost": {
    "context_tokens": 230
  }
}

Fetch it by URL: GET /api/v1/registry/bighardperson-computer-science-skills-collection-web-scr-13375a/manifest?version=1.0.0

Reviews

Star ratings from people who tried it. One review per account; edit yours any time.

No reviews yet. Install it, try it, and be the first to rate it.