douyin-video-puller

Imported from xuweizhengo/skills-hub (plugins/chinese-creator-skills/skills/douyin-video-puller/SKILL.md). Install upstream with npx skills add xuweizhengo/skills-hub --skill douyin-video-puller. Copyright stays with the author.

Douyin Video Puller

Overview

Use this skill to turn a public Douyin link into local artifacts: video files, metadata, keyframes, top-level comments, comment statistics, and a Markdown report.

The reliable path is not a plain API call. Let a real browser open the desktop Douyin page so Douyin generates msToken, a_bogus, verifyFp, and related parameters, then replay the signed comment URL outside the browser for pagination.

Quick Start

Run the bundled script from any workspace:

$skillDir = "<path-to-douyin-video-puller-skill>"
node "$skillDir\scripts\pull_douyin.mjs" "<Douyin share text or URL>" --out ".\douyin-output" --max-comments 1300

For comment pulls, first open https://www.douyin.com in the desktop browser and log in. If the desktop site is closed, logged out, or blocked, video and metadata can still succeed but comments may be partial.

The script accepts:

• Full copied share text containing a v.douyin.com URL
• https://www.douyin.com/video/<aweme_id>
• https://www.iesdouyin.com/share/video/<aweme_id>
• A bare aweme ID

Workflow

1. Resolve the aweme ID from the input.
2. Fetch the mobile iesdouyin.com/share/video/<id>/ page and parse window._ROUTER_DATA.
3. Save share_page_mobile.html and video_info.json.
4. Download the video from the share-page play_addr. Also try the no-watermark aweme/v1/play/ endpoint when a video_id is available.
5. Open https://www.douyin.com/video/<id> with a real browser, observe the signed comment/list request, and save signed_comment_request.json.
6. Replay the signed URL with browser cookies and paginate top-level comments.
7. De-duplicate comments by cid, save comments_structured.json, and write comments_stats.json.
8. Extract keyframes with FFmpeg when available.
9. Write pull_report.md and pull_status.json.

Important Limits

• This supports public videos only. Private, deleted, login-only, or region-limited videos can fail.
• Direct comment requests usually return blocked; use the browser-signed request path.
• Page comment count is not the same as retrievable top-level comments. It can include replies, folded comments, filtered comments, or items not exposed by web pagination.
• Top-level comments are the reliable target. Reply/comment-list-reply can require separate signatures and may return empty bodies or be session-sensitive.
• If the browser opens but the comment request fails in-page with ERR_CONNECTION_CLOSED, still inspect the signed request. Replaying the same signed URL can succeed.
• Comment signing failures are partial success by default. Keep the video, metadata, keyframes, and diagnostics, then retry after opening/logging in to desktop Douyin.

Environment

The script needs:

• Node.js 18+
• Microsoft Edge or Chrome installed
• Playwright package available to Node
• FFmpeg for keyframes and media probing

If Playwright is not installed in the current project, set one of these before running:

$env:PLAYWRIGHT_PACKAGE_PATH="<project>\node_modules\playwright"

or:

$env:PLAYWRIGHT_NODE_MODULES="<project>\node_modules"

The script also probes common local package locations and system Edge/Chrome paths.

Output

Default output root is a local douyin-output style directory; each run writes into <out>/<aweme_id>/.

Read references/output-schema.md before changing downstream analysis code that consumes the outputs.

Main files:

• video_info.json: normalized metadata from the share page.
• video_try_nowm.mp4: no-watermark endpoint attempt when available.
• video_watermark.mp4: share-page playback URL download.
• comments_structured.json: de-duplicated top-level comments with raw payloads preserved.
• comments_stats.json: high-level counts, keyword hits, IP label counts, and high-like comments.
• pull_report.md: concise human-readable summary.
• pull_status.json: success or partial-success status for the run.
• comment_error.json: written only when comment signing or pagination fails.
• keyframe_*.jpg: sampled frames.

Troubleshooting

• If iteminfo or old APIs return encrypt_data_miss, ignore them and parse the share page instead.
• If the mobile share page shows zero comments but metadata has a nonzero comment count, use the desktop douyin.com/video/<id> browser-signed request path.
• If no signed comment request is observed, save the page screenshot/logs and report that comments could not be pulled in this environment.
• If comment signing fails, rerun the same command after opening/logging in to desktop Douyin. Do not use --skip-comments for the retry.
• Use --skip-comments only to refresh video/metadata/keyframes. It preserves an existing comments_structured.json when present.
• Use --fail-on-comment-error only when a workflow should fail hard if comments cannot be pulled.
• If pagination returns repeated hot comments, keep cid de-duplication enabled.
• If comments stop before the page count, report both the page count and fetched top-level count.