How to Scrape YouTube Data: API, yt-dlp, Comments, Transcripts

November 9, 2026

YouTube is the one big platform where the official API covers most public data for free. So the order is simple: use the YouTube Data API for videos, channels and comments, use open-source tools for transcripts and edge cases, and use a hosted API when quota or setup gets in the way. Every number below comes from Google's own docs, checked today.

What you can collect, and where it comes from

Sources: quota costs, commentThreads, captions.download and the videos resource. All non-search calls share 10,000 units a day per project, and quotas reset at midnight Pacific Time.

Step 1: get an API key and pull a channel

1. Create a project in the Google Cloud console and enable "YouTube Data API v3".
2. Create an API key under Credentials and restrict it to that API.
3. Run the script below. It finds a channel by handle, walks its uploads playlist and fetches stats in batches of 50.

import requests

KEY = "YOUR_GOOGLE_API_KEY"
API = "https://www.googleapis.com/youtube/v3/"

def yt(method, **params):
    params["key"] = KEY
    r = requests.get(API + method, params=params, timeout=30)
    r.raise_for_status()
    return r.json()

ch = yt("channels", part="contentDetails,statistics", forHandle="@veritasium")["items"][0]
uploads = ch["contentDetails"]["relatedPlaylists"]["uploads"]

ids, token = [], None
while True:
    page = yt("playlistItems", part="contentDetails", playlistId=uploads,
              maxResults=50, pageToken=token)
    ids += [i["contentDetails"]["videoId"] for i in page["items"]]
    token = page.get("nextPageToken")
    if not token:
        break

videos = []
for i in range(0, len(ids), 50):
    videos += yt("videos", part="snippet,statistics,contentDetails",
                 id=",".join(ids[i:i + 50]))["items"]

Quota math. A channel with 500 uploads costs 1 (channel) + 10 (playlist pages) + 10 (video batches) = 21 units. At 10,000 units a day, you can pull about 470 such channels daily. The trap is search: Google moved search.list into its own bucket on 1 June 2026, and it allows about 100 searches a day, so never use search to list a channel's videos. The forHandle parameter has existed since 31 January 2024 (revision history). For more on listing uploads, see how to get all videos from a YouTube channel.

Two newer changes matter for stats work. On 3 June 2026 Google added videos.batchGetStats, which returns views, likes, comments and duration for a list of videos at 1 unit a call and needs no login for public videos (docs). And on 27 August 2026 Google said YouTube will count public views "the moment a video begins to play". If you compare view counts from before and after late August 2026, mark the break in your charts.

Step 2: scrape YouTube comments

def comments(video_id, max_pages=5):
    token = None
    for _ in range(max_pages):
        page = yt("commentThreads", part="snippet", videoId=video_id,
                  maxResults=100, order="time", pageToken=token)
        for t in page["items"]:
            c = t["snippet"]["topLevelComment"]["snippet"]
            yield {"author": c["authorDisplayName"], "text": c["textOriginal"],
                   "likes": c["likeCount"], "published": c["publishedAt"],
                   "replies": t["snippet"]["totalReplyCount"]}
        token = page.get("nextPageToken")
        if not token:
            break

Each page is 1 unit for up to 100 top-level comments, so 10,000 comments cost about 100 units. Use order="relevance" for the comments YouTube ranks highest. Replies need a separate comments.list call per thread. For analysis methods, see YouTube comment analysis.

Step 3: scrape YouTube transcripts

The Data API only downloads captions for videos the logged-in user can edit, at 200 units each. For anyone else's videos, the common free tool is youtube-transcript-api (MIT, about 8,600 stars):

from youtube_transcript_api import YouTubeTranscriptApi

fetched = YouTubeTranscriptApi().fetch("VIDEO_ID", languages=["en"])
text = " ".join(s.text for s in fetched)

Pass the video id, not the URL. Its README warns that YouTube has started blocking most IP addresses known to belong to cloud providers, so it tends to work from a laptop and fail from a server. Our YouTube transcripts guide goes deeper.

yt-dlp for metadata, comments and subtitles

yt-dlp (release 2026.10.26) reads video pages directly. Useful commands:

# One video's metadata as JSON, no download
yt-dlp -J "https://www.youtube.com/watch?v=VIDEO_ID" > video.json

# A channel's uploads as a fast, flat list
yt-dlp --flat-playlist -J "https://www.youtube.com/@veritasium/videos" > uploads.json

# Comments and English auto-captions, no video file
yt-dlp --skip-download --write-info-json --write-comments \
  --write-auto-subs --sub-langs en "https://www.youtube.com/watch?v=VIDEO_ID"

Comments go into the info JSON. yt-dlp's README says full YouTube support now needs its yt-dlp-ejs component plus a JavaScript runtime, and only deno is enabled by default. Install deno first or some formats and fields go missing.

The YouTube rules that shape your design

• 30-day storage. YouTube's developer policies say most API data may be stored "for no longer than 30 calendar days", after which you must delete or refresh it. Plan a refresh job from day one.
• No mixing. The same policies forbid API clients to scrape YouTube or use "any technology other than YouTube API Services to access or retrieve API Data". If your product uses the API, keep scraped data out of it.
• No invented metrics. You may not replace API data with your own calculated substitutes, and any metric you add must be labeled as yours.

This is not legal advice. Terms and law change, and the answer depends on where you are and what you do with the data. Our guide to the main web scraping cases has the background.

Which route fits

Doing it with ScrapeIO

ScrapeIO's YouTube channel videos API, comments API, transcript API and search API run without a Google project or daily quota. Most cost 1 credit; search costs 2. The full list is on the YouTube platform page, and the YouTube MCP server exposes them to AI agents.

FAQ

Is it legal to scrape YouTube?

YouTube's terms bar accessing the service "using any automated means (such as robots, botnets or scrapers)" without its permission, except search engines that follow robots.txt, and its developer policies bar API clients from scraping. Reading public data can still be lawful depending on where you are and what you do. Get advice for commercial use.

Is the YouTube Data API free?

Yes. Google lists no fee; the limit is 10,000 units a day per project, plus a separate bucket for search. See Is the YouTube API free?

How many YouTube API calls can I make per day?

Most reads cost 1 unit, so roughly 10,000 calls, each returning up to 50 videos or 100 comments. Search is limited to about 100 calls a day.

Can I scrape YouTube without the API?

Yes, with yt-dlp or youtube-transcript-api, which read public pages. They need upkeep and may be blocked from cloud servers.

How do I get a YouTube video's transcript?

For your own videos, use captions.download. For other videos, use youtube-transcript-api, yt-dlp with --write-auto-subs, or a transcript API.

Sources

Checked on 9 November 2026 from each vendor's own pages. Prices and limits change; follow the links before you decide.

• https://developers.google.com/youtube/v3/determine_quota_cost
• https://developers.google.com/youtube/v3/docs/channels/list
• https://developers.google.com/youtube/v3/docs/commentThreads/list
• https://developers.google.com/youtube/v3/docs/captions/download
• https://developers.google.com/youtube/v3/docs/videos
• https://developers.google.com/youtube/v3/docs/videos/batchGetStats
• https://developers.google.com/youtube/v3/revision_history
• https://developers.google.com/youtube/terms/developer-policies
• https://www.youtube.com/t/terms
• https://github.com/jdepoix/youtube-transcript-api
• https://github.com/yt-dlp/yt-dlp