Bluesky Scrapers and the Free AT Protocol API (2026)

November 8, 2026

Bluesky is the friendliest major network for data work. It runs on the open AT Protocol, and much of its data is readable through a public API with no key at all. For many projects you do not need a scraper. This guide shows the free route first, its real limits, and when a paid API still saves you time.

The free public API

Bluesky's API hosts guide says many app endpoints are public and need no authentication. You call them on https://public.api.bsky.app, which adds caching and is the host Bluesky asks developers to use for public web use cases. Two calls cover most needs:

# Profile: followersCount, followsCount, postsCount, description
curl "https://public.api.bsky.app/xrpc/app.bsky.actor.getProfile?actor=bsky.app"

# Latest posts with likeCount, repostCount, replyCount, quoteCount
curl "https://public.api.bsky.app/xrpc/app.bsky.feed.getAuthorFeed?actor=bsky.app&limit=50"

The feed response includes a cursor. Pass it back as cursor= to get the next page. Posts are identified by an at:// URI, and accounts by a DID that stays the same when the handle changes, so store the DID as your key.

Rate limits

Bluesky's rate limit guide separates services:

Requests over a limit get HTTP 429. Read the rate limit headers and back off. Reuse one session instead of logging in for every job, or you will hit the session cap first.

Search needs a login

In our test on 8 November 2026, calling app.bsky.feed.searchPosts on the public host without a token returned HTTP 403. Profile and feed calls worked. To search posts, log in with an app password and send the request through your account's PDS, which the hosts guide says proxies app requests to the AppView. The atproto Python SDK handles this for you:

from atproto import Client

client = Client()
client.login("you.bsky.social", "APP_PASSWORD")
res = client.app.bsky.feed.search_posts({"q": "open source", "limit": 25})
for p in res.posts:
    print(p.author.handle, p.like_count, p.record.text[:80])

The firehose and Jetstream

If you need everything as it happens, stream it. The firehose guide explains that com.atproto.sync.subscribeRepos streams every repository change over a WebSocket, encoded as CBOR. For most projects the simpler option is Jetstream, which sends JSON and lets you filter by collection:

wss://jetstream2.us-east.bsky.network/subscribe?wantedCollections=app.bsky.feed.post

A stream gives you creates and deletes. Honor the deletes: if a user removes a post, remove it from your store too.

Which method for which job

Common mistakes

• Using the handle as your key. Handles change. Resolve to the DID once and key on it.
• Logging in for every job. Session creation is capped at 30 per 5 minutes per account. Keep one session and refresh it.
• Polling for volume. If you call search every minute to catch everything, switch to Jetstream and filter locally. It is cheaper for both sides.
• Ignoring the cursor. Feeds page with a cursor, not a page number. Stop when the cursor is missing.
• Forgetting other servers. Bluesky is one operator on an open network, and its docs note that other providers are likely to set different rate limits.

When a paid API is worth it

Be honest about this one: for Bluesky alone, the free API is excellent. A paid API makes sense when Bluesky is one of several networks in the same pipeline, when you want the same field names everywhere instead of followersCount here and follower_count there, or when an AI agent should call one tool for every platform.

Doing it with ScrapeIO

ScrapeIO has a Bluesky profile API, a Bluesky user posts API and a Bluesky post API that returns a post with its direct replies. They return handle, followers, likes and created_at like every other platform on ScrapeIO, so your Bluesky code and your TikTok code read the same.

Sources

Checked on 8 November 2026 from each vendor's own pages. Prices and limits change; follow the links before you decide.

• https://github.com/bluesky-social/bsky-docs/blob/main/docs/advanced-guides/api-directory.mdx
• https://github.com/bluesky-social/bsky-docs/blob/main/docs/advanced-guides/rate-limits.md
• https://github.com/bluesky-social/bsky-docs/blob/main/docs/advanced-guides/firehose.mdx
• https://github.com/MarshalX/atproto
• https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md
• https://github.com/mikf/gallery-dl/blob/master/docs/supportedsites.md