How to scrape an X (Twitter) thread

November 8, 2026

A thread is a chain of posts where the author replies to themselves. Collecting it means finding the first post, then every follow-up by the same author, in order. Replies from other people live in the same conversation, so the job is mostly filtering. Here are two methods.

How X models a conversation

X gives every post a conversation_id. All replies in a thread share the original post's ID, "no matter how deep the thread goes". Each reply also has referenced_tweets with type replied_to and the parent's ID, and in_reply_to_user_id. X

Method 1: conversation_id search (official API)

Fetch the post you have, with tweet.fields=conversation_id,author_id,created_at,referenced_tweets. The conversation_id is the root post.
Search conversation_id:ROOT_ID from:AUTHOR on recent search, and page with next_token.
Sort the results by created_at ascending, and put the root first.

GET https://api.x.com/2/tweets/search/recent
  ?query=conversation_id:1234567890 from:author
  &tweet.fields=created_at,referenced_tweets,public_metrics
  &max_results=100

Recent search covers the last 7 days. For older threads, use full-archive search, which X makes available to pay-per-use customers. source Each post read costs $0.005, so a 20-post thread is about $0.10. source

Drop from:AUTHOR if you want the whole discussion, including other people's replies.

Method 2: walk the replies

If your tool has a "post" call and a "replies to a post" call but no conversation search, walk the chain:

Fetch the root post and note the author's handle.
Fetch the replies to the root. Keep the ones written by the same author. That is the next post in the thread.
Repeat on that post until no reply from the author is found.
If you started from a post in the middle, first follow the "replied to" parent upward until you reach the root.

// Pseudo-code that works with any post + replies API.
async function thread(rootUrl) {
  const root = await getPost(rootUrl);
  const out = [root];
  let cur = root;
  while (true) {
    const replies = await getAllReplies(cur.url);   // follow the cursor
    const next = replies
      .filter((r) => r.author.handle === root.author.handle)
      .sort((a, b) => a.created_at.localeCompare(b.created_at))[0];
    if (!next) break;
    out.push(next);
    cur = next;
  }
  return out;
}

Edge cases

Branches. An author sometimes replies twice to the same post. Taking the earliest reply follows the main line; keep the others as branches.
Deleted posts break the chain. If a step finds nothing, try a conversation search before you stop.
Quote posts are not part of the thread, even if the author quotes themselves.
Long posts and media. Store the full text and media URLs per post, not just the first line.
Reply volume. On viral threads, the replies to each post can run to thousands. Stop paging once you find the author's reply if you only want the thread.

Finding threads worth collecting

If you do not have a thread URL yet, start from the author. List their recent posts, and look for posts that are replies to their own earlier post, or posts whose text starts with a numbering pattern such as "1/". On the official API, you can also search from:author -is:retweet and look for posts where conversation_id equals the post's own ID: those are thread roots or standalone posts. source For research, pick threads with high views per follower; they show what the author's audience values.

What it costs

Method 1 costs one search page per 100 posts, billed per post read on the official API. Method 2 costs one post call, plus at least one replies call per post in the thread, more when you page through busy replies. A 15-post thread with quiet replies is about 16 calls. A 15-post thread on a viral account can be several times that, which is why stopping early matters.

Turning it into text

Join the posts with blank lines, number them, and keep each post's URL and metrics. That gives you a clean document for reading, summarizing or quoting with attribution.

Doing it with ScrapeIO

Method 2 maps to two endpoints: the X post API for the root and the X replies API for each step, both 1 credit a call. Every post carries author.handle, created_at, likes and views in plays. To find threads in the first place, use the X user posts API. Our study of 414 top X posts used the same endpoints.

Sources

Checked on 8 November 2026 from each vendor's own pages. Prices and limits change; follow the links before you decide.

100 free credits, no card. Credits never expire.

Get API key