How to scrape X (Twitter) Search with Go

Search X posts. X search operators work. This tutorial gets it with a short Go program and the X (Twitter) Search API. Each successful call costs 1 credit; errors and not-found answers are free.

1. Get an API key

Sign in to get a key and 100 free credits, no card. Then put the key in an environment variable so it stays out of your code:

export SCRAPEIO_API_KEY="your_key_here"

2. Set up Go

go mod init scrape

3. Write the code

Save this as main.go.

package main

import (
	"encoding/json"
	"fmt"
	"net/http"
	"net/url"
	"os"
)

func main() {
	params := url.Values{}
	params.Set("query", "coffee")
	var items []any
	for {
		req, _ := http.NewRequest("GET", "https://scrapeio.com/v1/twitter/search"+"?"+params.Encode(), nil)
		req.Header.Set("x-api-key", os.Getenv("SCRAPEIO_API_KEY"))
		res, err := http.DefaultClient.Do(req)
		if err != nil {
			panic(err)
		}
		if res.StatusCode != 200 {
			panic(fmt.Sprintf("HTTP %d", res.StatusCode))
		}
		var body struct {
			Data struct {
				Items   []any  `json:"items"`
				Cursor  string `json:"cursor"`
				HasMore bool   `json:"has_more"`
			} `json:"data"`
		}
		json.NewDecoder(res.Body).Decode(&body)
		res.Body.Close()
		items = append(items, body.Data.Items...)
		if !body.Data.HasMore || body.Data.Cursor == "" || len(items) >= 100 {
			break
		}
		params.Set("cursor", body.Data.Cursor)
	}
	fmt.Println(len(items), "items")
	out, _ := json.MarshalIndent(items[:min(3, len(items))], "", "  ")
	fmt.Println(string(out))
}

The loop follows data.cursor until data.has_more is false, and stops at 100 items. Each page is one call, so it costs 1 credit.

4. Run it

go run main.go

The response has the same envelope on every endpoint: success, cached, credits_used, credits_remaining and data. Shared fields such as handle, followers, plays, likes and created_at have the same names on every platform.

Parameters

Name	Type	Description
`query` (required)	string	Search text.
`type`	top, latest	Ranking. Default: top.
`cursor`	string	Cursor from the previous page's data.cursor.

Errors and what you pay

Status	Meaning	Charged
`200`	Success	Yes, unless cached
`400`	A parameter is missing or wrong	No
`401`	The key is missing or invalid	No
`402`	Not enough credits	No
`403`	The account or post is private	No
`404`	Not found on the platform	No
`429`	More than 300 requests a minute	No
`502`	Every source is down; retry shortly	No

Same endpoint in other languages

Python · Node.js · JavaScript · TypeScript · Rust · Java · C# · PHP · Ruby · Kotlin · Swift · R

Prefer no code?

Connect the MCP server to Claude Code, Cursor or Codex and ask in plain English. The agent uses the twitter_search tool for you.

100 free credits, no card. Credits never expire.

Get API key