LinkedIn Post Scrapers: Collect Public Posts by URL or Keyword

November 8, 2026

You want LinkedIn posts: a competitor's company updates, top posts on a topic, or the full text and stats of a post someone shared with you. The official API won't give you these unless you wrote them. This post covers the limits, the public routes, and a method for learning from top posts.

The official Posts API is for your own content

LinkedIn's Posts API needs r_organization_social to read an organization's posts, and that permission is "Restricted to organizations in which the authenticated member has" an admin or content role. Reading a member's posts needs r_member_social, which is "restricted and is available to approved users only" Posts API docs. In short: you can read posts from Pages you run, not from anyone else.

Three public routes

Keyword search over public posts depends on what is publicly indexed, so it finds popular public posts well and misses posts shared only with connections. That is the right boundary.

Method: reverse-engineer top posts on a topic

1. Search your topic, for example "product-led growth", for the last month.
2. Collect the top 50 posts with reactions and comments.
3. For each, record: first line (the hook), length, format (text, image, document, video), whether it ends with a question, and comments per reaction.
4. Compare the top 10 with the bottom 10. The differences are your writing rules.

Things to measure in the hook:

• Characters before the first line break. Short first lines are easier to read before "see more".
• Does it contain a number?
• Is it a claim, a question, or a story opener?

import os, requests

KEY = os.environ["SCRAPEIO_API_KEY"]
H = {"x-api-key": KEY}

r = requests.get("https://scrapeio.com/v1/linkedin/search/posts",
                 params={"query": "product-led growth", "date_posted": "last-month"},
                 headers=H, timeout=60)
data = r.json()["data"]
print(list(data)[:10])     # check the shape once

posts = data.get("posts") or data.get("items") or []
for p in posts[:50]:
    text = p.get("text") or p.get("commentary") or ""
    hook = text.split("\n", 1)[0]
    print(len(hook), any(c.isdigit() for c in hook), hook[:80])

LinkedIn results are passed through in the source's own shape. Print the keys once and adjust the field names in the loop.

Metrics that compare fairly

• Reactions grow with the author's audience. Compare posts from the same author, or normalize by follower count.
• Comments per reaction shows how much a post starts a conversation, and it is less tied to audience size.
• Reposts are rare and strong. A post with many reposts carried an idea people wanted to be seen sharing.
• Age. Most engagement on a LinkedIn post comes in its first days. Don't compare a post from this morning with one from last month.

Video posts

For video posts, the transcript is the content. Pull it and read the first 15 seconds of speech: that is the spoken hook. yt-dlp also lists a LinkedIn extractor if you only need a few video files yt-dlp supported sites.

Tracking a company's posts

For competitor posts, page through the company's feed once a week and store each post's URL, date, text and counts. Re-read posts from the last 14 days on each run, since their counts are still growing. After that, the numbers are mostly settled, and you can stop updating them. A simple sheet with one row per post and one column per week of counts is enough to see which topics keep earning after the first day.

Keep it about content

Post research is about ideas and formats. You don't need to store the authors' profiles to learn from their posts. LinkedIn's User Agreement also restricts scraping and copying content section 8.2, so quote sparingly, credit authors when you cite them, and never republish posts as your own.

How to do it with ScrapeIO

The LinkedIn post search API takes a keyword and an optional date_posted filter (last hour to last year) and pages with a cursor. The LinkedIn post API returns one public post by URL, the LinkedIn company posts API lists a company's posts, and the LinkedIn post transcript API returns what a video post says. Each call is 1 credit.

Sources

Checked on 8 November 2026 from each vendor's own pages. Prices and limits change; follow the links before you decide.

• https://learn.microsoft.com/en-us/linkedin/marketing/community-management/shares/posts-api
• https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md
• https://www.linkedin.com/legal/user-agreement