---
name: socialcrawl
description: Fetch social media and web data through one unified API — profiles, posts, search, transcripts and prediction markets across 21 platforms. Use when a task needs data from Reddit, YouTube, GitHub, Bluesky, Hacker News, the App Store, Google Play or Polymarket, or when a URL from any of those needs resolving into structured data.
---

# SocialCrawl

One key, one envelope, 100 endpoints across 21 platforms.

## Auth

Every request is a GET with your key in the `x-api-key` header:

```bash
curl "http://datatrend.nl/v1/reddit/subreddit?name=programming" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY"
```

Get a key with 100 free credits (no card) at http://datatrend.nl/dash/signup

## Self-onboarding (for agents)

No human needed — bootstrap your own key:

```bash
curl -X POST http://datatrend.nl/v1/signup \
  -H "Content-Type: application/json" \
  -d '{"email":"you@example.com"}'
# -> { "api_key": "sc_...", "credits": 100 }  (key shown once)
```

## The envelope — identical on every platform

```json
{
  "success": true,
  "platform": "reddit",
  "endpoint": "/v1/reddit/subreddit",
  "data": { "items": [ { "post": {...}, "computed": {...} } ] },
  "credits_used": 1,
  "credits_remaining": 99,
  "request_id": "req-...",
  "cached": false
}
```

Posts always carry `author`, `content`, `engagement` and `computed`
(`engagement_rate`, `estimated_reach`, `language`, `content_category`) —
so one parser handles every platform.

## Catalog

| Platform | Endpoints |
|---|---|
| `hackernews` | top(), search(query), item(id), user(handle), story(id), story/comments(id), profile(handle) |
| `github` | user(handle), repo(repo), repos(handle), profile(handle), profile/repos(handle, sort, per_page, page), repo/readme(url), repo/releases(url, per_page, page), repo/issues(url, state, labels, sort, per_page, page), issue(url), issue/comments(url, per_page, page), search(query, sort, order, per_page, page) |
| `appstore` | app(id), search(query), app-search(query, country, depth), app-info(app_id, country), app-reviews(app_id, country, sort_by), app-list(app_collection, country, depth) |
| `reddit` | subreddit(name, sort), search(query), post(url), user(handle), subreddit/details(subreddit, url), post/comments(url, cursor), subreddit/search(subreddit, query, sort, timeframe, cursor) |
| `bluesky` | profile(handle), feed(handle), search(query), user/posts(handle, user_id), post(url) |
| `polymarket` | markets(), market(slug), search(query) |
| `googleplay` | app(id, lang), app-search(query, country, language), app-info(app_id, language) |
| `youtube` | video(url), transcript(url, lang), search(query, region), channel(handle, channelId, url), channel/videos(handle, channelId), video/transcript(url, language) |
| `tiktok` | user(), videos(), video(video_id) |
| `instagram` | account(), media(), discover(username) |
| `twitter` | profile(handle), user/tweets(handle), tweet(url), search/recent(query) |
| `linkedin` | company(url), company/posts(url), post/comments(url) |
| `facebook` | profile(url), profile/posts(url), post(url), post/comments(url) |
| `pinterest` | account(), boards(), pin(url) |
| `amazon` | shop(url), product-search(query, country, depth), product(asin, country), reviews(asin, country), sellers(asin, country) |
| `google` | search(query, region, date_posted, page), ad(url), adlibrary/advertisers/search(query, region), company/ads(domain, advertiser_id, topic, region, start_date, end_date, platform, format, get_ad_details, cursor), business/info(keyword, cid, place_id, location_name, language_name), business/extended-reviews(keyword, cid, place_id, location_name, language_name, depth), business/updates(keyword, cid, location_name, language_name), business/questions(keyword, cid, place_id, location_name, language_name, depth), hotels/search(keyword, location_name, language_name, check_in, check_out), hotels/info(hotel_identifier, location_name, language_name) |
| `google_shopping` | product-search(query, country, language, depth, price_min, price_max, sort_by), product(product_id, gid, data_docid, country, language), reviews(gid, product_id, data_docid, depth, country, language), sellers(product_id, gid, data_docid, country, language) |
| `perplexity` | research(query) |
| `tiktokshop` | product(url, region), product/reviews(url, product_id, region), products(url, sort_by, region), search(query, page, region), user/showcase(handle, region, cursor) |
| `prism` | share-of-voice(brand, competitors), brand-mentions(brand), review-integrity(appstore_id, googleplay_id), viral-analysis(topic), sentiment-report(brand), crisis-radar(brand) |
| `marketprice` | lookup(ean, country) |

## Shortcuts worth knowing

- **Don't know the platform?** `GET /v1/lookup?url=<any supported URL>` resolves it for you.
- **Discovery:** `GET /v1/discover` returns the live catalog; `GET /v1/openapi.json` is the full spec.
- **Slow endpoints** (`youtube/transcript`, `googleplay/app`) return `202` with a `job_id` — poll `GET /v1/jobs/{id}` until status is `done`.
- **Safe retries:** send `Idempotency-Key: <uuid>`; a replay within 24h returns the first response and costs 0 credits.
- **Costs:** most endpoints cost 1 credit; transcripts cost 5. Cache hits, failed calls and `GET /v1/credits/balance` cost 0.

## Errors

Errors use the same envelope with `success: false` and an `error` object
carrying `type`, `message`, `status` and `doc_url`. Common cases:
`401` bad key · `402` out of credits · `429` rate limited (token bucket,
retry shortly) · `502` upstream failed (credits refunded automatically).