Self-hosting

Run the service and its documentation from the same Vercel project.

Run locally

git clone https://github.com/pc-style/x-md.git
cd x-md
bun install
bun run dev

The Vite server runs the landing page and local API handlers. Run bun run docs:dev separately for the Blume documentation server.

Build and deploy

bun run build
vercel --prod

The build generates Blume docs, then the Vite landing page into dist. The included Vercel configuration serves /docs, the static assets, and the API rewrites together.

Live search provider

Latest and Top try FxTwitter, then the custom-built live search provider, then an optional Firecrawl web fallback. Photos, Videos, and Users use the live provider directly and return 503 when it is not configured.

The provider is configured through X_SEARCH_SESSIONS_JSON; the expected shape is defined in lib/xsearch.ts. For local development, use the gitignored accounts.local.json. Treat its contents as secrets and keep them out of version control.

The provider budgets its own upstream capacity per 15-minute window, shared across post and user searches. Failed attempts and page walks count. Unhealthy entries are taken out of rotation in the current process and retried later; health is per instance.

Your own upstream, or a pool of them

Profiles, posts, and bulk imports read https://api.fxtwitter.com by default. That public instance allows about 1000 requests a minute per source IP, which caps a bulk import’s sustained fan-out around 16 chains. Point x.md at your own FxEmbed deployment instead — FxEmbed is itself a Cloudflare Worker, and Workers is its documented self-host path:

FXTWITTER_BASE_URL=https://fx.example.workers.dev

Or give it a pool. Requests rotate across every base; one that answers 429 sits out for its Retry-After while the walk continues on the others, and the import route allows 32 chains per base:

FXTWITTER_BASE_URL=https://fx-a.example.workers.dev,https://fx-b.example.workers.dev,https://api.fxtwitter.com

How high concurrency can go on your own instance depends on that deployment’s capacity and X’s own per-token limits, which FxEmbed manages; x.md still backs off on any 429 it returns. bench/RESULTS-scale.md in the repository is the reference run against the public instance; bun scripts/benchmark-import.ts --scale reproduces it against whatever FXTWITTER_BASE_URL names, so you can measure your own pool.

The caps in lib/import.ts (IMPORT_MAX_POSTS, IMPORT_MAX_CONCURRENCY_PER_BASE) and the allowances in lib/quotas.ts are the only limits; there are no hosted-only ones.

The post archive

Bulk imports store what they walk, per account, so repeat imports only fetch the gap. With the shared Redis below configured the archive is durable across instances and deploys; without it, it lives in process memory for the life of the function. Keys are hist:{handle}:index, hist:{handle}:ids (a sorted set by post time), and hist:{handle}:posts. Budget roughly 2 KB per post; a 3000-post account is about 6 MB.

Deploying the fast stack

The shape that removes every artificial ceiling:

1. FxEmbed on Cloudflare Workers — your own upstream, no per-IP cap. FxEmbed routes by Host, so put your API hostname in its API_HOST_LIST. It works best with X account credentials (credentials.json: authToken + csrfToken per account, the same pair as x.md’s search sessions) and spreads requests across them; without credentials it runs on guest access with lower limits. Its capacity is therefore roughly the number of X accounts behind it, and one Cloudflare account on the free tier covers 100k requests a day.
2. x.md on Vercel with Fluid Compute — the import function is declared with a 120 s maxDuration in vercel.json; Fluid allows it and keeps streaming responses open.
3. Upstash Redis via the Vercel marketplace — the archive and the rate-limit counters.

Set FXTWITTER_BASE_URL to your pool (public instance last), deploy, and verify with one import followed by ?index=true.

Shared counters

Attach Upstash Redis through the Vercel marketplace or CLI. The service automatically uses either environment-variable pair:

• KV_REST_API_URL and KV_REST_API_TOKEN
• UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN

These share per-IP and provider counters across instances. Without them, counters fall back to process memory. Live-provider checks fail closed when the counter store is unavailable, protecting the shared pool; the general request limiter still fails open. Redeploy after connecting environment variables.

Historical statistics

PostHog is optional. Connect its Vercel Marketplace integration to the production environment, choose your data region, and review its billing terms. It supplies NEXT_PUBLIC_POSTHOG_PROJECT_TOKEN and NEXT_PUBLIC_POSTHOG_HOST; the API reads these on the server only. Alternatively, set POSTHOG_PROJECT_TOKEN and POSTHOG_HOST (for example, https://eu.i.posthog.com). Redeploy after configuring them.

Production API functions send a request_completed event with the route category, method, HTTP status, duration in milliseconds, application-cache result, key-validation status, public/key access, and degraded-search flag. Use these for request volume by endpoint, error and 429 rates, latency percentiles, and application-cache hit rate. Failed requests also send request_failed with the error class and the raised error code (for example firecrawl_error). A request’s events, including request_completed and request_failed, go to PostHog in one batch call when the response finishes, in the background with a three-second timeout; failures do not change API responses. Events recorded after that flush go in their own call. endpoint_called is the exception: it is sent on arrival, so a function that crashes or times out is still counted. The high-volume events are sampled and carry sample_rate, so weight their counts by its inverse: endpoint_called is kept for one request in ten, and each cache_hit and cache_miss is kept one time in ten on its own. Volume and cache hit rate stay exact on request_completed.

request_completed events contain no search text, URLs, client IPs, bearer tokens, or key labels. Verified keys use a pseudonymous ID derived from their internal ID and the PostHog project token; rotating that token changes the pseudonyms. Public requests all share the anonymous ID. Each request_completed also carries caller_id: the key pseudonym for verified keys, otherwise a hash of the client IP under a salt derived from the server’s Redis token (KV_REST_API_TOKEN or UPSTASH_REDIS_REST_TOKEN) that changes every UTC day, so you can count unique callers per day but not link them across days. Without that token there is no anonymous caller_id. Person profiles, geolocation enrichment, and session replay are not enabled by this instrumentation.

These statistics cover completed function responses, not requests served by Vercel’s CDN, aborted requests, or platform errors before the handler runs. They are not billing-grade totals. Local development, previews, admin routes, and CORS preflights are excluded. Redis remains the source of truth for live limits and capacity; PostHog is only for historical analysis.

Homepage and docs web analytics

Set VITE_POSTHOG_KEY to the project token and VITE_POSTHOG_HOST to its ingestion host or managed reverse proxy (this deployment uses https://p.pcstyle.dev). These are public build-time values, not personal API credentials. Rebuild after changing them. Capture requires a production build with VERCEL_ENV=production; local development and previews do not send events. The SDK’s ui_host points to EU PostHog. In PostHog’s Web Analytics settings, add your public site URL (for this deployment, https://fast.mdfromx.com).

The homepage and the /docs pages send pageviews and page leaves, including navigations between docs pages; the homepage also sends conversion_requested and skill_install_command_copied. Anonymous browser IDs persist in local storage to measure returning visitors; session and browser/device properties support Web Analytics. No person profiles are created. Event URLs contain only the origin and the page path, without fragments or query strings. Traffic sources are kept: the referrer reduced to its origin, the referring domain, and the utm_source, utm_medium, utm_campaign, utm_content, and utm_term tags; every other query parameter is dropped. Uncaught errors are sent as $exception events carrying only the exception type, message, and stack frames, with query strings and fragments removed from any URL inside them. Form contents, automatic clicks, and recordings are excluded. The proxy receives the browser’s source IP, while event properties disable IP-based enrichment.

This covers the homepage and docs only, not converted posts, the about, contact, privacy, and terms pages, or admin. API events remain separate from browser visits. Vercel Web Analytics on the homepage and docs is unchanged. The admin dashboard includes no analytics scripts.

Error tracking

Unexpected errors that escape an API handler are captured with posthog-node and flushed before the original error is rethrown. They use the server analytics token and ingestion host above, with the same production-only gate. Exception properties keep the type, message, and stack frames; URL queries, fragments, and source context are removed. Request headers, bodies, and caller identities are not attached.

For browser source maps, set the build-only variables POSTHOG_API_KEY (a personal key permitted to upload error-tracking artifacts), POSTHOG_PROJECT_ID, and POSTHOG_API_HOST (for EU projects, https://eu.posthog.com). The API host is separate from the ingestion host in POSTHOG_HOST. bun run build uploads the landing and docs bundle maps and removes them from the output. Builds without upload credentials do not generate source maps.

To test browser capture locally, use the same project token for VITE_POSTHOG_KEY and POSTHOG_PROJECT_TOKEN, then run VERCEL_ENV=production bun run build followed by bun run preview. Check .env.local after running the wizard: it overrides .env, and an older token sends events to a different project. This local build enables capture without deploying the site. Keep ordinary development and Vercel previews excluded from capture.

Optional configuration