Skip to content

README.md

Latest commit

 

History

History
306 lines (216 loc) · 10.6 KB

README.md

File metadata and controls

306 lines (216 loc) · 10.6 KB
Raw

x-api-scraper-cli

@twexapi-dev/x-api-scraper-cli is a Node.js command-line client for TwexAPI Twitter/X endpoints. Use the x-api-scraper command to fetch profiles, search tweets, look up timelines, and call TwexAPI read endpoints from the terminal with an API key.

Install

npm install -g @twexapi-dev/x-api-scraper-cli
x-api-scraper --help

Requires Node.js 18 or newer. npm: https://www.npmjs.com/package/@twexapi-dev/x-api-scraper-cli

Quick facts

  • npm package: @twexapi-dev/x-api-scraper-cli
  • Command: x-api-scraper
  • Category: TwexAPI command-line client
  • Runtime: Node.js 18 or newer
  • Best for: developers, AI agents, research teams, and automation workflows
  • Main use cases: Twitter/X profile lookup, tweet search, timelines, followers/following, lists, articles lookup, and global trending tweets
  • API provider: TwexAPI
  • Default API base URL: https://api.twexapi.io
  • Authentication: Bearer API key
  • Interfaces: terminal CLI, raw API path calls, Claude/Codex/OpenClaw skill usage
  • License: MIT
  • Dashboard: TwexAPI dashboard

What it does

  • Calls TwexAPI Twitter/X read endpoints from the terminal with a stable x-api-scraper command.
  • Saves reusable app configs for API keys and base URLs.
  • Supports generic HTTP-style requests to any TwexAPI path.
  • Provides convenience commands for users, search, followers/following, lists, tweets, articles, profile, timeline, and global trending tweets.
  • Prints dry-run previews for request payloads before sending them.
  • Masks secrets in config output and dry-run previews.

Who it is for

  • Developers who want a terminal-first client for Twitter/X data retrieval.
  • AI agent builders who need deterministic commands, JSON output, and dry-run previews.
  • Growth and research teams that monitor Twitter/X profiles, search terms, followers, lists, and global trending tweets.
  • Operators who need saved app configs for repeatable read workflows without rewriting API request code.

AI agent use cases

x-api-scraper-cli is designed to be easy for AI coding agents and workflow agents to use:

  • Fetch structured Twitter/X data as JSON for downstream summarization or enrichment.
  • Preview requests with --dry-run before sending them.
  • Call unsupported endpoints through the raw x-api-scraper <path> form.
  • Use saved app names instead of pasting credentials into every command.
  • Install and invoke the repository as a Claude/Codex/OpenClaw-style skill.

Common workflows

Fetch Twitter/X user data

x-api-scraper --app prod about elonmusk
x-api-scraper --app prod users elonmusk sama

Search Twitter/X content

x-api-scraper --app prod search tweets "founder" "ai" --sort Latest
x-api-scraper --app prod search users "openai"

Get global trending tweets

x-api-scraper --app prod trending tweets --country "United States" --topic "Sports" --content "NFL" --count 50

Look up tweets and timelines

x-api-scraper --app prod tweet lookup 123 --summary
x-api-scraper --app prod tweet thread 123
x-api-scraper --app prod timeline user elonmusk

Run a raw TwexAPI request

x-api-scraper --app prod -X POST -d '["elonmusk","sama"]' /twitter/users

Why it is different

  • It combines convenience commands with raw path access, so users are not blocked when an endpoint is not wrapped yet.
  • It focuses public docs on API-key read workflows for fetch, search, profile, and timeline.
  • It treats AI-agent safety as a first-class workflow through --dry-run, secret masking, and saved app names.
  • It includes skill/plugin metadata so agents can discover how to install, configure, and run the CLI.
  • It keeps the implementation small and dependency-free, with separate modules for parsing, config, request execution, and command routing.

Use as a skill

Install from a GitHub skill installer:

npx skills add twexapi-dev/x-api-scraper-cli

Install from ClawHub:

npx clawhub@latest install x-api-scraper-cli

Install as a Claude Code marketplace:

/plugin marketplace add twexapi-dev/x-api-scraper-cli
/plugin install x-api-scraper-cli@x-api-scraper-cli

For manual Claude Code installs, this repository also includes:

.claude/skills/x-api-scraper-cli/

Then install the CLI:

npm install -g @twexapi-dev/x-api-scraper-cli

In skills-enabled environments:

  • use /x-api-scraper-cli after installing the Claude plugin command
  • use $x-api-scraper-cli in Codex/OpenClaw-style environments
  • or ask the agent to use x-api-scraper for installation, API-key setup, dry-run previews, or endpoint calls

Quick start

Get your API key from the TwexAPI dashboard:

TwexAPI dashboard

Save an app config and make a first request:

x-api-scraper auth apps add --name prod --api-key "twitterx_..."
x-api-scraper auth apps use prod
x-api-scraper --app prod about elonmusk

If you are developing from this repository, use:

node ./bin/x-api-scraper.js --help

If you want a local executable command while developing from source:

npm link
x-api-scraper --help

Config management

When a request is missing an API key, the CLI will include that URL in the error output.

Security notes

  • The CLI reads X_API_SCRAPER_KEY, X_API_SCRAPER_BASE_URL, and X_API_SCRAPER_CONFIG_DIR from the environment.
  • The CLI reads and writes persistent config in ~/.x-api-scraper/config.json by default, or in the directory set by --config-dir or X_API_SCRAPER_CONFIG_DIR.
  • Saved app configs may contain API keys in plain JSON on disk.
  • Avoid storing long-lived credentials on shared machines or CI runners. Prefer an isolated config directory when testing.

Save an app config:

x-api-scraper auth apps add --name prod --api-key "twitterx_..."
x-api-scraper auth apps list
x-api-scraper auth apps use prod

Inspect config:

x-api-scraper config show
x-api-scraper config path

By default config is stored in ~/.x-api-scraper/config.json. For testing, you can isolate it:

x-api-scraper --config-dir ./.x-api-scraper-local config show

Request examples

Query an endpoint directly:

x-api-scraper /twitter/elonmusk/about

Send a JSON body with a POST:

x-api-scraper -X POST -d '["elonmusk","sama"]' /twitter/users

Preview a request without sending it:

x-api-scraper --app prod --dry-run users elonmusk sama
x-api-scraper --app prod --dry-run about elonmusk
x-api-scraper --app prod --dry-run search tweets "ai" --count 20

Convenience commands

# Twitter/X User and Search
x-api-scraper --app prod users elonmusk sama           # Lookup user profiles
x-api-scraper --app prod about elonmusk                # Fetch detailed user info
x-api-scraper --app prod search tweets "founder" "ai"  # Search tweets by page
x-api-scraper --app prod search users "openai"         # Search for users
x-api-scraper --app prod search hashtags ai openai     # Search tweets by hashtag
x-api-scraper --app prod search cashtags TSLA          # Search tweets by cashtag
x-api-scraper --app prod followers elonmusk            # List user followers (v3)
x-api-scraper --app prod followers verified elonmusk   # List verified followers (v3)
x-api-scraper --app prod following elonmusk            # List user following (v3)

# Twitter/X Communities
x-api-scraper --app prod community search --query "ai"
x-api-scraper --app prod community get 123
x-api-scraper --app prod community tweets 123 --type Latest

# Twitter/X Lists
x-api-scraper --app prod list search --query "ai"      # Search for public lists
x-api-scraper --app prod list members 123456789        # List members of a list
x-api-scraper --app prod list subscribers 123456789    # List subscribers of a list

# Twitter/X Articles
x-api-scraper --app prod article markdown 123          # Get x article as Markdown
x-api-scraper --app prod article lookup 123 456        # Batch lookup x articles

# Twitter/X Profile & Timeline
x-api-scraper --app prod timeline user elonmusk        # Fetch user timeline page

# Twitter/X Global Trending
x-api-scraper --app prod trending tweets --country "United States"
x-api-scraper --app prod trending tweets --country "United States" --topic "Sports" --content "NFL" --count 50

# Twitter/X Tweets
x-api-scraper --app prod tweet lookup 123 --summary    # Batch tweet lookup (id:bool)
x-api-scraper --app prod tweet thread 123              # Get a tweet thread
x-api-scraper --app prod tweet replies 123             # Get replies by page

FAQ

What is x-api-scraper-cli?

@twexapi-dev/x-api-scraper-cli is a Node.js command-line client for TwexAPI Twitter/X endpoints. Public docs focus on API-key read workflows: user lookup, tweet search, timelines, followers/following, lists, articles lookup, and global trending tweets.

Does x-api-scraper-cli require an API key?

Yes. Read requests require a TwexAPI API key passed with --api-key, X_API_SCRAPER_KEY, or a saved app config created with x-api-scraper auth apps add.

Do these docs cover write actions?

No. Public documentation focuses on API-key read workflows (fetch, search, profile, timeline). Other capabilities are outside the scope of this getting-started guide.

Is x-api-scraper-cli suitable for AI agents?

Yes. It exposes deterministic commands, JSON responses, dry-run previews, saved app configs, and skill/plugin metadata for Claude, Codex, and OpenClaw-style environments.

Can x-api-scraper-cli call endpoints that do not have convenience commands?

Yes. Use the raw path form, for example x-api-scraper /twitter/elonmusk/about or x-api-scraper -X POST -d '["elonmusk"]' /twitter/users.

Limitations

  • A TwexAPI API key is required for real API requests.
  • Saved app configs are stored as plain JSON on disk.
  • The CLI is a terminal client, not a web dashboard or hosted API service.

Project layout

bin/x-api-scraper.js         # thin executable entrypoint
src/index.js           # main boot flow
src/parser.js          # global option parsing
src/config.js          # config load/save and auth helpers
src/request.js         # HTTP execution and dry-run preview
src/commands.js        # command routing and endpoint mapping
src/help.js            # help text
src/constants.js       # defaults and option metadata
src/utils.js           # shared helpers

Notes

  • Global options such as --app, --api-key, and --dry-run should be placed before the command.
  • For unsupported endpoints, use the generic x-api-scraper <path> form.
  • The CLI masks secrets in config output and dry-run previews.