The Data Collector
io.github.MarcinDudekDev/the-data-collector
Documentation
Installation
Remote MCP endpoint: streamable-http:https://frog03-20494.wykr.es/.well-known/mcp.json
Local setup:
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
uvicorn server:app --host 0.0.0.0 --port 8001
Docker:
docker build -t the-data-collector .
docker run -p 8001:8001 --env-file .env the-data-collector
Configuration
Environment variables: APIFY_TOKEN, API_KEY, BASE_URL, PAY_TO, PRICE_ATOMIC.
Capabilities
Search Bluesky posts, Substack newsletter articles, and Hacker News stories, with structured JSON output and x402/API-key access modes.
Limitations
Requires APIFY_TOKEN for scraping. Paid access uses x402 micropayments on Base. API-key auth is optional for regular use.