Project Overview: cloudflare-tweets


I decided to keep cloudflare-tweets as a local, script-first monitoring system: fetch tweets, store them in DuckDB, and generate summaries without adding service infrastructure. That direction is consistent with the repo’s baseline ("Initial commit: DuckDB-based Twitter monitoring") and the most recent session prompt ("what is this project?"), which pushed us to make the operating model explicit.

What We Built

  • A Python-based monitor for Cloudflare DevRel Twitter/X activity, with data persisted in tweets.duckdb.
  • A shared CLI/search path (twitter-search + twitter_search_client.py) used for tweet retrieval workflows.
  • A reporting flow in summary.py for daily, multi-day, and date-specific summaries.
  • A combined operational entrypoint (run-daily.py) intended for scheduled execution.
  • A clear local-first structure: scripts and data live together, with archived outputs and session memory tracked in-repo.

Why We Built It

  • We needed reliable visibility into Cloudflare DevRel activity without standing up a new backend service.
  • DuckDB gives us a practical middle ground: SQL querying, local durability, and low operational overhead.
  • We intentionally constrained scope to Cloudflare/DevRel accounts to reduce token usage and avoid noisy side-tracking.
  • We documented operational decisions in MEMORY.md so future runs are repeatable and less dependent on tribal knowledge.

How It Works

  • fetch.py runs fetch cycles for tracked accounts (and can add new handles/groups), then writes to DuckDB tables for accounts, tweets, and rate-limit state.
  • twitter-search provides direct search/user-tweets commands for ad hoc inspection and automation.
  • summary.py reads stored tweets and emits markdown summaries for the selected time window.
  • run-daily.py chains fetch + summary for cron-style hourly operation.
  • Auth is environment-driven, and prior runbook fixes capture real-world issues (shebang portability, non-interactive shell env loading, and correct xAI vs X API usage) so unattended runs are more stable.