Project Overview: cloudflare-tweets
I decided to keep cloudflare-tweets as a local, script-first monitoring system: fetch tweets, store them in DuckDB, and generate summaries without adding service infrastructure. That direction is consistent with the repo’s baseline ("Initial commit: DuckDB-based Twitter monitoring") and the most recent session prompt ("what is this project?"), which pushed us to make the operating model explicit.
What We Built
- A Python-based monitor for Cloudflare DevRel Twitter/X activity, with data persisted in
tweets.duckdb. - A shared CLI/search path (
twitter-search+twitter_search_client.py) used for tweet retrieval workflows. - A reporting flow in
summary.pyfor daily, multi-day, and date-specific summaries. - A combined operational entrypoint (
run-daily.py) intended for scheduled execution. - A clear local-first structure: scripts and data live together, with archived outputs and session memory tracked in-repo.
Why We Built It
- We needed reliable visibility into Cloudflare DevRel activity without standing up a new backend service.
- DuckDB gives us a practical middle ground: SQL querying, local durability, and low operational overhead.
- We intentionally constrained scope to Cloudflare/DevRel accounts to reduce token usage and avoid noisy side-tracking.
- We documented operational decisions in
MEMORY.mdso future runs are repeatable and less dependent on tribal knowledge.
How It Works
fetch.pyruns fetch cycles for tracked accounts (and can add new handles/groups), then writes to DuckDB tables for accounts, tweets, and rate-limit state.twitter-searchprovides direct search/user-tweets commands for ad hoc inspection and automation.summary.pyreads stored tweets and emits markdown summaries for the selected time window.run-daily.pychains fetch + summary for cron-style hourly operation.- Auth is environment-driven, and prior runbook fixes capture real-world issues (shebang portability, non-interactive shell env loading, and correct xAI vs X API usage) so unattended runs are more stable.