Calyx is a group calendar that finds events you'll actually go to. This document describes how it works: the weekly pipeline, the ranking model, the data model, the dashboard, and how it is run. It is rendered at calyx.arthgupta.dev/about.
Status: beta. One real user plus a handful of friends. Supported cities: Boston (default), New York, Chicago, Bay Area, Bangalore. No backwards compatibility promises; schemas and routes change without migrations.
Sister docs: CLAUDE.md (build, deploy, conventions), todo.txt (work queue),
ADMIN_TODO.md (operator chores).
Two halves that share one events table:
The two halves meet on the Discover page: group events render alongside discovered ones, and RSVPs from group-mates boost discovered events in ranking.
┌──────────────── weekly pipeline (cron, Sat 9am) ────────────────┐
│ │
Spotify ────┤ │
YouTube ────┼─▶ Ingest ─▶ Extract interests ─▶ Discover events ─▶ Rank ─▶ Deliver
Gmail ────┤ (snapshot (LLM, per user) (30+ sources, (prefilter (email +
│ cache) parallel, dedup) + 7-dim) SQLite)
└─────────────────────────────────────────────────────────────────┘
│
┌────────────────────────┘
▼
Browser / phone ──▶ FastAPI dashboard ──▶ calyx.db (SQLite)
Calendar apps ──▶ /u/<token>/feed.ics
Gmail ◀── daily digest (8am), admin digest (Sun 10am)
Everything is one Python package (src/calyx/), one SQLite file (calyx.db),
and one FastAPI process. There is no queue, no worker pool, no external
database. The pipeline is a CLI (uv run calyx) invoked by cron; the dashboard
is a long-running uvicorn process managed by launchd.
| Layer | Technology |
|---|---|
| Language | Python 3.12, uv for packaging |
| Web | FastAPI + uvicorn, server-rendered HTML in f-strings, vanilla JS |
| Storage | SQLite (WAL mode), one file |
| LLM | OpenRouter, Anthropic-compatible endpoint. Default model openai/gpt-6-luna for every call |
| Gmail SMTP (STARTTLS), Jinja2 templates, premailer for inline CSS | |
| Auth | Google OAuth (OpenID Connect), 1-year cookie |
| Hosting | 8 GB MacBook at home, Cloudflare Tunnel to calyx.arthgupta.dev |
src/calyx/main.py runs the stages below in sequence, once per active user
(--all-users), and records per-call LLM cost in the costs table. A failure in
any stage for one user emails the operator and continues to the next user.
Pulls recent activity from each connected source into a list of ActivityItems.
| Source | What is read | Module |
|---|---|---|
| Spotify | Top artists and tracks (short, medium and long term), recently played | ingest/spotify.py |
| YouTube | Liked videos, subscriptions, watch history when the user uploads a Google Takeout export | ingest/youtube.py |
| Gmail | Newsletters from a sender whitelist (newsletter_senders.txt), used both for taste and as an event source |
ingest/gmail.py |
OAuth tokens live per user under state/tokens/. Tokens die regularly (Google
revokes unverified-app grants after a week of inactivity). To keep one dead
token from erasing a source's signal, every successful ingest is snapshotted
into ingest_cache, keyed by user and a versioned fingerprint of the assigned
credential paths. When a connected source returns zero items, a matching
snapshot at most 30 days old can be reused, flagged (cached from <date>) in
ingest_stats. Disconnected providers and legacy snapshots from before
credential isolation are not replayed. Legacy rows are retained for recovery.
Without usable activity, the profile falls back to manual interests plus the
taste stack.
extract/interests.py sends the activity list to the model and gets back an
InterestProfile: a list of (name, category, weight, evidence) plus a short
summary. Interest extraction runs separately for each user; profiles and costs
are saved with that user’s run.
Manual interests (user_interests table, falling back to a user-assigned file;
the legacy my_interests.txt belongs only to user 1)
are merged in first and the ranker is told they outrank listening-history
interests. When every ingest source is dead and nothing is cached, the
profile is built from manual interests plus the Elo taste stack
(taste_items, taste_matchups) alone.
events/aggregator.py loads a shared public catalog for the user's city.
Public sources use the canonical city coordinates from cities.py; personal
home coordinates are used later for ranking and artist-show distance filtering.
On a cache miss, sources run concurrently with asyncio.gather, each wrapped
so one failure cannot take down the run.
Actual event payloads are stored in event_source_snapshots in the shared
application database, keyed by source, city and cache format version. Successful
nonempty snapshots last 24 hours, including across CLI invocations and deploys.
Empty results and failures get a one-hour cooldown, so they are not retried for
every user. Raised source failures may reuse a snapshot up to seven days old,
with an explicit warning; events from previous days are excluded on read.
Sources that swallow their own errors and return an empty list receive the
empty-result cooldown. Fetches, cache hits, age and cooldowns are logged.
The older source_cache table is freshness-reporting metadata only and is
updated on actual collection, not on each user's cache hit.
The CLI holds an OS file lock next to the resolved shared database to prevent overlapping pipeline runs. User settings are copied from immutable defaults for every account. User activity, interests and rankings remain private.
City-agnostic sources (run for every city): Eventbrite (API, JSON-LD scrape
fallback), Meetup (Next.js __APOLLO_STATE__), Luma (lat/lon API, discover-page
fallback), Ticketmaster (shared nearby and far-future catalogs, with Spotify artist
filtering applied per user), Bandsintown (shared worldwide catalogs per artist,
then filtered within 100 km of each user), Resident Advisor (GraphQL by area ID).
Boston-only sources: Dice.fm, MIT, Harvard, Harvard CMSA, Northeastern, MassArt, BU, Suffolk, BC, Berklee, Tufts, Brandeis, The Boston Calendar, ICA, MFA, MIT List, Gardner, Harvard Art Museums, Museum of Science, Boston Public Library, Coolidge Corner, Bowery Presents, BSO, Boston.gov, and a curated Outdoor list. Universities come through Localist, Trumba or Tribe Events endpoints; museums and venues are scrapers or JSON APIs.
Bangalore-only sources: AllEvents, Townscript, Bangalore International Centre, Atta Galatta, Jagriti Theatre, Sabha, Alliance Française.
Newsletter emails are a source too: the model extracts dated events from the whitelisted newsletters ingested in stage 1. Newsletter events are extracted separately per user and never enter shared source or detail-page caches.
After fetching:
difflib.SequenceMatcher
on title and venue; near-duplicates keep the record with the most filled-in
fields.calyx.db) so the map
view and distance logistics work. Cache access is serialized across worker
threads. Transient network failures are not cached as permanent misses;
successful empty lookups are retried after a day.SourceStat. The admin page and Sunday digest
surface this; silent scraper breakage is the most common failure.Undated Boston Calendar listings (venue promos, roundups) are dropped at the source. Only curated Outdoor spots may be undated.
ranking/ranker.py is a two-pass scorer with a layer of editorial rules on top.
Pass 1, prefilter. Only runs when there are more than 150 events. Batches
of 100 titles go to the cheap model, which returns the IDs worth a full look.
Three classes bypass the prefilter entirely because they rarely match taste
keywords but are exactly what should surface: talks and lectures (_TALK_RE),
marquee events (city-specific regex for festivals, marathons, major openings),
and events by a Spotify artist. A failed prefilter batch passes everything
through rather than dropping it.
Before scoring, events during weekday work hours are dropped, except talks: a lunch seminar or a 4pm colloquium is attendable.
Pass 2, full ranking. Batches of 40 events go to the model with the
interest profile, season hints, the user's upcoming RSVPs (calendar density)
and the city's system prompt. Each event comes back with seven scores from 0
to 15 and a vibe label:
| Dimension | Question it answers |
|---|---|
| interest | How well does this match the profile? |
| social | Is it fun in a room full of people? |
| urgency | Is this a rare or one-time occasion? |
| logistics | Is it easy to get to at a reasonable hour? |
| friend | Could you bring someone who doesn't share the interest? |
| discovery | Would this expand taste rather than confirm it? |
| quality | Is the venue or organizer good? |
Dimensions are combined by vibe-specific weights, so a lecture is not penalized for being hard to bring a friend to and a concert is not penalized for a loose interest match:
| Weight | social | intellectual | mixed |
|---|---|---|---|
| interest | 1.5 | 3.5 | 2.5 |
| social | 2.5 | 0.5 | 1.5 |
| urgency | 1.5 | 1.5 | 1.5 |
| logistics | 1.5 | 1.5 | 1.5 |
| friend | 2.5 | 0.5 | 1.5 |
| discovery | 0.5 | 1.5 | 1.0 |
| quality | 1.0 | 2.0 | 1.0 |
The weighted sum is normalized to 0 to 100. Distance from home adjusts the logistics component after scoring.
Post-scoring adjustments, applied in order:
steering table): block, done, more, less,
pause per keyword or category, set from one-click email links.The ranker is intentionally opinionated. It leads with ideas and rare occasions rather than reproducing the listening history back as concerts.
A separate ranking/ml_model.py distills model scores into a small
scikit-learn regressor over deterministic features (source, category, hour,
distance, price). It is an offline experiment trained by
scripts/train_model.py and is not part of the pipeline.
ranking/bucket_list.py picks a few seasonal activities from
bucket_list.txt to append to the digest.
| Job | Schedule | What it sends |
|---|---|---|
| Weekly pipeline | Saturday 9am | Full digest: top picks, bucket-list ideas, one-tap RSVP and steer links |
| Daily digest | 8am daily | Today's picks from the latest run plus group events |
| Admin digest | Sunday 10am | Source health, cost, retro notes, TODOs |
| Post-event rating | manual (scripts/send_ratings.py) |
"Did you go? Rate it" |
Every link in an email carries ?u=<user_token> so a tap records an RSVP
without a sign-in detour. Rankings are also persisted to rankings, which is
what the Discover page reads.
Cost: a few cents per full run per user on gpt-6-luna.
Manual group events live in the same events table as discovered ones, with
source='manual', a group_id, created_by and free-text notes. This is
what lets one RSVP endpoint, one iCal feed and one Discover list handle both
kinds.
/api/extract-event-url). Or pick a
quick chip ("Tonight 7pm") and type a title. Adding auto-RSVPs the adder as
going and emails the other members unless they have muted the group./group/<id>/join/<code>. Authed visitors join on load;
anonymous visitors see a preview and a Google button, and the invite code
rides through the OAuth next parameter so they land joined./share/event/<gid>/<eid>/<code> is a public preview with
Going, Maybe and No buttons. Tapping one bounces through Google if needed,
then joins the group and records the RSVP in one step. A dynamic OG image is
rendered with Pillow for chat unfurls.event_comments) on the group page./u/<token>/feed.ics is the one feed per user. It
always includes events you are going to and events any group-mate is going or
maybe to. Two settings toggles add your own maybes and up to two top
recommendations per day. Summary prefixes ([Group], (?), ★) distinguish
them in the calendar app.One SQLite database, WAL mode, accessed through db.py (plain sqlite3, no
ORM). Tables by area:
| Area | Tables |
|---|---|
| Users | users (email, name, phone, city, home lat/lon, work hours, OAuth tokens, user_token), user_interests, user_bucket_list, app_settings |
| Pipeline | runs, events, rankings, costs, source_stats, ingest_stats, ingest_cache, source_cache, daily_picks, retro_log |
| Feedback | rsvps, attended, impressions, steering, taste_items, taste_matchups |
| Groups | groups, group_members, event_comments |
| Other | travel_plans, gcal_events |
Key relationships:
run belongs to a user and owns a set of events (via run_id) and
rankings (one row per event per run with the seven dimension scores, the
composite, vibe, keep and the model's match reasons).events.event_id is the stable per-source ID. Manual events use
grp_evt_<rowid>. RSVPs key on event_id, not the run, so they survive
re-ranking.user_id. There is no tenancy
beyond that.recom_token cookie for one year. Every page calls
_get_current_user(request) and redirects to /login if it needs a user.?u=<user_token>. The handler resolves the user from
the token and also sets the cookie, so later navigation is signed in./u/<token>/feed.ics. Calendar clients cannot OAuth, so
the URL is the credential.src/calyx/dashboard/app.py holds nearly every route. Pages are f-string HTML
wrapped by _layout(title, body, current_user), which supplies the shared
stylesheet, nav and Open Graph tags. There is no front-end build step; each
page's JS is inline.
| Route | Purpose |
|---|---|
/ |
Redirects to /calendar |
/calendar |
Discover: ranked events in list, week and map views, filter chips, natural-language search, RSVP from the card. Group events appear with a group pill. Works signed out with the default profile |
/e/<event_id> |
Single discovered-event page |
/groups, /group/<id> |
Your groups; a group's events, RSVPs, chat, add and edit event, members and settings |
/group/<id>/join/<code>, /share/event/... |
Invite and share landings (section 4) |
/taste-profile |
"You": extracted interests, Elo matchups, ratings, and settings (home, work hours, integrations, feed toggles) |
/about |
This document |
/landing, /login, /welcome |
Marketing page, Google sign-in, onboarding |
/u/<token>/feed.ics, /event/<id>.ics |
Calendar feeds and single-event downloads |
/admin, /admin/sources |
Run health, cost, per-source success rate and sparklines |
/api/search |
Natural-language search over the ranked set, with web search fallback |
/api/rsvp, /api/steer, /api/rate, /api/attend* |
Feedback endpoints, each also reachable from email via ?u= |
/api/group/<id>/* |
Group mutations: add, edit, delete event, rename, leave, mute, kick, delete |
/api/profile/* |
Settings updates, interest paste, YouTube Takeout upload |
Testing is black-box: scripts/smoke_test.sh checks status codes on every
route and scripts/browser_test.py drives real Chromium through the main pages
after JS runs. Both must pass before a dashboard change ships.
calyx.arthgupta.dev; start.sh keeps the tunnel and the
static arthgupta.dev site alive in tmux.dev.arthgupta.calyx.dashboard, KeepAlive)
runs uvicorn from a separate checkout at ~/recom-deploy pinned to
origin/main. Data (calyx.db, state/, .env) stays in ~/recom/calyx.
scripts/deploy.sh fetches, resets, syncs dependencies and restarts.scripts/install_cron.sh) run the weekly,
daily and admin jobs from the ~/recom/calyx working tree. Logs land in
state/{cron,daily,admin}.log.db-backup
branch on GitHub..env with the CALYX_ prefix.
Only the OpenRouter key is required; a missing event API key skips that
source. Model slugs are overridable per call type.Sage green and terracotta on white, sharp corners, Inter.
| Token | Value | Use |
|---|---|---|
| Sage | #4a6741 |
Logo, active nav, primary buttons, links, high-score badges |
| Sage dark | #3a5334 |
Hover and pressed states |
| Sage light / tint | #edf2eb / #f4f7f3 |
Active nav background, tags, hover fills |
| Terracotta | #c4734f |
Social vibe, "show more", price, alerts |
| Steel blue | #5b7fa5 |
Mixed vibe |
| Text | #111 body, #555 secondary, #888 tertiary |
|
| Borders | #e0e0e0 cards, #eee separators, #ccc inputs |
border-radius: 0 everywhere except toggles, map markers and
avatars. No shadows, no gradients on the dashboard, no lift on hover.#e0e0e0 border, 24px padding. Event cards carry a
3px left border colored by score; going events get a 4px sage border and a
#f8faf7 fill.#e8ede7 with sage text; below
50 #f5f5f5 with gray text.#4a6741 to #2d3f27).