How it works

DeanReach is one Python application with three jobs: find academic decision-makers on public university pages, keep a history that stops the same person, department or institution from being emailed twice, and send a small number of personalised emails a day through the Gmail API. Everything is driven from a web admin; nothing needs a developer once it is deployed.

Components

Discovery, step by step

1. Fetch. Each source page is downloaded with a polite user agent, a 12 second timeout, a 3 MB cap and hand-checked redirects. Only public hosts are fetched (private ranges, loopback and link-local addresses are refused), which matters because admins can paste any URL.
2. Extract. University sites do not share a layout, so every page goes through a stack of extractors and the results are merged by email address:
3. jsonld: schema.org Person nodes (the cleanest signal when a site publishes them)
4. table: a directory table whose header row names a Name column plus a Title or Email column
5. cards: repeated blocks anchored on a mailto: link (profile cards, list items)
6. dl: definition lists (name in dt, role and email in dd)
7. prose: narrative paragraphs ("Dean of Business Marcus Bell (mbell@u.edu) oversees...") Emails written as name [at] school [dot] edu are rebuilt and flagged. The run records which extractors contributed to each page.
8. Model fallback. When a page yields nothing structured, or only low-confidence people, and ANTHROPIC_API_KEY is set, the page's visible text goes to Claude with a strict JSON schema (discovery/llm.py). The model's list is merged into the heuristic list: matches by email are confirmed (which raises their confidence), new emails are added at a lower base score. The model never overrides a title or an email the heuristics already found.
9. Classify. Titles are matched against the target titles (regular expressions, longest pattern first, so "associate dean" is not "dean"). Disciplines are tagged from keyword lists, checking the department before the title, the page hint, and the page title.
10. Score. A confidence score from 0 to 100 decides whether a new person may be emailed automatically:

Below the threshold (Settings, default 80) a contact waits in Needs Review. 6. Reconcile. Each person is looked up by email anywhere in the database, then by normalised name at the same institution. Known people get their last-seen date and any changed title or department recorded ("seen before, title updated"). New people become contacts with a status: eligible, or needs review with the reason (no email, low confidence, same name as someone at another institution), or invalid for role mailboxes.

Duplicate prevention over time

• Email is the primary key for a person. Case and stray punctuation are normalised.
• Names are normalised (honorifics, credentials, "Last, First" order, middle initials) before comparison, so "Dr. Susan M. Bright" and "Bright, Susan" are the same person.
• A page that lists the same person twice (leadership block plus A to Z table) yields one contact; the run reports "7 unique people after merging by email".
• The same name at another institution is never emailed automatically; it goes to review with a link to the existing record (a move, or a namesake).
• Previously contacted is permanent. Once emailed, a contact never re-enters the automatic queue. A manual Cooldown status with a date is the only way back, and only after the cooldown expires.
• Institution and department cooldowns and caps stop a second email to the same unit within the configured window, and cap how many people at one institution or department are ever emailed.
• Exclusions (email, domain, institution, person, department) are checked at send time, so one added at 09:00 blocks the 09:05 send.
• The review queue lets an admin merge two records; history moves to the record that is kept and the duplicate becomes invalid.

The send gate

outreach/eligibility.py runs the same ordered checks before every automated send and on every contact page ("Send check, right now"). The first failing check is the reason shown:

paused, contact status, email present, role mailbox, exclusions (email, domain, institution, person, department), institution blocked, institution cooldown, department cooldown, institution cap, department cap, discipline switched off or unknown, confidence threshold, daily cap, monthly cap, send window (scheduled runs only).

Outreach

Candidates are ordered by target-title priority (1 first), then confidence, then discovery date. For each cleared contact the run picks the enabled template whose discipline matches (else the default), renders the merge fields, attaches the CV, builds an RFC 2822 message, base64url-encodes it and POSTs {"raw": ...} to users.messages.send. The Gmail message id and thread id, the rendered email and the request preview are stored in the history. The contact becomes "previously contacted", the institution's last-contacted date moves, and the day's counter goes up. Failures are stored too, with the error.

Schedule

Two cron strings in Settings (defaults: discovery daily at 06:00, outreach 09:00 and 14:00 on weekdays, in the configured timezone). APScheduler runs them inside the web process when SCHEDULER_ENABLED=1, or in python -m deanreach.worker as a separate service. Scheduled outreach also respects the send window; manual runs from the dashboard do not.

Layout

app.py                     ASGI entrypoint (uvicorn app:app)
deanreach/
  config.py                environment variables, .env loading
  models.py  db.py         schema, engine, demo seeding
  settings.py              typed settings stored as key/value rows
  seed.py  fixtures/       demo data and six sample university pages
  discovery/               fetch, extract, classify, dedupe, llm, run
  outreach/                eligibility, templating, mime, gmail, run
  scheduler.py  worker.py  cron jobs
  export.py                CSV and Excel
  web/                     FastAPI app, routes, Jinja templates, CSS
docs/                      these documents (also rendered in the app)
tests/                     pytest suite
scripts/                   backup, restore, sample CV generator
Dockerfile  docker-compose.yml