# A TIME Call Reports — Staff Guide & Troubleshooting

A plain-English guide to the ATIME monthly call-activity reporting tool: what it is,
how it runs on the server, and what to do when something breaks.

> **TL;DR** — On the 1st of every month the server pulls A TIME's phone call records
> and emails **two** branded reports per region (Cleveland, Florida, NYC): last
> month's activity, and year-to-date (on Jan 1, the YTD slot becomes the prior
> year's **Annual** report instead). Right now every report goes to
> **provisioning@pressone.net** for review. A generic org-wide report also exists
> but is **not** part of the automatic monthly run — see Section 4 to send it.
> If something's wrong, 90% of the time the answer is in the logs:
> `journalctl -u ns-atime-reports.service -n 100`.

---

## 1. What this tool does

Each month it automatically:

1. **Pulls** new call detail records (CDRs) from the NetSapiens phone platform
   (`api.cloudsession.io`) into a local database, and refreshes the live extension →
   department cache so brand-new staff/queues are classified immediately.
2. **Keeps** a rolling 18 months of history (older data is pruned).
3. **Analyzes** every call — which **Reporting Division** it reached (the nine
   classic helplines plus A TIME's other programs — 24 divisions in total), whether
   it was answered or went to voicemail, and where the caller was calling from.
   Inbound (family calling in) and outbound (staff follow-up) are always reported
   separately, never blended together.
4. **Builds** an **A TIME-branded** HTML report + Excel workbook (colors, font, and
   the org logo pulled from atime.org — see the Branding note in Section 6), **per
   region** (Cleveland, Florida, New York City) — **two** reports each: last month,
   and year-to-date (Annual on Jan 1). A **generic** org-wide version (no regional
   section) also exists but has to be requested explicitly (Section 4).
5. **Emails** each report to the recipients configured for it.

The reports are for **donor / fundraising** use — they show the scale and reach of
A TIME's helplines. Keep the tone respectful; this serves families dealing with
infertility.

## 2. Where everything lives (server)

| Thing | Location |
|---|---|
| Server | `root@172.16.1.40` (hostname `ponjsecdev1`, a.k.a. `ns-scripts.pressone.net`) — on the office VPN |
| Code | `/opt/ns-scripts/atime-reports/` (git repo, branch `master`) |
| Python venv | `/opt/ns-scripts/atime-reports/venv/` (not in git) |
| Database (18 mo of CDRs) | `/opt/ns-scripts/atime-reports/data/atime_cdrs.db` |
| Staff/queue mapping | `/opt/ns-scripts/atime-reports/data/UserListWithReportingDept.xlsx` |
| Secrets (API token, email key, recipients) | `/opt/ns-scripts/atime-reports/secrets.json` (chmod 600, **never** in git) |
| Generated reports & email previews | `/opt/ns-scripts/atime-reports/output/` |
| Scheduler | systemd: `ns-atime-reports.timer` → `ns-atime-reports.service` |
| Logs | `journalctl -u ns-atime-reports.service` |
| Source of truth (code) | your Mac: `~/ClaudProjects/Netsapiens-Scripts/` → GitHub `shripald/netsapiens-scripts` |

Everything runs as `root` from the venv Python:
`/opt/ns-scripts/atime-reports/venv/bin/python`.

## 3. How it's scheduled

A systemd **timer** fires at **06:00 on the 1st of each month** and runs
`report.py --monthly`, which does: pull → prune → build → email.

```bash
systemctl status  ns-atime-reports.timer      # is it active?
systemctl list-timers ns-atime-reports.timer  # when did/does it run?
journalctl -u ns-atime-reports.service -n 100 # what happened last run
```

Each firing sends **two reports per region**:
1. **Last month** on its own (e.g. "June 2026") — a single month, so it has no
   month-by-month trend section.
2. **Year-to-date**, Jan 1 through the end of last month. On **Jan 1**, this window
   is the entire prior year, so it goes out once labeled **"`<year>` Annual"**
   instead of a duplicate "YTD through December."

So a typical month produces 6 emails (3 regions × 2 reports); January produces 6
as well, but the second one of each pair is the Annual report rather than a YTD.

## 4. Everyday tasks

All commands run on the server, from `/opt/ns-scripts/atime-reports/`:
```bash
cd /opt/ns-scripts/atime-reports
PY=./venv/bin/python
```

**Change who receives the reports** — edit `secrets.json`, the `recipients` block.
No restart needed (it's read each run).
```json
"recipients": {
  "generic":   ["board@atime.org"],
  "cleveland": ["cleveland-donor@example.org"],
  "florida":   ["fl-donor@example.org"],
  "nyc":       ["nyc-donor@example.org"]
}
```

**Run a report right now, without emailing** (writes files to `output/`):
```bash
$PY report.py --ytd                     # all three regions, year-to-date
$PY report.py --ytd --region cleveland  # one region, year-to-date
$PY report.py --ytd --generic           # generic org-wide (not sent automatically — see above)
$PY report.py --start 2026-06-01 --end 2026-06-30   # any custom date range
```

**Re-send this month's emails by hand:**
```bash
$PY report.py --monthly                 # full monthly run: pull + prune + BOTH
                                         # reports (last month + YTD/Annual) for
                                         # all 3 regions, emailed
$PY report.py --ytd --region florida --email   # just re-send Florida's YTD report
```

**Do a safe practice run that does NOT send email** (writes `.eml` previews to
`output/` instead):
```bash
ATIME_EMAIL_DRY_RUN=1 $PY report.py --monthly
```

**Sanity-check the numbers** (independent recompute; should say ALL MATCH):
```bash
$PY verify.py 2026
```

## 5. Troubleshooting — "an email didn't arrive / something's wrong"

**Always start here:**
```bash
journalctl -u ns-atime-reports.service -n 150 --no-pager
```
Look for the last run's lines: `pull_range done`, `prune_older_than`, `[sent:postmark]`
for each region, or an error/traceback.

### The reports didn't get emailed at all
1. **Did the timer fire?** `systemctl list-timers ns-atime-reports.timer`. If it's not
   listed or shows no recent run, re-enable it (Section 7). If the server was off at
   06:00 on the 1st, `Persistent=true` makes it run at next boot — or just run
   `$PY report.py --monthly` by hand.
2. **Did it send but you didn't get it?** Check the **Postmark Activity feed**
   (postmarkapp.com → your server → Activity) for the message + delivery status.
   Also check spam. Confirm the address in `secrets.json` `recipients`.
3. **Log shows `[dry-run] would email …`** → dry-run mode is on. Make sure
   `ATIME_EMAIL_DRY_RUN` is **not** set and `ATIME_EMAIL_API_KEY` is present in
   `secrets.json`. (Dry-run writes `output/PREVIEW_*.eml` instead of sending.)
4. **Log shows `[skip] no recipients for …`** → that region has no addresses in
   `secrets.json` → add them.

### Email send errors (Postmark)
- `422 … Sender signature not confirmed` / **unverified from-address** → the
  `ATIME_EMAIL_FROM` address (`reports@pressone.net`) must be a **verified sender
  signature** in Postmark. Verify it in the Postmark dashboard.
- `422 … inactive recipient` → Postmark suppressed that address after a bounce; remove
  the suppression in Postmark, fix the address.
- `401 Unauthorized` → wrong/rotated **Postmark API key** in `secrets.json`
  (`ATIME_EMAIL_API_KEY`).

### The pull failed / no fresh data
Symptoms in the log: `[!] offset 0 attempt … Missing required secret 'ATIME_NS_TOKEN'`,
repeated `attempt 1/3 … 2/3 … 3/3`, or `401`.
- **`Missing required secret 'ATIME_NS_TOKEN'`** → `secrets.json` is missing the NS
  token (or the file isn't readable). Check `cat secrets.json` and file perms (600, owned
  by root).
- **`401` / auth errors from the API** → the NetSapiens token was rotated or expired.
  Get a new token, put it in `secrets.json` (`ATIME_NS_TOKEN`), re-run.
- **Timeouts / connection errors** → server can't reach `api.cloudsession.io`. Test:
  `curl -sI https://api.cloudsession.io`. Check network/DNS.
- **Important:** a failed pull does **not** stop the run — it reports on the existing
  database and logs the error. So "reports came but the numbers look stale" can mean the
  pull is quietly failing. Check the log for pull errors.

### Report numbers look wrong or empty
- **Everything is zero / very low** → the database may be empty or the mapping file is
  missing. Check:
  ```bash
  $PY -c "import pull,config;c=pull.connect();print('rows',pull.row_count(c))"
  ls -la data/UserListWithReportingDept.xlsx
  ```
  If rows are 0, re-seed (Section 6). If the xlsx is missing, copy it back (Section 6).
- **A division looks wrong / a person's calls are miscounted** → division is decided by
  the caller's/queue's **NetSapiens department** field, run through the department →
  Reporting Division matrix in `config.py` (`DEPT_CROSSWALK`). Two possible fixes:
  - The department itself is wrong in NetSapiens → fix it in the NS admin portal for
    that user/queue; the next `--pull`/`--monthly` run picks it up automatically
    (no xlsx edit needed).
  - The department is right but maps to the wrong division, or a new department was
    added that isn't in the matrix yet → add/fix its entry in `_CROSSWALK_RAW` in
    `config.py` (a code change: edit on your Mac, commit, push, `git pull` on the
    server — Section 6).
  - `data/UserListWithReportingDept.xlsx`'s "Reporting Department" column is only a
    legacy fallback now, used solely when a user's department is blank or not in the
    matrix — updating it will not override a department that's already mapped.
- **Numbers don't match expectations** → run `$PY verify.py <year>`; it recomputes
  independently and flags any mismatch.

### Disk filling up
The DB is pruned to 18 months automatically, but check:
```bash
df -h /            # disk space
du -sh /opt/ns-scripts/atime-reports/data /opt/ns-scripts/atime-reports/output
```
`output/` accumulates report files — safe to clean old ones:
`find output -type f -mtime +120 -delete`.

## 6. Occasional maintenance

**Deploy a code change** (never edit files directly on the server):
```
# On your Mac:  edit → commit → push to GitHub, then on the server:
cd /opt/ns-scripts && git pull
# The tool is a batch job (no service to restart). Optionally re-run:
cd atime-reports && ./venv/bin/python report.py --ytd --generic
```

**New hires / new queues**: nothing to do here. As long as the new user or queue has
a **department** set in NetSapiens, the next scheduled run's live department-cache
refresh picks it up automatically and classifies it via the matrix in `config.py`.
Only touch the xlsx if you're deliberately overriding a division for someone whose
department is blank or not yet in the matrix:
```bash
# From your Mac (the xlsx is NOT in git — it holds staff PII):
scp UserListWithReportingDept.xlsx root@172.16.1.40:/opt/ns-scripts/atime-reports/data/
```

**Re-seed / rebuild the database from scratch** (e.g., corruption):
```bash
cd /opt/ns-scripts/atime-reports
mv data/atime_cdrs.db data/atime_cdrs.db.bak      # keep a backup
./venv/bin/python pull.py range "2025-01-01T00:00:00-05:00" "2026-12-31T23:59:59-05:00"
```

**Rotate a secret** — edit `secrets.json`, save. Nothing to restart.

**Add a new region** — this is a code change: add the region (name + area codes) to
`REGIONS` in `config.py`, commit/push/pull, add its recipients to `secrets.json`.

**Branding** (if A TIME redesigns their site / changes their logo): colors, font,
and the header logo all live in `config.BRAND` and the `_LOGO_SVG` constant in
render_html.py, both sourced from atime.org's live theme CSS and logo asset — not
hand-picked. Re-pull from the live site rather than eyeballing new hex values by
hand: fetch `wp-content/themes/a-time/assets/css/main.css` for the color tokens
and `wp-content/themes/a-time/assets/images/main-logo.svg` for the logo, splice
the SVG in programmatically (not retyped — the path data is long and a manual
transcription error would be easy to miss). Their heading font is a paid webfont
(P22MackinacPro) we don't have a license for; Inter substitutes for it. This is a
code change: edit on your Mac, commit, push, `git pull` on the server.

## 7. Turn the monthly run on / off

```bash
systemctl enable --now  ns-atime-reports.timer   # turn ON  (and start timing)
systemctl disable --now ns-atime-reports.timer   # turn OFF (stop the monthly run)
systemctl start ns-atime-reports.service         # run it once, right now
```

## 8. Escalation

- **Code / logic / new features:** Shripal Daphtary (shripal.daphtary@pressone.net).
- **Email delivery:** Postmark dashboard (account owner).
- **NetSapiens API / token:** PressONE platform admin.
- Full technical reference: `README.md` in this folder.