# Config ID Refactor Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Replace the `domain` config lookup key with `id` throughout all routes and tests, so one NS domain can have multiple widget/SMS configs.

**Architecture:** Pure rename/refactor — no new logic. All four routes (`/webresponder`, `/click-to-call`, `/widget.js`, `/update-domain`) and all three test files are updated to use `id` instead of `domain` as the config lookup key. The `_DOMAIN_RE` regex already handles the new format (`pressone.net-sales` etc.) — no change needed there.

**Tech Stack:** Python 3, Flask, pytest.

---

## What changes and where

| Location | Old | New |
|---|---|---|
| `/webresponder` | reads `AccountDomain` from POST body | reads `id` from URL query param (`?id=`) |
| `/click-to-call` | reads `domain` from JSON body | reads `id` from JSON body |
| `/widget.js` | reads `?domain=` query param | reads `?id=` query param |
| `/update-domain` | form field `domain`, returns `{"domain":...}` | form field `id`, returns `{"id":...}` |
| `_WIDGET_JS_TEMPLATE` | `var DOMAIN = ...` / `{ domain: DOMAIN }` | `var CONFIG_ID = ...` / `{ id: CONFIG_ID }` |
| `_render_widget_js` | `domain` param, `"domain"` key | `config_id` param, `"config_id"` key |
| `_UPDATE_REQUIRED_FIELDS` | contains `"domain"` | contains `"id"` |
| `test_app.py` | posts `AccountDomain=testco.net` | posts to `/webresponder?id=testco.net` |
| `test_click_to_call.py` | `CTC_FAKE_DOMAINS` key `"testco.net"`, body `{domain: ...}`, `?domain=` | key `"testco.net-sales"`, body `{id: ...}`, `?id=` |
| `test_update_domain.py` | `VALID_PAYLOAD["domain"]`, response `{"domain":...}` | `VALID_PAYLOAD["id"]`, response `{"id":...}` |

---

### Task 1: Update tests first (they will fail), then update app.py

This is a single TDD task — update all three test files, confirm they fail, then update app.py to make them pass.

**Files:**
- Modify: `Netsapiens-Scripts/webresponder/tests/test_app.py`
- Modify: `Netsapiens-Scripts/webresponder/tests/test_click_to_call.py`
- Modify: `Netsapiens-Scripts/webresponder/tests/test_update_domain.py`
- Modify: `Netsapiens-Scripts/webresponder/app.py`

---

**Step 1: Update `tests/test_app.py`**

Replace the entire file with:

```python
import pytest
import requests as req_lib
from unittest.mock import MagicMock, patch

from app import app


FAKE_DOMAINS = {
    "testco.net": {
        "api_domain": "testco.net",
        "user": "2000",
        "from_number": "+18135550001",
        "message": "Hours: M-F 9am-5pm. Address: 1 Test St, Tampa FL.",
    }
}


@pytest.fixture
def client():
    app.testing = True
    return app.test_client()


def _post(client, config_id="testco.net", **overrides):
    data = {"NmsAni": "+18135559999", **overrides}
    return client.post(f"/webresponder?id={config_id}", data=data)


def _get(client, config_id="testco.net", **overrides):
    qs = {"NmsAni": "+18135559999", "id": config_id, **overrides}
    return client.get("/webresponder", query_string=qs)


# --- happy path ---

def test_happy_path_returns_200(client):
    with patch("app.load_domains", return_value=FAKE_DOMAINS), \
         patch("app.send_sms") as mock_sms:
        mock_sms.return_value = MagicMock()
        resp = _post(client)
    assert resp.status_code == 200


def test_happy_path_xml_says_texted(client):
    with patch("app.load_domains", return_value=FAKE_DOMAINS), \
         patch("app.send_sms") as mock_sms:
        mock_sms.return_value = MagicMock()
        resp = _post(client)
    assert b"texted you" in resp.data
    assert b"Hangup" in resp.data


def test_happy_path_content_type_is_xml(client):
    with patch("app.load_domains", return_value=FAKE_DOMAINS), \
         patch("app.send_sms") as mock_sms:
        mock_sms.return_value = MagicMock()
        resp = _post(client)
    assert "text/xml" in resp.content_type


def test_happy_path_calls_send_sms_with_correct_args(client):
    with patch("app.load_domains", return_value=FAKE_DOMAINS), \
         patch("app.send_sms") as mock_sms:
        mock_sms.return_value = MagicMock()
        _post(client)
    mock_sms.assert_called_once_with(
        "testco.net",
        "2000",
        "+18135550001",
        "18135559999",  # + stripped before passing to NS SMS API
        "Hours: M-F 9am-5pm. Address: 1 Test St, Tampa FL.",
    )


# --- error: missing caller ---

def test_missing_caller_returns_error_xml(client):
    with patch("app.load_domains", return_value=FAKE_DOMAINS):
        resp = _post(client, NmsAni="")
    assert resp.status_code == 200
    assert b"could not send" in resp.data
    assert b"Hangup" in resp.data


# --- error: unknown id ---

def test_unknown_id_returns_error_xml(client):
    with patch("app.load_domains", return_value=FAKE_DOMAINS):
        resp = _post(client, config_id="unknown.net")
    assert resp.status_code == 200
    assert b"could not send" in resp.data


# --- error: SMS API failure ---

def test_sms_http_error_returns_error_xml(client):
    mock_response = MagicMock()
    mock_response.status_code = 400
    mock_response.text = "Bad Request"
    http_err = req_lib.HTTPError(response=mock_response)

    with patch("app.load_domains", return_value=FAKE_DOMAINS), \
         patch("app.send_sms", side_effect=http_err):
        resp = _post(client)
    assert resp.status_code == 200
    assert b"could not send" in resp.data


def test_sms_connection_error_returns_error_xml(client):
    with patch("app.load_domains", return_value=FAKE_DOMAINS), \
         patch("app.send_sms", side_effect=req_lib.ConnectionError("timeout")):
        resp = _post(client)
    assert resp.status_code == 200
    assert b"could not send" in resp.data


# --- GET method (NS default) ---

def test_get_happy_path(client):
    with patch("app.load_domains", return_value=FAKE_DOMAINS), \
         patch("app.send_sms") as mock_sms:
        mock_sms.return_value = MagicMock()
        resp = _get(client)
    assert resp.status_code == 200
    assert b"texted you" in resp.data


def test_get_unknown_id_returns_error_xml(client):
    with patch("app.load_domains", return_value=FAKE_DOMAINS):
        resp = _get(client, config_id="unknown.net")
    assert resp.status_code == 200
    assert b"could not send" in resp.data
```

---

**Step 2: Update `tests/test_click_to_call.py`**

Make these targeted changes (do NOT rewrite the whole file):

1. Change `CTC_FAKE_DOMAINS` key from `"testco.net"` to `"testco.net-sales"`:
```python
CTC_FAKE_DOMAINS = {
    "testco.net-sales": {
        "api_domain": "testco.net",
        "queue": "500",
        "branding": {
            "button_text": "Call Us",
            "button_color": "#000000",
            "company_name": "TestCo",
            "logo_url": "",
        },
    }
}
```

2. Change `_ctc_post` helper body from `{"domain": "testco.net", ...}` to `{"id": "testco.net-sales", ...}`:
```python
def _ctc_post(client, **overrides):
    body = {"id": "testco.net-sales", "phone_number": "8135551234", **overrides}
    return client.post(
        "/click-to-call",
        data=json_module.dumps(body),
        content_type="application/json",
    )
```

3. Change `test_click_to_call_unknown_domain_returns_404` — replace `domain="unknown.net"` with `id="unknown.net"`:
```python
def test_click_to_call_unknown_domain_returns_404(ctc_client):
    with patch("app.load_domains", return_value=CTC_FAKE_DOMAINS):
        resp = _ctc_post(ctc_client, id="unknown.net")
    assert resp.status_code == 404
    assert resp.get_json()["status"] == "error"
```

4. Change `test_click_to_call_domain_missing_queue_returns_404` — update the domains dict key and the `_ctc_post` call:
```python
def test_click_to_call_domain_missing_queue_returns_404(ctc_client):
    domains_without_queue = {"testco.net-sales": {"api_domain": "testco.net", "branding": {}}}
    with patch("app.load_domains", return_value=domains_without_queue):
        resp = _ctc_post(ctc_client)
    assert resp.status_code == 404
```

5. Change the three widget.js tests — replace `?domain=testco.net` with `?id=testco.net-sales` and `?domain=unknown.net` with `?id=unknown.net`:
```python
def test_widget_js_returns_javascript_content_type(ctc_client):
    with patch("app.load_domains", return_value=CTC_FAKE_DOMAINS):
        resp = ctc_client.get("/widget.js?id=testco.net-sales")
    assert resp.status_code == 200
    assert "javascript" in resp.content_type


def test_widget_js_includes_branding_values(ctc_client):
    with patch("app.load_domains", return_value=CTC_FAKE_DOMAINS):
        resp = ctc_client.get("/widget.js?id=testco.net-sales")
    body = resp.get_data(as_text=True)
    assert "Call Us" in body
    assert "#000000" in body
    assert "TestCo" in body
    assert "/click-to-call" in body


def test_widget_js_unknown_domain_returns_404(ctc_client):
    with patch("app.load_domains", return_value=CTC_FAKE_DOMAINS):
        resp = ctc_client.get("/widget.js?id=unknown.net")
    assert resp.status_code == 404
```

---

**Step 3: Update `tests/test_update_domain.py`**

Make these targeted changes:

1. Change `VALID_PAYLOAD["domain"]` → `VALID_PAYLOAD["id"]`:
```python
VALID_PAYLOAD = {
    "update_secret": "test-secret-for-pytest",
    "id": "acme.com",
    "api_domain": "acme.com",
    "user": "2000",
    "from_number": "18135550001",
    "message": "Hours: M-F 9am-5pm",
    "queue": "500@acme.com",
    "button_text": "Call Us",
    "button_color": "#ff0000",
    "company_name": "Acme Corp",
    "logo_url": "",
}
```

2. Change `test_update_domain_happy_path_returns_200` assertion:
```python
def test_update_domain_happy_path_returns_200(ud_client):
    with patch("app.load_domains", return_value=dict(EXISTING_DOMAINS)), \
         patch("app.save_domains") as mock_save:
        resp = _post(ud_client)
    assert resp.status_code == 200
    assert resp.get_json() == {"status": "ok", "id": "acme.com"}
```

3. Change `test_update_domain_invalid_domain_format_returns_400` — `"domain"` → `"id"`:
```python
def test_update_domain_invalid_domain_format_returns_400(ud_client):
    payload = {**VALID_PAYLOAD, "id": "not a domain!!"}
    resp = _post(ud_client, payload)
    assert resp.status_code == 400
```

---

**Step 4: Run tests to verify they FAIL**

```bash
cd Netsapiens-Scripts/webresponder && venv/bin/pytest -v 2>&1 | tail -20
```
Expected: multiple failures (routes still use old `domain`/`AccountDomain` param names)

---

**Step 5: Update `app.py`**

Make these changes in order:

**5a. `_WIDGET_JS_TEMPLATE` — rename JS variable and fetch body key (line ~102):**

Old:
```javascript
  var DOMAIN = %(domain)s;
```
New:
```javascript
  var CONFIG_ID = %(config_id)s;
```

Old:
```javascript
      body: JSON.stringify({ domain: DOMAIN, phone_number: input.value })
```
New:
```javascript
      body: JSON.stringify({ id: CONFIG_ID, phone_number: input.value })
```

**5b. `_render_widget_js` function (line ~161):**

Old:
```python
def _render_widget_js(domain, config):
    branding = config.get("branding") or {}
    values = {
        "button_text": json.dumps(branding.get("button_text", "Call Us Now"), ensure_ascii=True),
        "button_color": json.dumps(branding.get("button_color", "#1a73e8"), ensure_ascii=True),
        "company_name": json.dumps(branding.get("company_name", ""), ensure_ascii=True),
        "endpoint": json.dumps("https://ns-scripts.pressone.net/click-to-call", ensure_ascii=True),
        "domain": json.dumps(domain, ensure_ascii=True),
    }
    return _WIDGET_JS_TEMPLATE % values
```
New:
```python
def _render_widget_js(config_id, config):
    branding = config.get("branding") or {}
    values = {
        "button_text": json.dumps(branding.get("button_text", "Call Us Now"), ensure_ascii=True),
        "button_color": json.dumps(branding.get("button_color", "#1a73e8"), ensure_ascii=True),
        "company_name": json.dumps(branding.get("company_name", ""), ensure_ascii=True),
        "endpoint": json.dumps("https://ns-scripts.pressone.net/click-to-call", ensure_ascii=True),
        "config_id": json.dumps(config_id, ensure_ascii=True),
    }
    return _WIDGET_JS_TEMPLATE % values
```

**5c. `/webresponder` route — replace AccountDomain lookup with `?id=` query param:**

Old:
```python
    caller = request.values.get("NmsAni", "").strip()
    domain = request.values.get("AccountDomain", "").strip()

    log.info(
        "webresponder: caller=%s domain=%s digits=%s call_id=%s",
        caller,
        domain,
        request.values.get("Digits"),
        request.values.get("OrigCallID"),
    )

    if not caller:
        log.warning("NmsAni missing or empty")
        return xml_error()

    if not domain or not _DOMAIN_RE.match(domain):
        log.warning("invalid or missing AccountDomain: %s", domain)
        return xml_error()

    try:
        domains = load_domains()
    except Exception as exc:
        log.error("failed to load domains.json: %s", exc)
        return xml_error()

    config = domains.get(domain)
    if not config:
        log.warning("domain not configured: %s", domain)
        return xml_error()
```
New:
```python
    caller = request.values.get("NmsAni", "").strip()
    config_id = request.args.get("id", "").strip()

    log.info(
        "webresponder: caller=%s id=%s digits=%s call_id=%s",
        caller,
        config_id,
        request.values.get("Digits"),
        request.values.get("OrigCallID"),
    )

    if not caller:
        log.warning("NmsAni missing or empty")
        return xml_error()

    if not config_id or not _DOMAIN_RE.match(config_id):
        log.warning("invalid or missing id: %s", config_id)
        return xml_error()

    try:
        domains = load_domains()
    except Exception as exc:
        log.error("failed to load domains.json: %s", exc)
        return xml_error()

    config = domains.get(config_id)
    if not config:
        log.warning("id not configured: %s", config_id)
        return xml_error()
```

**5d. `/click-to-call` route — replace `domain` with `id`:**

Old:
```python
    domain = (payload.get("domain") or "").strip()
    raw_phone = (payload.get("phone_number") or "").strip()
    forwarded_for = request.headers.get("X-Forwarded-For", "")
    client_ip = forwarded_for.split(",")[0].strip() if forwarded_for else (request.remote_addr or "")

    log.info("click-to-call: domain=%s phone=%s ip=%s", domain, raw_phone, client_ip)

    if not domain or not _DOMAIN_RE.match(domain):
        return _ctc_error("Unknown domain.", 404)

    try:
        domains = load_domains()
    except Exception as exc:
        log.error("failed to load domains.json: %s", exc)
        return _ctc_error("Service temporarily unavailable.", 500)

    config = domains.get(domain)
    if not config or not config.get("queue"):
        log.warning("click-to-call: domain not configured for queue: %s", domain)
        return _ctc_error("Unknown domain.", 404)
```
New:
```python
    config_id = (payload.get("id") or "").strip()
    raw_phone = (payload.get("phone_number") or "").strip()
    forwarded_for = request.headers.get("X-Forwarded-For", "")
    client_ip = forwarded_for.split(",")[0].strip() if forwarded_for else (request.remote_addr or "")

    log.info("click-to-call: id=%s phone=%s ip=%s", config_id, raw_phone, client_ip)

    if not config_id or not _DOMAIN_RE.match(config_id):
        return _ctc_error("Unknown id.", 404)

    try:
        domains = load_domains()
    except Exception as exc:
        log.error("failed to load domains.json: %s", exc)
        return _ctc_error("Service temporarily unavailable.", 500)

    config = domains.get(config_id)
    if not config or not config.get("queue"):
        log.warning("click-to-call: id not configured for queue: %s", config_id)
        return _ctc_error("Unknown id.", 404)
```

Also update the two rate limiter and log lines below:

Old:
```python
    if not _domain_limiter.allow(f"domain:{domain}"):
        log.warning("click-to-call: rate limit exceeded for domain=%s", domain)
```
New:
```python
    if not _domain_limiter.allow(f"id:{config_id}"):
        log.warning("click-to-call: rate limit exceeded for id=%s", config_id)
```

Old:
```python
        log.info("click-to-call: originated call to %s for domain=%s", phone, domain)
```
New:
```python
        log.info("click-to-call: originated call to %s for id=%s", phone, config_id)
```

**5e. `/widget.js` route — replace `?domain=` with `?id=`:**

Old:
```python
    domain = (request.args.get("domain") or "").strip()

    if not domain or not _DOMAIN_RE.match(domain):
        return Response("// unknown domain", status=404, content_type="application/javascript")

    try:
        domains = load_domains()
    except Exception as exc:
        log.error("failed to load domains.json: %s", exc)
        return Response("// service unavailable", status=500, content_type="application/javascript")

    config = domains.get(domain)
    if not config or not config.get("queue"):
        return Response("// unknown domain", status=404, content_type="application/javascript")

    return Response(_render_widget_js(domain, config), content_type="application/javascript")
```
New:
```python
    config_id = (request.args.get("id") or "").strip()

    if not config_id or not _DOMAIN_RE.match(config_id):
        return Response("// unknown id", status=404, content_type="application/javascript")

    try:
        domains = load_domains()
    except Exception as exc:
        log.error("failed to load domains.json: %s", exc)
        return Response("// service unavailable", status=500, content_type="application/javascript")

    config = domains.get(config_id)
    if not config or not config.get("queue"):
        return Response("// unknown id", status=404, content_type="application/javascript")

    return Response(_render_widget_js(config_id, config), content_type="application/javascript")
```

**5f. `/update-domain` route — replace `domain` with `id`:**

Old:
```python
_UPDATE_REQUIRED_FIELDS = [
    "domain", "api_domain", "queue", "button_text", "button_color", "company_name",
]
```
New:
```python
_UPDATE_REQUIRED_FIELDS = [
    "id", "api_domain", "queue", "button_text", "button_color", "company_name",
]
```

Old:
```python
    domain = payload["domain"].strip()
    if not _DOMAIN_RE.match(domain):
        log.warning("update-domain: invalid domain format: %s", domain)
        return {"status": "error", "message": "Invalid domain format."}, 400

    try:
        domains = load_domains()
    except Exception as exc:
        log.error("update-domain: failed to load domains.json: %s", exc)
        return {"status": "error", "message": "Service temporarily unavailable."}, 500

    existing = domains.get(domain, {})
    ...
    domains[domain] = entry

    try:
        save_domains(domains)
        log.info("update-domain: upserted domain=%s", domain)
        return {"status": "ok", "domain": domain}, 200
```
New:
```python
    config_id = payload["id"].strip()
    if not _DOMAIN_RE.match(config_id):
        log.warning("update-domain: invalid id format: %s", config_id)
        return {"status": "error", "message": "Invalid id format."}, 400

    try:
        domains = load_domains()
    except Exception as exc:
        log.error("update-domain: failed to load domains.json: %s", exc)
        return {"status": "error", "message": "Service temporarily unavailable."}, 500

    existing = domains.get(config_id, {})
    ...
    domains[config_id] = entry

    try:
        save_domains(domains)
        log.info("update-domain: upserted id=%s", config_id)
        return {"status": "ok", "id": config_id}, 200
```

---

**Step 6: Run tests to verify they PASS**

```bash
cd Netsapiens-Scripts/webresponder && venv/bin/pytest -v 2>&1 | tail -10
```
Expected: 47 passed

**Step 7: Commit**

```bash
git add Netsapiens-Scripts/webresponder/app.py \
        Netsapiens-Scripts/webresponder/tests/test_app.py \
        Netsapiens-Scripts/webresponder/tests/test_click_to_call.py \
        Netsapiens-Scripts/webresponder/tests/test_update_domain.py
git commit -m "refactor: replace domain config key with id throughout all routes and tests"
```

---

### Task 2: Update `domains.json` example, push, deploy

**Files:**
- Modify: `Netsapiens-Scripts/webresponder/domains.json`

**Step 1: Update `domains.json` to use new id-based keys**

Replace the contents of `Netsapiens-Scripts/webresponder/domains.json` with:

```json
{
  "pressone.net-sms": {
    "api_domain": "pressone.net",
    "user": "2000",
    "from_number": "19294873999",
    "message": "Our hours are Mon-Fri 9am-5pm. We are located at 123 Main St, Tampa FL 33601."
  },
  "pressone.net-sales": {
    "api_domain": "pressone.net",
    "queue": "QUEUEEXT@pressone.net",
    "branding": {
      "button_text": "Call Us Now",
      "button_color": "#1a73e8",
      "company_name": "PressOne",
      "logo_url": ""
    }
  }
}
```

**Step 2: Run full test suite**

```bash
cd Netsapiens-Scripts/webresponder && venv/bin/pytest -v
```
Expected: 47 passed

**Step 3: Commit and push**

```bash
git add Netsapiens-Scripts/webresponder/domains.json
git commit -m "chore: update domains.json example to use new id-based config keys"
git push
```

**Step 4: Deploy to server**

```bash
# On the server:
cd /opt/ns-scripts && git pull

# Update domains.json on server to new format
# Edit /opt/ns-scripts/ClaudProjects/Netsapiens-Scripts/webresponder/domains.json
# Change key "pressone.net" to "pressone.net-sales" (or whatever id you want)
# and add "api_domain": "pressone.net" field if not present

systemctl restart ns-webresponder
```

**Step 5: Update the widget embed tag and test page**

The test page at `/opt/ns-scripts/test_widget.html` needs updating:
```bash
sed -i 's/widget.js?domain=pressone.net/widget.js?id=pressone.net-sales/' \
  /opt/ns-scripts/test_widget.html
```

Test it:
```bash
curl -s "http://127.0.0.1:5100/widget.js?id=pressone.net-sales" | grep CONFIG_ID
```

**Step 6: Update Zoho Form**

In Zoho Forms, rename the `domain` field to `id` in both the field name and the webhook mapping.

Also update the NS portal webresponder URL to include `?id=pressone.net-sms`:
```
https://ns-scripts.pressone.net/webresponder?id=pressone.net-sms
```