# HTTP API reference

All consumer endpoints are mounted under `/pulsar/api/`. Auth is the
same `req.user`-populating stack sgapps.io uses for every other service
(session cookie or bearer token). Response envelope is the SGApps
standard `{status: 'ok'|'error', ...}`.

## POST /pulsar/api/fetch

Submit a fetch request. Returns immediately with a `jobId` the caller
polls via `GET /pulsar/api/job/:id`.

### Request body

```json
{
  "url":              "<string, required, http:// or https://>",
  "method":           "GET|HEAD|POST|PUT|DELETE|PATCH|OPTIONS",
  "headers":          { "<k>": "<v>", ... },
  "body":             "<string | null>",
  "bodyBase64":       false,
  "timeoutMs":        10000,
  "maxResponseBytes": 2097152,
  "allowInsecureTLS": false,
  "allowPrivateIP":   false,
  "rangeRequest":     { "start": 0, "end": 65535 },
  "followRedirects":  true
}
```

| Field | Type | Default | Notes |
|---|---|---|---|
| `url` | string | — | **Required.** Scheme ∈ {http, https}. SSRF classifier rejects forbidden hosts + instance-metadata IPs regardless of flags. Private IPs are rejected unless `allowPrivateIP: true`. |
| `method` | string | `"GET"` | Must be in the allow-list above. |
| `headers` | object | `{}` | Browser-forbidden names (Host, Content-Length, Connection, Cookie, Origin, Referer, …) are stripped. CRLF-injecting values are dropped. |
| `body` | string / null | `null` | Only sent for non-GET/HEAD methods. ≤ `requestBodyMax` (default 1 MiB). |
| `bodyBase64` | bool | `false` | Set to `true` if `body` is base64-encoded binary. |
| `timeoutMs` | number | `10000` | Agent-side exec timeout. Capped at `timeoutMsMax` (default 30 s). |
| `maxResponseBytes` | number | `2097152` (2 MiB) | Agent + server both crop at this. Capped at `maxResponseBytesHard` (default 16 MiB). |
| `allowInsecureTLS` | bool | `false` | Lets the agent skip TLS cert validation. Only dispatched to agents whose capability is `accept`. |
| `allowPrivateIP` | bool | `false` | Required to target `127.0.0.0/8`, `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, link-local, CGNAT. |
| `rangeRequest` | object / null | `null` | `{start, end}` byte range. Mapped to `Range: bytes=<start>-<end>` on the outbound. |
| `followRedirects` | bool | `true` | When `false`, redirect responses are passed through to the caller. |

### Response (202, synchronous "accepted")

```json
{
  "status": "ok",
  "data": {
    "jobId":        "<ObjectId>",
    "state":        "pending",
    "retryAfterMs": 500
  }
}
```

### Error responses

| HTTP | `error.code` | When |
|---|---|---|
| 400 | `bad-request` / `bad-url` / `private-ip-blocked` | malformed body, scheme not http/https, private IP without flag |
| 401 | `auth-required` | no session / no bearer |
| 413 | `payload-too-large` | `body` exceeds `requestBodyMax` |
| 500 | `internal-error` | Mongo / server bug |
| 503 | `no-active-agents` | caller has 0 approved + pinged-in-last-60s devices |

## GET /pulsar/api/job/:id

Poll a job. The caller MUST own the job (same user as the submitter).

### Response — pending

```json
{"status":"ok","data":{"state":"pending","retryAfterMs":500}}
```

### Response — dispatched

```json
{"status":"ok","data":{"state":"dispatched","retryAfterMs":1500}}
```

### Response — done

```json
{
  "status": "ok",
  "data": {
    "state": "done",
    "result": {
      "status":           200,
      "statusText":       "OK",
      "headers":          { "<k>": "<v>" },
      "body":             "<string, utf8 OR base64>",
      "bodyBase64":       false,
      "cropped":          false,
      "cropOriginalSize": null,
      "elapsedMs":        182,
      "bytesWritten":     12
    }
  }
}
```

`bodyBase64: true` when the agent detected non-UTF-8 bytes in the first
1 KiB of the response (binary heuristic). Decode as `atob()` or
`Buffer.from(..., 'base64')`.

### Response — failed

```json
{
  "status": "ok",
  "data": {
    "state": "failed",
    "error": {
      "code":    "<see codes below>",
      "message": "<human-readable>"
    }
  }
}
```

Error codes agents may emit:

| Code | Meaning |
|---|---|
| `timeout` | Agent exec timeout (`timeoutMs`) hit before upstream responded |
| `dns` | Could not resolve the hostname |
| `tls` | TLS handshake failed (and `allowInsecureTLS: false`) |
| `refused` | Connection refused |
| `oversize` | Upstream body exceeded `maxResponseBytes` and the agent could not crop safely |
| `fetch-error` | Generic browser `fetch()` failure (CORS blocked, network layer, …) |
| `agent-refused-override` | A job with `allowInsecureTLS: true` or `allowPrivateIP: true` landed on an agent that refuses it. Pulsar may re-dispatch to another agent silently; the job is only marked failed if nobody accepts. |
| `cancelled` | Caller called `DELETE /pulsar/api/job/:id` |

### Response — expired

```json
{"status":"ok","data":{"state":"expired"}}
```

Returned once the row's `resultReadDeadlineAt` is in the past (default
`requestTimeoutMs` = 60 s). The caller stops polling and either submits
again or gives up.

## DELETE /pulsar/api/job/:id

Cancel a pending or dispatched job. The row flips to `state: 'failed'`
with `error: {code: 'cancelled'}`.

### Response

```json
{"status":"ok","data":{"state":"failed"}}
```

404 if the row is already settled or missing.

## Error codes

### Server-side (consumer API)

| Code | HTTP | Context |
|---|---|---|
| `auth-required` | 401 | No `req.user` populated |
| `bad-request` | 400 | malformed body |
| `missing-url` | 400 | `url` not provided |
| `bad-method` | 400 | `method` not in allow-list |
| `body-must-be-string` | 400 | `body` is not a string |
| `bad-url` | 400 | SSRF classifier flagged as unparseable / forbidden |
| `private-ip-blocked` | 400 | SSRF: private IP without `allowPrivateIP` |
| `bad-range` | 400 | `rangeRequest.start`/`end` invalid |
| `payload-too-large` | 413 | request body exceeds `requestBodyMax` |
| `no-active-agents` | 503 | caller has 0 approved + online devices |
| `bad-job-id` | 400 | `:id` not a 24-hex ObjectId |
| `not-found` | 404 | job unknown or not owned by caller |
| `internal-error` | 500 | server bug — check logs |

### Agent-side (via `error.code` in `done`/`failed` response)

See the table under **GET /pulsar/api/job/:id → failed**.