gatehouse-api/docs/cli-testing.md

# CLI Integration Tests

This document describes the integration tests for `client/gatehouse-cli.py`.

## Approach

The CLI is tested as a **real subprocess** (not mocked functions) against a **real Flask HTTP server** with an in-memory/file-based SQLite database. Each test is fully self-contained, using UUID-based names to avoid collisions on the shared database.

Key decisions:

- **Subprocess + real server**: The CLI is invoked via `subprocess.run([sys.executable, "client/gatehouse-cli.py", ...])` with `SIGN_URL` and `HOME` env vars pointing to the test server and an isolated temp directory. This catches real-world issues like argparse mismatches, import errors, and API contract drift.
- **Module-scoped server**: A single Flask server (random port, `sqlite:///<tempfile>`) is started once per module via a `cli_server` fixture and shared across all tests. Data isolation is achieved by UUID-scoped names rather than per-test databases.
- **Real crypto**: Certificate-signing tests use real Ed25519 keypairs generated by `ssh-keygen`. The `sshkey_tools` library validates the public key, so fake-looking keys (random hex / padding) are rejected by the server.
- **DB-level bypasses**: For cert-signing tests, `_mark_key_verified()` sets `verified=True` directly in the database (same pattern as `test_ssh_workflows.py`), avoiding the complex `ssh-keygen -Y sign` flow.

## Output Handling

The CLI emits all messages via the `coloredlogs` library, which writes to **stderr**, not stdout. Occasional `print()` calls go to stdout. The `_output()` helper combines both streams:

```python
def _output(result: subprocess.CompletedProcess) -> str:
    return result.stdout + result.stderr
```

All text assertions use `in _output(result)`.

## Fixtures

| Fixture | Scope | Description |
|---------|-------|-------------|
| `cli_server` | module | Flask server on `127.0.0.1` with random port; yields `(server_url, app)` |
| `home_dir` | function | Isolated temp directory for `~/.gatehouse` and `~/.ssh` |

## Test Inventory (10 tests)

### 1. `test_cli_add_key`
**Flags:** `-a -k <pubkey>`

Generates a real Ed25519 keypair with `ssh-keygen`, registers a user, caches the token, then runs the CLI to add the key. Verifies the key-and-verify flow produces an "added successfully" message. Skips if `ssh-keygen` is not available.

### 2. `test_cli_list_keys`
**Flags:** `--list-keys`

Adds two fake SSH keys via the API, then runs the CLI to list them. Asserts both key ID prefixes appear in the output.

### 3. `test_cli_remove_key`
**Flags:** `--remove-key <id>`

Adds a key via the API, then runs the CLI to delete it by ID. Asserts "removed successfully" in the output.

### 4. `test_cli_list_orgs`
**Flags:** `--list-orgs`

Creates two organizations where the test user is an owner, then runs the CLI to list them. Asserts both org names appear in the output.

### 5. `test_cli_request_cert_single_org`
**Flags:** `-r`

Sets up a single organization with a user-type CA, a principal ("deploy"), a principal membership for the user, and a verified real SSH key. Runs the CLI to request a certificate. Asserts "Certificate signed successfully" and the principal name appear in the output.

### 6. `test_cli_request_cert_multi_org_no_org`
**Flags:** `-r` (with multiple orgs)

Creates two organizations, each with a CA and one with a principal membership. Runs `-r` without `--org-id`. Asserts exit code 1 and a "multiple organizations" error message.

### 7. `test_cli_request_cert_multi_org_with_org`
**Flags:** `-r --org-id <id>` (with multiple orgs)

Same setup as #6 but passes `--org-id` to disambiguate. Asserts exit code 0 and "Certificate signed successfully".

### 8. `test_cli_install_known_hosts`
**Flags:** `--install-known-hosts`

Creates a host-type CA, then runs the CLI to install the Host CA public key. Asserts the success message and verifies the public key was written to `~/.ssh/known_hosts`.

### 9. `test_cli_clear_cache`
**Flags:** `--clear-cache`

Writes a dummy token cache file, then runs the CLI to clear it. Asserts the cache file is deleted and "Cached token removed" appears in output. The server URL is irrelevant (no network calls).

### 10. `test_cli_check_cert`
**Flags:** `-c`

Runs the CLI to check for a certificate file that does not exist (no prior setup needed). Asserts exit code 1 and "Certificate does not exist" in output. The server URL is irrelevant (no network calls).

## Running

```bash
# Full suite
pytest tests/integration/test_cli.py -v

# Single test
pytest tests/integration/test_cli.py::TestCLI::test_cli_request_cert_single_org -xvs

# Without coverage noise
pytest tests/integration/test_cli.py --no-cov
```

## Notes

- The `_gen_pubkey()` helper generates fake-looking keys (random hex + base64 padding) that are accepted by the API for storage but **rejected** by `sshkey_tools.PublicKey.from_string()` during cert signing. Use `_real_pubkey()` (which invokes actual `ssh-keygen`) for tests that need cryptographically valid keys.
- `_add_principal_member()` must be called whenever a user needs access to a principal. Even org owners are not automatically members — the CLI's `fetch_my_principals()` reads `my_principals`, not `all_principals`.
- The `cli_server` fixture creates tables once (`db.create_all()`) and reuses the database file across all tests. UUID-suffixed names (via `_email()`, `uuid.uuid4().hex`) prevent collisions.