gatehouse-api/docs/cli-testing.md

CLI Integration Tests

This document describes the integration tests for client/gatehouse-cli.py.

Approach

The CLI is tested as a real subprocess (not mocked functions) against a real Flask HTTP server with an in-memory/file-based SQLite database. Each test is fully self-contained, using UUID-based names to avoid collisions on the shared database.

Key decisions:

• Subprocess + real server: The CLI is invoked via subprocess.run([sys.executable, "client/gatehouse-cli.py", ...]) with SIGN_URL and HOME env vars pointing to the test server and an isolated temp directory. This catches real-world issues like argparse mismatches, import errors, and API contract drift.
• Module-scoped server: A single Flask server (random port, sqlite:///<tempfile>) is started once per module via a cli_server fixture and shared across all tests. Data isolation is achieved by UUID-scoped names rather than per-test databases.
• Real crypto: Certificate-signing tests use real Ed25519 keypairs generated by ssh-keygen. The sshkey_tools library validates the public key, so fake-looking keys (random hex / padding) are rejected by the server.
• DB-level bypasses: For cert-signing tests, _mark_key_verified() sets verified=True directly in the database (same pattern as test_ssh_workflows.py), avoiding the complex ssh-keygen -Y sign flow.

Output Handling

The CLI emits all messages via the coloredlogs library, which writes to stderr, not stdout. Occasional print() calls go to stdout. The _output() helper combines both streams:

def _output(result: subprocess.CompletedProcess) -> str:
    return result.stdout + result.stderr

All text assertions use in _output(result).

Fixtures

Fixture	Scope	Description
cli_server	module	Flask server on 127.0.0.1 with random port; yields (server_url, app)
home_dir	function	Isolated temp directory for ~/.gatehouse and ~/.ssh

Test Inventory (10 tests)

1. test_cli_add_key

Flags: -a -k <pubkey>

Generates a real Ed25519 keypair with ssh-keygen, registers a user, caches the token, then runs the CLI to add the key. Verifies the key-and-verify flow produces an "added successfully" message. Skips if ssh-keygen is not available.

2. test_cli_list_keys

Flags: --list-keys

Adds two fake SSH keys via the API, then runs the CLI to list them. Asserts both key ID prefixes appear in the output.

3. test_cli_remove_key

Flags: --remove-key <id>

Adds a key via the API, then runs the CLI to delete it by ID. Asserts "removed successfully" in the output.

4. test_cli_list_orgs

Flags: --list-orgs

Creates two organizations where the test user is an owner, then runs the CLI to list them. Asserts both org names appear in the output.

5. test_cli_request_cert_single_org

Flags: -r

Sets up a single organization with a user-type CA, a principal ("deploy"), a principal membership for the user, and a verified real SSH key. Runs the CLI to request a certificate. Asserts "Certificate signed successfully" and the principal name appear in the output.

6. test_cli_request_cert_multi_org_no_org

Flags: -r (with multiple orgs)

Creates two organizations, each with a CA and one with a principal membership. Runs -r without --org-id. Asserts exit code 1 and a "multiple organizations" error message.

7. test_cli_request_cert_multi_org_with_org

Flags: -r --org-id <id> (with multiple orgs)

Same setup as #6 but passes --org-id to disambiguate. Asserts exit code 0 and "Certificate signed successfully".

8. test_cli_install_known_hosts

Flags: --install-known-hosts

Creates a host-type CA, then runs the CLI to install the Host CA public key. Asserts the success message and verifies the public key was written to ~/.ssh/known_hosts.

9. test_cli_clear_cache

Flags: --clear-cache

Writes a dummy token cache file, then runs the CLI to clear it. Asserts the cache file is deleted and "Cached token removed" appears in output. The server URL is irrelevant (no network calls).

10. test_cli_check_cert

Flags: -c

Runs the CLI to check for a certificate file that does not exist (no prior setup needed). Asserts exit code 1 and "Certificate does not exist" in output. The server URL is irrelevant (no network calls).

Running

# Full suite
pytest tests/integration/test_cli.py -v

# Single test
pytest tests/integration/test_cli.py::TestCLI::test_cli_request_cert_single_org -xvs

# Without coverage noise
pytest tests/integration/test_cli.py --no-cov

Notes

• The _gen_pubkey() helper generates fake-looking keys (random hex + base64 padding) that are accepted by the API for storage but rejected by sshkey_tools.PublicKey.from_string() during cert signing. Use _real_pubkey() (which invokes actual ssh-keygen) for tests that need cryptographically valid keys.
• _add_principal_member() must be called whenever a user needs access to a principal. Even org owners are not automatically members — the CLI's fetch_my_principals() reads my_principals, not all_principals.
• The cli_server fixture creates tables once (db.create_all()) and reuses the database file across all tests. UUID-suffixed names (via _email(), uuid.uuid4().hex) prevent collisions.