Updated ZeroTier network membership flow and logic

docs/zerotier-device-membership.md

@@ -0,0 +1,165 @@
+# ZeroTier Device Membership
+
+## Overview
+
+This document covers the ZeroTier device membership model — how devices are registered, approved, activated, and deactivated on ZeroTier networks. It explains the schema design, the distinction between `active` and `approved`, the user activation flow, and audit coverage.
+
+## Schema Design
+
+The core model is `NetworkAccessRequest` (table: `network_access_requests`). It replaces the legacy two-table approach (`UserNetworkApproval` + `DeviceNetworkMembership`) with a single per-device, per-network row.
+
+### `network_access_requests` table
+
+| Column | Type | Purpose |
+|---|---|---|
+| `id` | UUID | Primary key |
+| `organization_id` | FK -> organizations | Org scope |
+| `user_id` | FK -> users | Requesting user |
+| `device_id` | FK -> devices | Target device |
+| `portal_network_id` | FK -> portal_networks | Target network |
+| `granted_by_user_id` | FK -> users (nullable) | Approving manager |
+| `grant_type` | enum | `requested` or `assigned` |
+| `status` | enum | `pending`, `approved`, `rejected`, `revoked`, `suspended` |
+| `active` | boolean | Currently authorized on controller? |
+| `justification` | text | User's reason for requesting |
+| `join_seen` | boolean | Controller observed the device join |
+| `deleted_at` | timestamp (nullable) | Soft delete support |
+
+**Unique constraint:** `(user_id, device_id, portal_network_id, deleted_at)` — ensures exactly one active record per device per network.
+
+### Supporting models
+
+| Model | Table | Purpose |
+|---|---|---|
+| `Device` | `devices` | A user-registered ZeroTier node (10-char node ID) |
+| `PortalNetwork` | `portal_networks` | A managed ZeroTier network (scoped to org) |
+| `ActivationSession` | `activation_sessions` | Temporary authorization window (TTL-based) |
+| `ZeroTierMembership` | `zerotier_memberships` | Cache of controller-side state |
+| `KillSwitchEvent` | `kill_switch_events` | Append-only audit of kill switch actions |
+
+### Entity relationships
+
+```
+Device (1) ──> (N) NetworkAccessRequest (N) <── (1) PortalNetwork
+                           │
+                     (1 or 0)
+                           │
+                    ActivationSession
+                           │
+                     (1 or 0)
+                           │
+                   ZeroTierMembership
+```
+
+## `active` vs `approved` — the key distinction
+
+These are orthogonal concepts:
+
+| Concept | Represents | Set by | Persists across sessions? |
+|---|---|---|---|
+| **`status = approved`** | Administrative permission to use the network | Manager approval | Yes — once approved, stays approved until revoked |
+| **`active = True`** | Device is currently authorized on the ZT controller | User activation | No — toggled on/off per session |
+
+A request can be in any of these states:
+
+| `status` | `active` | Meaning |
+|---|---|---|
+| `approved` | `false` | Manager said yes, but user hasn't activated yet |
+| `approved` | `true` | Manager said yes, device is actively connected |
+| `pending` | `false` | Awaiting manager decision |
+| `rejected` / `revoked` / `suspended` | `false` | Access denied or removed |
+
+The `active` flag is **not** a persistent grant — it's a run-time operational state tied to an `ActivationSession` with a finite TTL (default 8 hours).
+
+## User activation flow — "turning on" ZeroTier
+
+```
+User                         API                           Service                    ZT Controller
+ │                            │                              │                            │
+ │  POST /orgs/<id>/approvals │                              │                            │
+ ├───────────────────────────>│ request_access()             │                            │
+ │                            ├──> creates NetworkAccessRequest (status=PENDING)          │
+ │                            ├──> _ensure_zerotier_member()                              │
+ │                            │    └──> provisions member (de-authorized) ────────────────>│
+ │                            │                              │                            │
+ │  <── 201 { status: pending }                              │                            │
+ │                            │                              │                            │
+ │  [Admin approves]          │                              │                            │
+ │  POST /orgs/<id>/approvals │                              │                            │
+ │  /<id>/approve             │                              │                            │
+ ├───────────────────────────>│ approve_request()            │                            │
+ │                            ├──> sets status=APPROVED      │                            │
+ │  <── 200 { status: approved }                             │                            │
+ │                            │                              │                            │
+ │  POST /orgs/<id>/memberships/<id>/activate                │                            │
+ │  ─────────────────────────>│ activate_request()           │                            │
+ │    "turn on ZeroTier"      ├──> creates ActivationSession (TTL=8h)                    │
+ │                            ├──> sets request.active=True  │                            │
+ │                            ├──> _authorize_in_zerotier()  │                            │
+ │                            │    └──> authorizes member ───────────────────────────────>│
+ │                            │                              │                            │
+ │  <── 200 { active: true, session: {...} }                 │                            │
+```
+
+### Endpoints
+
+| Method | Path | Action |
+|---|---|---|
+| `POST` | `/organizations/<id>/devices` | Register a device (prerequisite) |
+| `POST` | `/organizations/<id>/approvals` | Request network access |
+| `POST` | `/organizations/<id>/approvals/<id>/approve` | Admin approves |
+| `POST` | `/organizations/<id>/memberships/<id>/activate` | **Turn on ZeroTier** |
+| `POST` | `/organizations/<id>/memberships/activate-all` | Bulk-activate all approved |
+| `POST` | `/organizations/<id>/memberships/<id>/deactivate` | Turn off ZeroTier |
+| `POST` | `/organizations/<id>/devices/<id>/join-network/<id>` | Direct join (open networks) |
+| `POST` | `/organizations/<id>/kill-switch` | Emergency deactivation |
+
+## Session expiry and audit
+
+When the `ActivationSession` TTL expires, the reconciliation worker handles it:
+
+```
+Reconciliation worker (runs every 2 min)
+    │
+    ├── reconcile_expired_activations()
+    │    └── for each expired ActivationSession:
+    │         ├── _deauthorize_in_zerotier() ───> ZT controller de-authorizes member
+    │         ├── sets request.active = False
+    │         └── logs audit event
+    │
+    └── reconcile_all()
+         └── for each network:
+              ├── sync ZeroTierMembership cache
+              └── detect/repair drift (portal vs controller state mismatch)
+```
+
+Every authorization state change is audited:
+
+| Event | When |
+|---|---|
+| `zt.approval.requested` | User requests access |
+| `zt.approval.granted` | Manager approves/assigns |
+| `zt.approval.rejected` | Manager rejects |
+| `zt.approval.revoked` | Manager revokes |
+| `zt.membership.activated` | User activates (session created) |
+| `zt.membership.deactivated` | User deactivates (session ended) |
+| `zt.member.authorized` | ZT controller authorize call succeeds |
+| `zt.member.deauthorized` | ZT controller de-authorize call succeeds |
+| `zt.activation.expired` | Session expired by reconciliation worker |
+| `zt.kill_switch.activated` | Admin triggers kill switch |
+
+All audit entries are stored in the `audit_logs` table with `organization_id`, `user_id`, `resource_type`, `resource_id`, `ip_address`, and `extra_data` (JSON) for full traceability.
+
+## Key source files
+
+| File | Purpose |
+|---|---|
+| `gatehouse_app/models/zerotier/network_access_request.py` | `NetworkAccessRequest` model |
+| `gatehouse_app/models/zerotier/activation_session.py` | `ActivationSession` model |
+| `gatehouse_app/models/zerotier/zerotier_membership.py` | `ZeroTierMembership` model |
+| `gatehouse_app/services/network_access_service.py` | Core business logic |
+| `gatehouse_app/services/zerotier_reconciliation_service.py` | Reconciliation worker logic |
+| `gatehouse_app/api/v1/zerotier.py` | API endpoints |
+| `gatehouse_app/utils/constants.py` | Audit action enums |
+| `gatehouse_app/jobs/zerotier_reconciliation_job.py` | Scheduled job entry point |
+| `migrations/versions/merge_approval_membership_tables.py` | Schema migration |

gatehouse_app/api/v1/zerotier.py

@@ -924,16 +924,25 @@ def admin_list_memberships(org_id):
 @require_admin
 @full_access_required
 def admin_delete_membership(org_id, membership_id):
-    """Hard-delete a membership and remove it from ZeroTier (admin only)."""
+    """Force-delete a membership and remove it from ZeroTier (admin only).
+
+    Handles the full lifecycle: deactivates if active, removes the member
+    from the ZeroTier controller, and hard-deletes the DB record.
+    """
     org, err = _org_check(org_id)
     if err:
         return err
 
     try:
-        network_access_service.hard_delete_request(membership_id)
+        network_access_service.admin_force_delete_request(
+            membership_id,
+            admin_user_id=g.current_user.id,
+        )
         return api_response(message="Request permanently deleted")
     except ApprovalNotFoundError as e:
         return api_response(success=False, message=str(e), status=404, error_type=e.error_type)
+    except AppValidationError as e:
+        return api_response(success=False, message=str(e.message), status=400, error_type=e.error_type)
 
 
 # ── ZeroTier Controller ───────────────────────────────────────────────────────

gatehouse_app/services/network_access_service.py

@@ -760,7 +760,7 @@ def join_network_for_device(
     ).first()
 
     if existing:
-        if existing.status in (ApprovalState.APPROVED, ApprovalState.PENDING):
+        if existing.status == ApprovalState.PENDING or (existing.status == ApprovalState.APPROVED and existing.active):
             raise ApprovalAlreadyExistsError("Already have access or pending request.")
         # Re-open
         existing.status = ApprovalState.APPROVED
@@ -861,3 +861,62 @@ def hard_delete_request(
 
     db.session.delete(request)
     db.session.commit()
+
+
+def admin_force_delete_request(
+    request_id: str,
+    admin_user_id: str,
+) -> None:
+    """Force-delete a network access request (admin only).
+
+    Handles the full lifecycle: deactivates if active, removes the member
+    from the ZeroTier controller entirely, then hard-deletes the DB record.
+    Does NOT require the request to be soft-deleted first.
+    """
+    request = NetworkAccessRequest.query.filter(
+        NetworkAccessRequest.id == request_id,
+    ).first()
+    if not request:
+        raise ApprovalNotFoundError(f"Request {request_id} not found.")
+
+    # Deactivate if active
+    if request.active:
+        deactivate_request(
+            request_id,
+            reason="manual_revoke",
+            deactivated_by_user_id=admin_user_id,
+        )
+
+    # Remove from ZeroTier controller entirely
+    device = Device.query.get(request.device_id)
+    network = PortalNetwork.query.get(request.portal_network_id)
+    if device and network:
+        try:
+            zt.delete_network_member(
+                network.zerotier_network_id,
+                device.node_id,
+                organization_id=request.organization_id,
+            )
+        except Exception as exc:
+            logger.warning(
+                f"[admin_force_delete] Could not remove {device.node_id} "
+                f"from ZT network {network.zerotier_network_id}: {exc}"
+            )
+
+    # Hard-delete from DB
+    db.session.delete(request)
+    db.session.commit()
+
+    AuditService.log_action(
+        action=AuditAction.ZT_REQUEST_REVOKED,
+        user_id=admin_user_id,
+        organization_id=request.organization_id,
+        resource_type="network_access_request",
+        resource_id=request.id,
+        metadata={
+            "target_user_id": request.user_id,
+            "force_delete": True,
+        },
+        description=f"Network access request force-deleted by admin for user {request.user_id}",
+        success=True,
+    )

gatehouse_app/services/portal_network_service.py

@@ -304,16 +304,15 @@ def delete_network(network_id: str, user_id: str) -> None:
 
 
 def get_network_members(network_id: str) -> list:
-    """Return all approved and active NetworkAccessRequests for a network."""
+    """Return all approved NetworkAccessRequests for a network (active or inactive)."""
     from gatehouse_app.models import NetworkAccessRequest
     from gatehouse_app.utils.constants import ApprovalState
 
     return NetworkAccessRequest.query.filter(
         NetworkAccessRequest.portal_network_id == network_id,
         NetworkAccessRequest.status == ApprovalState.APPROVED,
-        NetworkAccessRequest.active == True,
         NetworkAccessRequest.deleted_at.is_(None),
-    ).all()
+    ).order_by(NetworkAccessRequest.created_at.desc()).all()
 
 
 def get_network_pending_requests(network_id: str) -> list:

tests/integration/test_zerotier.py

@@ -203,6 +203,80 @@ class TestZeroTierMembership:
             assert exc.status_code in (400, 500)
 
 
+class TestZeroTierJoinNetwork:
+    """Test joining a network with a registered device."""
+
+    @patch("gatehouse_app.services.network_access_service._ensure_zerotier_member")
+    def test_rejoin_after_deactivation_positive(
+        self,
+        mock_ensure_member,
+        integration_client,
+        create_test_user,
+        create_test_org,
+        create_test_membership,
+        integration_app,
+    ):
+        """TEST: ZT-15 — Re-join network after deactivation.
+
+        WHAT:  User with deactivated membership (APPROVED + active=False)
+               POSTs to join-network for the same device and network.
+        WHY:   Deactivated memberships should not block re-join attempts.
+        EXPECTED: 201 Created (re-opens existing request), not 409 Conflict.
+        """
+        from gatehouse_app.models.zerotier.device import Device
+        from gatehouse_app.models.zerotier.portal_network import PortalNetwork
+        from gatehouse_app.models.zerotier.network_access_request import NetworkAccessRequest
+        from gatehouse_app.extensions import db as _db
+        from gatehouse_app.utils.constants import ApprovalState, NetworkRequestMode
+
+        user = create_test_user(password="MyPassword123!")
+        org = create_test_org()
+        create_test_membership(user["id"], org["id"], OrganizationRole.MEMBER)
+
+        with integration_app.app_context():
+            device = Device(
+                user_id=user["id"],
+                organization_id=org["id"],
+                node_id="1234567890",
+                device_nickname="Test Device",
+                hostname="test-device",
+            )
+            _db.session.add(device)
+            _db.session.flush()
+            device_id = device.id
+
+            portal_network = PortalNetwork(
+                organization_id=org["id"],
+                name="Test Network",
+                owner_user_id=user["id"],
+                zerotier_network_id="a84ac5c10a6e4c7e",
+                request_mode=NetworkRequestMode.OPEN,
+            )
+            _db.session.add(portal_network)
+            _db.session.flush()
+            portal_network_id = portal_network.id
+
+            deactivated_request = NetworkAccessRequest(
+                organization_id=org["id"],
+                user_id=user["id"],
+                device_id=device_id,
+                portal_network_id=portal_network_id,
+                status=ApprovalState.APPROVED,
+                active=False,
+            )
+            _db.session.add(deactivated_request)
+            _db.session.commit()
+
+        integration_client.auth.login(email=user["email"], password="MyPassword123!")
+        result = integration_client.post(
+            f"/organizations/{org['id']}/devices/{device_id}/join-network/{portal_network_id}",
+        )
+        data = assert_success(result, "joined network successfully")
+        assert result.get("code") == 201
+        assert "membership" in data
+        mock_ensure_member.assert_called_once()
+
+
 class TestAdminUserDevices:
     """Test admin endpoint to list devices for a specific user."""
 
@@ -343,3 +417,215 @@ class TestAdminUserDevices:
         with pytest.raises(ApiError) as exc_info:
             integration_client.get(f"/organizations/{org['id']}/users/{non_existent_id}/devices")
         assert exc_info.value.status_code == 404
+
+
+class TestAdminForceDeleteMembership:
+    """Test admin force-delete of network access requests."""
+
+    @patch("gatehouse_app.services.zerotier_api_service.deauthorize_member")
+    @patch("gatehouse_app.services.zerotier_api_service.delete_network_member")
+    def test_force_delete_active_membership_positive(
+        self,
+        mock_delete_member,
+        mock_deauthorize_member,
+        integration_client,
+        create_test_user,
+        create_test_org,
+        create_test_membership,
+        integration_app,
+    ):
+        """TEST: ZT-16 — Admin force-deletes an active membership.
+
+        WHAT:  Admin calls DELETE admin/memberships/<id> on an active,
+               non-soft-deleted request. Should deactivate, remove from ZT,
+               and hard-delete the DB record in one step.
+        EXPECTED: 200 OK, request no longer exists in DB.
+        """
+        from gatehouse_app.models.zerotier.device import Device
+        from gatehouse_app.models.zerotier.portal_network import PortalNetwork
+        from gatehouse_app.models.zerotier.network_access_request import NetworkAccessRequest
+        from gatehouse_app.extensions import db as _db
+        from gatehouse_app.utils.constants import ApprovalState, NetworkRequestMode
+
+        admin = create_test_user(password="AdminPass123!")
+        user = create_test_user(password="UserPass123!")
+        org = create_test_org()
+        create_test_membership(admin["id"], org["id"], OrganizationRole.ADMIN)
+        create_test_membership(user["id"], org["id"], OrganizationRole.MEMBER)
+
+        with integration_app.app_context():
+            device = Device(
+                user_id=user["id"],
+                organization_id=org["id"],
+                node_id="1234567890",
+                device_nickname="Test Device",
+                hostname="test-device",
+            )
+            _db.session.add(device)
+            _db.session.flush()
+            device_id = device.id
+
+            portal_network = PortalNetwork(
+                organization_id=org["id"],
+                name="Test Network",
+                owner_user_id=admin["id"],
+                zerotier_network_id="a84ac5c10a6e4c7e",
+                request_mode=NetworkRequestMode.OPEN,
+            )
+            _db.session.add(portal_network)
+            _db.session.flush()
+            portal_network_id = portal_network.id
+
+            request = NetworkAccessRequest(
+                organization_id=org["id"],
+                user_id=user["id"],
+                device_id=device_id,
+                portal_network_id=portal_network_id,
+                status=ApprovalState.APPROVED,
+                active=True,
+            )
+            _db.session.add(request)
+            _db.session.commit()
+            request_id = request.id
+
+        integration_client.auth.login(email=admin["email"], password="AdminPass123!")
+        result = integration_client.delete(
+            f"/organizations/{org['id']}/admin/memberships/{request_id}",
+        )
+        assert_success(result, "permanently deleted")
+
+        # Verify request is gone from DB
+        with integration_app.app_context():
+            deleted = NetworkAccessRequest.query.get(request_id)
+            assert deleted is None
+
+        mock_deauthorize_member.assert_called_once()
+        mock_delete_member.assert_called_once()
+
+    @patch("gatehouse_app.services.zerotier_api_service.delete_network_member")
+    def test_force_delete_soft_deleted_membership_positive(
+        self,
+        mock_delete_member,
+        integration_client,
+        create_test_user,
+        create_test_org,
+        create_test_membership,
+        integration_app,
+    ):
+        """TEST: ZT-17 — Admin force-deletes an already-soft-deleted membership.
+
+        WHAT:  Admin calls DELETE admin/memberships/<id> on a soft-deleted
+               request. Should still work (remove from ZT, hard-delete).
+        EXPECTED: 200 OK.
+        """
+        from datetime import datetime, timezone
+        from gatehouse_app.models.zerotier.device import Device
+        from gatehouse_app.models.zerotier.portal_network import PortalNetwork
+        from gatehouse_app.models.zerotier.network_access_request import NetworkAccessRequest
+        from gatehouse_app.extensions import db as _db
+        from gatehouse_app.utils.constants import ApprovalState, NetworkRequestMode
+
+        admin = create_test_user(password="AdminPass123!")
+        user = create_test_user(password="UserPass123!")
+        org = create_test_org()
+        create_test_membership(admin["id"], org["id"], OrganizationRole.ADMIN)
+        create_test_membership(user["id"], org["id"], OrganizationRole.MEMBER)
+
+        with integration_app.app_context():
+            device = Device(
+                user_id=user["id"],
+                organization_id=org["id"],
+                node_id="1234567890",
+                device_nickname="Test Device",
+                hostname="test-device",
+            )
+            _db.session.add(device)
+            _db.session.flush()
+            device_id = device.id
+
+            portal_network = PortalNetwork(
+                organization_id=org["id"],
+                name="Test Network",
+                owner_user_id=admin["id"],
+                zerotier_network_id="a84ac5c10a6e4c7e",
+                request_mode=NetworkRequestMode.OPEN,
+            )
+            _db.session.add(portal_network)
+            _db.session.flush()
+            portal_network_id = portal_network.id
+
+            request = NetworkAccessRequest(
+                organization_id=org["id"],
+                user_id=user["id"],
+                device_id=device_id,
+                portal_network_id=portal_network_id,
+                status=ApprovalState.APPROVED,
+                active=False,
+                deleted_at=datetime.now(timezone.utc),
+            )
+            _db.session.add(request)
+            _db.session.commit()
+            request_id = request.id
+
+        integration_client.auth.login(email=admin["email"], password="AdminPass123!")
+        result = integration_client.delete(
+            f"/organizations/{org['id']}/admin/memberships/{request_id}",
+        )
+        assert_success(result, "permanently deleted")
+
+        # Verify request is gone from DB
+        with integration_app.app_context():
+            deleted = NetworkAccessRequest.query.get(request_id)
+            assert deleted is None
+
+        mock_delete_member.assert_called_once()
+
+    def test_force_delete_non_existent_membership_negative(
+        self,
+        integration_client,
+        create_test_user,
+        create_test_org,
+        create_test_membership,
+    ):
+        """TEST: ZT-18 — Admin force-deletes a non-existent membership.
+
+        WHAT:  Admin calls DELETE admin/memberships/<non_existent_id>.
+        EXPECTED: 404 Not Found.
+        """
+        import uuid
+
+        admin = create_test_user(password="AdminPass123!")
+        org = create_test_org()
+        create_test_membership(admin["id"], org["id"], OrganizationRole.ADMIN)
+
+        integration_client.auth.login(email=admin["email"], password="AdminPass123!")
+        with pytest.raises(ApiError) as exc_info:
+            integration_client.delete(
+                f"/organizations/{org['id']}/admin/memberships/{uuid.uuid4()}",
+            )
+        assert exc_info.value.status_code == 404
+
+    def test_force_delete_non_admin_negative(
+        self,
+        integration_client,
+        create_test_user,
+        create_test_org,
+        create_test_membership,
+    ):
+        """TEST: ZT-19 — Non-admin cannot force-delete a membership.
+
+        WHAT:  A MEMBER calls DELETE admin/memberships/<id>.
+        EXPECTED: 403 Forbidden.
+        """
+        import uuid
+
+        member = create_test_user(password="MemberPass123!")
+        org = create_test_org()
+        create_test_membership(member["id"], org["id"], OrganizationRole.MEMBER)
+
+        integration_client.auth.login(email=member["email"], password="MemberPass123!")
+        with pytest.raises(ApiError) as exc_info:
+            integration_client.delete(
+                f"/organizations/{org['id']}/admin/memberships/{uuid.uuid4()}",
+            )
+        assert exc_info.value.status_code == 403