Commit Graph
74 Commits
Author SHA1 Message Date
JamesBhattarai 812a1318e9 Feat: Add community plane
- Implemented relevant ping ping routes, regions, cloudflare, auth as a proper abstract to Core for a community cloud
2026-09-15 05:46:18 +05:45
JamesBhattarai b25c6cd24e Feat: Add cloud plane
- Plane with OIDC users, projects and VDCs with per-VDC roles, SSH key vault, Cloudflare exposure, TLS certificates, and a gateway that forwards allowlisted calls to core with the VDC as tenant_id.
- Also added docker compose for setups
2026-09-02 11:15:25 +05:45
JamesBhattarai ca29ba96e3 Chore: Add xcloudify_shared package
API response envelope, logging, core client, Cloudflare tunnel manager, secret encryption, OIDC token verification, the APP_ENV dev-bypass switch, SSH key routes, and the core gateway allowlist are reusable components
2026-09-01 13:18:08 +05:45
JamesBhattarai efda0f4e8f Chore: Squash core migrations into one baseline 2026-08-31 20:32:01 +05:45
JamesBhattarai e5b2a54db8 Feat: Worker for community nodes
A packaged worker binary and node docker-compose for hardware we don't own, authenticated only with its host secret.
Added build scripts for the binary and NSController image, uninstaller.
2026-08-30 16:41:21 +05:45
JamesBhattarai 2438e0d0e4 Fix: Worker and libvirt reliability
- Give each NIC its own guest PCI slot; a shared 0x05 default made libvirt reject VMs with more than one network
- VM reconcile no longer treats a domain as orphaned when its assigned host row is the same physical machine
2026-08-29 20:06:55 +05:45
JamesBhattarai 118585a465 Feat: Add VM start, stop and restart 2026-08-28 17:10:54 +05:45
JamesBhattarai 9ee533e817 Feat: Pin VM placement to a host and VM retries idempotent
- Pin a workload to one host, which a community lease needs
- A VM's networks, volumes, ports, metadata and GPUs are created once and reused on every retry.
2026-08-27 15:03:11 +05:45
JamesBhattarai cafc2f2383 Fix: Sign VNC console tickets per VM and verify them in the proxy 2026-08-24 10:19:02 +05:45
JamesBhattarai 92d9c8c9d2 Feat: Authenticate workers with a per-host secret
Core accepts XCLOUDIFY_API_KEY for Full-Access or a host's own enrollment secret limited to the routes a worker calls, for its own
data only.
2026-08-23 11:47:47 +05:45
JamesBhattarai 4c83aff64c Chore(Refactor): Replace VDC with an opaque tenant_id and region_id
Workloads, Network, Volume, Pod, Server group usees tenant_id for isolation with region_id for placement.
DNS zones and worker DNS paths are tenant-scoped and rooted at BASE_DOMAIN.
2026-08-22 19:34:23 +05:45
JamesBhattarai 5ea418f86c Feat (Chore): Remove tenancy, identity, certificate API, Cloudflare from core
Universes, projects, VDCs, region access, permissions, SSH keys, Cloudflare and TLS certificates are product concerns and moves to the cloud/community plane.
Core keeps only infrastructure.
2026-08-19 12:52:52 +05:45
JamesBhattarai 7fea4a5bcc Chore: Move services into core
- Move app/, worker/, websocket_server/, migrations/, tests and the services to core/, which becomes a self-contained root
- .gitmodules path for the novnc submodule rewritten by git mv.
- Initial Step to split into core and cloud/community
2026-08-17 22:15:19 +05:45
JamesBhattarai eba78fa92b Chore: removed uis and interfaces
The management panel for cloud (backend/hyperscale/community) will be handled seperatly via React/Typescript frontend (Secured and More convinient).
Cleaning the repo!
2026-08-16 23:02:36 +05:45
JamesBhattarai 7ec2e4d47b Feat: Implement NAT for Private network
Added enable_nat bool for networks to allow nats
Implemented docker network bridge, to allow nat with tenancy seperated as `enable_icc:false`
2026-08-08 17:41:15 +02:00
JamesBhattarai b1442e277d Feat: Zerotier & user-meatadata implementation
- User can now add zerotier id during vm creation which will add the node to zerotier via cloud-config
- VM creation supports population of user-data cloud-config
2026-08-08 17:37:17 +02:00
JamesBhattarai 64368a0cd1 Fix: skip api-key check for CORS OPTIONS Preflight
Preflight requests gate for X-Internal-Xcloudify-API made the browser see a non-2xx preflight
2026-08-07 02:13:23 +02:00
JamesBhattarai 4c49f5defb Feat: Implemented X-Internal-Xcloudify-API 2026-08-06 11:30:30 +02:00
JamesBhattarai 9f15566434 Fix: Added OVS Filter 2026-08-03 23:55:12 +02:00
JamesBhattarai 7642bdd99e Fix: Thread congestion to handle pingpong in worker
Move Libvirt VM creation entirely into the worker thread instead of instantiating it on the main thread, preventing ping-pong responses to/from the master from being blocked.
2026-08-02 16:48:23 +02:00
JamesBhattarai 667e690a2b Fix: removed deletion blocker of NS 2026-07-26 12:11:30 +02:00
JamesBhattarai ba311216e3 Feat: VM reconsilation 2026-07-23 08:21:56 +02:00
JamesBhattarai c9b9a17102 Fix: Failover VM Create <-> DNS Chain
Task was depending on DNS of other failed worker with a serial chain which caused VM Creation to waiting.
2026-07-17 01:48:41 +02:00
JamesBhattarai 124775e29a Feat: added workload hosts failover 2026-07-17 11:50:00 +02:00
JamesBhattarai 96cdf56c2f Fix: Worker Reconnect Attempt
Check the connection status of reconnect and loop retry drop to start
2026-07-14 00:41:56 +02:00
JamesBhattarai a68c1c8347 Feat: Br-wan Flat Networking + Fix DNS/DHCP
Implemented Encapsulation type for Flat
Fix DHCP stateless route for metadata-server
Fix default subnet mask  and gateway default to NSController
2026-07-13 01:11:40 +02:00
JamesBhattarai 4f0d057094 Fix: added missing flask_cors package 2026-07-13 01:11:18 +02:00
JamesBhattarai c486b35e2a Feat: Retry diff Hosts on Workload failure 2026-06-11 13:38:56 +05:45
JamesBhattarai 38f5601945 Feat: Implemented VM Workload Error Store 2026-06-11 11:20:36 +05:45
JamesBhattarai 2fd12eb237 Fix: NS docker-bridge Container
Previously ns controller pod injected by XCF_DEFAULT_OVS_BRIDGE making it ovs-bridge and not attaching docker bridge.
Added DNS (Essential for cloudflared side container)
2026-06-08 15:00:30 +05:45
JamesBhattarai 780bb6019e Feat: added VM allocation retry 2026-06-08 01:48:09 +05:45
JamesBhattarai 2a32013791 FIx: swagger /apisec.json 404 and Chore: vm POST doc + Modularity
Flasgger uses find('---') substring search to detect YAML in docstrings.
Numpy-style section underlines (-------, -----------, etc.) inside route
handler docstrings were triggering YAML parsing on plain English text,
crashing the spec view with a TypeError on every request to /apispec.json.
2026-06-07 15:58:06 +05:45
JamesBhattarai db82e33715 Chore: added workload host tag in scheduling VM 2026-06-07 01:15:02 +05:45
JamesBhattarai 292641fcf0 Fix(Feat): dispatch nscontroller if doesn't exist when scheduling vm 2026-06-01 23:30:28 +05:45
JamesBhattarai 0d3d2f0749 Fix(Feat): VxLAN and Stray ports 2026-06-01 22:11:38 +05:45
JamesBhattarai f7fb5a9daa Fix: Worker_settings overwrite & Watchdog Celery, Scripts
Added watchdog for celery in dev mode to apply changes
Fix Overwritten of worker settings. Priority keys over .env overwrites
Chore: Script fixes of master,worker
2026-06-01 09:38:39 +05:45
JamesBhattarai 446fed5df0 Fix: nscontroller port type casing 2026-05-25 20:59:57 +05:45
JamesBhattarai 8727f26ceb Chore: added release gpu script /vfio 2026-05-25 20:58:04 +05:45
JamesBhattarai 9618f702d2 Chore: hard fail vnc and FIx: port type acryonm mismatch/gpu-release 2026-05-25 20:58:04 +05:45
JamesBhattarai 7b0a195a11 Chore: script changes 2026-05-25 19:07:25 +05:45
JamesBhattarai b7dbcaf315 Fix: remove hardcoded vnc vm port & remove consts 2026-05-25 19:07:25 +05:45
JamesBhattarai 0e59fee995 Fix(Chore): gpu infra, passthrough 2026-05-25 19:00:47 +05:45
JamesBhattarai 71379950bc Chore: Remove unnecessary constants 2026-05-25 19:00:47 +05:45
JamesBhattarai bb9feac770 Chore(VM): Openstack Cloud VM type for Cloud Init 2026-05-24 19:38:40 +05:45
JamesBhattarai f1d47cb3a2 Feat(Fix): Implemented Mac, Network ID for MDS lookup 2026-05-24 19:38:40 +05:45
JamesBhattarai 0cba37d3bf Chore(Fix)-infrautils: remove redundant api prefix 2026-05-24 19:38:40 +05:45
JamesBhattarai e60271d1b9 Fix: make pod_id nullable
Essential for vm creation (as no seperate table for vm)
Exception handle for pod_id enforcer
2026-05-24 17:29:41 +05:45
JamesBhattarai ec106a992e Chore/Fix(network): MDS and Network Dispatch
autostart mds in nscontroller
added missing docker volume/packages for socket mounts for ns
added network id env for nscontroller
add missing module of sdn_helper
error handle when ovs-ofctl is empty
2026-05-24 17:22:01 +05:45
JamesBhattarai cc5f91728a Chore(images): enable format for image's user input 2026-05-24 17:16:10 +05:45
JamesBhattarai 38ddcb5d4d Chore: appropos removed constants
Removed redundant code level constatnt with repo based runtime_urls.py
Added run-master, run-worker.sh scripts for convinient clean, builds
2026-05-23 19:20:18 +05:45
JamesBhattarai 7b685d3580 Chore(Fix): Throw Error when checksum not supplied for volume with image remote/http* 2026-05-23 11:14:50 +05:45
JamesBhattarai bf47406b59 Chore(Fix): added volume http image handler and Removed NorthSouth IP Filter 2026-05-23 02:15:53 +05:45
JamesBhattarai 3ac855cd81 Fix: ovs sdn worker package link + Container-host worker socket 2026-05-23 02:14:27 +05:45
JamesBhattarai a5531e317a Chore(Fix): Worker Docker and Setup
Added docker volume for worker's code
Added pcie and qemu docker package
Added worker_settings local volume bootstrap path
2026-05-23 02:13:05 +05:45
JamesBhattarai 93b3784805 Chore: /ws prefix + pciutils in docker 2026-05-22 09:37:33 +05:45
JamesBhattarai 18b2ec006c Fix(Chore): Infra utils
missing package, docker race state
throw errors in metadata proxy failure in worker
merge all the migrations
add missing field in VmMetadata
2026-05-22 09:34:07 +05:45
JamesBhattarai 80975fb953 Feat: Implemented Metadata Sockets
Added APIs to expose the metadata endpoints (from api server)
Proxy in worker hosts that listen on sockets and forwards metadata requests to API
Proxy in NSController to forward metadata requests via socket to its worker host.
2026-05-19 02:15:11 +00:00
JamesBhattarai 11c0e000fe Feat(Fix): allow same NS IP across hosts, and L2 Flow isolation
Chore: OVS Flow Code Refractor
2026-05-19 02:15:11 +00:00
JamesBhattarai 35106a2ed7 Feat: GPU VM & GPU-Dynamic Passthrough
API to register GPU (worker does at initialization), list all GPUs (to fetch during creation menu)
Schedular to Match with relevant hosts filter
A Table to store GPU models
 A table to map gpu to pcie to  workload and host
Dynamic binding unbinding of vfio, nvidia driver at runtime for passthrough
2026-05-16 03:59:00 +00:00
JamesBhattarai 1aa35b3b7b Feat: implemented templating 2026-05-14 11:25:43 +05:45
JamesBhattarai f4de833cb6 Fix: Redundant add of WorloadHostResources for GPU Discovery Storation per host 2026-05-14 10:12:27 +05:45
JamesBhattarai 405c5a1093 Feat: Implemented LVM API & Debug Volume Fits Image
API can call volumes.type = lvm
worker_settings.json require  "LVM_VG_NAME"
Check/Debug if image fits into  volume size (local/lvm)
2026-05-06 10:19:59 +05:45
JamesBhattarai b9cf5f23ff Feat: Implemented Logging Streams 2026-05-06 01:35:54 +05:45
JamesBhattarai c58454c3f5 Feat: DHCP via NSController w OVSFlow & Fix: NSController Delete Schedulars; Fix: NSController Per Network
Implement NS Controller API for Dispatch & Delete to Per Host Per Network
 Chore: Used_ips filter to show deleted (network.py:106) while creating new
DHCP templated w DNSMasq (Mapped Port to IP in NSController)
Protect NSControllers from Delete during Reconsilation
2026-05-05 23:52:42 +05:45
JamesBhattarai fe888efcb8 Feat(Chore): Implement DHCP, NS-Controller IP Store in DB
Also preper ip passed in param when creating port (essentail for NS-Controller) later use.
2026-05-05 23:52:42 +05:45
JamesBhattarai 5e6b8e74f6 Feat: Implemented SSH Key Management 2026-05-05 23:52:42 +05:45
JamesBhattarai bbdd90bcbf Chore(VM): added Serial Console for Libvirt Debug 2026-04-22 07:54:41 +05:45
JamesBhattarai f7b58c8c1c Fix(Revert): Clash of Port attachment in Libvirt 2026-04-22 07:54:41 +05:45
JamesBhattarai 9485e1bcc2 Feat: Add GPU Scheduling + PCIE Discovery Enabled
Warning! : It uses Static, Hardcoded at libvirt_xml_builder.py

Implemented GPUCapableHosts Filter
Santize GPU Payload from User Input
Add Endpoint to register PCIE Devices
Seed the all regions with pcie filter of AMD and NVIDIA GPU
2026-04-22 07:54:37 +05:45
JamesBhattarai bbf71012ea FIX(api): correct WorkloadHostFixedResource field access in _enrich_host_data
Previous code accessed resource.resource_type and resource.quantity which
never existed on WorkloadHostFixedResource. The correct fields are type,
model, manufacturer and physical_address, matching the model definition
and DB schema since the initial migration.

Bug introduced  in 1731e2c.
2026-04-22 07:53:44 +05:45
JamesBhattarai bb357fcbd1 Feat(Fix): VM Networking, Volume,
VM:
Enabled Filter VM Scheduling by  HasSufficientResources
Implemented to Record Resource Usage by WorkloadResourceUsage; creates new record on basis of resource_type (CPU,RAM)

Volume:
Implemented Local Volume vm spin (Fallback Local and reject other volume types for spinning vm),
Fix: hardcode to image for image backed volume in vm payload :298

Network:
Added error handle for port creation, bridge heirarchy selection (network to stored)
2026-04-22 07:53:37 +05:45
JamesBhattarai 6401cded78 Fix: normalize image location_type from 'Local'/'HTTP' to 'image'/'http'
Worker tasks depends on fixed enums location_type of Image [Ref:worker_tasks/volume.py:188]
2026-04-22 07:46:42 +05:45
JamesBhattarai 3c50a66706 Feat(Compute): Server Groups & Feat(DB): allembic manage.py
Added Affinity/Anti-Affinity Rules for VM & Containers
Allow to see current status, rollback, history of db migrations
2026-04-04 15:53:51 +05:45
JamesBhattarai 99028f40b1 Chore(Feat): added Infra utils
Added Swagger/OpenAPI
Added .env.examples
2026-04-02 19:19:16 +05:45