Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ scripts/utm/images/

# IDE and editor
.cursor/
.refs/

# Build artifacts
api
18 changes: 18 additions & 0 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,8 @@ Common settings:
| `acme.dns_provider` | DNS provider for ACME challenges | _(empty)_ |
| `acme.cloudflare_api_token` | Cloudflare API token | _(empty)_ |
| `build.docker_socket` | Path to Docker socket | `/var/run/docker.sock` |
| `hypervisor.default` | Default hypervisor type (`cloud-hypervisor`, `firecracker`, `qemu`, `vz`) | `cloud-hypervisor` |
| `hypervisor.firecracker_binary_path` | Optional runtime path to external Firecracker binary | _(empty = embedded)_ |

Environment variables can also override any config key using `__` as the nesting separator (e.g. `CADDY__LISTEN_ADDRESS` overrides `caddy.listen_address`).

Expand Down Expand Up @@ -256,6 +258,22 @@ make dev

The server will start on port 8080 (configurable via `port` in config.yaml).

### Shared-Machine Local Config

When developing on a shared host, avoid global paths like `/etc/hypeman` and `/var/lib/hypeman`.
Use a workspace-local config and data directory instead:

```bash
mkdir -p .tmp/hypeman-data
cp config.example.yaml .tmp/hypeman.config.yaml

# Edit .tmp/hypeman.config.yaml:
# data_dir: /absolute/path/to/your/repo/.tmp/hypeman-data
# jwt_secret: dev-secret

CONFIG_PATH="$(pwd)/.tmp/hypeman.config.yaml" make dev
```

### Setting Up the Builder Image (for Dockerfile builds)

The builder image is required for `hypeman build` to work. When `build.builder_image` is unset or empty, the server will automatically build and push the builder image on startup using Docker. This is the easiest way to get started — just ensure Docker is available and run `make dev`. If a build is requested while the builder image is still being prepared, the server returns a clear error asking you to retry shortly.
Expand Down
54 changes: 45 additions & 9 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
SHELL := /bin/bash
.PHONY: oapi-generate generate-vmm-client generate-wire generate-all dev build build-linux test test-linux test-darwin install-tools gen-jwt download-ch-binaries download-ch-spec ensure-ch-binaries build-caddy-binaries build-caddy ensure-caddy-binaries release-prep clean build-embedded
.PHONY: oapi-generate generate-vmm-client generate-wire generate-all dev build build-linux test test-linux test-darwin install-tools gen-jwt download-ch-binaries download-firecracker-binaries download-ch-spec ensure-ch-binaries ensure-firecracker-binaries build-caddy-binaries build-caddy ensure-caddy-binaries release-prep clean build-embedded

# Directory where local binaries will be installed
BIN_DIR ?= $(CURDIR)/bin
Expand All @@ -13,6 +13,7 @@ OAPI_CODEGEN_VERSION ?= v2.5.1
AIR ?= $(BIN_DIR)/air
WIRE ?= $(BIN_DIR)/wire
XCADDY ?= $(BIN_DIR)/xcaddy
TEST_TIMEOUT ?= 600s

# Install oapi-codegen
$(OAPI_CODEGEN): | $(BIN_DIR)
Expand Down Expand Up @@ -50,6 +51,24 @@ download-ch-binaries:
@chmod +x lib/vmm/binaries/cloud-hypervisor/v*/*/cloud-hypervisor
@echo "Binaries downloaded successfully"

# Firecracker version to embed
FIRECRACKER_VERSION := v1.14.2
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-existent Firecracker version v1.14.2 referenced

High Severity

FIRECRACKER_VERSION is set to v1.14.2, but this release does not exist on the Firecracker GitHub releases page (the latest is v1.14.1). The download-firecracker-binaries target will fail with a 404 when curl tries to fetch the tarball, breaking build-linux, dev-linux, test-linux, and release-prep.

Additional Locations (1)

Fix in Cursor Fix in Web


# Download Firecracker binaries
download-firecracker-binaries:
@echo "Downloading Firecracker binaries..."
@mkdir -p lib/hypervisor/firecracker/binaries/firecracker/$(FIRECRACKER_VERSION)/{x86_64,aarch64}
@echo "Downloading $(FIRECRACKER_VERSION) for x86_64..."
@curl -L "https://github.com/firecracker-microvm/firecracker/releases/download/$(FIRECRACKER_VERSION)/firecracker-$(FIRECRACKER_VERSION)-x86_64.tgz" \
| tar -xzO "release-$(FIRECRACKER_VERSION)-x86_64/firecracker-$(FIRECRACKER_VERSION)-x86_64" \
> lib/hypervisor/firecracker/binaries/firecracker/$(FIRECRACKER_VERSION)/x86_64/firecracker
@echo "Downloading $(FIRECRACKER_VERSION) for aarch64..."
@curl -L "https://github.com/firecracker-microvm/firecracker/releases/download/$(FIRECRACKER_VERSION)/firecracker-$(FIRECRACKER_VERSION)-aarch64.tgz" \
| tar -xzO "release-$(FIRECRACKER_VERSION)-aarch64/firecracker-$(FIRECRACKER_VERSION)-aarch64" \
> lib/hypervisor/firecracker/binaries/firecracker/$(FIRECRACKER_VERSION)/aarch64/firecracker
@chmod +x lib/hypervisor/firecracker/binaries/firecracker/$(FIRECRACKER_VERSION)/*/firecracker
@echo "Firecracker binaries downloaded successfully"

# Caddy version and modules
CADDY_VERSION := v2.10.2
CADDY_DNS_MODULES := --with github.com/caddy-dns/cloudflare
Expand Down Expand Up @@ -153,6 +172,22 @@ ensure-ch-binaries:
$(MAKE) download-ch-binaries; \
fi

# Check if Firecracker binaries exist, download if missing
.PHONY: ensure-firecracker-binaries
ensure-firecracker-binaries:
@ARCH=$$(uname -m); \
if [ "$$ARCH" = "x86_64" ]; then \
FC_ARCH=x86_64; \
elif [ "$$ARCH" = "aarch64" ] || [ "$$ARCH" = "arm64" ]; then \
FC_ARCH=aarch64; \
else \
echo "Unsupported architecture: $$ARCH"; exit 1; \
fi; \
if [ ! -f lib/hypervisor/firecracker/binaries/firecracker/$(FIRECRACKER_VERSION)/$$FC_ARCH/firecracker ]; then \
echo "Firecracker binaries not found, downloading..."; \
$(MAKE) download-firecracker-binaries; \
fi

# Check if Caddy binaries exist, build if missing
.PHONY: ensure-caddy-binaries
ensure-caddy-binaries:
Expand Down Expand Up @@ -191,7 +226,7 @@ else
$(MAKE) build-linux
endif

build-linux: ensure-ch-binaries ensure-caddy-binaries build-embedded | $(BIN_DIR)
build-linux: ensure-ch-binaries ensure-firecracker-binaries ensure-caddy-binaries build-embedded | $(BIN_DIR)
go build -tags containers_image_openpgp -o $(BIN_DIR)/hypeman ./cmd/api

# Build all binaries
Expand All @@ -212,7 +247,7 @@ dev:
fi

# Linux development mode with hot reload
dev-linux: ensure-ch-binaries ensure-caddy-binaries build-embedded $(AIR)
dev-linux: ensure-ch-binaries ensure-firecracker-binaries ensure-caddy-binaries build-embedded $(AIR)
@rm -f ./tmp/main
$(AIR) -c .air.toml

Expand All @@ -227,15 +262,15 @@ else
endif

# Linux tests (as root for network capabilities)
test-linux: ensure-ch-binaries ensure-caddy-binaries build-embedded
test-linux: ensure-ch-binaries ensure-firecracker-binaries ensure-caddy-binaries build-embedded
@VERBOSE_FLAG=""; \
TEST_PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$$PATH"; \
if [ -n "$(VERBOSE)" ]; then VERBOSE_FLAG="-v"; fi; \
if [ -n "$(TEST)" ]; then \
echo "Running specific test: $(TEST)"; \
sudo env "PATH=$$TEST_PATH" "DOCKER_CONFIG=$${DOCKER_CONFIG:-$$HOME/.docker}" go test -tags containers_image_openpgp -run=$(TEST) $$VERBOSE_FLAG -timeout=300s ./...; \
sudo env "PATH=$$TEST_PATH" "DOCKER_CONFIG=$${DOCKER_CONFIG:-$$HOME/.docker}" go test -tags containers_image_openpgp -run=$(TEST) $$VERBOSE_FLAG -timeout=$(TEST_TIMEOUT) ./...; \
else \
sudo env "PATH=$$TEST_PATH" "DOCKER_CONFIG=$${DOCKER_CONFIG:-$$HOME/.docker}" go test -tags containers_image_openpgp $$VERBOSE_FLAG -timeout=300s ./...; \
sudo env "PATH=$$TEST_PATH" "DOCKER_CONFIG=$${DOCKER_CONFIG:-$$HOME/.docker}" go test -tags containers_image_openpgp $$VERBOSE_FLAG -timeout=$(TEST_TIMEOUT) ./...; \
fi

# macOS tests (no sudo needed, adds e2fsprogs to PATH)
Expand All @@ -250,10 +285,10 @@ test-darwin: build-embedded sign-vz-shim
if [ -n "$(TEST)" ]; then \
echo "Running specific test: $(TEST)"; \
PATH="/opt/homebrew/opt/e2fsprogs/sbin:$(PATH)" \
go test -tags containers_image_openpgp -run=$(TEST) $$VERBOSE_FLAG -timeout=300s $$PKGS; \
go test -tags containers_image_openpgp -run=$(TEST) $$VERBOSE_FLAG -timeout=$(TEST_TIMEOUT) $$PKGS; \
else \
PATH="/opt/homebrew/opt/e2fsprogs/sbin:$(PATH)" \
go test -tags containers_image_openpgp $$VERBOSE_FLAG -timeout=300s $$PKGS; \
go test -tags containers_image_openpgp $$VERBOSE_FLAG -timeout=$(TEST_TIMEOUT) $$PKGS; \
fi

# Generate JWT token for testing
Expand All @@ -277,14 +312,15 @@ e2e-build-test:
clean:
rm -rf $(BIN_DIR)
rm -rf lib/vmm/binaries/cloud-hypervisor/
rm -rf lib/hypervisor/firecracker/binaries/firecracker/
rm -rf lib/ingress/binaries/
rm -f lib/system/guest_agent/guest-agent
rm -f lib/system/init/init
rm -f lib/hypervisor/vz/vz-shim/vz-shim

# Prepare for release build (called by GoReleaser)
# Downloads all embedded binaries and builds embedded components
release-prep: download-ch-binaries build-caddy-binaries build-embedded
release-prep: download-ch-binaries download-firecracker-binaries build-caddy-binaries build-embedded
go mod tidy

# =============================================================================
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
</p>

<p align="center">
<strong>Run containerized workloads in VMs, powered by <a href="https://github.com/cloud-hypervisor/cloud-hypervisor">Cloud Hypervisor</a>.</strong>
<strong>Run containerized workloads in VMs, powered by Cloud Hypervisor, Firecracker, QEMU, and Apple Virtualization.framework.</strong>
<img alt="GitHub License" src="https://img.shields.io/github/license/kernel/hypeman">
<a href="https://discord.gg/FBrveQRcud"><img src="https://img.shields.io/discord/1342243238748225556?logo=discord&logoColor=white&color=7289DA" alt="Discord"></a>
</p>
Expand All @@ -22,7 +22,7 @@
## Requirements

### Linux
**KVM** virtualization support required. Supports Cloud Hypervisor and QEMU as hypervisors.
**KVM** virtualization support required. Supports Cloud Hypervisor, Firecracker, and QEMU as hypervisors.

### macOS
**macOS 11.0+** on Apple Silicon. Uses Apple's Virtualization.framework via the `vz` hypervisor.
Expand Down
6 changes: 4 additions & 2 deletions cmd/api/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,8 @@ type CapacityConfig struct {

// HypervisorConfig holds hypervisor settings.
type HypervisorConfig struct {
Default string `koanf:"default"`
Default string `koanf:"default"`
FirecrackerBinaryPath string `koanf:"firecracker_binary_path"`
}

// GPUConfig holds GPU-related settings.
Expand Down Expand Up @@ -297,7 +298,8 @@ func defaultConfig() *Config {
},

Hypervisor: HypervisorConfig{
Default: "cloud-hypervisor",
Default: "cloud-hypervisor",
FirecrackerBinaryPath: "",
},

GPU: GPUConfig{
Expand Down
8 changes: 8 additions & 0 deletions config.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@ data_dir: /var/lib/hypeman
# Server configuration
# port: 8080

# =============================================================================
# Hypervisor Configuration
# =============================================================================
# hypervisor:
# default: cloud-hypervisor
# # Optional: use a custom Firecracker binary path instead of the embedded one.
# # firecracker_binary_path: /usr/local/bin/firecracker

# =============================================================================
# Network Configuration
# =============================================================================
Expand Down
1 change: 1 addition & 0 deletions lib/hypervisor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Hypeman originally supported only Cloud Hypervisor. This abstraction layer allow
| Hypervisor | Platform | Process Model | Control Interface |
|------------|----------|---------------|-------------------|
| Cloud Hypervisor | Linux | External process | HTTP API over Unix socket |
| Firecracker | Linux | External process | HTTP API over Unix socket |
| QEMU | Linux | External process | QMP over Unix socket |
| vz | macOS | Subprocess (vz-shim) | HTTP API over Unix socket |

Expand Down
8 changes: 8 additions & 0 deletions lib/hypervisor/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,14 @@ type NetworkConfig struct {
IP string
MAC string
Netmask string
// DownloadBps limits host->guest bandwidth in bytes/sec (0 = unlimited).
// Hypeman enforces this host-side via TAP shaping for all hypervisors.
// Firecracker also maps it to per-interface API rate limiters.
DownloadBps int64
// UploadBps limits guest->host bandwidth in bytes/sec (0 = unlimited).
// Hypeman enforces this host-side via TAP shaping for all hypervisors.
// Firecracker also maps it to per-interface API rate limiters.
UploadBps int64
}

// VMInfo contains current VM state information
Expand Down
35 changes: 35 additions & 0 deletions lib/hypervisor/firecracker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Firecracker Hypervisor

This package implements Firecracker support behind the common `lib/hypervisor` interfaces:

- `Starter` (`process.go`): starts/restores a Firecracker process and waits for the API socket.
- `Firecracker` client (`firecracker.go`): configures boot, controls lifecycle, and manages snapshots.
- `VsockDialer` (`vsock.go`): host-initiated vsock connections via Firecracker's UDS handshake.
- Config translation (`config.go`): maps `hypervisor.VMConfig` to Firecracker API models.

## Binaries

Like Cloud Hypervisor, Firecracker binaries are embedded and extracted into `data_dir` at runtime.

- Embedded source path: `lib/hypervisor/firecracker/binaries/firecracker/<version>/<arch>/firecracker`
- Download helper: `make download-firecracker-binaries`
- Runtime override: `hypervisor.firecracker_binary_path` (uses external binary instead of embedded)

## VM State Mapping

`mapVMState()` maps Firecracker `GET /` state strings to internal states:

- `Not started` -> `created`
- `Running` -> `running`
- `Paused` -> `paused`

These strings are validated against Firecracker's source/spec:

- `src/vmm/src/vmm_config/instance_info.rs`
- `src/firecracker/swagger/firecracker.yaml`

## Rate Limits

Instance bandwidth limits are still instance-level API inputs, but are propagated into
per-interface `hypervisor.NetworkConfig` so Firecracker can program device rate limiters.
Host-level traffic shaping remains handled by Hypeman's network manager.
Loading
Loading