Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,23 @@ azldev.toml # Root config — includes distro/ and base
│ ├── distro.toml # Includes all *.distro.toml
│ ├── fedora.distro.toml # Fedora: dist-git URIs, lookaside, version branches
│ └── mock/ # Mock build environment configs
├── specs/ # Rendered specs (generated by `azldev comp render`)
│ └── <first-char>/<name>/ # Per-component: spec, patches, scripts (no source tarballs)
└── external/schemas/
└── azldev.schema.json # Authoritative schema for all TOML config files
```

## Rendered Specs (`specs/`)

The `specs/` directory contains rendered spec files generated by `azldev comp render`. These are the final specs with all overlays applied — ready for check-in. After adding or modifying components/overlays, re-render:

```bash
azldev comp render -p <name> # Single component
azldev comp render -a # All components (slow)
```

To inspect what a component's spec looks like after overlays, read `specs/<first-char>/<name>/<name>.spec` directly — no need to run `prep-sources` just to view the result. Use `prep-sources` when you need the full source tree (tarballs) or want to diff pre/post overlay output for debugging.

## Key Concepts

**Components** = unit of packaging (→ one or more RPMs). Spec sources: upstream (default, from Fedora dist-git), local, or pinned upstream. See [`comp-toml.instructions.md`](instructions/comp-toml.instructions.md#spec-source-types) for syntax.
Expand All @@ -52,6 +65,8 @@ Run all commands from the repo root (where `azldev.toml` lives). If the terminal
| Add a component | `azldev comp add` |
| Build a component | `azldev comp build -p <name> -q` |
| Build chain (auto-publish to local repo) | `azldev comp build --local-repo-with-publish ./base/out -p <a> -p <b> -q` |
| Render many specs for check-in | `azldev comp render -a` |
| Render a single component | `azldev comp render -p <name>` |
| Prepare sources (apply overlays) | `azldev comp prep-sources -p <name> --force -o <dir> -q` |
| Prepare sources (skip overlays) | `azldev comp prep-sources -p <name> --skip-overlays --force -o <dir> -q` |
| Build, keep env on failure | `azldev comp build -p <name> --preserve-buildenv on-failure -q` |
Expand Down
113 changes: 113 additions & 0 deletions .github/instructions/pr-check-workflows.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
---
applyTo: ".github/workflows/**"
description: ALWAYS review these instructions when reading or modifying PR check workflows.
---

# PR Check Workflow Guidelines

## Fork-PR-safe pattern: stub + reusable

Problem: `pull_request` triggers on fork PRs run without secrets and with a read-only token. `pull_request_target` runs with write access but checks out the BASE ref by default — easy to footgun into RCE if you then check out PR code with full privileges.

Pattern:

1. **Stub workflow** on the default branch — triggered by `pull_request_target`, guards on repo owner, calls the reusable workflow. This is the only file GitHub will load from the base branch, so it locks the entrypoint.
2. **Reusable workflow** (`workflow_call`) holds the real logic. Lives on the PR branch, so contributors can iterate on it.
3. Stub passes `pull-requests: write` / `contents: read` only. Reusable declares its own minimum permissions.

Never check out PR code into a privileged job and then execute it on the host. Either:
- Run untrusted code inside a container with no secrets mounted, **or**
- Keep the privileged job read-only (lint, comment-post) and isolate code execution to a separate unprivileged job.

## Prefer in-container for anything that executes PR code

If the check builds, renders, or runs PR code, do the whole thing inside the build container. Mock is a critical component of many azldev workflows. It requires many privileges to run successfully. It also is not available in Ubuntu, which is the default runner image for GitHub Actions.

- Mount the PR checkout read-only when possible; if writes are needed (e.g. `git add -N`), mount rw but don't leak host paths or secrets.
- Produce **all outputs** (reports, patches, diffs) inside the container and write them to a bind-mounted output dir. Host-side steps then only read these artifacts (json report, patch files, etc.)
- This eliminates a huge class of config-driven git RCE vectors (`core.fsmonitor`, `core.sshCommand`, hook files, etc.) because the host never runs git against PR-controlled config.

### Container config

The shared runner image is [`.github/workflows/containers/azldev-runner.Dockerfile`](../workflows/containers/azldev-runner.Dockerfile). It's a minimal Azure Linux base with `mock`, `git`, `python3`, and `sudo` — enough to run any `azldev` subcommand. Reuse it rather than building a per-check image; add extras via a derived `FROM localhost/azldev-runner` stage if a check genuinely needs more.

`azldev` itself is **not** baked into the image. The workflow installs it on the host (`go install …/azldev@main`) and bind-mounts `$(go env GOPATH)/bin` into the container as `/gobin`. This means that all stages of the workflow use the same version of `azldev`.

Build it with the caller's UID so bind-mounted writes don't end up root-owned:

```yaml
- name: Build azldev runner
run: |
docker build \
--build-arg UID=$(id -u) \
-t localhost/azldev-runner \
-f .github/workflows/containers/azldev-runner.Dockerfile \
.github/workflows/containers/
```

#### Bind-mount conventions

| Mount | Mode | Purpose |
| ----- | ---- | ------- |
| `pr-head/` → `/workdir` | rw | PR checkout. rw because `azldev` writes to `specs/`, `base/build/`, etc. |
| `<host-output-dir>/` → `/output` | rw | Trusted-shape outputs (JSON reports, patches, ...) the container produces for the host to consume after the run. |
| `$(go env GOPATH)/bin` → `/gobin` | ro | `azldev` binary from the trusted base checkout. |
| `.github/workflows/scripts/` → `/scripts` | ro | Helper scripts from the trusted base checkout. |

Always set `-e PATH=/gobin:…` so `/gobin/azldev` wins over anything the image might provide.

#### Sandbox flags (minimum viable for `mock`)

```yaml
docker run --rm \
--cap-add=SYS_ADMIN \
--security-opt seccomp=unconfined \
--security-opt apparmor=unconfined \
...
```

Why each one is needed:

- **`--cap-add=SYS_ADMIN`** — `mock` sets up mount namespaces for its chroot. Without this you get `mount … exit status 32` during chroot init.
- **`--security-opt seccomp=unconfined`** — `mock` uses syscalls (`unshare`, `pivot_root`, etc.) that Docker's default seccomp profile blocks.
- **`--security-opt apparmor=unconfined`** — `ubuntu-latest` ships the `docker-default` AppArmor profile, which blocks `mount -t tmpfs` on paths under `/var/lib/mock` **even with `SYS_ADMIN` granted**. This is the confusing one; symptom is the same `exit status 32` after seccomp is already unconfined.

Avoid `--privileged` — it grants every capability and removes cgroup restrictions, which is a much bigger blast radius than the three flags above.

`--security-opt no-new-privileges` would be nice but `mock`'s `userhelper` needs setuid, which that flag blocks.

#### Running commands in the container

Use `bash -eu -o pipefail -c '…'` as the entrypoint invocation so a failure inside the heredoc actually fails the step:

```yaml
localhost/azldev-runner \
bash -eu -o pipefail -c '
azldev component render -q -a --clean-stale -O json > /output/render.json
python3 /scripts/check_rendered_specs.py \
--specs-dir "$(azldev config dump -q -f json | jq -r .project.renderedSpecsDir)" \
--report /output/report.json \
--patch /output/rendered-specs.patch
'
```

Use single-quotes around the `-c` payload so host-side `${{ … }}` interpolation doesn't leak into the container script. If you need to pass a host value in, use `-e VAR=…` and reference `"$VAR"` inside — same script-injection concern as any other shell step.

## Shell hardening in workflow steps

- Start every multi-line `run:` with `set -euo pipefail`.
- Quote **every** expansion involving a workflow input, matrix value, or file path: `"${VAR}"`, not `$VAR`.
- Never interpolate `${{ github.event.pull_request.* }}` directly into a shell script — assign to an `env:` var first, then reference as `"$VAR"`. Direct interpolation is a classic script-injection vector.
- For paths that must stay inside the repo, resolve with `realpath -m` and verify they start with the repo root prefix before use.

## Markdown / HTML injection in PR comments

- Escape any PR-controlled string (file paths, error messages) before dropping into Markdown.
- Prefer code spans (`` `path` ``) or fenced blocks for anything path-like.
- Don't embed raw diff content in the comment body — upload as an artifact and link to it.

## zizmor / pedantic linting

Workflows are linted with `zizmor --pedantic`.

Use `# zizmor: ignore[<rule>]` comments as an absolute last resort, and provide a comprehensive justification for why the rule is being ignored.
32 changes: 32 additions & 0 deletions .github/instructions/rendered-specs.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
applyTo: "specs/**/*"
description: ALWAYS refer to this when working with rendered spec files (`*.spec`) in the `specs/` directory.
---

# Rendered Spec Files (`specs/*.spec`)

## What are rendered specs?

Rendered specs are generated by the `azldev comp render` command based on the component definitions and overlays. They are output to the `specs/` directory and should not be edited directly.

They are meant to be consumed by downstream processes (e.g., build pipelines) and are the source of truth for all subsequent steps. Any changes to the spec should be made via the component definition and overlays, not by editing the rendered spec.

## Changing a rendered spec

To change a rendered spec, modify the component's `.comp.toml` and/or its overlays. Then re-run the render command to regenerate the spec:

```bash
# VERY SLOW - Re-render all specs, removes any stale specs that are no longer defined in the components
azldev comp render -a --clean-stale -O json
```

```bash
# Small set, will NOT remove stale specs, faster for iterative development
azldev comp render <component-1> <component-2> -O json
```

```bash
# Custom output directory, useful for debugging. When not using the automatically configured spec directory, --force is
# required to delete and re-create the output folders if they already exist.
azldev comp render <component> -O json -o ./base/build/work/scratch/rendered-specs --force
```
3 changes: 2 additions & 1 deletion .github/instructions/spec.instructions.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
applyTo: "**/*.spec"
applyTo: "base/**/*.spec"
description: Read this when working with spec files (`*.spec`) that are hand-maintained in the Azure Linux repo (not rendered).
---

# RPM Spec Files (`*.spec`)
Expand Down
2 changes: 1 addition & 1 deletion .github/prompts/azl-add-component.prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@ Follow the workflow in the [skill-add-component skill](../skills/skill-add-compo
- Needs overlays or customizations → create `${input:component_name}/${input:component_name}.comp.toml`
- Needs extensive changes overlays can't handle → forked local spec (**last resort**, requires explicit user sign-off)
6. Add overlays with meaningful `description` fields explaining *why* each change is needed
7. Validate: `azldev comp prep-sources -p ${input:component_name} --force -o base/build/work/scratch/${input:component_name}-post -q` (with overlays) and diff against the skip-overlays output
7. Render and verify: `azldev comp render -p ${input:component_name}` and inspect `specs/` output. For deeper debugging, diff pre/post overlay output with `prep-sources`.
8. Build: `azldev comp build -p ${input:component_name} -q`
9. Smoke-test the built RPMs in a mock chroot
10 changes: 8 additions & 2 deletions .github/prompts/azl-debug-component.prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,21 @@ First, determine the error category:
- Follow the `skill-mock` workflow: install RPMs in a mock chroot, verify contents, check dependencies
- Common causes: missing Requires, wrong file paths, permission issues

**When in doubt**, start with a `prep-sources` pre/post diff to determine if the issue is overlay-related:
**When in doubt**, start with a render to determine if the issue is overlay-related:

```bash
azldev comp render -p ${input:component_name}
```

If `render` fails, the issue is overlay-related (category 1). For deeper debugging, diff pre/post overlay output:

```bash
azldev comp prep-sources -p ${input:component_name} --skip-overlays -o base/build/work/scratch/${input:component_name}-pre --force
azldev comp prep-sources -p ${input:component_name} -o base/build/work/scratch/${input:component_name}-post --force
diff -r base/build/work/scratch/${input:component_name}-pre base/build/work/scratch/${input:component_name}-post
```

If `prep-sources` itself fails, the issue is overlay-related (category 1). If it succeeds but `comp build` fails, it's a build issue (category 2).
If both render and `prep-sources` succeed but `comp build` fails, it's a build issue (category 2).

## Fix

Expand Down
4 changes: 4 additions & 0 deletions .github/prompts/azl-update-component.prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ Use structural patterns from [comp-toml.instructions.md](../instructions/comp-to
- `config` — build config changes (`build.defines`, `build.without`)
3. **Apply changes** to the `.comp.toml` file
4. **Verify overlays still apply:**
```bash
azldev comp render -p ${input:component_name}
```
Inspect `specs/` output. For deeper debugging, diff pre/post overlay output:
```bash
azldev comp prep-sources -p ${input:component_name} --skip-overlays -o base/build/work/scratch/${input:component_name}-pre --force
azldev comp prep-sources -p ${input:component_name} -o base/build/work/scratch/${input:component_name}-post --force
Expand Down
14 changes: 13 additions & 1 deletion .github/skills/skill-add-component/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,16 @@ Overlays are vastly preferable to maintaining a forked spec, they get automatic

## Validate

After adding overlays or customizations, render the spec to verify:

```bash
azldev comp render -p <name>
# Inspect the result
cat specs/<first-char>/<name>/<name>.spec
```

For deeper debugging (diffing pre/post overlay output with full sources):

> Use a temp dir for `prep-sources` output. Use `--force` to overwrite an existing output dir.

`prep-sources -o <dir>` writes to a user-specified directory (NOT `base/out/` — that's for `comp build` output).
Expand All @@ -77,9 +87,11 @@ Overlays are vastly preferable to maintaining a forked spec, they get automatic
azldev comp prep-sources -p <name> --skip-overlays --force -o base/build/work/scratch/<name>-pre -q
azldev comp prep-sources -p <name> --force -o base/build/work/scratch/<name>-post -q
diff -r base/build/work/scratch/<name>-pre base/build/work/scratch/<name>-post
```

```bash
# Test build (RPMs land in base/out/ per project.toml output-dir)
azldev comp build -p <name> -q
```

For testing the built RPMs, see the [`skill-mock`](../skill-mock/SKILL.md) skill. New components always need a smoke-test. For the full inner loop cycle (investigate → modify → verify → build → test → inspect), see [`skill-build-component`](../skill-build-component/SKILL.md).
For testing the built RPMs, see the [`skill-mock`](../skill-mock/SKILL.md) skill. New components always need a smoke-test. For the full inner loop cycle (investigate → modify → render → build → test → inspect), see [`skill-build-component`](../skill-build-component/SKILL.md).
28 changes: 21 additions & 7 deletions .github/skills/skill-build-component/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,26 +49,40 @@ Build foundational packages first (e.g., `azurelinux-rpm-config`), then dependen
The standard cycle for investigating, modifying, and verifying components:

```
investigate → modify → verify → build → test → inspect
investigate → modify → render → build → test → inspect
```

| Step | Command | What to check |
|------|---------|---------------|
| **Investigate** | `prep-sources --skip-overlays --force -o base/build/work/scratch/<name>-pre` | Upstream spec/sources as-is |
| **Compare** | `prep-sources --force -o base/build/work/scratch/<name>-post` + `diff -r ...-pre ...-post` | Current overlay effect |
| **Investigate** | Read `specs/<first-char>/<name>/<name>.spec` or `prep-sources --skip-overlays --force -o base/build/work/scratch/<name>-pre` | Upstream spec/sources as-is |
| **Compare** | `prep-sources --force -o base/build/work/scratch/<name>-post` + `diff -r ...-pre ...-post` | Current overlay effect (deep debug) |
| **Modify** | Edit `*.comp.toml` (overlays, defines, without) | — |
| **Verify** | `prep-sources --force -o base/build/work/scratch/<name>-post` | Overlay applies cleanly |
| **Verify** | `comp render -p <name>` + inspect `specs/<first-char>/<name>/` | Overlay applies cleanly (fast path) |
| **Build** | `comp build -p <name>` | RPMs appear in `base/out/` |
| **Test** | `adv mock shell --add-package base/out/<name>*.rpm` | Package installs, binary runs, basic functionality works |
| **Inspect** | `comp build --preserve-buildenv always` + `adv mock shell` | BUILDROOT contents, file lists |

> **Prefer `comp render` for quick verification.** It's faster than `prep-sources` since it skips downloading source tarballs. Use `prep-sources` when you need the full source tree or want to diff pre/post overlay output for debugging.

> Use a temp dir for `prep-sources` output. Use `--force` to overwrite an existing output dir.

> Package builds are often very long, so adjust command timeouts accordingly when using shell tools to run builds, or use background mode if available.

## Debugging Build Failures

### 1. Diff sources pre/post overlay
### 1. Render and inspect the spec

The fastest way to verify overlays applied correctly:

```bash
azldev comp render -p <name>
# Inspect the result
cat specs/<first-char>/<name>/<name>.spec
```

### 2. Diff sources pre/post overlay (deep debug)

When you need to understand exactly what upstream provides vs. what overlays change:

```bash
azldev comp prep-sources -p <name> --skip-overlays --force -o base/build/work/scratch/<name>-pre -q
Expand All @@ -78,14 +92,14 @@ diff -r base/build/work/scratch/<name>-pre base/build/work/scratch/<name>-post

This reveals whether overlays apply as intended or whether upstream changed.

### 2. Preserve build environment on failure
### 3. Preserve build environment on failure

```bash
azldev comp build -p <name> --preserve-buildenv on-failure -q
# Use `always` to inspect even successful builds
```

### 3. Enter mock shell (deep debug)
### 4. Enter mock shell (deep debug)

For testing built RPMs or inspecting the chroot, see the [`skill-mock`](../skill-mock/SKILL.md) skill. Quick reference:

Expand Down
Loading
Loading