Skip to content

feat(kubernetes): support driver config passthrough#1744

Open
elezar wants to merge 6 commits into
mainfrom
poc/kubernetes-driver-config-1589
Open

feat(kubernetes): support driver config passthrough#1744
elezar wants to merge 6 commits into
mainfrom
poc/kubernetes-driver-config-1589

Conversation

@elezar
Copy link
Copy Markdown
Member

@elezar elezar commented Jun 4, 2026

Summary

Adds the driver_config path end to end: CLI raw JSON, gateway driver-key selection, and Kubernetes driver-local typed config consumption. The Kubernetes driver now supports pod scheduling fields, agent container resources, and pod.runtime_class_name mapped to PodSpec runtimeClassName.

The typed SandboxTemplate runtime_class_name remains highest precedence. Kubernetes driver_config pod.runtime_class_name overrides the gateway default_runtime_class_name when the typed template field is absent.

Related Issue

Discussion POC for #1589.

Related to #1492.

Does not close either reference.

Changes

  • Adds a pre-POC Docker test cleanup that switches sandbox template fixtures to ..Default::default().
  • Adds experimental --driver-config-json CLI plumbing and forwards the selected driver block through the gateway to the compute driver.
  • Adds Kubernetes driver-local config handling for pod scheduling fields, agent container resources, and pod.runtime_class_name.
  • Maps pod.runtime_class_name to PodSpec runtimeClassName with precedence over default_runtime_class_name.
  • Documents the experimental CLI surface and Kubernetes config shape in the driver README and published docs.
  • Notes in the Kubernetes driver that the typed config schema should remain driver-local and may later move to driver-local proto definitions.

Testing

  • cargo test -p openshell-driver-kubernetes
  • mise run pre-commit
  • Unit tests added/updated
  • E2E tests added/updated (not run; covered by unit tests and pre-commit for this PR)

Checklist

  • Follows Conventional Commits format
  • Commits are signed off (DCO)
  • Documentation updated (driver README and published docs)
  • Architecture docs updated (not applicable; published/reference docs updated instead)

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Jun 4, 2026

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

@drew drew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good to me. I say we ship it and then add podman/docker volume configs, and gpu configs.

Once those are in maybe we can work on config validation and more concretely defining schemas that can be validated by drivers.

Comment on lines +98 to +102
struct KubernetesPodDriverConfig {
node_selector: BTreeMap<String, String>,
tolerations: Vec<serde_json::Value>,
priority_class_name: String,
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also add runtime_class_name here

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I made a note of this after the discussions in the community slack yesterday. Let me add it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@elezar elezar force-pushed the poc/kubernetes-driver-config-1589 branch from f99337f to 7c6bff6 Compare June 5, 2026 07:15
@elezar elezar changed the title feat(kubernetes): prototype driver config passthrough feat(kubernetes): support driver config passthrough Jun 5, 2026
@elezar elezar marked this pull request as ready for review June 5, 2026 07:16
@elezar elezar requested review from a team, derekwaynecarr, maxamillion and mrunalp as code owners June 5, 2026 07:16
Signed-off-by: Evan Lezar <elezar@nvidia.com>
@elezar elezar force-pushed the poc/kubernetes-driver-config-1589 branch from 4e4ed33 to ce91dfe Compare June 5, 2026 09:29
@elezar elezar enabled auto-merge (squash) June 5, 2026 09:30
@elezar elezar disabled auto-merge June 5, 2026 09:31
elezar added 5 commits June 5, 2026 11:35
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
@elezar elezar force-pushed the poc/kubernetes-driver-config-1589 branch from ce91dfe to 5189f78 Compare June 5, 2026 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants