-
Notifications
You must be signed in to change notification settings - Fork 276
adds guest manager interfaces + implementation for guest operations #2598
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rawahars
wants to merge
1
commit into
microsoft:main
Choose a base branch
from
rawahars:feat/vm-package-guestmanager
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # VM Package | ||
|
|
||
| This directory defines the utility VM (UVM) contracts and separates responsibilities into three layers. The goal is to keep | ||
| configuration, host-side management, and guest-side actions distinct so each layer can evolve independently. | ||
|
|
||
| 1. **Builder**: constructs an HCS compute system configuration used to create a VM. | ||
| 2. **VM Manager**: manages host-side VM configuration and lifecycle (NICs, SCSI, VPMem, etc.). | ||
| 3. **Guest Manager**: intended for guest-side actions (for example, mounting a disk). | ||
|
|
||
| **Note that** this layer does not store UVM host or guest side state. That will be part of the orchestration layer above it. | ||
|
|
||
| ## Packages and Responsibilities | ||
|
|
||
| - `internal/vm` | ||
| - Interface definitions and shared types. | ||
| - `UVM` interface exposes APIs for lifecycle, resources, and device management of running VM. | ||
| - `Builder` exposes APIs for shaping the configuration used to create a VM. | ||
| - `GuestOps` defines guest-side operations executed via the GCS connection. | ||
| - `internal/vm/builder` | ||
| - Concrete implementation of `Builder` for building `hcsschema.ComputeSystem` documents. | ||
| - Provides a fluent API for configuring all aspects of the VM document. | ||
| - Presently, this package is tightly coupled with HCS backend. | ||
| - `internal/vm/vmmanager` | ||
| - Concrete implementation of `UVM` for running and managing a UVM instance. | ||
| - Presently, this package is tightly coupled with HCS backend and only runs HCS backed UVMs. | ||
| - Owns lifecycle calls (start/terminate/close) and host-side modifications (NICs, SCSI, VPMem, pipes, VSMB, Plan9). | ||
| - Allows creation of the UVM using `vmmanager.Create` which takes a `Builder` and produces a running UVM. | ||
| - `internal/vm/guestmanager` | ||
| - Guest-side actions executed via the GCS connection. | ||
| - Implements `vm.GuestOps` which provides APIs for network, storage, devices, security policy, and layers management within guest. | ||
| - Translates guest operations into GCS modify requests. | ||
|
|
||
| ## Typical Flow | ||
|
|
||
| 1. Build the config using the builder interfaces. | ||
| 2. Create the VM using the VM manager. | ||
| 3. Use manager interfaces for lifecycle and host-side changes. | ||
| 4. Use guest manager interfaces for in-guest actions. | ||
|
|
||
| ## Example (High Level) | ||
|
|
||
| ``` | ||
| builder, _ := builder.New("owner", vm.Linux) | ||
|
|
||
| // Configure the VM document. | ||
| builder.Memory().SetMemoryLimit(1024) | ||
| builder.Processor().SetProcessorConfig(&vm.ProcessorConfig{Count: 2}) | ||
| // ... other builder configuration | ||
|
|
||
| // Create and start the VM. | ||
| uvm, _ := vmmanager.Create(ctx, "uvm-id", builder) | ||
| _ = uvm.LifetimeManager().Start(ctx) | ||
|
|
||
| // Apply host-side updates. | ||
| _ = uvm.NetworkManager().AddNIC(ctx, nicID, endpointID, macAddr) | ||
| ``` | ||
|
|
||
| ## Layer Boundaries (Quick Reference) | ||
|
|
||
| - **Builder**: static, pre-create configuration only. No host mutations. | ||
| - **VM Manager**: host-side changes and lifecycle operations on an existing UVM. | ||
| - **Guest Manager**: guest-side actions, scoped to work that requires in-guest context (GCS-backed). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| package vm | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| "github.com/Microsoft/hcsshim/internal/cow" | ||
| "github.com/Microsoft/hcsshim/internal/gcs" | ||
| hcsschema "github.com/Microsoft/hcsshim/internal/hcs/schema2" | ||
| "github.com/Microsoft/hcsshim/internal/protocol/guestrequest" | ||
| "github.com/Microsoft/hcsshim/internal/protocol/guestresource" | ||
| ) | ||
|
|
||
| // ConfigOption defines a function that modifies the GCS connection config. | ||
| type ConfigOption func(*gcs.GuestConnectionConfig) error | ||
|
|
||
| // GuestOps is an interface to interact with | ||
| type GuestOps interface { | ||
| // CreateConnection creates a connection to the guest. | ||
| CreateConnection(ctx context.Context, opts ...ConfigOption) error | ||
| // CloseConnection closes the connection to the guest. | ||
| CloseConnection() error | ||
| // Manager returns an interface to manage general operations in the guest. | ||
| Manager() Manager | ||
| // LayersManager returns an interface to manage combined layers in the guest. | ||
| LayersManager() LayersManager | ||
| // GuestNetworkManager returns an interface to manage networking in the guest. | ||
| GuestNetworkManager() GuestNetworkManager | ||
| // DirectoryManager returns an interface to manage mapped directories in the guest. | ||
| DirectoryManager() DirectoryManager | ||
| // SecurityPolicyManager returns an interface to manage guest security policy operations. | ||
| SecurityPolicyManager() SecurityPolicyManager | ||
| // GuestDeviceManager returns an interface to manage device operations in the guest. | ||
| GuestDeviceManager() GuestDeviceManager | ||
| // BlockCIMsManager returns an interface to manage WCOW block CIMs in the guest. | ||
| BlockCIMsManager() BlockCIMsManager | ||
| // ScsiManager returns an interface to manage SCSI-related operations in the guest. | ||
| ScsiManager() ScsiManager | ||
| } | ||
|
|
||
| type Manager interface { | ||
| // Capabilities returns the guest's declared capabilities. | ||
| Capabilities() gcs.GuestDefinedCapabilities | ||
| // CreateContainer creates a container within guest using ID `cid` and `config`. | ||
| // Once the container is created, it can be managed using the returned `gcs.Container` interface. | ||
| // `gcs.Container` uses the underlying guest connection to issue commands to the guest. | ||
| CreateContainer(ctx context.Context, cid string, config interface{}) (*gcs.Container, error) | ||
| // CreateProcess creates a process in the guest. | ||
| // Once the process is created, it can be managed using the returned `cow.Process` interface. | ||
| // `cow.Process` uses the underlying guest connection to issue commands to the guest. | ||
| CreateProcess(ctx context.Context, settings interface{}) (cow.Process, error) | ||
| // DumpStacks requests a stack dump from the guest and returns it as a string. | ||
| DumpStacks(ctx context.Context) (string, error) | ||
| // DeleteContainerState removes persisted state for the container identified by `cid` from the guest. | ||
| DeleteContainerState(ctx context.Context, cid string) error | ||
| } | ||
|
|
||
| // LayersManager exposes combined layer operations in the guest. | ||
| type LayersManager interface { | ||
| AddCombinedLayers(ctx context.Context, settings interface{}) error | ||
| AddCWCOWCombinedLayers(ctx context.Context, settings interface{}) error | ||
| RemoveCombinedLayers(ctx context.Context, settings interface{}) error | ||
| RemoveCWCOWCombinedLayers(ctx context.Context, settings interface{}) error | ||
| } | ||
|
|
||
| // GuestNetworkManager exposes guest network operations. | ||
| type GuestNetworkManager interface { | ||
| AddNetworkNamespace(ctx context.Context, settings interface{}) error | ||
| RemoveNetworkNamespace(ctx context.Context, settings interface{}) error | ||
| AddNetworkInterface(ctx context.Context, adapterID string, requestType guestrequest.RequestType, settings interface{}) error | ||
| AddNetworkInterfaceLCOW(ctx context.Context, settings *guestresource.LCOWNetworkAdapter) error | ||
| RemoveNetworkInterface(ctx context.Context, adapterID string, requestType guestrequest.RequestType, settings interface{}) error | ||
| RemoveNetworkInterfaceLCOW(ctx context.Context, settings *guestresource.LCOWNetworkAdapter) error | ||
| } | ||
|
|
||
| // DirectoryManager exposes mapped directory operations in the guest. | ||
| type DirectoryManager interface { | ||
| AddMappedDirectory(ctx context.Context, settings *hcsschema.MappedDirectory) error | ||
| AddMappedDirectoryLCOW(ctx context.Context, settings guestresource.LCOWMappedDirectory) error | ||
| RemoveMappedDirectoryLCOW(ctx context.Context, settings guestresource.LCOWMappedDirectory) error | ||
| } | ||
|
|
||
| // SecurityPolicyManager exposes guest security policy operations. | ||
| type SecurityPolicyManager interface { | ||
| AddSecurityPolicy(ctx context.Context, settings interface{}) error | ||
| InjectPolicyFragment(ctx context.Context, settings guestresource.SecurityPolicyFragment) error | ||
| } | ||
|
|
||
| // GuestDeviceManager exposes guest device operations. | ||
| type GuestDeviceManager interface { | ||
| AddVPCIDevice(ctx context.Context, vmBusGUID string) error | ||
| AddVPMemDevice(ctx context.Context, settings guestresource.LCOWMappedVPMemDevice) error | ||
| RemoveVPMemDevice(ctx context.Context, settings guestresource.LCOWMappedVPMemDevice) error | ||
| } | ||
|
|
||
| // BlockCIMsManager exposes guest WCOW block CIM operations. | ||
| type BlockCIMsManager interface { | ||
| AddWCOWBlockCIMs(ctx context.Context, settings *guestresource.CWCOWBlockCIMMounts) error | ||
| RemoveWCOWBlockCIMs(ctx context.Context, settings *guestresource.CWCOWBlockCIMMounts) error | ||
| } | ||
|
|
||
| // ScsiManager exposes guest SCSI operations. | ||
| type ScsiManager interface { | ||
| AddMappedVirtualDisk(ctx context.Context, settings interface{}) error | ||
| AddMappedVirtualDiskForContainerScratch(ctx context.Context, settings interface{}) error | ||
| RemoveMappedVirtualDisk(ctx context.Context, settings interface{}) error | ||
| RemoveSCSIDevice(ctx context.Context, settings guestresource.SCSIDevice) error | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| //go:build windows | ||
|
|
||
| package guestmanager | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| hcsschema "github.com/Microsoft/hcsshim/internal/hcs/schema2" | ||
| "github.com/Microsoft/hcsshim/internal/protocol/guestrequest" | ||
| "github.com/Microsoft/hcsshim/internal/protocol/guestresource" | ||
| "github.com/Microsoft/hcsshim/internal/vm" | ||
| "github.com/pkg/errors" | ||
| ) | ||
|
|
||
| // BlockCIMsManager returns the guest block CIMs manager. | ||
| func (gm *guestManager) BlockCIMsManager() vm.BlockCIMsManager { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you have managers why dont they have the fn's that relate to what they do? vm.BlockCIMManager().AddBlockCIM(settings) ? vm.AddBlockCIM(settings) is what we have today. What is the point of the manager then? |
||
| return gm | ||
| } | ||
|
|
||
| // AddWCOWBlockCIMs adds WCOW block CIM mounts in the guest. | ||
| func (gm *guestManager) AddWCOWBlockCIMs(ctx context.Context, settings *guestresource.CWCOWBlockCIMMounts) error { | ||
| request := &hcsschema.ModifySettingRequest{ | ||
| GuestRequest: guestrequest.ModificationRequest{ | ||
| ResourceType: guestresource.ResourceTypeWCOWBlockCims, | ||
| RequestType: guestrequest.RequestTypeAdd, | ||
| Settings: settings, | ||
| }, | ||
| } | ||
|
|
||
| err := gm.modify(ctx, request.GuestRequest) | ||
| if err != nil { | ||
| return errors.Wrap(err, "failed to add WCOW block CIMs") | ||
| } | ||
|
|
||
| return nil | ||
| } | ||
|
|
||
| // RemoveWCOWBlockCIMs removes WCOW block CIM mounts in the guest. | ||
| func (gm *guestManager) RemoveWCOWBlockCIMs(ctx context.Context, settings *guestresource.CWCOWBlockCIMMounts) error { | ||
| request := &hcsschema.ModifySettingRequest{ | ||
| GuestRequest: guestrequest.ModificationRequest{ | ||
| ResourceType: guestresource.ResourceTypeWCOWBlockCims, | ||
| RequestType: guestrequest.RequestTypeRemove, | ||
| Settings: settings, | ||
| }, | ||
| } | ||
|
|
||
| err := gm.modify(ctx, request.GuestRequest) | ||
| if err != nil { | ||
| return errors.Wrap(err, "failed to remove WCOW block CIMs") | ||
| } | ||
|
|
||
| return nil | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| //go:build windows | ||
|
|
||
| package guestmanager | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| hcsschema "github.com/Microsoft/hcsshim/internal/hcs/schema2" | ||
| "github.com/Microsoft/hcsshim/internal/protocol/guestrequest" | ||
| "github.com/Microsoft/hcsshim/internal/protocol/guestresource" | ||
| "github.com/Microsoft/hcsshim/internal/vm" | ||
|
|
||
| "github.com/pkg/errors" | ||
| ) | ||
|
|
||
| // LayersManager returns the guest layers manager. | ||
| func (gm *guestManager) LayersManager() vm.LayersManager { | ||
| return gm | ||
| } | ||
|
|
||
| // AddCombinedLayers adds combined layers in the guest. | ||
| func (gm *guestManager) AddCombinedLayers(ctx context.Context, settings interface{}) error { | ||
| modifyRequest := &hcsschema.ModifySettingRequest{ | ||
| GuestRequest: guestrequest.ModificationRequest{ | ||
| ResourceType: guestresource.ResourceTypeCombinedLayers, | ||
| RequestType: guestrequest.RequestTypeAdd, | ||
| Settings: settings, | ||
| }, | ||
| } | ||
|
|
||
| err := gm.modify(ctx, modifyRequest.GuestRequest) | ||
| if err != nil { | ||
| return errors.Wrap(err, "AddCombinedLayers failed") | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // AddCWCOWCombinedLayers adds combined layers in CWCOW guest. | ||
| func (gm *guestManager) AddCWCOWCombinedLayers(ctx context.Context, settings interface{}) error { | ||
| modifyRequest := &hcsschema.ModifySettingRequest{ | ||
| GuestRequest: guestrequest.ModificationRequest{ | ||
| ResourceType: guestresource.ResourceTypeCWCOWCombinedLayers, | ||
| RequestType: guestrequest.RequestTypeAdd, | ||
| Settings: settings, | ||
| }, | ||
| } | ||
|
|
||
| err := gm.modify(ctx, modifyRequest.GuestRequest) | ||
| if err != nil { | ||
| return errors.Wrap(err, "AddCWCOWCombinedLayers failed") | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // RemoveCombinedLayers removes combined layers in the guest. | ||
| func (gm *guestManager) RemoveCombinedLayers(ctx context.Context, settings interface{}) error { | ||
| modifyRequest := &hcsschema.ModifySettingRequest{ | ||
| GuestRequest: guestrequest.ModificationRequest{ | ||
| ResourceType: guestresource.ResourceTypeCombinedLayers, | ||
| RequestType: guestrequest.RequestTypeRemove, | ||
| Settings: settings, | ||
| }, | ||
| } | ||
|
|
||
| err := gm.modify(ctx, modifyRequest.GuestRequest) | ||
| if err != nil { | ||
| return errors.Wrap(err, "RemoveCombinedLayers failed") | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // RemoveCWCOWCombinedLayers removes combined layers in CWCOW guest. | ||
| func (gm *guestManager) RemoveCWCOWCombinedLayers(ctx context.Context, settings interface{}) error { | ||
| modifyRequest := &hcsschema.ModifySettingRequest{ | ||
| GuestRequest: guestrequest.ModificationRequest{ | ||
| ResourceType: guestresource.ResourceTypeCWCOWCombinedLayers, | ||
| RequestType: guestrequest.RequestTypeRemove, | ||
| Settings: settings, | ||
| }, | ||
| } | ||
|
|
||
| err := gm.modify(ctx, modifyRequest.GuestRequest) | ||
| if err != nil { | ||
| return errors.Wrap(err, "RemoveCWCOWCombinedLayers failed") | ||
| } | ||
| return nil | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| //go:build windows | ||
|
|
||
| package guestmanager | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| hcsschema "github.com/Microsoft/hcsshim/internal/hcs/schema2" | ||
| "github.com/Microsoft/hcsshim/internal/protocol/guestrequest" | ||
| "github.com/Microsoft/hcsshim/internal/protocol/guestresource" | ||
| "github.com/Microsoft/hcsshim/internal/vm" | ||
| "github.com/pkg/errors" | ||
| ) | ||
|
|
||
| // GuestDeviceManager returns the guest device manager. | ||
| func (gm *guestManager) GuestDeviceManager() vm.GuestDeviceManager { | ||
| return gm | ||
| } | ||
|
|
||
| // AddVPCIDevice adds a VPCI device in the guest. | ||
| func (gm *guestManager) AddVPCIDevice(ctx context.Context, vmBusGUID string) error { | ||
| request := &hcsschema.ModifySettingRequest{ | ||
| GuestRequest: guestrequest.ModificationRequest{ | ||
| ResourceType: guestresource.ResourceTypeVPCIDevice, | ||
| RequestType: guestrequest.RequestTypeAdd, | ||
| Settings: guestresource.LCOWMappedVPCIDevice{ | ||
| VMBusGUID: vmBusGUID, | ||
| }, | ||
| }, | ||
| } | ||
|
|
||
| err := gm.modify(ctx, request.GuestRequest) | ||
| if err != nil { | ||
| return errors.Wrap(err, "failed to add VPCI device") | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // AddVPMemDevice adds a VPMem device in the guest. | ||
| func (gm *guestManager) AddVPMemDevice(ctx context.Context, settings guestresource.LCOWMappedVPMemDevice) error { | ||
| request := &hcsschema.ModifySettingRequest{ | ||
| GuestRequest: guestrequest.ModificationRequest{ | ||
| ResourceType: guestresource.ResourceTypeVPMemDevice, | ||
| RequestType: guestrequest.RequestTypeAdd, | ||
| Settings: settings, | ||
| }, | ||
| } | ||
|
|
||
| err := gm.modify(ctx, request.GuestRequest) | ||
| if err != nil { | ||
| return errors.Wrap(err, "failed to add VPMem device") | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // RemoveVPMemDevice removes a VPMem device in the guest. | ||
| func (gm *guestManager) RemoveVPMemDevice(ctx context.Context, settings guestresource.LCOWMappedVPMemDevice) error { | ||
| request := &hcsschema.ModifySettingRequest{ | ||
| GuestRequest: guestrequest.ModificationRequest{ | ||
| ResourceType: guestresource.ResourceTypeVPMemDevice, | ||
| RequestType: guestrequest.RequestTypeRemove, | ||
| Settings: settings, | ||
| }, | ||
| } | ||
|
|
||
| err := gm.modify(ctx, request.GuestRequest) | ||
| if err != nil { | ||
| return errors.Wrap(err, "failed to remove VPMem device") | ||
| } | ||
| return nil | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you need combine and combine wcow? What is different?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh its C WCOW. I get it. This isn't lcow/wcow difference its confidential vs not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it have to be C 'W' COW? Or can it just be CombineConfidentialLayers() ? Is there something special here from the guest request perspective?