gRPC server for Bicep #1330

andyleejordan · 2025-12-18T00:16:44Z

This adds a new binary dscbicep to the DSC build which is a small, self-contained Rust executable hosting a gRPC server. The methods are defined in bicep.proto from Bicep's extensibility APIs (hence the addition of protoc as a build dependency). It's actually fairly similar to the MCP server, in that it's simply invoking find/get/set/delete resource functions via dsc-lib.

Using bicep local-deploy with this is actually done through a "Bicep extension" as defined in the project bicep-types-dsc. In essence, it's an OCI artifact with two things: the type information in a big JSON file and this dscbicep binary. When all put together and hosted on an OCI registry, Bicep will auto-restore the extension and just deploy the DSC resources defined in any arbitrary set of Bicep files, handling all Bicep functionality and orchestration itself.

While modern Windows technically supports UDS, many libraries (like Rust's Tokio) do not. So unfortunately is much more straightforward to always use named pipes on Windows. ## Description  ## Example Usage PowerShell/DSC#1330 ## Checklist - [x] I have read and adhere to the [contribution guide](https://github.com/Azure/bicep/blob/main/CONTRIBUTING.md). ###### Microsoft Reviewers: [Open in CodeFlow](https://microsoft.github.io/open-pr/?codeflow=https://github.com/Azure/bicep/pull/18712) --------- Co-authored-by: Anthony Martin <38542602+anthony-c-martin@users.noreply.github.com>

andyleejordan

Notes for reviewers.

andyleejordan · 2026-01-14T20:58:37Z

.github/workflows/rust.yml

+      - name: Add WinGet links to PATH
+        if: matrix.platform == 'windows-latest'
+        run: |
+          "$env:LOCALAPPDATA\\Microsoft\\WinGet\\Links" | Out-File -Append -Encoding utf8 -FilePath $env:GITHUB_PATH


This was necessary for Protobuf because protoc.exe is made available by WinGet in this local app data path, which on GitHub images is not already in the PATH, and that is unlike the rest of our dependencies (Node.js is already installed system-wide and so in the PATH, tree-sitter isn't installed by installs via Cargo, this was our first installation on GitHub Actions using WinGet that doesn't run a system installer).

I think you should have this environment change in the build.helpers.psm1 for protobuf tool install instead of a one-off here

This is specific to GitHub Actions. When you install Protobuf via WinGet on Windows normally, protoc is found in the PATH. It appears that GitHub Actions Windows images don't correctly include the WinGet path.

.pipelines/DSC-Official.yml

dscbicep/proto/bicep.proto

dscbicep/src/main.rs

andyleejordan · 2026-01-14T21:03:28Z

dscbicep/src/main.rs

+                properties: result.after_state.to_string(),
+                status: None,
+            }),
+            error_data: None,


Once in use I suspect we may want to transform the error data a little better, though returning Err above works just fine.

andyleejordan · 2026-01-14T21:38:42Z

dscbicep/src/main.rs

+    #[cfg(windows)]
+    if let Some(pipe_name) = pipe {
+        // TODO: This named pipe code is messy and honestly mostly generated. It
+        // does work, but most of the problem lies in minimal Windows support
+        // inside the Tokio library (and no support for UDS).
+        use std::pin::Pin;
+        use std::task::{Context, Poll};
+        use tokio::io::{AsyncRead, AsyncWrite};
+        use tokio::net::windows::named_pipe::ServerOptions;
+        use tonic::transport::server::Connected;
+
+        // Wrapper to implement Connected trait for NamedPipeServer
+        struct NamedPipeConnection(tokio::net::windows::named_pipe::NamedPipeServer);
+
+        impl Connected for NamedPipeConnection {
+            type ConnectInfo = ();
+
+            fn connect_info(&self) -> Self::ConnectInfo {
+                ()
+            }
+        }
+
+        impl AsyncRead for NamedPipeConnection {
+            fn poll_read(
+                mut self: Pin<&mut Self>,
+                cx: &mut Context<'_>,
+                buf: &mut tokio::io::ReadBuf<'_>,
+            ) -> Poll<std::io::Result<()>> {
+                Pin::new(&mut self.0).poll_read(cx, buf)
+            }
+        }
+
+        impl AsyncWrite for NamedPipeConnection {
+            fn poll_write(
+                mut self: Pin<&mut Self>,
+                cx: &mut Context<'_>,
+                buf: &[u8],
+            ) -> Poll<std::io::Result<usize>> {
+                Pin::new(&mut self.0).poll_write(cx, buf)
+            }
+
+            fn poll_flush(
+                mut self: Pin<&mut Self>,
+                cx: &mut Context<'_>,
+            ) -> Poll<std::io::Result<()>> {
+                Pin::new(&mut self.0).poll_flush(cx)
+            }
+
+            fn poll_shutdown(
+                mut self: Pin<&mut Self>,
+                cx: &mut Context<'_>,
+            ) -> Poll<std::io::Result<()>> {
+                Pin::new(&mut self.0).poll_shutdown(cx)
+            }
+        }
+
+        // Windows named pipes must be in the format \\.\pipe\{name}
+        let full_pipe_path = format!(r"\\.\pipe\{}", pipe_name);
+        tracing::info!("Starting Bicep gRPC server on named pipe: {full_pipe_path}");
+
+        // Create a stream that accepts connections on the named pipe
+        let incoming = async_stream::stream! {
+            // Track whether this is the first instance
+            let mut is_first = true;
+
+            loop {
+                let pipe = if is_first {
+                    ServerOptions::new()
+                        .first_pipe_instance(true)
+                        .create(&full_pipe_path)
+                } else {
+                    ServerOptions::new()
+                        .create(&full_pipe_path)
+                };
+
+                let server = match pipe {
+                    Ok(server) => server,
+                    Err(e) => {
+                        tracing::error!("Failed to create named pipe: {}", e);
+                        break;
+                    }
+                };
+
+                is_first = false;
+
+                tracing::debug!("Waiting for client to connect to named pipe...");
+                match server.connect().await {
+                    Ok(()) => {
+                        tracing::info!("Client connected to named pipe");
+                        yield Ok::<_, std::io::Error>(NamedPipeConnection(server));
+                    }
+                    Err(e) => {
+                        tracing::error!("Failed to accept connection: {}", e);
+                        break;
+                    }
+                }
+            }
+        };
+
+        Server::builder()
+            .add_service(BicepExtensionServer::new(service))
+            .serve_with_incoming(incoming)
+            .await?;
+
+        return Ok(());
+    }


See the first comment. This works, it's just a lot more verbose than I would've liked as I did indeed have to relay on Copilot to generate the boilerplate necessary to map one interface to another. Since the Rust community has been arguing about Unix socket support on Windows for half a decade, it's not coming, so we have to use named pipes, which is more Windows-native, but not nearly as well supported in Rust/tonic/tokio. If this can be simplified, please let me know. I was just happy to get it working.

andyleejordan · 2026-01-14T21:39:18Z

dscbicep/src/main.rs

+    // Default to HTTP server on [::1]:50051 if no transport specified
+    let addr = http.unwrap_or_else(|| "[::1]:50051".to_string()).parse()?;
+    tracing::info!("Starting Bicep gRPC server on HTTP: {addr}");
+
+    let reflection_service = tonic_reflection::server::Builder::configure()
+        .register_encoded_file_descriptor_set(proto::FILE_DESCRIPTOR_SET)
+        .build_v1()
+        .unwrap();
+
+    Server::builder()
+        .add_service(reflection_service)
+        .add_service(BicepExtensionServer::new(service))
+        .serve(addr)
+        .await?;


The aforementioned "debug" HTTP server where requests can be sent to DSC over gRPC without Bicep at all (which is really neat).

andyleejordan · 2026-01-14T21:40:45Z

dscbicep/build.rs

+
+    tonic_prost_build::configure()
+        .build_client(false)
+        .file_descriptor_set_path(&descriptor_path)


This is used for the reflection in the HTTP server.

andyleejordan · 2026-01-14T21:41:47Z

dscbicep/Cargo.toml

+[package]
+name = "dscbicep"
+version = "0.1.0"
+edition = "2021"
+
+[[bin]]
+name = "dscbicep"
+path = "src/main.rs"


I have no strong opinions on the name, version, or file structure other than suggesting that changes be made after this PR, as the extension has to incorporate the binary.

andyleejordan · 2026-01-14T21:43:33Z

lib/dsc-lib/src/dscresources/command_resource.rs

+    // Try to use existing runtime first (e.g. from gRPC or MCP server)
+    match tokio::runtime::Handle::try_current() {
+        Ok(handle) => {
+            tokio::task::block_in_place(|| {
+                handle.block_on(run_async)
+            })
+        },
+        // Otherwise create a new runtime
+        Err(_) => {
+            tokio::runtime::Builder::new_multi_thread()
+                .enable_all()
+                .build()
+                .unwrap()
+                .block_on(run_async)
+        }
+    }


This is the only real change inside of DSC. Since under gRPC there's already a runtime, we need to re-use it (and tell it that we're spawning a blocking task so that it manages the threads correctly and doesn't deadlock). Whereas when the DSC CLI invokes these resources, it spins up a runtime just for the sake of threading the processes (yes I know that sounds funny).

Copilot

Pull request overview

This PR adds a new dscbicep binary that implements a gRPC server for Bicep's local-deploy extensibility APIs. The server enables Bicep to invoke DSC resource operations (get, set, delete) via gRPC, allowing DSC resources to be deployed through Bicep's orchestration.

Changes:

Implements gRPC server for Bicep extensibility with Unix socket, Windows named pipe, and HTTP transport support
Refactors tokio runtime handling in command_resource.rs to support nested async contexts
Adds protobuf build dependencies and installation logic to build scripts

Reviewed changes

Copilot reviewed 13 out of 15 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
dscbicep/src/main.rs	New gRPC server implementation with Bicep extension service
dscbicep/proto/bicep.proto	Protobuf definitions for Bicep extensibility protocol
dscbicep/build.rs	Build script for protobuf code generation
dscbicep/Cargo.toml	Dependencies for gRPC and protobuf support
lib/dsc-lib/src/dscresources/command_resource.rs	Refactored to reuse existing tokio runtime when available
build.ps1	Added protobuf installation step
build.helpers.psm1	New Install-Protobuf function for cross-platform protobuf installation
Cargo.toml	Added dscbicep workspace member and gRPC dependencies
.pipelines/DSC-Official.yml	Added protobuf-compiler to CI/CD pipeline
.github/workflows/rust.yml	Added WinGet PATH configuration for Windows builds

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

dscbicep/src/main.rs

lib/dsc-lib/src/dscresources/command_resource.rs

dscbicep/src/main.rs

SteveL-MSFT · 2026-01-14T23:44:37Z

.github/workflows/rust.yml

+      - name: Add WinGet links to PATH
+        if: matrix.platform == 'windows-latest'
+        run: |
+          "$env:LOCALAPPDATA\\Microsoft\\WinGet\\Links" | Out-File -Append -Encoding utf8 -FilePath $env:GITHUB_PATH


I think you should have this environment change in the build.helpers.psm1 for protobuf tool install instead of a one-off here

.pipelines/DSC-Official.yml

dscbicep/proto/bicep.proto

SteveL-MSFT · 2026-01-14T23:48:32Z

dscbicep/src/main.rs

+
+        tracing::debug!("CreateOrUpdate called for {resource_type}@{version:?}: {properties}");
+
+        let mut dsc = DscManager::new();


I think it would be better to create it as a singleton

dscbicep/src/main.rs

SteveL-MSFT · 2026-01-14T23:57:15Z

dscbicep/src/main.rs

+        let mut dsc = DscManager::new();
+        let Some(resource) = dsc.find_resource(&DiscoveryFilter::new(&resource_type, version.as_deref(), None)).unwrap_or(None) else {
+            return Err(Status::not_found("Resource not found"));
+        };
+
+        resource
+            .delete(&identifiers)
+            .map_err(|e| Status::aborted(e.to_string()))?;


This might be where the design is inverted between bicep and DSC. In DSC, delete is called if set doesn't support _exist, but in this case, delete might have to call set with _exist=false to get the same operation. Although maybe this is something the DSC engine could perform.

SteveL-MSFT · 2026-01-14T23:59:07Z

dscbicep/src/main.rs

+        // TODO: Return actual Bicep type definitions...yet the extension already has these?
+        // Perhaps this is where we can dynamically get them from the current system.
+        Err(Status::unimplemented("GetTypeFiles not yet implemented"))


Since the types are only useful for Bicep, perhaps write a small CLI in C# (AOT) to generate the type information and this exe can call out to that? This can be done later

SteveL-MSFT · 2026-01-15T00:00:43Z

dscbicep/src/main.rs

+        return Ok(());
+    }
+
+    // Default to HTTP server on [::1]:50051 if no transport specified


Where did 50051 come from? Is there a risk of conflict?

That's the default gRPC port number (and this is only used in debug/test scenarios, not when connecting via Bicep over a socket or named pipe).

This is imported from: https://github.com/Azure/bicep/blob/main/src/Bicep.Local.Rpc/extension.proto There may be a better way to sync this dependency, but as far as I can tell they're usually just copied like this.

And tonic-prost, and tonic-prost-build... Adds a simple build.rs script for compiling the Protobuf files. Requires the protoc binary: https://protobuf.dev/installation/

Copilot was helpful, then reconciled against this example: https://github.com/hyperium/tonic/blob/master/examples/src/routeguide/server.rs

Since the Bicep extension is very unforgiving on the CLI.

Until Bicep will actually send this.

Upgraded to a "Resource not found" error!

Since the gRPC (and MCP) servers already start one.

Now output works!

andyleejordan · 2026-01-15T19:50:15Z

@SteveL-MSFT changes implemented except DscManager is still per-function since it would otherwise need to be locked and therefore seems to me like it would constrain parallel execution, please let me know if I'm misunderstanding that though. I tried sharing an instance in the BicepExtensionService struct but using it meant putting it into a mutex.

andyleejordan force-pushed the bicep-gRPC branch from 2e40dc5 to c3a918b Compare December 18, 2025 09:39

This was referenced Dec 20, 2025

Always use named pipes on Windows for gRPC Azure/bicep#18712

Merged

Unable to invoke bicep modules in bicep config files #1269

Open

andyleejordan force-pushed the bicep-gRPC branch 5 times, most recently from 712aeed to 46d1cea Compare January 13, 2026 23:07

andyleejordan changed the title ~~WIP: gRPC server for Bicep~~ gRPC server for Bicep Jan 13, 2026

andyleejordan force-pushed the bicep-gRPC branch 3 times, most recently from 60b2211 to 73f3692 Compare January 14, 2026 20:43

andyleejordan marked this pull request as ready for review January 14, 2026 20:54

andyleejordan requested review from SteveL-MSFT and michaeltlombardi January 14, 2026 20:55

andyleejordan force-pushed the bicep-gRPC branch from 73f3692 to 612615f Compare January 14, 2026 21:09

andyleejordan commented Jan 14, 2026

View reviewed changes

SteveL-MSFT requested a review from Copilot January 14, 2026 23:42

Copilot started reviewing on behalf of SteveL-MSFT January 14, 2026 23:43 View session

Copilot AI reviewed Jan 14, 2026

View reviewed changes

SteveL-MSFT requested changes Jan 15, 2026

View reviewed changes

andyleejordan added 8 commits January 15, 2026 11:41

Update .gitignore for tree-sitter files

e598a1d

Add bicep.proto file

bde1fc7

This is imported from: https://github.com/Azure/bicep/blob/main/src/Bicep.Local.Rpc/extension.proto There may be a better way to sync this dependency, but as far as I can tell they're usually just copied like this.

Basic CLI plumbing for Bicep gRPC server

92b6e01

Add Rust tonic and prost packages for gRPC

1868e58

And tonic-prost, and tonic-prost-build... Adds a simple build.rs script for compiling the Protobuf files. Requires the protoc binary: https://protobuf.dev/installation/

Implement stub Bicep gRPC server

4b79f4f

Copilot was helpful, then reconciled against this example: https://github.com/hyperium/tonic/blob/master/examples/src/routeguide/server.rs

Refactor to stand-alone dscbicep binary

8039b6c

Since the Bicep extension is very unforgiving on the CLI.

Always attach debugger

6d74da5

Until Bicep will actually send this.

Start to implement create_or_update gRPC method

3793117

Upgraded to a "Resource not found" error!

andyleejordan added 23 commits January 15, 2026 11:43

Place the dscbicep binary

34d5875

Reuse Tokio runtime in invoke_command

8382304

Since the gRPC (and MCP) servers already start one.

Add tracing

403c2fa

Don't build gRPC client

169eeb6

Basic implementation of all methods

90b7b1f

Check environment for tracing and debug settings

b93d598

Add a default HTTP server for debugging with grpcurl

9900616

Add tonic-reflection package

c693ecc

Add tonic reflection service

841cd13

Refactor use std::

093a165

Implement fuller return responses

621a34d

Fix bugs

df36a0a

Unwrap DSC result structs to property bags

1f37f5d

Now output works!

Simplify and remove direct serde dependency

4266b52

Clean up error handling

177d0b0

Slight socket code clean up

fd310e6

Hacky support for named pipes

c7c1d06

Use tokio correctly by informing it we're about to block

cd87700

Update to use DiscoveryFilter

01629a6

Ensure Protobuf is available in build

aeed883

Use Test-CommandAvailable for protoc and node

d9d2b9e

Add WinGet links to PATH on Windows

254eed3

Localize strings in dscbicep

873c479

andyleejordan force-pushed the bicep-gRPC branch from 9c7d41b to 873c479 Compare January 15, 2026 19:44


		tracing::debug!("CreateOrUpdate called for {resource_type}@{version:?}: {properties}");

		let mut dsc = DscManager::new();

gRPC server for Bicep #1330

Are you sure you want to change the base?

gRPC server for Bicep #1330

Uh oh!

Conversation

andyleejordan commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andyleejordan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andyleejordan commented Jan 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

andyleejordan commented Dec 18, 2025 •

edited

Loading