feat(plugins): allow plugin defintion by pha91 · Pull Request #680 · cloudnative-pg/charts

pha91 · 2025-10-07T09:35:01Z

Allow to add plugins to the Cluster definition fixes #634
Prepare migration to the new Barman Cloud Plugin including the ObjectStore.

…ugin Signed-off-by: Philipp Hamann <84906475+pha91@users.noreply.github.com>

eriksjolund · 2025-10-07T15:48:57Z

...uster/test/postgresql-cluster-configuration/01-non_default_configuration_cluster-assert.yaml

    limits:
      cpu: 100m
-      memory: 256Mi
+      memory: 256Mi  


trailing spaces after memory: 256Mi

Signed-off-by: Philipp Hamann <84906475+pha91@users.noreply.github.com>

Const-antine · 2025-10-17T00:55:49Z

As far as I can see, the new objectStore.yaml template does not support the spec.retentionPolicy knob. More information on migrating this functionality here.

pha91 · 2025-10-17T07:33:40Z

Added the retentionPolicy to the ObjectStore. Thanks @Const-antine for the reminder

Signed-off-by: Philipp Hamann <84906475+pha91@users.noreply.github.com>

ChrisTomAlxHitachi · 2025-11-04T12:58:00Z

@itay-grudev Is there anything pending on this PR?

eriksjolund · 2025-11-04T13:29:58Z

I tried out the PR with such a values file:

type: postgresql

version:
  # -- PostgreSQL major version to use
  postgresql: "16"
  # -- If using TimescaleDB, specify the version
  timescaledb: "2.15"
  # -- If using PostGIS, specify the version
  postgis: "3.4"

mode: standalone

cluster:
  # -- Number of instances
  instances: 3

  storage:
    size: 4Gi
    storageClass: ""

  resources:
    limits:
      cpu: 500m
      memory: 1Gi
    requests:
      cpu: 500m
      memory: 1Gi

  plugins:
     - name: barman-cloud.cloudnative-pg.io
       enabled: true
       isWALArchiver: true
       parameters:
          barmanObjectName: my-postgres-cluster-object-store
          serverName:

backups:
  enabled: true
  endpointURL: "https://s3.example.com"

  destinationPath: "s3://mybucket/testcnpg1"
  provider: s3
  secret:
    # -- Whether to create a secret for the backup credentials
    create: false
    # -- Name of the backup credentials secret
    name: netapp-s3-creds

  scheduledBackups:
    - name: daily-backup
      schedule: "0 2 14 * * *"

  # -- Retention policy for backups
  retentionPolicy: "30d"

It worked. One thing though, I was not able to use

    accessKey: "AWS_ACCESS_KEY_ID"
    secretKey: "AWS_SECRET_ACCESS_KEY"

(but that's not so important).

To make it work I had to use

          barmanObjectName: my-postgres-cluster-object-store

together with

helm upgrade --install -n my-namespace my-postgres -f ~/values.yaml  ./cluster

It seems barmanObjectName should be set to my-postgres + -cluster-object-store

(I used my-postgres as helm release name)

Regarding the software design:

I haven't really thought it through yet, but maybe it would be better to store the S3 configurations under under parameters: together with a name field? Such S3 configurations could then be referenced by name under backup: and under recovery:. Has anyone else been thinking about this too?

pha91 · 2025-11-20T07:21:24Z

Hi @eriksjolund,
just to understand your issue with the accessKey & secretKey:
Where did you set the key?

    accessKey: "AWS_ACCESS_KEY_ID"
    secretKey: "AWS_SECRET_ACCESS_KEY"

As a parameter in the plugin definiton section or in the barmanObjectStore config?
Because I've not changed the behaviour of the barmanObjectStoreConfig function in the template where the keys should be set.

The idea was to use that config in the BarmanObjectStore CRD and reference this object in the plugin like documented here: Defining ObjectStore

eriksjolund · 2025-11-24T14:51:01Z

I have one configuration that is working fine:

helm  --kubeconfig  ~/Downloads/cluster.yaml upgrade --install  -n testing my-postgres -f ~/values.yaml  ./cluster

The cluster, scheduledbackup and the backup are created just fine.

As a test I wanted to see if there is a way to modify ~/values.yaml so that the names in my secret netapp-s3-creds could be changed from

data:
  ACCESS_KEY_ID: retracted
  ACCESS_SECRET_KEY: retracted

to

data:
  AWS_ACCESS_KEY_ID: retracted
  AWS_SECRET_ACCESS_KEY: retracted

My idea was to add:

 % diff -U6 -u  ~/values.yaml ~/values2.yaml
--- /Users/myuser/values.yaml   2025-11-03 16:58:22
+++ /Users/myuser/values2.yaml  2025-11-24 15:16:12
@@ -37,17 +37,20 @@
 backups:
   enabled: true
   endpointURL: "https://s3.example.com"

   destinationPath: "s3://mybucket/testcnpg"
   provider: s3
+  s3:
+    accessKey: "AWS_ACCESS_KEY_ID"
+    secretKey: "AWS_SECRET_ACCESS_KEY"
   secret:
     # -- Whether to create a secret for the backup credentials
     create: false
     # -- Name of the backup credentials secret
     name: netapp-s3-creds

   scheduledBackups:
     - name: daily-backup
       schedule: "0 2 14 * * *"

   # -- Retention policy for backups

When I tried it out, the backup failed with the error message plugin failed rpc error: code = Unknown desc = missing key ACCESS_KEY_ID, inside secret netapp-s3-creds

For details, see this command

% kubectl  --kubeconfig  ~/Downloads/cluster.yaml get -n testing backup.postgresql.cnpg.io  | grep failed
my-postgres-cluster-daily-backup-20251124141718            6m25s   my-postgres-cluster            plugin   failed      rpc error: code = Unknown desc = missing key ACCESS_KEY_ID, inside secret netapp-s3-creds

(I don't know if it should be possible change the names in the secret)

pha91 · 2025-12-09T10:08:54Z

@itay-grudev do you need any other features or informations to merge this PR?

charts/cluster/templates/objectStore.yaml

…same namespace Signed-off-by: Philipp Hamann <84906475+pha91@users.noreply.github.com>

5h4k4r · 2025-12-18T07:18:55Z

Hello maintainers,
This PR would unblock a use case we’re currently depending on.
No rush of course — just wanted to share that it would be very valuable for us.
Thanks!

pschichtel · 2026-01-04T22:54:39Z

thanks @tamcore for pointing me to this, I somehow didn't see it when checking for existing PRs.

I personally feel like this chart would be better served by keeping it simpler. Instead of trying to manage every object and property in detail, a few more general options (list of plugins, list of externalClusters, ...) would be simpler to use, maintain and extend. Some charts have *Spec properties to simply passthrough underlying resources, if the helm chart would otherwise just duplicate it (e.g. prometheus or rook).

Umbrella charts are a nice way to introduce additional resources like the ObjectStore here. #775 I think is the minimal change to get at least backups going via cnpg-i. I have that running in a cluster of mine.

silanosa · 2026-01-22T16:19:21Z

@itay-grudev Could we please get this merged and released? This would enable us to complete the migration to the Barman Cloud Plugin and would be really appreciated.

itay-grudev · 2026-01-22T19:16:55Z

I'll put it on my task list for next week. There are tests so it would make the job easier.

itboy87 · 2026-02-06T08:45:55Z

Hi @itay-grudev when can we expect this PR to be merged? This is quite critical for us at the moment, as we’re currently unable to use the backup plugin.

canaykin · 2026-02-13T16:25:15Z

Greetings,
We are using an s3 compatible storage provider as explained here which requires following env vars to function correctly:

cluster:
  env:
    - name: AWS_REQUEST_CHECKSUM_CALCULATION
      value: when_required
    - name: AWS_RESPONSE_CHECKSUM_VALIDATION
      value: when_required

In the current chart with legacy barman object store, this can be achieved by setting them under cluster.env . It would be really cool if you can keep this functionality by adding the option to define env vars for the ObjectStore via spec.instanceSidecarConfiguration:

apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
...
spec:
  instanceSidecarConfiguration:
    env:
      - name: AWS_REQUEST_CHECKSUM_CALCULATION
        value: when_required
      - name: AWS_RESPONSE_CHECKSUM_VALIDATION
        value: when_required

Thank you!

ProbstDJakob

I've looked through the PR (except for the files not containing a comment) and have some suggestions with the most important being the way the ObjectStore CR receives its name. This has also pointed out by @eriksjolund but seems to have been somewhat lost in the review comments.

ProbstDJakob · 2026-02-17T14:20:34Z

charts/cluster/templates/_helpers.tpl

+{{- $hasPlugin := false }}
+{{- if .Values.cluster.plugins }}
+  {{- range .Values.cluster.plugins }}
+    {{- if and (eq .name "barman-cloud.cloudnative-pg.io") .enabled }}


In the CRD enabled is set to true by default, thus this may be changed to something like this:

Suggested change

{{- if and (eq .name "barman-cloud.cloudnative-pg.io") .enabled }}

{{- if and (eq .name "barman-cloud.cloudnative-pg.io") (ne .enabled false) }}

Furthermore I guess this should also check if isWALArchiver is true (but I am not sure):

Suggested change

{{- if and (eq .name "barman-cloud.cloudnative-pg.io") .enabled }}

{{- if and (eq .name "barman-cloud.cloudnative-pg.io") (ne .enabled false) .isWALArchiver }}

Depending whether the first or no suggestion gets applied, the suggestion in #680 (comment) needs to get adapted (twice; in both nested template definitions).

ProbstDJakob · 2026-02-17T14:37:20Z

charts/cluster/templates/_helpers.tpl

+{{/*
+Check if barman-cloud plugin exists and is enabled
+*/}}
+{{- define "cluster.useBarmanCloudPlugin" -}}
+{{- $hasPlugin := false }}
+{{- if .Values.cluster.plugins }}
+  {{- range .Values.cluster.plugins }}
+    {{- if and (eq .name "barman-cloud.cloudnative-pg.io") .enabled }}
+      {{- $hasPlugin = true }}
+    {{- end }}
+  {{- end }}
+{{- end }}
+{{- $hasPlugin }}
+{{- end }}


If the suggestion with the ObjectStore name will be accepted the following could be helpful:

Suggested change

{{/*

Check if barman-cloud plugin exists and is enabled

*/}}

{{- define "cluster.useBarmanCloudPlugin" -}}

{{- $hasPlugin := false }}

{{- if .Values.cluster.plugins }}

{{- range .Values.cluster.plugins }}

{{- if and (eq .name "barman-cloud.cloudnative-pg.io") .enabled }}

{{- $hasPlugin = true }}

{{- end }}

{{- end }}

{{- end }}

{{- $hasPlugin }}

{{- end }}

{{/*

Patch the plugins list to contain auto generated information and return it formatted as JSON

*/}}

{{- define "cluster.patchPlugins" -}}

{{- $plugins := list }}

{{- range default list (deepCopy .Values.cluster.plugins) }}

{{- if and (eq .name "barman-cloud.cloudnative-pg.io") (ne .enabled false) .isWALArchiver }}

{{- $_ := set . "parameters" (merge (default dict .parameters) (dict "barmanObjectName" (printf "%s-object-store" (include "cluster.fullname" .)))) }}

{{- end }}

{{- $plugins = append $plugins . }}

{{- end }}

{{- toJson $plugins }}

{{- end }}

{{/*

Check if barman-cloud plugin exists, is enabled, is WAL archiver, and return the plugin parameters (or null) formatted as JSON

*/}}

{{- define "cluster.findBarmanCloudPlugin" -}}

{{- range include "cluster.patchPlugins" . | fromJsonArray }}

{{- if and (eq .name "barman-cloud.cloudnative-pg.io") (ne .enabled false) .isWALArchiver }}

{{- toJson .parameters }}

{{- break }}

{{- end }}

{{- end }}

{{- end }}

Usage would then be as follows:

{{ if include "cluster.findBarmanCloudPlugin" . | fromJson }}

or:

{{ include "cluster.findBarmanCloudPlugin" . | fromJson | .barmanObjectName | quote }}

ProbstDJakob · 2026-02-17T15:33:35Z

charts/cluster/templates/cluster.yaml

+  {{- with .Values.cluster.plugins }}
+  plugins:
+    {{- toYaml . | nindent 4}}
+  {{- end }}


If the suggestion with the ObjectStore name and the corresponding change within the _helpers.tpl will be accepted this must be changed to:

Suggested change

{{- with .Values.cluster.plugins }}

plugins:

{{- toYaml . | nindent 4}}

{{- end }}

plugins: {{ include "cluster.patchPlugins" . }}

ProbstDJakob · 2026-02-17T15:39:13Z

charts/cluster/templates/objectStore.yaml

+apiVersion: barmancloud.cnpg.io/v1
+kind: ObjectStore
+metadata:
+  name: {{ include "cluster.fullname" $  }}-object-store


The name is currently "hardcoded" but must also be specified within the plugin declaration. This creates the problem, that the user must know/predict the name to be used and if that ever changes (either by changing what cluster.fullname renders, the suffix/prefix, the "full" name), one must adapt their values even though they should not need to. Within the _helpers.tpl and cluster.yaml are suggestions which would fix this and set a default barmanObjectName plugin parameter if non is given. If this change is accepted, the name should be change to:

Suggested change

name: {{ include "cluster.fullname" $ }}-object-store

name: {{ include "cluster.findBarmanCloudPlugin" . | fromJson | .barmanObjectName | quote }}

Possibly save the output of include "cluster.findBarmanCloudPlugin" . | fromJson in a variable so that it must not be computed twice (for the name and the if in line one).

ProbstDJakob · 2026-02-17T16:02:17Z

charts/cluster/templates/scheduled-backups.yaml

+{{- if (eq (include "cluster.useBarmanCloudPlugin" $context ) "true") }}
+  pluginConfiguration:
+    name: barman-cloud.cloudnative-pg.io
+  method: plugin
+{{ else }}
+  method: {{ .method }}
+{{- end }}


I am not sure if this should be "hardcoded" in case the barman cloud plugin is enabled. In case it is possible to still use volumeSnapshot as method even though the barman cloud plugin is enabled (I do not know if this is possible) the following may be better:

Suggested change

{{- if (eq (include "cluster.useBarmanCloudPlugin" $context ) "true") }}

pluginConfiguration:

name: barman-cloud.cloudnative-pg.io

method: plugin

{{ else }}

method: {{ .method }}

{{- end }}

{{- $barmanCloudPluginParameters := include "cluster.findBarmanCloudPlugin" . | fromJson }}

{{- if or (eq .method "plugin") (and (eq .method "") (or .pluginConfiguration $barmanCloudPluginParameters)) }}

method: plugin

{{- $pluginConfiguration := default dict (deepCopy .pluginConfiguration) }}

{{- if $barmanCloudPluginParameters }}

{{- $_ := set $pluginConfiguration "name" (default "barman-cloud.cloudnative-pg.io" $pluginConfiguration.name) }}

{{- end }}

pluginConfiguration: {{ toJson $pluginConfiguration }}

{{ else }}

method: {{ default "barmanObjectStore" .method | quote }}

{{- end }}

ProbstDJakob · 2026-02-17T16:11:32Z

charts/cluster/values.schema.json

                            },
                            "method": {
                                "type": "string"
                            },


In case the suggestion within the scheduled-backups.yaml will be accepted, the following property should be added:

Suggested change

},

"pluginConfiguration": {

"type": "object",

"properties": {

"name": {

"type": "string"

},

"parameters": {

"type": "object"

"additionalProperties": {

"type": "string"

}

}

}

},

ProbstDJakob · 2026-02-17T16:22:44Z

charts/cluster/values.yaml

      # -- Backup owner reference
      backupOwnerReference: self
      # -- Backup method, can be `barmanObjectStore` (default) or `volumeSnapshot`
      method: barmanObjectStore


In case the suggestion within the scheduled-backups.yaml will be accepted, this should be changed to the following:

Suggested change

method: ""

pluginConfiguration:

name: ""

parameters: {}

ProbstDJakob · 2026-02-17T16:34:20Z

charts/cluster/README.md

 | backups.s3.region | string | `""` |  |
 | backups.s3.secretKey | string | `""` |  |
 | backups.scheduledBackups[0].backupOwnerReference | string | `"self"` | Backup owner reference |
 | backups.scheduledBackups[0].method | string | `"barmanObjectStore"` | Backup method, can be `barmanObjectStore` (default) or `volumeSnapshot` |


In case the suggestion within the scheduled-backups.yaml will be accepted, this should be changed to the following (possibly also sort the table entries):

Suggested change

| backups.scheduledBackups[0].method | string | `"plugin"` if `backups.scheduledBackups[0].pluginConfiguration.name` is not empty or the barman cloud plugin is added to `cluster.plugins`, otherwise ``"barmanObjectStore"` | Backup method, can be `barmanObjectStore` (conditional default), `plugin` (conditional default) or `volumeSnapshot` |

| backups.scheduledBackups[0].pluginConfiguration.name | string | `"barman-cloud.cloudnative-pg.io"` if the barman cloud plugin is added to `cluster.plugins` and `backups.scheduledBackups[0].method` defaults to/equals `"plugin"` | The name of the plugin to use for the backup schedule |

| backups.scheduledBackups[0].pluginConfiguration.parameters | object | `{}` | Parameters to pass to the plugin |

pha91 requested a review from itay-grudev as a code owner October 7, 2025 09:35

dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. chart( cluster ) Related to the cluster chart labels Oct 7, 2025

pha91 marked this pull request as draft October 7, 2025 10:56

pha91 force-pushed the feat/cluster-plugin-support branch from ef1fe69 to fff1e84 Compare October 7, 2025 12:54

feat(plugins): add pluging including backup logic for barman cloud pl…

3118f67

…ugin Signed-off-by: Philipp Hamann <84906475+pha91@users.noreply.github.com>

pha91 force-pushed the feat/cluster-plugin-support branch from fff1e84 to 3118f67 Compare October 7, 2025 12:54

pha91 marked this pull request as ready for review October 7, 2025 13:07

eriksjolund reviewed Oct 7, 2025

View reviewed changes

pha91 added 2 commits October 8, 2025 07:42

fix(test): remove trailing spaces

7782a65

Signed-off-by: Philipp Hamann <84906475+pha91@users.noreply.github.com>

feat(recovery): use plugin to recover cluster when plugin is enabled

b24dec1

Signed-off-by: Philipp Hamann <84906475+pha91@users.noreply.github.com>

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Oct 8, 2025

featue(objectStore): add retentionPolicy to ObjectStore

0289e16

Signed-off-by: Philipp Hamann <84906475+pha91@users.noreply.github.com>

pha91 force-pushed the feat/cluster-plugin-support branch from 5a84d88 to 0289e16 Compare October 17, 2025 07:37

mikeshootzz mentioned this pull request Nov 17, 2025

Add CNPG cluster chart as vshnpostgresql vshn/appcat-charts#14

Closed

5 tasks

Merge branch 'main' into feat/cluster-plugin-support

a4a30b7

Merge branch 'main' into feat/cluster-plugin-support

f3a73a2

tamcore reviewed Dec 11, 2025

View reviewed changes

charts/cluster/templates/objectStore.yaml Show resolved Hide resolved

fix(objectStore): ensure cluster and objectstore will be deployed to …

0b68ca7

…same namespace Signed-off-by: Philipp Hamann <84906475+pha91@users.noreply.github.com>

tamcore mentioned this pull request Jan 4, 2026

Add support for cnpg-i plugins #775

Open

jdesnoes mentioned this pull request Feb 2, 2026

The plugins is missing in the cluster helm chart #634

Open

ProbstDJakob suggested changes Feb 17, 2026

View reviewed changes

	{{- if and (eq .name "barman-cloud.cloudnative-pg.io") .enabled }}
	{{- if and (eq .name "barman-cloud.cloudnative-pg.io") (ne .enabled false) }}

	name: {{ include "cluster.fullname" $ }}-object-store
	name: {{ include "cluster.findBarmanCloudPlugin" . \| fromJson \| .barmanObjectName \| quote }}

+                            },
+                            "pluginConfiguration": {
+                              "type": "object",
+                              "properties": {
+                                "name": {
+                                  "type": "string"
+                                },
+                                "parameters": {
+                                  "type": "object"
+                                  "additionalProperties": {
+                                    "type": "string"
+                                  }
+                                }
+                              }
+                            },

+      method: ""
+      pluginConfiguration:
+        name: ""
+        parameters: {}

+| backups.scheduledBackups[0].method | string | `"plugin"` if `backups.scheduledBackups[0].pluginConfiguration.name` is not empty or the barman cloud plugin is added to `cluster.plugins`, otherwise ``"barmanObjectStore"` | Backup method, can be `barmanObjectStore` (conditional default), `plugin` (conditional default) or `volumeSnapshot` |
+| backups.scheduledBackups[0].pluginConfiguration.name | string | `"barman-cloud.cloudnative-pg.io"` if the barman cloud plugin is added to `cluster.plugins` and `backups.scheduledBackups[0].method` defaults to/equals `"plugin"` | The name of the plugin to use for the backup schedule |
+| backups.scheduledBackups[0].pluginConfiguration.parameters | object | `{}` | Parameters to pass to the plugin |

Conversation

pha91 commented Oct 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Const-antine commented Oct 17, 2025

Uh oh!

pha91 commented Oct 17, 2025

Uh oh!

ChrisTomAlxHitachi commented Nov 4, 2025

Uh oh!

eriksjolund commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pha91 commented Nov 20, 2025

Uh oh!

eriksjolund commented Nov 24, 2025

Uh oh!

pha91 commented Dec 9, 2025

Uh oh!

Uh oh!

5h4k4r commented Dec 18, 2025

Uh oh!

pschichtel commented Jan 4, 2026

Uh oh!

silanosa commented Jan 22, 2026

Uh oh!

itay-grudev commented Jan 22, 2026

Uh oh!

itboy87 commented Feb 6, 2026

Uh oh!

canaykin commented Feb 13, 2026

Uh oh!

ProbstDJakob left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

Comments

eriksjolund commented Nov 4, 2025 •

edited

Loading