You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm having some issues with the helm chart in clusters recently, not sure exactly when it failed but I'm currently on boky/postfix:v5.1.0 and the mail container
mail ★★★★★ POSTFIXSTARTINGUP (debian) ★★★★★
mail ‣ NOTESystemaccounts: postfix=100:102, opendkim=101:104.Carefulwhenswitchingdistros.
mail ‣ INFONotsettinganytimezoneforthecontainermail ‣ WARNToobroadpermissions (777) for /var/log. logrotatewilllikelycomplain.
mail ‣ DEBUG /tmpwritable.
mail ‣ INFOUsingplainlogformatforrsyslog.
mail ‣ INFORemoving /var/log/mail.logfrom /etc/logrotate.d/logrotate.confmailsed: cannotrename /etc/logrotate.d/sedQaZYSn: Deviceorresourcebusyexporter2026/01/1423:54:08Readinglogeventsfrom /var/log/mail.logexporter2026/01/1423:54:08Erroropeninglogsource: open /var/log/mail.log: nosuchfileordirectory
I tried rolling the mail container back to v4.4.0 with no luck as I wasn't able to roll the exporter container back either based on the tags available still using latest mutatable tag from 8 days ago
exporter2026/01/1500:34:33Readinglogeventsfrom /var/log/mail.logexporter2026/01/1500:34:33Erroropeninglogsource: open /var/log/mail.log: nosuchfileordirectory
Reacted by Jonathon Beauregard II and Markus Möslinger
Any specific reason you are overriding a bunch of things which are already provided in the default configuration? E.g. I see here you're specifying logrotate.conf, start-up probes, readiness probes...
I thought I was doing default tbh and I hadn't touched any probes on this one so I'm using whats in the chart currently and patching in domain and some scheduling decision items.
helm show values bokysan/mailreplicaCount: 1image:
repository: boky/postfix# tag: "" # If not specified uses chart's AppVersion as the tagpullPolicy: IfNotPresentimagePullSecrets: []nameOverride: ""fullnameOverride: ""serviceAccount:
create: true # Specifies whether a service account should be createdannotations: {} # Annotations to add to the service accountname: ""# The name of the service account to use. If not set and create is true, a name is generated using the fullname templateservice:
type: ClusterIPport: 587labels: {}annotations: {}# externalTrafficPolicy: Cluster# nodePort:# StatefulSets now require a headless service# https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#limitationsheadlessService:
enabled: truelabels: {}annotations: {}metrics:
enabled: falseservice:
labels: {}annotations: {}serviceMonitor:
enabled: falselabels: {}annotations: {}port: 9154path: /metricsimage:
repository: "boky/postfix-exporter"tag: "latest"# Warning: Metrics needs a file to work. If you're using this DO NOT override POSTFIX_maillog_file.# If you do, metrics *will not work*. If you need to log to file:# - use this file# - add another file to /etc/rsyslog.d/*.confmaillog: /var/log/mail.loglogrotate:
enabled: truelogrotate.conf: | /var/log/mail.log { copytruncate rotate 1 monthly minsize 1M compress missingok notifempty dateext olddir /var/log/ maxage 90 }resources: {}# If you do want to specify resources, uncomment the following# lines, adjust them as necessary, and remove the curly braces after 'resources:'.# limits:# cpu: 100m# memory: 128Mi# requests:# cpu: 100m# memory: 128Mi# Tell helm to restart (recreate) pods on every deploy. Setting this to true will inject# `date/deploy-date: <timestamp>` annotation into pod specification for StateFulset. This# ensures that the Pod is recreated with the new changes.## Feel free to disable this but doing so **IS IT YOUR OWN RISK**. By disabling this feature# you might run into issues where some changes are not reflected on the Pod. If you experience# this with this setting set to `false` the first thing to try out is to restart / delete# a pod and let `StatefulSet` recreate it.recreateOnRedeploy: trueresources:
{}# We usually recommend not to specify default resources and to leave this as a conscious# choice for the user. This also increases chances charts run on environments with little# resources, such as Minikube. If you do want to specify resources, uncomment the following# lines, adjust them as necessary, and remove the curly braces after 'resources:'.# limits:# cpu: 100m# memory: 128Mi# requests:# cpu: 100m# memory: 128Miautoscaling:
enabled: falseminReplicas: 1maxReplicas: 100targetCPUUtilizationPercentage: 80# targetMemoryUtilizationPercentage: 80labels: {}annotations: {}nodeSelector: {}tolerations: []affinity: {}extraVolumes: []extraVolumeMounts: []extraInitContainers: []extraEnv: []extraContainers: []deployment:
labels: {}annotations: {}pod:
securityContext: {}# Default setup# capabilities:# drop:# - ALL# readOnlyRootFilesystem: true# capabilities:# drop:# - ALL# readOnlyRootFilesystem: true# runAsNonRoot: true# runAsUser: 1000labels: {}annotations: {}container:
postfix:
securityContext: {}# capabilities:# drop:# - ALL# readOnlyRootFilesystem: truecerts:
# Auto-generate certificates for the server and mount them into Postfix volumecreate: false# Provide existing secret nameexistingSecret: ""# Define data which should be stored in a Secret# (and shared with the pod as environment variables)# secret:# hello: worldsecret: {}# Use an existing secret to share with the pod# as environment variables.existingSecret: ""# Define a secret which should be deployed together with the# chart amd mounted into a specific directory in the pod.mountSecret:
enabled: falsepath: /var/lib/secretdata:
{}# e.g.# dkim-private: |# -----BEGIN RSA PRIVATE KEY-----# < redacted ># -----END RSA PRIVATE KEY-----config:
general:
{}# e.g.# TZ:# FORCE_COLOR:# INBOUND_DEBUGGING:# ALLOWED_SENDER_DOMAINS:# ALLOW_EMPTY_SENDER_DOMAINS:# LOG_FORMAT:# RELAYHOST:# SASL_RELAYHOST: # when specified overwrites the RELAYHOST in the sasl_passwd file# RELAYHOST_USERNAME:# RELAYHOST_PASSWORD:# MASQUERADED_DOMAINS:# SMTP_HEADER_CHECKS:# DKIM_SELECTOR:# DKIM_AUTOGENERATE:postfix:
{}# e.g.# myhostname: "postfix"# smtp_tls_security_level: "encrypt"opendkim:
{}# e.g.# RequireSafeKeys: "yes"dns:
policy: ""# policy: "None"nameservers: []# - 8.8.8.8# - 8.8.4.4searches: []# searches:# - "default.svc.cluster.local"# - "svc.cluster.local"# - "cluster.local"options: []# options:# - name: ndots# value: "5"persistence:
enabled: trueaccessModes:
- ReadWriteOnceexistingClaim: ""size: 1GistorageClass: ""readinessProbe:
initialDelaySeconds: 10periodSeconds: 60failureThreshold: 6timeoutSeconds: 8exec:
command:
- sh
- -c
- /scripts/healthcheck.shlivenessProbe:
initialDelaySeconds: 5periodSeconds: 5failureThreshold: 2exec:
command:
- sh
- -c
- >- ps axf | fgrep -v grep | egrep -q '\{supervisord\}|/usr/bin/supervisord' && ps axf | fgrep -v grep | egrep -q '(/usr/lib/postfix/sbin/|/usr/libexec/postfix/)master'startupProbe:
initialDelaySeconds: 5periodSeconds: 5failureThreshold: 12exec:
command:
- sh
- -c
- >- ps axf | fgrep -v grep | egrep -q '\{supervisord\}|/usr/bin/supervisord' && ps axf | fgrep -v grep | fgrep -q "postfix-script" && ps axf | fgrep -v grep | fgrep -q 'opendkim'lifecycle:
postStart: {}preStopSleepSeconds: 0# Allow up to 2 minutes for Postfix to flush / empty the queue before shutting down the containerterminationGracePeriodSeconds: 120
I also tried to update to Version 5.1.0 today and stumbled over the same issue:
★★★★★ POSTFIX STARTING UP (debian) ★★★★★
‣ NOTE System accounts: postfix=100:102, opendkim=101:104. Careful when switching distros.
‣ NOTE Setting container timezone to: Europe/Berlin
‣ DEBUG /tmp writable.
‣ INFO Using json log format for rsyslog.
‣ INFO Removing /var/log/mail.log from /etc/logrotate.d/logrotate.conf
sed: cannot rename /etc/logrotate.d/sedeDC1C1: Device or resource busy
I have also a slightly changed logrotate config, but it´s running fine for a long time
I also have just run into this on 5.1.0, 5.0.1, 5.0.0. So anything past 4.4.0 which we have been running for a bit.
---
image:
repository: boky/postfix
tag: "v5.1.0" # If not specified uses chart's AppVersion as the tag
fullnameOverride: "mail"
# if the following is true, config.prefix.myhostname should not be set
# Requires get-public-ip init container
lookupHostName:
enabled: true
extraInitContainers: |
- name: init-subpaths
image: busybox:stable
command:
- sh
- -c
- |
mkdir -p /mnt/mail/spool && \
mkdir -p /mnt/mail/logs
volumeMounts:
- name: mail
mountPath: /mnt/mail
config:
general:
# TZ:
# INBOUND_DEBUGGING:
...
# DKIM_AUTOGENERATE:
postfix:
# e.g.
smtp_tls_security_level: "may"
smtpd_tls_security_level: "may"
smtpd_tls_cert_file: /var/run/certs/tls.cert
smtpd_tls_key_file: /var/run/certs/tls.key
maximal_queue_lifetime: "1d"
opendkim: {}
# e.g.
# RequireSafeKeys: "yes"
#This does not create the cert, but it does setup for reading the cert that is manualy made.
certs:
create: true
metrics:
enabled: true
serviceMonitor:
enabled: true
port: 9154
#This is what https://github.com/kumina/postfix_exporter is expecting, so setting as default
maillog: /var/log/mail.log
logrotate:
enabled: true
persistence:
enabled: true
size: 5Gi
storageClass: "local-path"
replicaCount: 5
...
★★★★★ POSTFIX STARTING UP (debian) ★★★★★
‣ NOTE System accounts: postfix=100:102, opendkim=101:104. Careful when switching distros.
‣ INFO Not setting any timezone for the container
‣ DEBUG /tmp writable.
‣ INFO Using plain log format for rsyslog.
‣ INFO Removing /var/log/mail.log from /etc/logrotate.d/logrotate.conf
sed: cannot rename /etc/logrotate.d/sediWncQD: Device or resource busy
However, if I leave the metrics block out (so don't enable that), then it does start successfully. I suspect that enabling that enables a different code path related to logrotate (which I understand the exporter needs) and because it is trying to modify a file in the image rather than on a mounted volume it triggers the behaviour which @origpumu notices above. Hopefully this helps you with debugging the root cause for this @bokysan!
This issue affected us when upgrading as well! I opened #266, which only uses sed when logrotate is writable (-> doesn't use it when mounted in Kubernetes).
@NerdyShawn While the fix hasn't been released yet, you can set the tag built from master branch (edge-alpine, edge-ubuntu or edge-debian/edge) in image.tag - this should already include the fix, even when used with the 5.1.0 chart version!
If you don't like floating tags, you could pin image.tag by setting it to edge-debian@sha256:fda621f6ff3980d397719b10916c75680308a7759ddead33695555f365a903d9 (or whatever variant/digest you want to use)
But be aware that the Helm Chart currently only seems to support digests via image.tag, using tagname@sha:256:digest format. Digest only, without a tag, would be rendered in Pod as :@, like boky/postfix:@sha256:fda621f6ff3980d397719b10916c75680308a7759ddead33695555f365a903d9.
Activity
bokysan commentedon Jan 15, 2026
Can you give me your
values.yamlto try to replicate the issue?NerdyShawn commentedon Jan 15, 2026
sure thing
bokysan commentedon Jan 15, 2026
Any specific reason you are overriding a bunch of things which are already provided in the default configuration? E.g. I see here you're specifying
logrotate.conf, start-up probes, readiness probes...Any reason not to leave it at default?
NerdyShawn commentedon Jan 15, 2026
I thought I was doing default tbh and I hadn't touched any probes on this one so I'm using whats in the chart currently and patching in domain and some scheduling decision items.
I'm sourcing the chart from here
Here is what that looks like
frankhetterich commentedon Jan 20, 2026
I also tried to update to Version 5.1.0 today and stumbled over the same issue:
I have also a slightly changed logrotate config, but it´s running fine for a long time
But i don´t think that this config has something todo with the startup issue, because when the startup script fails, postfix isn´t started yet.
qharnay-cc commentedon Jan 26, 2026
I encounter the same issue but did not realize this issue was open : I commented in a previous closed issue about logrotate : #246 (comment)
I also have a logrotate config in values.yaml
origpumu commentedon Jan 28, 2026
Has anyone tried this? https://unix.stackexchange.com/a/632478
jonathon2nd commentedon Jan 29, 2026
I also have just run into this on 5.1.0, 5.0.1, 5.0.0. So anything past 4.4.0 which we have been running for a bit.
bokysan commentedon Feb 2, 2026
Thank you for the reports -- I will look at this as soon as possible.
sivarama-p-raju commentedon Feb 13, 2026
@bokysan
Please note that I see the same issues as well on upgrading to
v5.1.0.Is there any workaround for this issue right now or will a fix be included in the next release ?
cnorthwood commentedon Feb 19, 2026
If it helps I get this on a fresh install into my k3s cluster, with the following values so I'm not overriding any of the logrotate config at all.
However, if I leave the metrics block out (so don't enable that), then it does start successfully. I suspect that enabling that enables a different code path related to logrotate (which I understand the exporter needs) and because it is trying to modify a file in the image rather than on a mounted volume it triggers the behaviour which @origpumu notices above. Hopefully this helps you with debugging the root cause for this @bokysan!
outofrange commentedon Feb 24, 2026
This issue affected us when upgrading as well! I opened #266, which only uses sed when logrotate is writable (-> doesn't use it when mounted in Kubernetes).
NerdyShawn commentedon Mar 5, 2026
This is still an issue for me in k8s. I'm running latest release chart so I think the issue should remain open.
outofrange commentedon Mar 6, 2026
@NerdyShawn While the fix hasn't been released yet, you can set the tag built from master branch (
edge-alpine,edge-ubuntuoredge-debian/edge) inimage.tag- this should already include the fix, even when used with the 5.1.0 chart version!If you don't like floating tags, you could pin
image.tagby setting it toedge-debian@sha256:fda621f6ff3980d397719b10916c75680308a7759ddead33695555f365a903d9(or whatever variant/digest you want to use)But be aware that the Helm Chart currently only seems to support digests via
image.tag, usingtagname@sha:256:digestformat. Digest only, without a tag, would be rendered in Pod as:@, likeboky/postfix:@sha256:fda621f6ff3980d397719b10916c75680308a7759ddead33695555f365a903d9.