Skip to content

logrotate device or resource busy #263

Closed

Description

@NerdyShawn

I'm having some issues with the helm chart in clusters recently, not sure exactly when it failed but I'm currently on boky/postfix:v5.1.0 and the mail container

mail ★★★★★ POSTFIX STARTING UP (debian) ★★★★★                                                            
mail ‣ NOTE  System accounts: postfix=100:102, opendkim=101:104. Careful when switching distros.         
mail ‣ INFO  Not setting any timezone for the container                                                  
mail ‣ WARN  Too broad permissions (777) for /var/log. logrotate will likely complain.                   
mail ‣ DEBUG /tmp writable.                                                                              
mail ‣ INFO  Using plain log format for rsyslog.                                                         
mail ‣ INFO  Removing /var/log/mail.log from /etc/logrotate.d/logrotate.conf                             
mail sed: cannot rename /etc/logrotate.d/sedQaZYSn: Device or resource busy                              
exporter 2026/01/14 23:54:08 Reading log events from /var/log/mail.log                                   
exporter 2026/01/14 23:54:08 Error opening log source: open /var/log/mail.log: no such file or directory 

I tried rolling the mail container back to v4.4.0 with no luck as I wasn't able to roll the exporter container back either based on the tags available still using latest mutatable tag from 8 days ago

exporter 2026/01/15 00:34:33 Reading log events from /var/log/mail.log                                  
exporter 2026/01/15 00:34:33 Error opening log source: open /var/log/mail.log: no such file or directory

Activity

  1. bokysan commented on Jan 15, 2026

    @bokysan
    Owner

    Can you give me your values.yaml to try to replicate the issue?

  2. NerdyShawn commented on Jan 15, 2026

    @NerdyShawn
    Author

    sure thing

      values:
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
              - matchExpressions:
                - key: kubernetes.io/arch
                  operator: In
                  values:
                  - arm64
        autoscaling:
          annotations: {}
          enabled: false
          labels: {}
          maxReplicas: 100
          minReplicas: 1
          targetCPUUtilizationPercentage: 80
        certs:
          create: false
          existingSecret: ""
        config:
          general:
            ALLOWED_SENDER_DOMAINS: somecooldomain.com
            RELAYHOST: smtp-relay.gmail.com:587
          opendkim: {}
          postfix: {}
        container:
          postfix:
            securityContext: {}
        deployment:
          annotations: {}
          labels: {}
        dns:
          nameservers: []
          options: []
          policy: ""
          searches: []
        existingSecret: ""
        extraContainers: []
        extraEnv: []
        extraInitContainers: []
        extraVolumeMounts: []
        extraVolumes: []
        fullnameOverride: ""
        headlessService:
          annotations: {}
          enabled: true
          labels: {}
        image:
          pullPolicy: IfNotPresent
          repository: boky/postfix
        imagePullSecrets: []
        lifecycle:
          postStart: {}
        livenessProbe:
          exec:
            command:
            - sh
            - -c
            - ps axf | fgrep -v grep | egrep -q '\{supervisord\}|/usr/bin/supervisord'
              && ps axf | fgrep -v grep | egrep -q '(/usr/lib/postfix/sbin/|/usr/libexec/postfix/)master'
          failureThreshold: 2
          initialDelaySeconds: 5
          periodSeconds: 5
        metrics:
          enabled: true
          image:
            repository: boky/postfix-exporter
            tag: latest
          logrotate:
            enabled: true
            logrotate.conf: |
              /var/log/mail.log {
                  copytruncate
                  rotate 1
                  monthly
                  minsize 1M
                  compress
                  missingok
                  notifempty
                  dateext
                  olddir /var/log/
                  maxage 90
              }
          maillog: /var/log/mail.log
          path: /metrics
          port: 9154
          resources: {}
          service:
            annotations: {}
            labels: {}
          serviceMonitor:
            annotations: {}
            enabled: false
            labels: {}
        mountSecret:
          data: {}
          enabled: false
          path: /var/lib/secret
        nameOverride: ""
        nodeSelector: {}
        persistence:
          accessModes:
          - ReadWriteOnce
          enabled: true
          existingClaim: ""
          size: 1Gi
          storageClass: ""
        pod:
          annotations: {}
          labels: {}
          securityContext: {}
        preStopSleepSeconds: 0
        readinessProbe:
          exec:
            command:
            - sh
            - -c
            - /scripts/healthcheck.sh
          failureThreshold: 6
          initialDelaySeconds: 10
          periodSeconds: 60
          timeoutSeconds: 8
        recreateOnRedeploy: true
        replicaCount: 1
        resources: {}
        secret: {}
        service:
          annotations:
            external-dns.alpha.kubernetes.io/ingress-hostname-source: defined-hosts-only
          labels:
            app: relay
          port: 587
          type: LoadBalancer
        serviceAccount:
          annotations: {}
          create: true
          name: ""
        startupProbe:
          exec:
            command:
            - sh
            - -c
            - ps axf | fgrep -v grep | egrep -q '\{supervisord\}|/usr/bin/supervisord'
              && ps axf | fgrep -v grep | fgrep -q "postfix-script" && ps axf | fgrep
              -v grep | fgrep -q 'opendkim'
          failureThreshold: 12
          initialDelaySeconds: 5
          periodSeconds: 5
        terminationGracePeriodSeconds: 120
        tolerations: []
  3. bokysan commented on Jan 15, 2026

    @bokysan
    Owner

    Any specific reason you are overriding a bunch of things which are already provided in the default configuration? E.g. I see here you're specifying logrotate.conf, start-up probes, readiness probes...

    Any reason not to leave it at default?

  4. NerdyShawn commented on Jan 15, 2026

    @NerdyShawn
    Author

    I thought I was doing default tbh and I hadn't touched any probes on this one so I'm using whats in the chart currently and patching in domain and some scheduling decision items.

    I'm sourcing the chart from here

    helm repo ls | grep -i post
    bokysan                  	https://bokysan.github.io/docker-postfix/  

    Here is what that looks like

    helm show values bokysan/mail
    replicaCount: 1
    
    image:
      repository: boky/postfix
      # tag: "" # If not specified uses chart's AppVersion as the tag
      pullPolicy: IfNotPresent
    
    imagePullSecrets: []
    nameOverride: ""
    fullnameOverride: ""
    
    serviceAccount:
      create: true # Specifies whether a service account should be created
      annotations: {} # Annotations to add to the service account
      name: "" # The name of the service account to use. If not set and create is true, a name is generated using the fullname template
    
    service:
      type: ClusterIP
      port: 587
      labels: {}
      annotations: {}
      # externalTrafficPolicy: Cluster
      # nodePort:
    
    # StatefulSets now require a headless service
    # https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#limitations
    headlessService:
      enabled: true
      labels: {}
      annotations: {}
    
    metrics:
      enabled: false
      service:
        labels: {}
        annotations: {}
      serviceMonitor:
        enabled: false
        labels: {}
        annotations: {}
      port: 9154
      path: /metrics
      image:
        repository: "boky/postfix-exporter"
        tag: "latest"
    
      # Warning: Metrics needs a file to work. If you're using this DO NOT override POSTFIX_maillog_file.
      # If you do, metrics *will not work*. If you need to log to file:
      # - use this file
      # - add another file to /etc/rsyslog.d/*.conf
      maillog: /var/log/mail.log
      logrotate:
        enabled: true
        logrotate.conf: |
          /var/log/mail.log {
              copytruncate
              rotate 1
              monthly
              minsize 1M
              compress
              missingok
              notifempty
              dateext
              olddir /var/log/
              maxage 90
          }
    
      resources: {}
      # If you do want to specify resources, uncomment the following
      # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
      # limits:
      #   cpu: 100m
      #   memory: 128Mi
      # requests:
      #   cpu: 100m
      #   memory: 128Mi
    
    # Tell helm to restart (recreate) pods on every deploy. Setting this to true will inject
    # `date/deploy-date: <timestamp>` annotation into pod specification for StateFulset. This
    # ensures that the Pod is recreated with the new changes.
    #
    # Feel free to disable this but doing so **IS IT YOUR OWN RISK**. By disabling this feature
    # you might run into issues where some changes are not reflected on the Pod. If you experience
    # this with this setting set to `false` the first thing to try out is to restart / delete
    # a pod and let `StatefulSet` recreate it.
    recreateOnRedeploy: true
    
    resources:
      {}
      # We usually recommend not to specify default resources and to leave this as a conscious
      # choice for the user. This also increases chances charts run on environments with little
      # resources, such as Minikube. If you do want to specify resources, uncomment the following
      # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
      # limits:
      #   cpu: 100m
      #   memory: 128Mi
      # requests:
      #   cpu: 100m
      #   memory: 128Mi
    
    autoscaling:
      enabled: false
      minReplicas: 1
      maxReplicas: 100
      targetCPUUtilizationPercentage: 80
      # targetMemoryUtilizationPercentage: 80
      labels: {}
      annotations: {}
    
    nodeSelector: {}
    tolerations: []
    affinity: {}
    extraVolumes: []
    extraVolumeMounts: []
    extraInitContainers: []
    extraEnv: []
    extraContainers: []
    
    deployment:
      labels: {}
      annotations: {}
    
    pod:
      securityContext: {}
      # Default setup
      #  capabilities:
      #    drop:
      #    - ALL
      #  readOnlyRootFilesystem: true
      # capabilities:
      #   drop:
      #   - ALL
      # readOnlyRootFilesystem: true
      # runAsNonRoot: true
      # runAsUser: 1000
      labels: {}
      annotations: {}
    
    container:
      postfix:
        securityContext: {}
        #  capabilities:
        #    drop:
        #    - ALL
        #  readOnlyRootFilesystem: true
    
    certs:
      # Auto-generate certificates for the server and mount them into Postfix volume
      create: false
      # Provide existing secret name
      existingSecret: ""
    
    # Define data which should be stored in a Secret
    # (and shared with the pod as environment variables)
    # secret:
    #   hello: world
    secret: {}
    
    # Use an existing secret to share with the pod
    # as environment variables.
    existingSecret: ""
    
    # Define a secret which should be deployed together with the
    # chart amd mounted into a specific directory in the pod.
    mountSecret:
      enabled: false
      path: /var/lib/secret
      data:
        {}
        # e.g.
        # dkim-private: |
        #   -----BEGIN RSA PRIVATE KEY-----
        #   < redacted >
        #   -----END RSA PRIVATE KEY-----
    
    config:
      general:
        {}
        # e.g.
        # TZ:
        # FORCE_COLOR:
        # INBOUND_DEBUGGING:
        # ALLOWED_SENDER_DOMAINS:
        # ALLOW_EMPTY_SENDER_DOMAINS:
        # LOG_FORMAT:
        # RELAYHOST:
        # SASL_RELAYHOST: # when specified overwrites the RELAYHOST in the sasl_passwd file
        # RELAYHOST_USERNAME:
        # RELAYHOST_PASSWORD:
        # MASQUERADED_DOMAINS:
        # SMTP_HEADER_CHECKS:
        # DKIM_SELECTOR:
        # DKIM_AUTOGENERATE:
      postfix:
        {}
        # e.g.
        # myhostname: "postfix"
        # smtp_tls_security_level: "encrypt"
      opendkim:
        {}
        # e.g.
        # RequireSafeKeys: "yes"
    
    dns:
      policy: ""
      # policy: "None"
      nameservers: []
        # - 8.8.8.8
        # - 8.8.4.4
      searches: []
      # searches:
      # - "default.svc.cluster.local"
      # - "svc.cluster.local"
      # - "cluster.local"
      options: []
      # options:
      #  - name: ndots
      #    value: "5"
    
    persistence:
      enabled: true
      accessModes:
        - ReadWriteOnce
      existingClaim: ""
      size: 1Gi
      storageClass: ""
    
    readinessProbe:
      initialDelaySeconds: 10
      periodSeconds: 60
      failureThreshold: 6
      timeoutSeconds: 8
      exec:
        command:
          - sh
          - -c
          - /scripts/healthcheck.sh
    livenessProbe:
      initialDelaySeconds: 5
      periodSeconds: 5
      failureThreshold: 2
      exec:
        command:
          - sh
          - -c
          - >-
            ps axf | fgrep -v grep | egrep -q '\{supervisord\}|/usr/bin/supervisord' &&
            ps axf | fgrep -v grep | egrep -q '(/usr/lib/postfix/sbin/|/usr/libexec/postfix/)master'
    startupProbe:
      initialDelaySeconds: 5
      periodSeconds: 5
      failureThreshold: 12
      exec:
        command:
          - sh
          - -c
          - >-
            ps axf | fgrep -v grep | egrep -q '\{supervisord\}|/usr/bin/supervisord' &&
            ps axf | fgrep -v grep | fgrep -q "postfix-script" &&
            ps axf | fgrep -v grep | fgrep -q 'opendkim'
    
    lifecycle:
      postStart: {}
    
    preStopSleepSeconds: 0
    
    # Allow up to 2 minutes for Postfix to flush / empty the queue  before shutting down the container
    terminationGracePeriodSeconds: 120
  5. frankhetterich commented on Jan 20, 2026

    @frankhetterich

    I also tried to update to Version 5.1.0 today and stumbled over the same issue:

    ★★★★★ POSTFIX STARTING UP (debian) ★★★★★
    ‣ NOTE  System accounts: postfix=100:102, opendkim=101:104. Careful when switching distros.
    ‣ NOTE  Setting container timezone to: Europe/Berlin
    ‣ DEBUG /tmp writable.
    ‣ INFO  Using json log format for rsyslog.
    ‣ INFO  Removing /var/log/mail.log from /etc/logrotate.d/logrotate.conf
    sed: cannot rename /etc/logrotate.d/sedeDC1C1: Device or resource busy
    

    I have also a slightly changed logrotate config, but it´s running fine for a long time

        maillog: /var/log/mail.log
        logrotate:
          enabled: true
          logrotate.conf: |
            /var/log/mail.log {
                rotate 5
                copytruncate
                daily
                minsize 100k
                compress
                missingok
                notifempty
                dateext
                olddir /var/log/
                maxage 7
            }
    

    But i don´t think that this config has something todo with the startup issue, because when the startup script fails, postfix isn´t started yet.

  6. qharnay-cc commented on Jan 26, 2026

    @qharnay-cc

    I encounter the same issue but did not realize this issue was open : I commented in a previous closed issue about logrotate : #246 (comment)

    I also have a logrotate config in values.yaml

  7. origpumu commented on Jan 28, 2026

    @origpumu
  8. jonathon2nd commented on Jan 29, 2026

    @jonathon2nd
    Contributor

    I also have just run into this on 5.1.0, 5.0.1, 5.0.0. So anything past 4.4.0 which we have been running for a bit.

    ---
    image:
      repository: boky/postfix
      tag: "v5.1.0" # If not specified uses chart's AppVersion as the tag
    fullnameOverride: "mail"
    
    # if the following is true, config.prefix.myhostname should not be set
    # Requires get-public-ip init container
    lookupHostName:
      enabled: true
    
    extraInitContainers: |
      - name: init-subpaths
        image: busybox:stable
        command:
          - sh
          - -c
          - |
            mkdir -p /mnt/mail/spool && \
            mkdir -p /mnt/mail/logs
        volumeMounts:
          - name: mail
            mountPath: /mnt/mail
    
    
    
    config:
      general:
        # TZ:
        # INBOUND_DEBUGGING:
    ...
        # DKIM_AUTOGENERATE:
      postfix:
        # e.g.
        smtp_tls_security_level: "may"
        smtpd_tls_security_level: "may"
        smtpd_tls_cert_file: /var/run/certs/tls.cert
        smtpd_tls_key_file: /var/run/certs/tls.key
        maximal_queue_lifetime: "1d"
      opendkim: {}
        # e.g.
        # RequireSafeKeys: "yes"
    
    #This does not create the cert, but it does setup for reading the cert that is manualy made.
    certs:
      create: true
    
    
    metrics: 
      enabled: true
      serviceMonitor:
        enabled: true
      port: 9154
      #This is what https://github.com/kumina/postfix_exporter is expecting, so setting as default
      maillog: /var/log/mail.log
      logrotate:
        enabled: true
    
    persistence:
      enabled: true
      size: 5Gi
      storageClass: "local-path"
    
    replicaCount: 5
    ...
    
    ★★★★★ POSTFIX STARTING UP (debian) ★★★★★
    ‣ NOTE  System accounts: postfix=100:102, opendkim=101:104. Careful when switching distros.
    ‣ INFO  Not setting any timezone for the container
    ‣ DEBUG /tmp writable.
    ‣ INFO  Using plain log format for rsyslog.
    ‣ INFO  Removing /var/log/mail.log from /etc/logrotate.d/logrotate.conf
    sed: cannot rename /etc/logrotate.d/sediWncQD: Device or resource busy
    
  9. bokysan commented on Feb 2, 2026

    @bokysan
    Owner

    Thank you for the reports -- I will look at this as soon as possible.

  10. sivarama-p-raju commented on Feb 13, 2026

    @sivarama-p-raju

    @bokysan
    Please note that I see the same issues as well on upgrading to v5.1.0.

    Is there any workaround for this issue right now or will a fix be included in the next release ?

  11. cnorthwood commented on Feb 19, 2026

    @cnorthwood

    If it helps I get this on a fresh install into my k3s cluster, with the following values so I'm not overriding any of the logrotate config at all.

          config:
            general:
              ALLOWED_SENDER_DOMAINS: coven.test.local
            postfix:
              myhostname: coven.test.local
          metrics:
            enabled: true
            serviceMonitor:
              enabled: true

    However, if I leave the metrics block out (so don't enable that), then it does start successfully. I suspect that enabling that enables a different code path related to logrotate (which I understand the exporter needs) and because it is trying to modify a file in the image rather than on a mounted volume it triggers the behaviour which @origpumu notices above. Hopefully this helps you with debugging the root cause for this @bokysan!

  12. outofrange commented on Feb 24, 2026

    @outofrange
    Contributor

    This issue affected us when upgrading as well! I opened #266, which only uses sed when logrotate is writable (-> doesn't use it when mounted in Kubernetes).

  13. NerdyShawn commented on Mar 5, 2026

    @NerdyShawn
    Author

    This is still an issue for me in k8s. I'm running latest release chart so I think the issue should remain open.

  14. outofrange commented on Mar 6, 2026

    @outofrange
    Contributor

    @NerdyShawn While the fix hasn't been released yet, you can set the tag built from master branch (edge-alpine, edge-ubuntu or edge-debian/edge) in image.tag - this should already include the fix, even when used with the 5.1.0 chart version!

    If you don't like floating tags, you could pin image.tag by setting it to edge-debian@sha256:fda621f6ff3980d397719b10916c75680308a7759ddead33695555f365a903d9 (or whatever variant/digest you want to use)

    But be aware that the Helm Chart currently only seems to support digests via image.tag, using tagname@sha:256:digest format. Digest only, without a tag, would be rendered in Pod as :@, like boky/postfix:@sha256:fda621f6ff3980d397719b10916c75680308a7759ddead33695555f365a903d9.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions