/sig auth

/cc @jingxu97

see also #57923 (specifically #57923 (comment))

I think this is what the fsGroup setting was intended to accomodate

edit: misread the description as multiple containers running as different users wanting access to the same secret.

this is asking for a way to have multiple processes in a single container running as different users not have access to portions of the same secret?

this is asking for a way to have multiple processes in a single container running as different users not have access to portions of the same secret?

Correct, multiple processes in a single container running as different users not have access to another or portions of the same secret.

This does not sound like a very common use case. Different processes running under different users are typically separated to different containers.

There are a handful of ways you could implement a custom solution within a container, but this doesn't sound like a use case we're likely to support in Kubernetes.
/priority awaiting-more-evidence

Different processes running under different users are typically separated to different containers.

Let's say there is only one process running in the container, and the container's security context has

          runAsNonRoot: true
          runAsUser: <user-id>

Currently, the secret file would still be created with root as the owner. Unless the file is world-readable, the main container process will not be able to read it. So in this scenario (which is not uncommon I would guess), there is little to no use for the defaultMode. Either the secret file needs to be created with the owner of runAsUser or you should be able to explicitly set the owner of the secret file.

I think that's a scenario that fsGroup would address. That said, I agree that it shouldn't be necessary (at least to manually add it) in this case.

This has a lot of overlap (maybe a dupe?) with #2630

I have read through #2630 where the focus was on a related issue of volumes from some kind of provisioner (hostPath, emptyDir, etc.). This thread seems to be focused on secrets and by extension configmaps.

I can say that some software is strongly opinionated (open ssh comes to mind, but I have seen java application servers also do this) about the combination of the uid of the process accessing a sensitive file (an ssh private key for example), the uid/gid of said file, and the user/group/other permissions of the file.

Currently using kubernetes 1.14.6 to the best of my knowledge it is not possible to satisfy the requirements of such opinionated software because it is not possible to for the user to set the uid of a secret/configmap mount point.

fsGroup is not always a solution because the software checks the uid which as of today is always 0. Providing a configuration option to set the mount uid via the yaml used to define the mount is clean because it is the same abstraction expected by a non-contanerized system. It then is left to the user to correctly align the value of the mount uid and the value of the runAsUser setting for the container itself, one could even make an argument that the mount should default to the runAsUser value because most of the time there is a single container in a Pod, and it makes sense that the process running in the container needs to access the mount.

I totally approve

Either the secret file needs to be created with the owner of runAsUser or you should be able to explicitly set the owner of the secret file.

At this time, being able to use secret to store ssh keys in a strict-security context (pod.securityContext.runAsNonRoot) is nearly impossible (or require tedious workaround like in https://stackoverflow.com/a/50426726/1620937 which is lame)

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

I do not believe this issue should go stale. I still have software that can not be run in a Pod because the software does uid checks of sensitive files. The software (rightly) refuses to run as uid 0 because it doesn't need to, and (rightly) refuses to use a sensitive (secrets) file it doesn't own and is not the only entity that can read said file.

/remove-lifecycle stale

@tallclair:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

I'd like to give this issue a shot - I will start working on a PoC and send updates here as soon as I have something working/reviewable.
/assign

@a-hilaly Thanks for picking this up! It should go through the KEP process before implementation though (feel free to prototype if it helps to figure out the design). https://github.com/kubernetes/enhancements

+1 needed for the mounted secret files

+1 also affected here. (ssh keys)

+1

any updates on this?

+1

Another common use case is the postgres client modules when using a .pgpass credentials file.
They check process user is the owner and the file mode is <-= 0600

Another common use case is the postgres client modules when using a .pgpass credentials file. They check process user is the owner and the file mode is <-= 0600

@wildk1w1 You can use fsGroup for this. Set the file permissions of the mounted files along with the owning user:

spec:
  volumes:
    - name: client-certs
      secret:
        secretName: postgres-client-certs
        defaultMode: 0600
    - name: ca-cert
      secret:
        secretName: postgres-cluster-cert
        defaultMode: 0600
  securityContext:
    fsGroup: 12345 # UID

/unassign a-hilaly
Please reassign yourself if you'd want to continue working on the issue.

Another common use case is the postgres client modules when using a .pgpass credentials file. They check process user is the owner and the file mode is <-= 0600

@wildk1w1 You can use fsGroup for this. Set the file permissions of the mounted files along with the owning user:

spec:
  volumes:
    - name: client-certs
      secret:
        secretName: postgres-client-certs
        defaultMode: 0600
    - name: ca-cert
      secret:
        secretName: postgres-cluster-cert
        defaultMode: 0600
  securityContext:
    fsGroup: 12345 # UID

this does not change owner of the file and with fsGroup set, it does not set the mode 0600. Just tested with 1.26.7 kubernetes.

How can we proceed here? fsUser would require API change. Is it doable without version change? Like v1/pod -> v2alpha/pod? Such a change would make many complications. But perhaps via featuregate, we could actually set the fsUser value to the one from either runAsUser or fsGroup? Any thoughts?