GCloud CLI Container as a Different User

Posted on June 30, 2023
Tags: docker, cloud, gcp

TL;DR

The key is to start the container with the HOME environment variable set to something writeable, such as /tmp/gcloud.

docker run --rm -it --user 1001 -e HOME=/tmp/gcloud gcr.io/google.com/cloudsdktool/cloud-sdk

Details

The GCP tools automatically setup a GCP config directory which includes a log of all the commands that are run. Typically this is in ~/.config/gcloud/.

Out of the box, the container has a user with UID=0 and UID=1000

root:x:0:0:root:/root:/bin/bash
...
cloudsdk:x:1000:1000::/home/cloudsdk:/bin/bash

Which means, if you either run it by default you end up with UID=0, which exists and has a writeable home, or you are the only user on your system, and start the container with --user 1000, you also end up with a writeable home, and everything works.

But, if you run it with an odd UID, so that your mounted volumes are written to with the correct UID, then you get:

WARNING: Could not setup log file in /.config/gcloud/logs, (Error: Could not create directory [/.config/gcloud/logs/2023.07.11]: Permission denied.

And worse, when you try to authenticate to run a command from within the container, glcoud has nowhere to put it.

But, the location of the directory default to be under the user’s home directory, and a reasonable place that is writeable by any container user is under /tmp, so we can set it to something like /tmp/gcloud.

docker run --rm -it --user 1001 -e HOME=/tmp/gcloud gcr.io/google.com/cloudsdktool/cloud-sdk

There is another variable called CLOUDSDK_CONFIG which can be used to set the config directory, however, there are other directories which are created, such as the GSUtil state directory that are created in the user’s home directory, and for which there is no easy override.

Additional

To get better stack traces for errors with gsutil add the -d flag.


$ gsutil -d cat gs://xxx.domain.com/notifications.json
***************************** WARNING *****************************
*** You are running gsutil with debug output enabled.
*** Be aware that debug output includes authentication credentials.
*** Make sure to remove the value of the Authorization header for
*** each HTTP request printed to the console prior to posting to
*** a public medium such as a forum post or Stack Overflow.
***************************** WARNING *****************************
gsutil version: 5.24
checksum: 75e3a997d45e5c15f113bc3a4f1c90c6 (OK)
boto version: 2.49.0
python version: 3.9.16 (main, Apr 27 2023, 20:02:17) [Clang 12.0.1 ]
OS: Linux 5.15.0-76-generic
multiprocessing available: True
using cloud sdk: True
pass cloud sdk credentials to gsutil: True
config path(s): /tmp/gcloud/legacy_credentials/[email protected]/.boto
gsutil path: /usr/lib/google-cloud-sdk/bin/gsutil
compiled crcmod: True
installed via package manager: False
editable install: False
shim enabled: False
Command being run: /usr/lib/google-cloud-sdk/platform/gsutil/gsutil -d cat gs://xxx.domain.com/notifications.json
config_file_list: ['/tmp/gcloud/legacy_credentials/[email protected]/.boto']
config: [('working_dir', '/mnt/pyami'), ('debug', '0'), ('https_validate_certificates', 'true')]
DEBUG: Exception stack trace:
    Traceback (most recent call last):
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 633, in _RunNamedCommandAndHandleExceptions
        return command_runner.RunNamedCommand(command_name,
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 421, in RunNamedCommand
        return_code = command_inst.RunCommand()
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/cat.py", line 159, in RunCommand
        return cat_helper.CatHelper(self).CatUrlStrings(self.args,
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/cat_helper.py", line 117, in CatUrlStrings
        for blr in self.command_obj.WildcardIterator(url_str).IterObjects(
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/wildcard_iterator.py", line 552, in IterObjects
        for blr in self.__iter__(bucket_listing_fields=bucket_listing_fields,
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/wildcard_iterator.py", line 189, in __iter__
        get_object = self.gsutil_api.GetObjectMetadata(
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 319, in GetObjectMetadata
        return self._GetApi(provider).GetObjectMetadata(bucket_name,
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 122, in _GetApi
        self._LoadApi(provider, api_selector)
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 140, in _LoadApi
        self.api_map[ApiMapConstants.API_MAP][provider][api_selector](
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 241, in __init__
        SetUpJsonCredentialsAndCache(self, logger, credentials=credentials)
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/gcs_json_credentials.py", line 155, in SetUpJsonCredentialsAndCache
        GetCredentialStoreFilename(), credential_store_key))
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/boto_util.py", line 203, in GetCredentialStoreFilename
        return os.path.join(GetGsutilStateDir(), 'credstore2')
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/boto_util.py", line 196, in GetGsutilStateDir
        system_util.CreateDirIfNeeded(config_file_dir)
      File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/system_util.py", line 112, in CreateDirIfNeeded
        os.makedirs(dir_path, mode)
      File "/usr/bin/../lib/google-cloud-sdk/platform/bundledpythonunix/lib/python3.9/os.py", line 225, in makedirs
        mkdir(name, mode)
    PermissionError: [Errno 13] Permission denied: '/.gsutil'
    
OSError: Permission denied.