TL;DR
The key is to start the container with the HOME environment variable set to something writeable, such as /tmp/gcloud.
docker run --rm -it --user 1001 -e HOME=/tmp/gcloud gcr.io/google.com/cloudsdktool/cloud-sdk
Details
The GCP tools automatically setup a GCP config directory which includes a log of all the commands that are run. Typically this is in ~/.config/gcloud/.
Out of the box, the container has a user with UID=0 and UID=1000
root:x:0:0:root:/root:/bin/bash
...
cloudsdk:x:1000:1000::/home/cloudsdk:/bin/bash
Which means, if you either run it by default you end up with UID=0, which exists and has a writeable home, or you are the only user on your system, and start the container with --user 1000, you also end up with a writeable home, and everything works.
But, if you run it with an odd UID, so that your mounted volumes are written to with the correct UID, then you get:
WARNING: Could not setup log file in /.config/gcloud/logs, (Error: Could not create directory [/.config/gcloud/logs/2023.07.11]: Permission denied.
And worse, when you try to authenticate to run a command from within the container, glcoud has nowhere to put it.
But, the location of the directory default to be under the user’s home directory, and a reasonable place that is writeable by any container user is under /tmp, so we can set it to something like /tmp/gcloud.
docker run --rm -it --user 1001 -e HOME=/tmp/gcloud gcr.io/google.com/cloudsdktool/cloud-sdk
There is another variable called CLOUDSDK_CONFIG which can be used to set the config directory, however, there are other directories which are created, such as the GSUtil state directory that are created in the user’s home directory, and for which there is no easy override.
Additional
To get better stack traces for errors with gsutil add the -d flag.
$ gsutil -d cat gs://xxx.domain.com/notifications.json
***************************** WARNING *****************************
*** You are running gsutil with debug output enabled.
*** Be aware that debug output includes authentication credentials.
*** Make sure to remove the value of the Authorization header for
*** each HTTP request printed to the console prior to posting to
*** a public medium such as a forum post or Stack Overflow.
***************************** WARNING *****************************
gsutil version: 5.24
checksum: 75e3a997d45e5c15f113bc3a4f1c90c6 (OK)
boto version: 2.49.0
python version: 3.9.16 (main, Apr 27 2023, 20:02:17) [Clang 12.0.1 ]
OS: Linux 5.15.0-76-generic
multiprocessing available: True
using cloud sdk: True
pass cloud sdk credentials to gsutil: True
config path(s): /tmp/gcloud/legacy_credentials/[email protected]/.boto
gsutil path: /usr/lib/google-cloud-sdk/bin/gsutil
compiled crcmod: True
installed via package manager: False
editable install: False
shim enabled: False
Command being run: /usr/lib/google-cloud-sdk/platform/gsutil/gsutil -d cat gs://xxx.domain.com/notifications.json
config_file_list: ['/tmp/gcloud/legacy_credentials/[email protected]/.boto']
config: [('working_dir', '/mnt/pyami'), ('debug', '0'), ('https_validate_certificates', 'true')]
DEBUG: Exception stack trace:
Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 633, in _RunNamedCommandAndHandleExceptions
return command_runner.RunNamedCommand(command_name,
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 421, in RunNamedCommand
return_code = command_inst.RunCommand()
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/cat.py", line 159, in RunCommand
return cat_helper.CatHelper(self).CatUrlStrings(self.args,
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/cat_helper.py", line 117, in CatUrlStrings
for blr in self.command_obj.WildcardIterator(url_str).IterObjects(
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/wildcard_iterator.py", line 552, in IterObjects
for blr in self.__iter__(bucket_listing_fields=bucket_listing_fields,
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/wildcard_iterator.py", line 189, in __iter__
get_object = self.gsutil_api.GetObjectMetadata(
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 319, in GetObjectMetadata
return self._GetApi(provider).GetObjectMetadata(bucket_name,
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 122, in _GetApi
self._LoadApi(provider, api_selector)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 140, in _LoadApi
self.api_map[ApiMapConstants.API_MAP][provider][api_selector](
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 241, in __init__
SetUpJsonCredentialsAndCache(self, logger, credentials=credentials)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/gcs_json_credentials.py", line 155, in SetUpJsonCredentialsAndCache
GetCredentialStoreFilename(), credential_store_key))
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/boto_util.py", line 203, in GetCredentialStoreFilename
return os.path.join(GetGsutilStateDir(), 'credstore2')
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/boto_util.py", line 196, in GetGsutilStateDir
system_util.CreateDirIfNeeded(config_file_dir)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/system_util.py", line 112, in CreateDirIfNeeded
os.makedirs(dir_path, mode)
File "/usr/bin/../lib/google-cloud-sdk/platform/bundledpythonunix/lib/python3.9/os.py", line 225, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/.gsutil'
OSError: Permission denied.