StorageException: Anonymous caller does not have storage.objects.get access - google-cloud-platform

Question

When trying to run the below code on CircleCI
fun getJsonFromCloudStorage(): ByteArrayInputStream {
val blobId = BlobId.of("my-company", "creds/my-company-creds.json")
val storage = StorageOptions.getDefaultInstance().service
val get = storage.get(blobId)
return get.getContent().inputStream()
}
it will throw the below error during the integration tests.
> Task :test FAILED
function.GetMetadataFromYouTubeTest > extractIncorrectId FAILED
java.lang.ExceptionInInitializerError
at function.GetMetadataFromYouTube.expand(GetMetadataFromYouTube.kt:17)
at function.GetMetadataFromYouTube.expand(GetMetadataFromYouTube.kt:14)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:491)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:299)
at function.GetMetadataFromYouTubeTest.extractIncorrectId(GetMetadataFromYouTubeTest.kt:71)
Caused by:
com.google.cloud.storage.StorageException: Anonymous caller does not have storage.objects.get access to cni-analytics/creds/cni-awesome.json.
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:220)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:414)
at com.google.cloud.storage.StorageImpl$5.call(StorageImpl.java:198)
at com.google.cloud.storage.StorageImpl$5.call(StorageImpl.java:195)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:89)
at com.google.cloud.RetryHelper.run(RetryHelper.java:74)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51)
at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:195)
at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:209)
at storage.CredentialHelper$Companion.getJsonFromCloudStorage(CredentialHelper.kt:18)
at service.YoutubeService.initialiseYouTube(YoutubeService.kt:50)
at service.YoutubeService.<init>(YoutubeService.kt:19)
at MainKt.<clinit>(main.kt:15)
... 6 more
Caused by:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 401 Unauthorized
{
"code" : 401,
"errors" : [ {
"domain" : "global",
"location" : "Authorization",
"locationType" : "header",
"message" : "Anonymous caller does not have storage.objects.get access to my-company/creds/my-company-creds.json.",
"reason" : "required"
} ],
"message" : "Anonymous caller does not have storage.objects.get access to my-company/creds/my-company-creds.json."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:411)
... 17 more
I followed their documentation.

Answer 1

They said this in their documentation:
Note: To use certain services (like Google Cloud Datastore), you will also need to set the CircleCI $GOOGLE_APPLICATION_CREDENTIALS environment variable to ${HOME}/gcloud-service-key.json.
Instead I set $GOOGLE_APPLICATION_CREDENTIALS in the CircleCI UI to /home/circleci/gcloud-service-key.json and it worked.
I'm assuming this is because I was trying to reference an environment variable from the UI so ${HOME} had not been set when it was setting this environment variable. Perhaps if this environment variable was set in the config.yml ${HOME} would resolve.