Google Cloud Functions Credentials for Local Development - google-cloud-platform

Question

I have a google cloud function. Within this function, I want to write files to GCS (google cloud storage), then get a signed URL of the file that is written to GCS and send that URL to the caller.
For local development, I run the functions locally using the functions-framework command:
functions-framework --source=.build/ --target=http-function --port 8082
When I want to write to GCS or get the signed URL, the cloud functions framework just tries to get the credentials from the signed-in gcloud CLI user. However, I want to point it to read the credentials from a service account. For all other gcloud development purposes, we have put the service account information in a local creds.json file and point the gcloud to read from that file.
Is there any way I can achieve this for functions? Meaning that when I start the functions locally (using functions-framework), I point it to the creds.json file to read the credentials from there?

Answer 1

All Google's SDKs, e.g., for GCS, make use of Application Default Credentials which you should be using instead of explicitly pathing to a key. If this is true for functions-framework, then exporting the variable should work.
The command gcloud auth application-default login is a better recommendation in that case, especially for testing the signed URL, because with that local credential as well as the Cloud Functions credential (through metadata server), the private key isn't present, and the signed URL must be called in a specific manner (provide token and the service account to be able to sign the URL).
Using gcloud auth application-default login creates Application Default Credentials, which have all the powers of the user's account and are persisted as a key called {HOME}/.config/gcloud/application-default_credentials.

Answer 2

Cloud Function Local Development Authentication
This is what I'm doing for local development of a google cloud function using Nodejs that is triggered by a pub/sub event. This function reads a file from a google cloud storage. This uses the Functions Framework Nodejs
TL;DR;
# shell A
gcloud auth application-default login
npm start
pub/sub event message
# shell B
curl -d "#mockPubSub.json" \
-X POST \
-H "Content-Type: application/json" \
http://localhost:8080
Greater Details
Cloud Function with Functions Framework
Docs: Functions Framework Nodejs
package.json
note the --target and --signature-type
{
...
"scripts": {
"start": "npx functions-framework --target=helloPubSub --signature-type=http"
},
"dependencies": {
"#google-cloud/debug-agent": "^7.0.0",
"#google-cloud/storage": "^6.0.0"
},
"devDependencies": {
"#google-cloud/functions-framework": "^3.1.2"
}
...
}
sample nodejs cloud function that downloads file into memory
/* modified from the sample
index.js
*/
const {Storage} = require('#google-cloud/storage');
function log(message, severity = 'DEBUG', payload) {
// Structured logging
// https://cloud.google.com/functions/docs/monitoring/logging#writing_structured_logs
if (!!payload) {
// If payload is an Error, get the stack trace.
if (payload instanceof Error && !!payload.stack) {
if (!!message ) {
message = message + '\n' + payload.stack;
} else {
message = payload.stack;
}
}
}
const logEntry = {
message: message,
severity: severity,
payload : payload
};
console.log(JSON.stringify(logEntry));
}
function getConfigFile(payload){
console.log("Get Config File from GCS")
const bucketName = 'some-bucket-in-a-project';
const fileName = 'config.json';
// Creates a client
const storage = new Storage();
async function downloadIntoMemory() {
// Downloads the file into a buffer in memory.
const contents = await storage.bucket(bucketName).file(fileName).download();
console.log(
`Contents of gs://${bucketName}/${fileName} are ${contents.toString()}.`
);
}
downloadIntoMemory().catch(console.error);
}
exports.helloPubSub = async (pubSubEvent, context) => {
/*
Read payload from the event and log the exception in App project if the payload cannot be parsed
*/
try {
const payload = Buffer.from(pubSubEvent.body.message.data, 'base64').toString()
const pubSubEventObj = JSON.parse(payload) ;
console.log("name: ", pubSubEventObj.name);
getConfigFile(pubSubEventObj)
} catch (err) {
log('failed to process payload: + payload \n' , 'ERROR', err);
}
};
Mock Message for Pub/Sub Event
blog reference, but I'm not using the emulator
myJson.json
{"widget": {
"debug": "on",
"window": {
"title": "Sample Konfabulator Widget",
"name": "main_window",
"width": 500,
"height": 500
},
"image": {
"src": "Images/Sun.png",
"name": "sun1",
"hOffset": 250,
"vOffset": 250,
"alignment": "center"
},
"text": {
"data": "Click Here",
"size": 36,
"style": "bold",
"name": "text1",
"hOffset": 250,
"vOffset": 100,
"alignment": "center",
"onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
}
}}
encode for Pub/sub message ( likely there's a better way )
cat myJson.json | grep -v % | base64
take that output and put it into value for the data key:
mockPubSub.json
{
"message": {
"attributes": {
"greeting": "Hello from the Cloud Pub/Sub Emulator!"
},
"data": "< put the output of the base64 from above here >",
"messageId": "136969346945"
},
"subscription": "projects/myproject/subscriptions/mysubscription"
}
Follow steps from the TL;DR: above.
Disclaimers
gcloud auth application-default login uses the permissions of the user who executes the command. So remember, in production, the service account the cloud function uses will need to read from the storage bucket.
while scrubbing this (i.e. renaming bits) and copying it over, I may have messed it up. Sorry if that is true.
this is all contrived if you are curious, my design is to take a message from a cloud scheduler that includes relevant details about what to read from the config.
this article explains a way to hot-reload the cloud function
It remains unclear why I'm using ``-signature-type=http` to mock message, but for now, I am.