I've got a cloud function that needs to install dependencies from a private pypi.
I've set up a secret called pypi_password for the project and given it the correct value.
I've set up a service account to do deployment and given that service account roles/secretmanager.secretAccessor.
I've added --update-secrets PYPI_PASSWORD=pypi_password:latest to my deploy command so the function should have access to the secret.
I've added --extra-index-url=https://account:${PYPI_PASSWORD}#pypi.my-company.com/pypi to my function's requirements.txt.
When I deploy, I briefly see
before the during-deployment pip install fails because my credentials aren't right.
To investigate, I've gone and added an extra ${PYPI_PASSWORD} in my requirements.txt to a portion that's not starred-out in deploy's printed outputs, like: --extra-index-url=https://${PYPI_PASSWORD}account:${PYPI_PASSWORD}#pypi.my-company.com/pypi.
What I see is that the value of ${PYPI_PASSWORD} is coming out as %7BPYPI_PASSWORD%!D(MISSING), which makes it seem like the environment variable doesn't exist.
But shouldn't the secret be accessible as an environment variable this way? This makes no sense to me, and I can't find a solution in the documentation.
I've discovered that I need to make a distinction between run time variables and build time variables.
Basically, by trying to access a Secret in a requirements.txt, I'm trying to use it before it's fully defined and available. The only variables accessible when the dependencies are being installed are build variables.
If I put my password in one of those, it works, so this means Secrets are only available at run time, which makes them kind of useless for this. (ahem, Google)
I'm not so happy about still having the password in plain text for anyone who can view the function, but at least this gets it out of the source code.
Related
I've created a public/private key pair as described here for the Google Cloud Platform (see graphic below)
The problem: I can't find a shred of documentation describing where to put it. This thing is not the typical SSH key pair, but rather a JSON file.
Where should it be stored on a mac to allow the gcloud command to authenticate and push to the GCP?
If you are authenticating locally with a service account to build/push with gcloud, you should set the environment variable on your mac terminal to point to the JSON key file.
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/service-account-file.json"
Once this environment variable is defined, all the requests will be authenticated against that Service Account using the key info from the json file.
Please consider looking at the doc below for reference:
https://cloud.google.com/docs/authentication/production
The CaioT answer is the right one if you want to use a service account key file locally.
However, the question shouldn't be asked because it's a bad practice to have service account key files. They have to be used in only few cases. Else, they are security weakness in your projects.
Have a higher look on this key file. At the end, it's only a file, stored on your mac (or elsewhere) without special security dispositions. You can copy it without any problem, edit it, copy the content. You can send it by email, push it in Git repository (might be public!)...
If you are several developers to work on the same project, it because quickly a mess to know who manage the keys. When you have a leak, it's hard to know which key has been used and need to be removed,...
So, have a closer look to this part of the documentation. I also wrote some articles to propose alternative to use them. Let me know if you are interested.
In Google shell which is a part of Google cloud, I set environment variable GOOGLE_APPLICATION_CREDENTIALS because It is need it for PHP NLP project [info: https://cloud.google.com/natural-language/docs/quickstart-client-libraries#client-libraries-install-php]. My project worked fine, but I notice that variable GOOGLE_APPLICATION_CREDENTIALS lasts on my sistem only one day. This is my third time that I am setting it. My project doesn't work when I am missing required variable. Am I doing something wrong?
EDIT:
It is default OS (Debian) when you create new App on Google App engine.
When I type help in Google shell I get info with:
Your 5GB home directory will persist across sessions, but the VM is ephemeral and will be reset
approximately 20 minutes after your session ends. No system-wide change will persist beyond that.
You are completely right, Cloud Shell is running on an ephemeral instance that resets some minutes after the session has ended, reason why you are losing the content of the environment variable you mentioned.
The documentation about limitations in Cloud Shell clearly states that it is intended for interactive use only, and any non-interactive session or intensive usage can be automatically terminated with (or without) a warning.
Therefore, and understanding from your question that you have a background script that is working with Cloud Natural Language, I would strongly advise you to move to a "real" instance of Compute Engine, in which you will have much more control about what is happening. This will allow more flexibility and you will be able to use a bigger machine type, given that Cloud Shell runs on a g1-small GCE instance which, in general, is not enough to run an application. Also, depending on your use case, you may even consider App Engine.
That being said, I have found that when constructing the LanguageClient instance, you may also not use Application Default Credentials and, instead, use the keyFile or keyFilePath variables (explained in the PHP Client Library reference) to pass the path to the JSON key directly to your code, instead of reading it from the environment variable.
Lets assume you are using Linux, make sure that:
The system is not being restarted, and if it is, make sure to set the environment variables accordingly (see how to set permantent environment variables)
I'm using AWS AppStream to stream a legacy .NET client. The app requires a parameter to start up correctly, which it gets via SessionContext passed into the create_streaming_url API call. I'd like to test this interaction locally without having to redeploy my app for every debug iteration as that takes well over half an hour. According to the AWS AppStream Docs session-context is stored in an environment variable that is only accessible via the AWS provided SessionContextRetriever.exe .NET application. The docs list the environment var as AppStream_Session_Context. I've tried setting this env var and running SessionContextRetriever.exe with no success. There is no documentation that I can find for SessionContextRetriever.exe but there's obviously something I'm missing here. Anybody have any experience with AppStream and session context?
The executable they provide doesn't come with a license, so I have to presume that it's copyrighted and licensed restrictively etc. So de-compiling it would be not be a good idea. But if somebody were to do such a thing, I would expect them to find something like
Console.Write(Environment.GetEnvironmentVariable("APPSTREAM_SESSION_CONTEXT", EnvironmentVariableTarget.Machine));
So I suggest that you try setting the environment variable at the system level for testing. That is, setting it in a script won't be visible to this executable because it's not looking at your current terminal session.
Setting the environment variable at the system level (using the Windows "Edit system environment variables) I see the output from this executable.
Run PS as Administrator:
PS C:\Users\Public\Apps> setx -m AppStream_Session_Context "Value"
PS C:\Users\Public\Apps> .\SessionContextRetriever.exe
Value
I have an application I am currently working on for which I am integrating Travis CI. I am running into the problem of API keys being accessed by Travis. Given below is my current setup (without Travis):
I have a config.py (and is git ignored) that has API keys for all my interfacing applications. I use ConfigParser to read this file and get the required keys.
Travis asks me to look at environment variables as an option to encrypt the keys and add them to .travis.yml. How would Travis know or what needs to be done in order to make travis know that a particular key belongs to a specific interfacing application. Does there need to be changes to the code?
You would read these important variables into your application as system variables.
However, this will only work for builds that are run against master. These environment variables aren't available for builds that are run as part of pull requests.
I have a problem in understanding the relationship between services and registry.
I have the task of taking my windows C++ program and transform it from simple application to a service.
I read that I need to produce some more functions as: start stop resume install.
The problem is:
Why I need the regisrty ?
how I enter the new program ?
Beside those method what I need to do with the registry? how I enter inside it ?
Do I need to write a script for entering the service ?
I read but I just didn't understand, any answear and or some good links to explanation will be appreciated.
Thanks,
I'm not aware of any documented relationship between services and the registry. Services can use the registry to store their settings, just like any other application, but they're not required to.
Formally, you don't need the registry. You simply need to install the service using the relevant API functions. As part of their implementation, the API functions create registry entries that the OS uses later to know when and how to start your service, but I don't think those keys are documented with any expectation that developers would modify them manually, so don't worry about them.
If your program uses the registry to store settings, though, you'll need to understand what account your service runs as, because that affects what areas of the registry your program has access to.
Install your service by calling CreateService. Do that in your program's installer. You can also make your service install itself when it detects itself being run with a certain command-line switch, such as -i. To uninstall your service, call OpenService and then DeleteService. In either case, you'll need to call OpenSCManager first. See MSDN for more on how to call those functions.
Alternatively, you can use the sc command to create and delete your service.
As I mentioned above, you don't need to do anything with the registry. Just install and uninstall your service with the API and let the OS take care of the rest.
You don't need to write any scripts to start your service. The OS already knows how to start it (because it's installed). If your service is something that users would want to start and stop frequently, then rather than use the service control panel then they can use the net or sc commands.
We use the registry to store command line parameters. The executable is passed a special parameter saying "you are a service, and here is your service name", and then the program knows to look in the registry and read the rest of the command line parameters from there. Honestly, I don't know why it was done this way, but I suspect that there's a limit on the length of service command line.
As Rob said however, services don't have to use the registry at all.