"It works on my machine."
I have a django app. I'm followed this tutorial. OAuth2 works great on my dev box like this:
$ curl -v -H "Authorization: OAuth c52676b24a63b79a564b4ed38db3ac5439e51d47" http://localhost:8000/api/v1/my-model/?format=json
My local dev app finds the header with this line of code:
auth_header_value = request.META.get('HTTP_AUTHORIZATION')
But when I deploy it to my ubuntu box running apache it doesn't.
I added the following to my authentication.py file so I could inspect the values in the log on the remote machine.
logging.error(request.GET)
logging.error(request.POST)
logging.error(request.META)
The header value is mysteriously missing from the output. So I just get 401s.
Did you turn on WSGIPassAuthorization?
http://modwsgi.readthedocs.org/en/latest/configuration-directives/WSGIPassAuthorization.html
Authorisation headers are not passed through by default as doing so
could leak information about passwords through to a WSGI application
which should not be able to see them when Apache is performing
authorisation.
Related
I have a Google Cloud Source Repository I want my application to download files from. I have a specific use case where I want to get files from a Google Cloud Source Repository programmatically- not GCS or another location.
I want to control permissions to the repo with standard Google IAM. Can I grant a GCP service account access to read from a Cloud Source Repository?
In bitbucket you can download a file directly from a private repo with a rest call like this: curl -s -S --user username:apppassword -L -O https://bitbucket.org/<ORG_NAME>/<REPO>/src/master/<FOLDER>/file.txt
How can I use a GSA to download a file like this from a private Google Cloud Source Repository?
I am doing this in code so I do not have access to ssh or curl or the gcloud cli. I'll be using python to fetch this file.
I was also looking if the SDK supports this. I did not see anything in the docs for a python API for interacting with Google Cloud Source Repositories this way. I'm wondering how I can pull down this file with the requests library or even something like GitPython while authenticating with the GSA.
EDIT
Per the comments I tried creating a token in python and gcloud, but it does not work. The token is generated fine, but file download doesn't work.
I tried this (and via python):
curl -s -S -H "Authorization: Bearer $(gcloud auth print-access-token)" -L -O https://source.cloud.google.com/MY_GCP_PROJECT/MY_REPO/master/README.md
This downloads a huge html page that seems to be showing auth errors.
Maybe the http path is wrong? What is the correct path to the file in the source repo via http GET?
I confirmed I have permissions because this works gcloud source repos clone MY_REPO --project=MY_PROJECT
EDIT
This is where I am right now, I can't figure out what the right URL is to point to a specific branch and file:
import google.auth
import google.auth.transport.requests
import requests
# Generate a token from current security context
creds, project = google.auth.default()
auth_req = google.auth.transport.requests.Request()
creds.refresh(auth_req)
# Set token in Authorization header of http request
headers = {'Authorization':'Bearer {}'.format(creds.token)}
# Repo URL with branch and file specified (trying to download README.md in the root of the repo)
# What is the right URL here?
url = "https://source.developers.google.com/p/<GCP PROJECT>/r/<REPO NAME>/<BRANCH NAME>/README.md"
response = requests.get(url, headers=headers)
# I get a big mess of html with auth errors
print(response.content)
If I use this URL "https://source.developers.google.com/<GCP PROJECT>/<REPO NAME>/<BRANCH NAME>/README.md" I get back a page that includes PERMISSION_DENIED: The caller does not have permission
I am following the GCP Speech-to-Text Quickstart. As best as I can tell, I have followed all setup criteria.
Enabled STT for my project.
Generated Service API keys and downloaded the JSON file.
Installed SDK and initialized.
In Windows CMD shell, I set GOOGLE_APPLICATION_CREDENTIALS to the downloaded JSON file.
In Windows CMD shell, I executed gcloud auth activate-service-account <my service email generated by GCP> --key-file= "mypath/JSON key file".
I executed gcloud auth list and I see my project account identifier.
I executed the example curl command:
curl -s -H "Content-Type: application/json" -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) https://speech.googleapis.com/v1/speech:recognize -d #sync-request.json
And get this error:
{
"error": {
"code": 401,
"message": "Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
"status": "UNAUTHENTICATED"
}
}
No where in the Quickstart steps does it mention OAuth
As a test, I executed:
gcloud auth application-default print-access-token
And got this:
(gcloud.auth.application-default.print-access-token) File "mypath/JSON key file" was not found.
Even though the file exists in the folder I specify.
Trying something else, I tried executing the Java example in the SDK. It creates a very simple SpeechClient with no credentials, which seems suspect. I made the GOOGLE_APPLICATION_CREDENTIALS env variable available to the example. I think the example uses gRCP, but not sure.
The example hangs at:
RecognizeResponse response = speech.recognize(config, audio);
Looking online, I found the likely suspect is bad authentication, which is the same as trying the CMD line example.
Any and all guidance is appreciated.
Did you run the curl command from the same directory where your JSON key file is located?
Google's documentation states the following:
Note that to pass a filename to curl you use the -d option (for
"data") and precede the filename with an # sign. This file should be
in the same directory in which you execute the curl command.
I have the answer to the CLI issue. A dumb mistake on my part. When I set GOOGLE_APPLICATION_CREDENTIALS I wrapped the pathname in double quotes. Sigh. I reset the env variable without the double quotes.
I could successfully run gcloud auth application-default print-access-token and it printed out the token.
I tried the curl command again with $(gcloud auth....) and got same error. Next, I tried the curl command replacing the $(gcloud auth....) with the token returned above and it worked!
Next, I need to resolve the Java example and I am good.
No need to be suspicious:
If you don't specify credentials when constructing the client, the client library will look for credentials via the environment variable GOOGLE_APPLICATION_CREDENTIALS.
In your java code try to print System.getenv("GOOGLE_APPLICATION_CREDENTIALS"), to verify it's set . Probably it's not, depending on how you are setting it in your IDE, or terminal.
I am trying to send a GET request through postman using file URI scheme. An example:
file:///absolute/path/to/file.txt
The request works properly in any browser, but postman is failing with message:
Could not get any response
This seems to be like an error connecting to file:///absolute/path/to/file.txt
Is there any way to make postman work with file URIs?
Postman does not support file URIs, because the Chrome App platform does not allow direct access to files.
It should be easy to start a small HTTP server yourself though.. if you are on Mac/Linux, you can serve files in a directory using Python:
$ cd /path/to/directory/
$ python -m SimpleHTTPServer 1234
You can then visit
http://localhost:1234/filename
which will serve up the file.
Good answer.
I am not acquainted with SimpleHTTPServer, so I used live-server instead.
I followed the instructions at 2. Install and Run a Local Web Server to install live-server.
Then:
cd /path/to/(myJSONfile.json)
live-server <myJSONfile.json>
and the JSON file was opened in my default web browser.
Finally, in Postman I made a normal GET request, but on the address
http://127.0.0.1:8080.
It all worked just as intended.
In addition to czardoz's answer, for Python 3, SimpleHttpServer has been replaced by http.server. So, to serve the files using Python 3.x:
$ cd/path-to-directory/
$ python -m http.server 1234
Then, browse the file using:
http://localhost:1234/filename
I'm just having a bit of trouble understanding why this command:
>appcfg.py -A adept-box-109804 update app.yaml
as given by the Try Google App Engine Now page does not work. I have downloaded the App Engine SDK for Python, and have Path set up to point to the location of appcfg.py, but running appcfg.py in my projects root directory does not work in the command line. I either have to navigate to the folder containing appcfg.py and do
>python appcfg.py help
or do
>python "C:\Program Files (x86)\Google\google_appengine\appcfg.py" help
to get a command to work from anywhere. I used the latter method to deploy my test app, but was just wondering if someone could explain why the command as given by the simple Google tutorial did not do anything. I also checked to make sure that .py files are automatically opened with the Python 2.7 interpreter, such that a file hello.py will be executed in the command line by simply typing
>hello.py
and it will output its print statement. On the other hand, using appcfg.py in a similar manner gives the same output no matter the arguments (please note I truncated the output, but rest assured that they are identical no matter the arguments:
C:\>appcfg.py help backends
Usage: appcfg.py [options] <action>
Action must be one of:
backends: Perform a backend action.
backends configure: Reconfigure a backend without stopping it.
backends delete: Delete a backend.
backends list: List all backends configured for the app.
backends rollback: Roll back an update of a backend.
backends start: Start a backend.
backends stop: Stop a backend.
backends update: Update one or more backends.
create_bulkloader_config: Create a bulkloader.yaml from a running application.
cron_info: Display information about cron jobs.
delete_version: Delete the specified version for an app.
download_app: Download a previously-uploaded app.
download_data: Download entities from datastore.
help: Print help for a specific action.
list_versions: List all uploaded versions for an app.
request_logs: Write request logs in Apache common log format.
resource_limits_info: Get the resource limits.
rollback: Rollback an in-progress update.
set_default_version: Set the default (serving) version.
start_module_version: Start a module version.
stop_module_version: Stop a module version.
update: Create or update an app version.
update_cron: Update application cron definitions.
update_dispatch: Update application dispatch definitions.
update_dos: Update application dos definitions.
update_indexes: Update application indexes.
update_queues: Update application task queue definitions.
upload_data: Upload data records to datastore.
vacuum_indexes: Delete unused indexes from application.
Use 'help <action>' for a detailed description.
C:\>appcfg.py help update
Usage: appcfg.py [options] <action>
Action must be one of:
backends: Perform a backend action.
backends configure: Reconfigure a backend without stopping it.
backends delete: Delete a backend.
backends list: List all backends configured for the app.
backends rollback: Roll back an update of a backend.
backends start: Start a backend.
backends stop: Stop a backend.
backends update: Update one or more backends.
create_bulkloader_config: Create a bulkloader.yaml from a running application.
cron_info: Display information about cron jobs.
delete_version: Delete the specified version for an app.
download_app: Download a previously-uploaded app.
download_data: Download entities from datastore.
help: Print help for a specific action.
list_versions: List all uploaded versions for an app.
request_logs: Write request logs in Apache common log format.
resource_limits_info: Get the resource limits.
rollback: Rollback an in-progress update.
set_default_version: Set the default (serving) version.
start_module_version: Start a module version.
stop_module_version: Stop a module version.
update: Create or update an app version.
update_cron: Update application cron definitions.
update_dispatch: Update application dispatch definitions.
update_dos: Update application dos definitions.
update_indexes: Update application indexes.
update_queues: Update application task queue definitions.
upload_data: Upload data records to datastore.
vacuum_indexes: Delete unused indexes from application.
Use 'help <action>' for a detailed description.
I finally tracked down the real reason, and it wasn't a bug with the AppEngine SDK. Rather it was with my Python interpreter, as I noticed it wasn't accepting arguments for any .py files. It turned out to be a registry error, located at [HKEY_CLASSES_ROOT\Applications\python.exe\shell\open\command] where I had to change the value from "C:\Python27\python.exe" "%1" to "C:\Python27\python.exe" "%1" %*
How this happened, whether it be the Python 2.7 installer, or maybe the AppEngine SDK, I'm not sure though.
Your confusion probably stems from mixing up 2 possible invocations styles:
python appcfg.py ...
appcfg.py ...
The 1st one can't make use of the fact that the location of the appcfg.py is in the path, it is just an argument to the python executable, which can not locate the appcfg.py file unless either:
it finds it in the current directory
the appcfg.py file is specified using a full path or a path relative to the current working directory from which python is invoked
This is the reason for which your 2nd and 3rd commands don't work as you'd expect. Using the 2nd invocation style instead should work if the location of the appcfg.py is in the path - just as your last command invocation does.
Key point to remember: the path configuration applies to the command executable only, not to its arguments (which BTW each executable may process as it wishes, some executables may combine arguments with the path configuration to obtain location of files).
Similarly appcfg.py itself (once successfully invoked using either of the 2 invocation styles) needs to be able to locate your app.yaml file specified as argument. It cannot do so unless either:
it finds it in the current directory
the app.yaml file (or its directory) is specified using a full path or a path relative to the current working directory from which appcfg.py is invoked
I suspect appcfg.py's inability to locate your app.yaml file may be the reason for which the 1st command you mentioned didn't work. If not you should provide details about the failure.
Regarding why the output of your last command is identical regardless of the arguments, I'm not sure, it could be a bug in the windows version of the SDK. In linux the output is different:
> appcfg.py help backends
Usage: appcfg.py [options] backends <directory> <action>
Perform a backend action.
The 'backends' command will perform a backends action.
Options:
-h, --help Show the help message and exit.
-q, --quiet Print errors only.
-v, --verbose Print info level logs.
--noisy Print all logs.
-s SERVER, --server=SERVER
The App Engine server.
-e EMAIL, --email=EMAIL
The username to use. Will prompt if omitted.
-H HOST, --host=HOST Overrides the Host header sent with all RPCs.
--no_cookies Do not save authentication cookies to local disk.
--skip_sdk_update_check
Do not check for SDK updates.
-A APP_ID, --application=APP_ID
Set the application, overriding the application value
from app.yaml file.
-M MODULE, --module=MODULE
Set the module, overriding the module value from
app.yaml.
-V VERSION, --version=VERSION
Set the (major) version, overriding the version value
from app.yaml file.
-r RUNTIME, --runtime=RUNTIME
Override runtime from app.yaml file.
-E NAME:VALUE, --env_variable=NAME:VALUE
Set an environment variable, potentially overriding an
env_variable value from app.yaml file (flag may be
repeated to set multiple variables).
-R, --allow_any_runtime
Do not validate the runtime in app.yaml
--oauth2 Ignored (OAuth2 is the default).
--oauth2_refresh_token=OAUTH2_REFRESH_TOKEN
An existing OAuth2 refresh token to use. Will not
attempt interactive OAuth approval.
--oauth2_access_token=OAUTH2_ACCESS_TOKEN
An existing OAuth2 access token to use. Will not
attempt interactive OAuth approval.
--authenticate_service_account
Authenticate using the default service account for the
Google Compute Engine VM in which appcfg is being
called
--noauth_local_webserver
Do not run a local web server to handle redirects
during OAuth authorization.
I had this problem, and is deepened in local variable python version that different from app engine python version.
So the solution is just to add before the script the current python version location:
C:\Python27\python.exe "C:\Program Files (x86)\Google\google_appengine\appcfg.py"
And it just return to work well.
I am trying to build a curl request replica using POSTMAN extension in Chrome
For even a simple POST -d, it adds a POSTMAN token. How can I avoid this? Thanks
curl -X POST -H Cache-Control:no-cache -H Postman-Token:494ce988-48f7-67b4-4b8c-90f63c4668f1 -d 'code=newcode' http://127.0.0.1:8000/snippets/6/
In the settings dialog, there is a 'Send postman-token header' option.
This is only available in the packaged app (https://chrome.google.com/webstore/detail/postman-rest-client-packa/fhbjgbiflinjbdggehcddcbncdddomop?hl=en), though, not in the in-chrome version.