I am using a GCE instance with one of the python/jupyter notebook images provided by GCP to do some data analytics. The data resides in BigQuery. The instance can use a service account to access BiqQuery by default, but for data protection reasons I have to use personal user accounts requesting the data.
I know can use an programmatic oauth flow in python to request credentials to authenticate to BigQuery with a personal account, but that flow has to be run interactively every time you restart the ipython kernel and involves opening that authorization URL in a browser and then pasting the secret, which is annoying.
Since I am using IAP to log onto the GCE instance with my account, is there a way to get personal credentials from the IAP to use for authentication to BigQuery automatically?
As I understand, IAP is like a login screen between user and the application (in your case, jupyter server running on GCE) for the purpose of authenticating user to the app, e.g. jupyter server.
IAP adds an id_token (JWT) as signed headers to your http requests after you authenticate, the JWT is specifically made for the IAP instance, if you base64 decode the jwt, you'll see the audience claim is set to the client_id of the IAP instance, and it can't be used for other purposes.
Also bigquery requires an access_token instead of an id_token. I think doing an oauth flow might be the only way. but happy to be corrected...
Related
I have a spring boot based web application which already authenticates the user. I would like to create some AWS APIs using AWS api gateway and a springboot rest app deployed on EC2. The user will log into the existing application and call the new AWS APIs from the browser ajax calls (like an SPA I guess but I have a server and can use client credentials/secrets if needed). What is the best way to handle security for the APIs. My first thought was to generate a JWT using a shared server side secret for hash and verify it with an AWS lambda. But this seems a bit non standard. Oauth2 might be better but might also be over kill. Not sure the performance would be good. Few requirements
token passed to the API should be a user specific token and have an expiration (and hence a refresh token)
The user is already logged into my web app and calling the api should not issue a login challenge.
AWS API Gateway should be able to verify the token before passing it to the application
Any token passed to the API should probably be generated on the logged in web application. Server knows the user is authenticated and should generate the user access token on behalf of the user. The AWS api should be able to figure out what privileges the user has based on the user principle or scopes in the token
I've thought about using Cognito AWS but I dont want to require the users to preexist in a user pool. Cognito seems geared more for the authentication rather than authorization and I would not be using the cognito login functionality. I dont know if its possible to use the oauth2/token endpoint alone with cognito.
I would like to store files on Google Cloud Storage and give devices access to the files via GET HTTP with basic authentication using username and password or something similar. Can this be done?
I will ask you a question to introduce my answer:
If you set a login/password in the basic authentication, which system will validate them?
Cloud Storage relies only on OAuth token. Nothing built-in, you have to build your custom authentication mechanism.
So, you need an additional service that check the credential and then provide a link to download the file.
I recommend you to
Create a serverless service, such as a Cloud Functions or a Cloud Run services
Perform a GET on it with your basic authentication to this service
The service check the authentication, you can use firestore database to store the credential/permissions/autorisations of the connected user
If the auth is OK, create a signed URL and return it with a 302 HTTP code (redirect)
I searched all over, tutorials, web, everybody jumps away without explaining(I understand why) the checkboxes in app client settings:
Enable sign-in API for server-based authentication
Only allow Custom Authentication
Enable username-password (non-SRP) flow for app-based authentication
The learn more link does not help me, lots of information and not so easy to understand, grasp. Can someone explain this settings?
Here is my take on the matter.
App client has several Auth Flow Configurations.
1. Enable username password auth for admin APIs for authentication (ALLOW_ADMIN_USER_PASSWORD_AUTH)
This enables Server-Side Authentication Flow. If you don't have an end-user app, but instead you're using a secure back end or server-side app.
2. Enable lambda trigger-based custom authentication (ALLOW_CUSTOM_AUTH)
This enables the Custom Authentication Flow. This can help you create a challenge/response-based authentication model using AWS Lambda triggers.
https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-challenge.html
Under User Pools -> Triggers you can see many lambda functions. You can use Create Auth Challenge, Define Auth Challenge and Verify Auth Challenge Response functions to create a custom authentication flow.
3. Enable username password based authentication (ALLOW_USER_PASSWORD_AUTH)
This enables Client Side Authentication Flow that uses user password-based authentication. In this flow, Cognito receives the password in the request.
You can use AWS Mobile SDK for Android, AWS Mobile SDK for iOS, or AWS SDK for JavaScript to implement this.
4. Enable SRP (secure remote password) protocol based authentication (ALLOW_USER_SRP_AUTH)
This is similar to the above flow in section 3. except for the password verification. This flow uses the SRP protocol to verify passwords.
http://srp.stanford.edu/whatisit.html
https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_UserPoolClientType.html
5. Enable refresh token based authentication (ALLOW_REFRESH_TOKEN_AUTH)
After successful authentication, Amazon Cognito returns user pool tokens(Three tokens) to your app. You can use the tokens to grant your users access to your own server-side resources, or to the Amazon API Gateway. Or, you can exchange them for temporary AWS credentials to access other AWS services.
The three tokens are ID Token(JWT), Access Token, Refresh Token. The refresh token can be used to retrieve new ID and access tokens. Once you login to a mobile app, you are not needed to log in each time when you close and open the application and this functionality is implemented using refresh tokens.
https://docs.aws.amazon.com/cognito/latest/developerguide/amazon-cognito-user-pools-using-tokens-with-identity-providers.html
What about Amazon Cognito hosted UI ?
App clients can be configured to use built-in Cognito webpages for signing up and signing in users. When using the hosted UI you can enable both the Authorization code grant and the Implicit code grant, and then use each grant as needed.
https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-app-idp-settings.html
Here is my attempt at explaining these options. Before that I would like to briefly mention about Oauth2, which is the protocol on which AWS Cognito is based.
In the context of AWS Cognito, Cognito itself is the Authentication (OAuth) server and also the Resource server (because we create users in Cognito user pool) and your app would be the Client (which sends the authentication request). The client has to first register itself with the OAuth server - this is what is being done in the "App clients" section of Cognito.
The recommended OAuth2 flow is Authorization Code Grant flow. In this flow,
i) The Client sends username/password to the OAuth Server.
ii) The OAuth server validates and calls back the client with a
authorization code.
iii) The Client again sends this code back to the OAuth server
iv) The OAuth server sends the tokens to the Client.
Please read the above linked article for more explanation on OAuth2.
Now explaining the options in Cognito App Client settings:
1. Enable sign-in API for server-based authentication
With this option, your client app can directly receive the tokens without having the additional step of first getting the authorization code.
There are Cognito APIs like AdminInitiateAuth, Admin-* which does this. However, these APIs require AWS admin credentials. Hence usually these calls are done by the backend server of the client app. The front-end can pass the username/password to the backend and the backend server can communicate with AWS Cognito and authorize the user.
2. Only allow Custom Authentication
Here you don't use the OAuth provided authorization code grant flow. Instead, you can define your own steps and challenges. Your client app can ask a secret question etc, before authenticating and giving tokens.
3. Enable username-password (non-SRP) flow for app-based authentication
This is the least safe flow. This skips the part of returning the authorization code and directly returns the tokens back to the client.
I hope this explains.
I have created a service account on Google Cloud Platform. I am using a standalone Java program which uses GCP Java Client API to get the Authentication Token ID by taking service account JSON file. By using the Token ID, I can create a compute instance.
I don't want to use Java API or gcloud tool, is there a REST API exists which takes required details and return Token ID?
I went through the GCP documentation and could not find any details for the authentication and authorization through REST using Service Account.
I have created a service account on Google Cloud Platform. I am using
a standalone Java program which uses GCP Java Client API to get the
Authentication Token ID by taking service account JSON file. By using
the Token ID, I can create a compute instance.
Using a service account is the correct and recommended method to authenticate and authorize software applications.
Note. The assumption here is that the software application is running on your systems under your control. If instead you are installing software on a user's desktop or system not under your control, then you would use Google OAuth 2.0 (Google Accounts) to obtain User Credentials to authorize your application.
Another method is to issue short-lived temporary credentials from a service account credential that are time limited. These short-lived credentials are created on your server and then handed to the client.
I don't want to use Java API or gcloud tool, is there a REST API
exists which takes required details and return Token ID?
You have the classic "Chicken or Egg" situation. You need credentials to authenticate and authorize otherwise anyone could create credentials. Google Service Account credentials provides this. To create service account credentials, use the Google Cloud Console or gcloud CLI to download the service account Json file.
I went through the GCP documentation and could not find any details
for the authentication and authorization through REST using Service
Account.
This question is confusing. Do you want to use Service Account credentials to authorize your Google API calls OR do you want to call a Google API to obtain credentials?
In the first case, once you create service account credentials, they are used to authorize your API calls. You add the Access Token to the HTTP header when making API calls.
For the second case, use Google OAuth 2.0 to obtain credentials. Google OAuth 2.0 uses Google Accounts for authentication. This method provides you with an Access Token (just like a service account) and a Refresh Token and Client ID token. You will need to add the Google Accounts user identity to your Google Cloud IAM which provides for authorization (privileges).
I have a backend that is serving android clients, authenticating them with IdToken sent from the android app.
Now, I need to authenticate a service running on aws that is using my apis. So I figured a service account would do the trick, using the private pem file to create a IdToken and send it along just as the android clients do. But I find no way of obtaining an IdToken with these credentials. Is this possible (preferrably in nodejs).
Or am I on the wrong path here?
I know this is older, but I found this question and it didn't lead me to the answer I ended up with.
I followed the guide in https://cloud.google.com/endpoints/docs/openapi/service-account-authentication#using_a_google_id_token with some mix of https://cloud.google.com/iap/docs/authentication-howto, which mentioned that the key to this was to include a target_audience claim in the generated JWT.
So, essentially I made a JWT that looked like:
{
"exp": 1547576771,
"iat": 1547575906,
"aud":"https://www.googleapis.com/oauth2/v4/token",
"target_audience": "https://example.com/",
"iss": EMAIL OF SERVICE ACCOUNT
}
and posted that to https://www.googleapis.com/oauth2/v4/token with params grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer and assertion=<THE JWT>
Without target_audience the endpoint gave me an access token, but with it I got an id_token instead.
Grettings since 2020
I had problems in Java for take ID_TOKEN of a Google Service Account. My project had two years and i were using GoogleCredentials, fromStream method and a JSON credential, but this class didn't gave me ID_TOKEN, only access_token on a not JWT format.
I solved because on this years Google updated here java code for authentication, for take ID_TOKEN you must use this library https://github.com/googleapis/google-auth-library-java
<dependency>
<groupId>com.google.auth</groupId>
<artifactId>google-auth-library-credentials</artifactId>
<version>0.20.0</version>
</dependency>
And then use ServiceAccountCredential
String credPath = "/path/to/svc_account.json";
ServiceAccountCredentials sourceCredentials = ServiceAccountCredentials
.fromStream(new FileInputStream(credPath));
When you create this class, itself will authenticate with google and have a access_token,refreshToken...
For extract ID_TOKEN you must use this function:
String audience = "http://localhost"; //Your server domain
IdToken idToken = credential.idTokenWithAudience(audience, new ArrayList<IdTokenProvider.Option>());
String id_token = idToken.getTokenValue();
And with this you have a JWT token.
I hope this help people like me,that are trying get ID_TOKEN.
You cannot use service accounts generated for Google Cloud APIs to directly authenticate against your own APIs. How will you know which service account private keys are valid and which have been revoked? Google does not expose this information.
Service accounts are rather meant for delegation of credentials. When you access Google Cloud platform service, you will be authenticating with your google account credentials. You will not want to provision the very same credentials everywhere your running code needs to access any of the Google cloud services (i.e. Cloud APIs). Instead you create service accounts whose scope can be reduced to a subset of the scope of your google account credentials. This way a particular piece of code can be limited to only a few set of APIs.
Service Accounts
A service account is a special account that can be used by services
and applications running on your Google Compute Engine instance to
interact with other Google Cloud Platform APIs. Applications can use
service account credentials to authorize themselves to a set of APIs
and perform actions within the permissions granted to the service
account and virtual machine instance.
What are service accounts?
Service accounts authenticate applications running on your virtual
machine instances to other Google Cloud Platform services. For
example, if you write an application that reads and writes files on
Google Cloud Storage, it must first authenticate to the Google Cloud
Storage API. You can create a service account and grant the service
account access to the Cloud Storage API. Then, you would update your
application code to pass the service account credentials to the Cloud
Storage API. In this way, your application authenticates seamlessly to
the API without embedding any secret keys or user credentials in your
instance, image, or application code.
I know where your confusion stems, it is because service account also have the same OAuth model you are used to.
You can use service accounts to get access tokens and refresh them as needed, but the scope of authentication is at the very maximum limited to the surface of the Google Cloud APIs. You will not be able to mix and match your APIs with that.
Alternative is to either build your own authentication model (which is not so clear from your question when you say authenticating them with IdToken sent from the android app) or rely on something like Cloud endpoints which you create and manage APIs along with API keys for authentication.
As you already mentioned in one of your comments, you can follow the Service-to-Service authentication guide which describes how you can use Google Cloud Service accounts to authenticate with your APIs running on Google Cloud Endpoint.
It supports using Google ID JWT tokens. The caller will have to send the JWT to Google Token endpoints to obtain a Google ID token and then use this Google ID token in all of your requests. This approach also has the advantage that you only have to whitelist the Google ID token server in your API configuration.