I am trying to develop a python script that gets different parameters of any AWS service (for EC2 e.g., those parameters would be operating system, billing type etc.). Where can I find a listing of all the available Filters that can be used with the get_products function in boto3 for each different supported Service?
Thanks in advance,
Andreas
Actually, there is no direct API or doc available for getting all the attributes. At least I didnt find any.
What you can do is combine various API calls:
You can use DescribeServices
, you get all the attributes of the all the services or if you want to have for one particular you can provide the name. Boto3 call describe_services
Returns the metadata for one service or a list of the metadata for all services
Then you need to use GetAttributeValues to determine the possible values of the attributes. Boto3 call get_attribute_values
And finally depending on the attributes collected in the earlier step you can build a filter for get_producs
Related
I am using #google-cloud/logging package to get logs from gcloud, and it works nicely, you can get logs, event (and query them if needed). But how I can get the same info as Logs Explorer? I mean different type of fields which can be queried and etc:
On this picture you see Log fields like, FUNCTION NAME which may be a list of values. And it seems that #google-cloud/logging can't get this meta (or fields info)? So is it possible to obtain it using some other APIs?
If I understand your question correctly, you're asking how Logs Viewer is determining the values that allows it to present you with the various log fields to filter|refine your log queries.
I suspect (don't know) that the viewer is building these lists from the properties as it parses the logs. This would suggest that, the lists are imperfect and that e.g. FUNCTION_NAME's would only appear once a log including the Function's name were parsed.
There is a way to enumerate definitive lists of GCP resources. This is done using list or equivalent methods available using service-specific libraries (SDKs) e.g. #google-cloud/functions.
The easiest way to understand what functionality is provided by a given Google service is to browse the service using Google's APIs Explorer. Here's Cloud Logging API v2 and here's Cloud Functions API.
You can prove to yourself that there's no method under Cloud Logging that allows enumeration of all a project's Cloud Functions. But there is a method in Cloud Functions projects.locations.functions.list. The latter returns a response body that includes a list of functions that are a type CloudFunction that have a name.
Another way to understand how these APIs ("libraries") are used is to add --log-http to any gcloud command to see what API calls are being made by the command.
I'm using a java client to retreive available keyrings from google KMS based on https://cloud.google.com/kms/docs/samples/kms-quickstart#kms_quickstart-java
below gcould command will list the available beys and versions for a specific keyRing
gcloud kms keys list --keyring keyring --location location
but I'm able to find any documentation about the java client to retrieve key names and versions from the keyring. is there any option available in gogle KMS java client ?
See Cloud KMS Client Library
The example in the page you referenced includes listKeyRings.
To enumerate a specific keyring's (crypto) keys, I think (!?) you can use listCryptoKeys to (also) get pages (!) of ListCryptoKeysPagedResponse that you should be able to iterateAll().
NOTE These examples don't (!?) fetch subsequent pages of results; you will need to do this.
Unless you're familiar with the auto-generated javadocs, navigating these APIs can be gnarly.
A few things to know:
Google (almost without exception) does an excellent job with its SDKs. If an API method exists, you can be very confident that the functionality is present in a Google SDK of your choosing; you just need to find it!
APIs Explorer is an excellent tool for understanding Google's APIs (it used to do a better job referencing SDKs too). In this case, Cloud KMS keyRings.list and keyRings.cryptoKeys.list not only document the method functionality but summarize the request|response objects.
Any (!) gcloud command can be tweaked with --log-http to show which underlying REST API calls are being made, i.e. gcloud kms keys list --keyring=${KEYRING} --location=${LOCATION} --log-http should (!) reference back to keyRings.cryptoKeys.list
I'm trying to retrieve below details using java SDK or CLI.
I found below CLI commands for that but in sample responses I don't see an appropriate CLI command for that.
describe-ops-items
get-ops-item
get-ops-metadata
get-ops-summary
list-ops-item-events
list-ops-item-related-items
list-ops-metadata
Trying out each CLI is not an option since this is very sensitive data and cannot request permission for all CLI methods..
Have you seen that there is a method list-compliance-summaries ?
From the official docs:
Returns a summary count of compliant and non-compliant resources for a compliance type. For example, this call can return State Manager associations, patches, or custom compliance types according to the filter criteria that you specify.
With deploying an AWS Lambda function (via Serverless Framework), and exposing it via a HTTPS endpoint in AWS API Gateway... is it possible to construct and set the API Gateway id and thus determine the first part of the HTTP endpoint for that Lambda function?
When deploying an AWS Lambda function and adding a HTTP event, I now get a random id as (the first hostname) in https://klv5e3c8z5.execute-api.eu-west-1.amazonaws.com/v1/fizzbuzz. New/fresh deployments receive new random string, that 10 character id.
Instead of using that, I would like to determine and set that id. (I will make sure that it's sufficiently unique myself, or deal with endpoint naming collisions myself.)
Reason for this is that in a separate Serverless project, I need to use that endpoint (and thus need to know that id). Instead of having it being determined by project 1 and then reading/retrieving that in project 2, I want to construct and set the endpoint in project 1 so that I can use the known endpoint in project 2 as well.
(A suggestion was to use a custom domain as an alternative/alias for that endpoint... but if possible I want don't want to introduce a new component in the mix, and a solution that does not include Cloud-it-might-take-30-minutes-to-create-a-domain-Front is better :-) )
If this isn't possible, I might need to use the approach as described at http://www.goingserverless.com/blog/api-gateway-url, mentioning that the endpoint is being exposed from one project via the CloudFormation stack, to be read from and used in the other project, but that introduces (a little latency and) a dependency in deploying the second project.
The "first hostname" you want to set is called "REST API id" and is generated by API Gateway when creating the API. The API used to create API's in API Gateway doesn't offer the ability to specify the REST API id, so no, there is no way to specify the id.
The reason for that is probably that these id's are used as part of a public facing domain name. As this domain name doesn't include an identifier for the AWS account it belongs to, the id's have to be globally unique, so AWS generates them to avoid collisions. As AWS puts it (emphasis by me):
For an edge-optimized API, the base URL is of the http[s]://*{restapi-id}*.execute-api.amazonaws.com/stage format, where {restapi-id} is the API's id value generated by API Gateway. You can assign a custom domain name (for example, apis.example.com) as the API's host name and call the API with a base URL of the https://apis.example.com/myApi format.
For the option to create a custom domain name you should consider that there is even some more complexity associated with it, as you must provision a matching SSL-certificate for the domain as well. While you can use ACM for that, there is currently the limitation that SSL-certificates for CloudFront distributions (which edge-optimized API Gateway API's use behind the scenes) need to be issued in us-east-1.
The option you already mentioned to export the API endpoint in the CloudFormation stack as output value and use that exported value in your other stack would work well. As you noted that'd create a dependency between the two stacks, so once you deployed project 2, which uses the output value from project 1, you can only delete the CloudFormation stack for project 1 after the project 2 stack is either deleted or updated to not use the exported value anymore. That can be a feature, but from your description it sounds like it wouldn't for your use case.
Something similar to exported stack output values would be to use some shared storage instead of making use of CloudFormation's exported output values features. What comes to mind here is the SSM Parameter Store, which offers some integration into CloudFormation. The integration makes it easy to read a parameter from the SSM Parameter Store in the stack of project 2. For writing the value to the Parameter Store in project 1 you'd need to use a custom resource in your CloudFormation template. There is at least one sample implementation for that available on Github.
As you can see there are multiple options available to solve your problem. Which one to choose depends on your projects needs.
Question: "is it possible to construct and set the API Gateway id?"
Answer: No (see the other answer to this question).
I was able to get the service endpoint of project 1 into the serverless.yml file of project 2 though, to finally construct the full URL of the service that I needed. I'm sharing this because it's an alternative solution that also works in my case.
In the serverless.yml of project 2, you can refer to the service endpoint of project 1 via service_url: "${cf:<service-name>-<stage>.ServiceEndpoint}". Example: "${cf:my-first-service-dev.ServiceEndpoint}".
CloudFront exposes the ServiceEndpoint that contains the full URL, so including the AWS Gateway REST API id.
More information in Serverless Framework documentation: https://serverless.com/framework/docs/providers/aws/guide/variables/#reference-cloudformation-outputs.
It seems that Serverless Framework is adding this ServiceEndpoint as stack output.
For example when I try to figure out what properties I can put into deployment manager for creating a bigquery table, I had to reference the REST API docs as the best place to find parameters and required fields.
Is there a good place from within gcloud command or online docs that are specific to deployment manager yamls? I would like to be able to reference required fields and optional fields for creating GCP resources. Currently it's very difficult to figure out.
From the documentation at: https://cloud.google.com/deployment-manager/docs/configuration/supported-resource-types
You can get a list of the supported resource types by running:
gcloud deployment-manager types list
That said the yaml reference from documentation on the that page looks pretty complete.
Edit: Refer to this github link for a list of deployment manager examples.
If anything you need is not described in the documentation/exemplary schemas there is a brutal walk around.
You can make an api call with developer console open (F12) and have a look on network activity where your call will be described with all used and available properties.
It will not provide any addtional information about implementation besides parameter's name itself, so you will have to follow rules covering alike parameter.