I am looking to see if there is an official API for retrieving metadata for information related to IAM permissions, services, etc.
The policy simulator console has pretty much everything I need. Looking at the HTTP requests made, there are requests that return, what looks like, a well defined JSON object.
For instance, to retrieve all of the available services, it calls:
https://policysim.aws.amazon.com/home/data/resource
and returns (shorted for brevity):
[
{
"name": "Amazon EC2",
"actionPrefix": "ec2",
"hasResource": true,
"arnFormat": "arn:aws:ec2:<region>:<account>:<resourceType>/<resourcePath>"
},
{
"name": "Amazon Sumerian",
"actionPrefix": "sumerian",
"hasResource": false,
"arnFormat": "arn:aws:sumerian:<region>:<account-id>:<resource-type>/<resource_name>"
},
{
"name": "Amazon Comprehend",
"actionPrefix": "comprehend",
"hasResource": false,
"arnFormat": "arn:${Partition}:comprehend:${Region}:${AccountId}:${ResourceType}/${ResourceName}"
},
...
{
"name": "AWS Elemental MediaStore",
"actionPrefix": "mediastore",
"hasResource": true,
"arnFormat": "arn:aws:mediastore:<Region>:<Account>:<Resource>"
}
]
And then for example, API Gateway, calling
https://policysim.aws.amazon.com/home/data/action?serviceName=Amazon+API+Gateway&servicePrefix=execute-api
returns:
[
{
"name": "InvalidateCache",
"evaluated": false,
"allowed": false,
"prefix": "execute-api",
"service": "Amazon API Gateway",
"serviceAware": true,
"resourceEnabled": true,
"deniedByOrganization": false,
"requiredResourceNames": [
"execute-api-general"
],
"supportedResources": [
{
"Name": "execute-api-general",
"ARN": "arn:aws:execute-api:${Region}:${Account}:${ApiId}/${Stage}/${Method}/${ApiSpecificResourcePath}",
"RegEx": "^arn:aws:execute-api:.+",
"isRequired": true
}
],
"scenarios": [],
"contextKeys": []
},
{
"name": "Invoke",
"evaluated": false,
"allowed": false,
"prefix": "execute-api",
"service": "Amazon API Gateway",
"serviceAware": true,
"resourceEnabled": true,
"deniedByOrganization": false,
"requiredResourceNames": [
"execute-api-general"
],
"supportedResources": [
{
"Name": "execute-api-general",
"ARN": "arn:aws:execute-api:${Region}:${Account}:${ApiId}/${Stage}/${Method}/${ApiSpecificResourcePath}",
"RegEx": "^arn:aws:execute-api:.+",
"isRequired": true
}
],
"scenarios": [],
"contextKeys": []
}
]
This is all of the information I am after. The list of service name and then the actions for each of them, along with some of that other metadata.
Obviously, I want to use an official API that would support IAM credentials and policies but wasn't able to find anything outside of the supported basic SimulatePolicy and context related API endpoints.
Thanks!
Pink
Just as a follow up in case anyone is looking for the same thing, according to AWS support, there is currently no official or supported API to retrieve this information.
Related
Say I have an SSM document like the below, and I want to be alerted when a run fails or doesn't finish for whatever reason:
{
"description": "Restores specified pg_dump backup to specified RDS/DB.",
"mainSteps": [
{
"action": "aws:runCommand",
"description": "Restores specified pg_dump backup to specified RDS/DB.",
"inputs": {
"DocumentName": "AWS-RunShellScript",
"Parameters": {
"commands": [
"blahblahblah"
],
"executionTimeout": "1800"
},
"Targets": [
{
"Key": "InstanceIds",
"Values": [
"i-xxxxxxxx"
]
}
]
},
"name": "DBRestorer",
"nextStep": "RunQueries"
},
Terraform documents show me that RunCommand documents should support a NotificationConfig where I can pass in my SNS topic ARN and declare what state transitions should trigger a message: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_maintenance_window_task#notification_config
However, I can't find any Amazon docs that actually include the use of a notification configuration in the document itself (not just the maintenance window, which I have set up as automation so it doesn't support it at the window level), so I'm not sure if it belongs as a sub-parameter, or whether to define it with camel case or dash separation.
Try this
{
"description": "Restores specified pg_dump backup to specified RDS/DB.",
"mainSteps": [
{
"action": "aws:runCommand",
"description": "Restores specified pg_dump backup to specified RDS/DB.",
"inputs": {
"DocumentName": "AWS-RunShellScript",
"NotificationConfig": {
"NotificationArn": "<<Replace this with a SNS Topic Arn>>",
"NotificationEvents": ["All"],
"NotificationType": "Invocation"
},
"ServiceRoleArn": "<<Replace this with an IAM role Arn that has access to SNS>>",
"Parameters": {
"commands": [
"blahblahblah"
],
"executionTimeout": "1800"
},
"Targets": [
{
"Key": "InstanceIds",
"Values": [
"i-xxxxxxxx"
]
}
]
},
"name": "DBRestorer",
"nextStep": "RunQueries"
},
...
]
}
Related documentation:
https://docs.aws.amazon.com/systems-manager/latest/userguide/automation-action-runcommand.html
https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_NotificationConfig.html#systemsmanager-Type-NotificationConfig-NotificationType
I am using flask-restx to build an app with a swagger UI and I trying to upload this swagger file as a documentation part in AWS API Gateway. Through this swagger UI, I am enabling the user to upload a CSV file for further data processing.
I have the following swagger json:
{
"swagger": "2.0",
"basePath": "/",
"paths": {
"/upload_profile/csv": {
"post": {
"responses": {
"200": {
"description": "Profile uploaded"
},
"400": {
"description": "Validation Error"
},
"401": {
"description": "Not authorized"
}
},
"operationId": "Get uploaded profiles from user",
"parameters": [
{
"name": "csv_file",
"in": "formData",
"type": "file",
"required": true,
"description": "CSV file"
}
],
"consumes": [
"multipart/form-data"
],
"tags": [
"upload_profile"
]
}
}
},
"info": {
"title": "Upload Profile",
"version": "0.0.1"
},
"produces": [
"application/json"
],
"consumes": [
"application/json"
],
"tags": [
{
"name": "upload_profile",
"description": "Uploading User Profiles"
}
],
"responses": {
"ParseError": {
"description": "When a mask can't be parsed"
},
"MaskError": {
"description": "When any error occurs on mask"
}
}
}
When I go to API Gateway --> Documentation --> Import Documentation and paste the json, I get the following error:
How can the following issue be solved? If formData isn't supported by API Gateway, is there an alternate for hosting the swagger UI?
The problem is that AWS API Gateway expects swagger/OpenAPI version 3, and your file is version 2. If you only want a way to host swagger UI for documentation/collaboration purposes, take a look at SwaggerHub https://swagger.io/tools/swaggerhub/.
But, if you really have to use AWS API Gateway, then you need to get spec in OpenAPI-3 format. Since the API is rather small, I'd suggest preparing OpenAPI-3 spec yourself (rather than generating it) and testing it locally via swagger UI.
As per the documentation, I should be able to get a list of users with a custom schema as long as the field in the schema has a value of ALL_DOMAIN_USERS in the readAccessType property. That is the exact set up I have in the admin console; Moreover, when I perform a get request to the schema get endpoint for the schema in question, I get confirmation that the schema fields are set to ALL_DOMAIN_USERS in the readAccessType property.
The problem is when I perform a users list request, I don't get the custom schema in the response. The request is the following:
GET /admin/directory/v1/users?customer=my_customer&projection=full&query=franc&viewType=domain_public
HTTP/1.1
Host: www.googleapis.com
Content-length: 0
Authorization: Bearer fakeTokena0AfH6SMD6jF2DwJbgiDZ
The response I get back is the following:
{
"nextPageToken": "tokenData",
"kind": "admin#directory#users",
"etag": "etagData",
"users": [
{
"externalIds": [
{
"type": "organization",
"value": "value"
}
],
"organizations": [
{
"department": "department",
"customType": "",
"name": "Name",
"title": "Title"
}
],
"kind": "admin#directory#user",
"name": {
"fullName": "Full Name",
"givenName": "Full",
"familyName": "Name"
},
"phones": [
{
"type": "work",
"value": "(999)999-9999"
}
],
"thumbnailPhotoUrl": "https://photolinkurl",
"primaryEmail": "user#domain.com",
"relations": [
{
"type": "manager",
"value": "user#domain.com"
}
],
"emails": [
{
"primary": true,
"address": "user#domain.com"
}
],
"etag": "etagData",
"thumbnailPhotoEtag": "photoEtagData",
"id": "xxxxxxxxxxxxxxxxxx",
"addresses": [
{
"locality": "Locality",
"region": "XX",
"formatted": "999 Some St Some State 99999",
"primary": true,
"streetAddress": "999 Some St",
"postalCode": "99999",
"type": "work"
}
]
}
]
}
However, if I perform the same request with a super admin user, I get an extra property in the response:
"customSchemas": {
"Dir": {
"fieldOne": false,
"fieldTwo": "value",
"fieldThree": value
}
}
My understanding is that I should get the custom schema with a non admin user as long as the custom schema fields are set to be visible by all domain users. This is not happening. I opened a support ticket with G Suite but the guy that provided "support", send me in this direction. I believe this is a bug or maybe I overlooked something.
I contacted G Suite support and in fact, this issue is a domain specific problem.
It took several weeks for the issue to be addressed by the support engineers at Google but it was finally resolved. The behaviour is the intended one now.
I want to delete several IAM Users in AWS Account, is there a way to backup them first so if something bad happen i can restore again?
If you have AWS Config snapshots enabled then the IAM user details will be captured as per example below.
Another option you can use is the IAM access advisor to determine last activity and what permissions the user has been performing. If nothing over a period of weeks/months then perhaps it is safe to delete.
https://aws.amazon.com/about-aws/whats-new/2019/06/now-use-iam-access-advisor-with-aws-organizations-to-set-permission-guardrails-confidently/
Also as mentioned earlier, you can also just disable any keys associated with that user and remove all inline/attached policies. Keep for a while and see what breaks.
{
"relatedEvents": [],
"relationships": [
{
"resourceId": "<########>",
"resourceName": "admins",
"resourceType": "AWS::IAM::Group",
"name": "Is attached to Group"
}
],
"configuration": {
"path": "/",
"userName": "<########>",
"userId": "<########>",
"arn": "arn:aws:iam::<########>:user/<########>",
"createDate": "2018-04-12T00:11:34.000Z",
"userPolicyList": [],
"groupList": [
"admins"
],
"attachedManagedPolicies": []
},
"supplementaryConfiguration": {},
"tags": {},
"configurationItemVersion": "1.3",
"configurationItemCaptureTime": "2018-04-12T00:26:56.332Z",
"configurationStateId": <########>,
"awsAccountId": "<########>",
"configurationItemStatus": "ResourceDiscovered",
"resourceType": "AWS::IAM::User",
"resourceId": "<########>",
"resourceName": "<########>",
"ARN": "arn:aws:iam::<########>:user/<########>",
"awsRegion": "global",
"availabilityZone": "Not Applicable",
"configurationStateMd5Hash": "",
"resourceCreationTime": "2018-04-12T00:11:34.000Z"
}
https://docs.aws.amazon.com/config/latest/developerguide/resource-config-reference.html
i have few lambdas that use different other services like SSM, athena, dynamodb, s3, SQS, SNS for my process. i am almost done with all my development and would love to monitor it visually. I use X-ray and cloud watch as my regular log monitoring and analysis. I feel cloud watch dashboards is not so efficient way to visualize my stuff with multiple services. So i did a lambda that pulls trace data from my X-ray traces and outputs a nested json file something like below.
[
{
"id": "4707a33e472",
"name": "test-lambda",
"start_time": 1524714634.098,
"end_time": 1524714672.046,
"parent_id": "1b9122bc",
"aws": {
"function_arn": "arn:aws:lambda:us-east-1:9684596:function:test-lambda",
"resource_names": [
"test-lambda"
],
"account_id": "9684596"
},
"trace_id": "1-5ae14c88-41dca52ccec8c7d",
"origin": "AWS::Lambda::Function",
"subsegments": [
{
"id": "ab6420197c",
"name": "S3",
"start_time": 1524714671.7148032,
"end_time": 1524714671.8333395,
"http": {
"response": {
"status": 200
}
},
"aws": {
"id_2": "No9Gemg5b9Y2XREorBG+6a1KLXX7S6O3HtPZ3f6vUuU5F1dQE0nIE1WmwmRRHIqCjI=",
"operation": "DeleteObjects",
"region": "us-east-1",
"request_id": "E2709BB91B8"
},
"namespace": "aws"
},
{
"id": "370e11d6d",
"name": "SSM",
"start_time": 1524714634.0991564,
"end_time": 1524714634.194922,
"http": {
"response": {
"status": 200
}
},
"aws": {
"operation": "GetParameter",
"region": "us-east-1",
"request_id": "f901ed67-4904-bde0-f9ad15cc558b"
},
"namespace": "aws"
},
{
"id": "8423bf21354",
"name": "DynamoDB",
"start_time": 1524714671.9744427,
"end_time": 1524714671.981935,
"http": {
"response": {
"status": 200
}
},
"aws": {
"operation": "UpdateItem",
"region": "us-east-1",
"request_id": "3AHBI44JRJ2UJ72V88CJPV5L4JVV4K6Q9ASUAAJG",
"table_name": "test-dynamodb",
"resource_names": [
"test-dynamodb"
]
},
I only posted the first few line of x-ray trace json output, but it's pretty large to post here. AWS quicksight doesn't support nested json, my question is, is there a way to visualize all my lambdas in a better way using quicksight. I am not allowed to use other third party monitoring systems. Need help with this