How to add service-account to instance template with deployment manager - google-cloud-platform

I am trying to create simple VM based on example given in here1.
I want to add custom service account 2 to this VM.
My config looks something like this
def GenerateConfig(context):
"""Create instance with disks."""
resources = [{
'type': 'compute.v1.instance',
'name': 'vm-' + context.env['deployment'],
'properties': {
'zone': context.properties['zone'],
'disks': [{
'deviceName': 'boot',
'type': 'PERSISTENT',
'boot': True,
'autoDelete': True,
'initializeParams': {
'diskName': 'disk-' + context.env['deployment'],
}
}],
'networkInterfaces': [{
'network': '...',
'subnetwork': '...',
'no-address': True,
}],
'tags':{
'items': [context.env['deployment']]
},
'service-account': ''.join(['custom-compute#',
context.env['project'],
'.iam.gserviceaccount.com']),
'scopes': ['https://www.googleapis.com/auth/devstorage.read_only',
'https://www.googleapis.com/auth/logging.write',
'https://www.googleapis.com/auth/monitoring.write',
'https://www.googleapis.com/auth/trace.append']
}
}]
return {'resources': resources}
I am able to successfully create the deployment. However when I describe the newly created instance it doesn't have any "service-account" associated with the vm.
I couldn't find any example of adding service-account to Deployment manager template. I have also tried to use "serviceAccount" key instead of 'service-account' without any success.
Does anyone knows what I am missing?

I found the reference DM reference docs.
The required changes were
'serviceAccounts': [{
'email': '....',
'scopes': '...'
}]

Related

How can I specify routing policy via the boto3 change_resource_record_sets function?

When I create a record in my hosted zone via the AWS Web Console, I can select the Routing Policy as "Simple".
When I try to create the same record programmatically via boto3, I seem to have no option to set a Routing Policy, and it is "Latency" by default.
What am I missing?
r53.change_resource_record_sets(
HostedZoneId=hz_id,
ChangeBatch={
'Changes': [{
'Action': 'UPSERT',
'ResourceRecordSet': {
'Name': root_domain,
'Type': 'A',
'Region': region,
'AliasTarget': {
'DNSName': f's3-website.{region}.amazonaws.com',
'EvaluateTargetHealth': False,
'HostedZoneId': s3_hz_id,
},
'SetIdentifier': str(uuid.uuid4())
}
}]
}
)
removing region and SetIdentifier works for me here - can't explain it though :)

Compute environments are not displayed in AWS console

Compute environments created via boto3 are not displayed in AWS console. I can see them in the batch_client.describe_compute_environments() call response:
{
'computeEnvironmentName': 'name',
'computeEnvironmentArn': 'arn:aws:batch:us-east-1:<ID>:compute-environment/ml-retraining-compute-env-second',
'ecsClusterArn': 'arn:aws:ecs:us-east-1:<ID>:cluster/ml-retraining-compute-env-second_Batch_b18fcd09-8d7e-351b-bc0f-13ffa83a6b15',
'type': 'MANAGED',
'state': 'ENABLED',
'status': 'INVALID',
'statusReason': "CLIENT_ERROR - The security group 'sg-2436d85c' does not exist",
'computeResources': {
'type': 'EC2',
'minvCpus': 0,
'maxvCpus': 512,
'desiredvCpus': 24,
'instanceTypes': [
'optimal'
],
'subnets': [
'subnet-fa22de86'
],
'securityGroupIds': [
'sg-2436d85c'
],
'instanceRole': 'arn:aws:iam::<ID>:instance-profile/ecsInstanceRole',
'tags': {
'component': 'ukai-training-pipeline',
'product': 'Cormorant',
'jira_project_team': 'CORPRJ',
'business_unit': 'Threat Systems Products',
'created_by': 'ml-pipeline'
}
},
'serviceRole': 'arn:aws:iam::<ID>:role/AWSBatchServiceRole'
}
but the Compute Environments table on the Batch page in AWS console UI does not show anything. The table is empty. When I try to create compute environment with the same name again via boto3 call, I get this response:
ERROR - Error setting compute environment: An error occurred
(ClientException) when calling the CreateComputeEnvironment operation: Object already exists.
Based on the comments, the issue was the use of different region in the console.
The solution was to change the region.

Can I get AMI details with out creating the EC2 instance from AMI in AWS

Is it possible to get AMI details like Operating System Type, Operating System Version and Softwares that were used to build the AMI without creating EC2 instance of it.
I know that I can get the details by creating a EC2 Instance from the AMI.
I what to get these details without creating EC2 instance.
Sometimes. It all depends on who created the AMI. In general, an AMI provides the following information:
'Architecture': 'i386'|'x86_64'|'arm64',
'CreationDate': 'string',
'ImageId': 'string',
'ImageLocation': 'string',
'ImageType': 'machine'|'kernel'|'ramdisk',
'Public': True|False,
'KernelId': 'string',
'OwnerId': 'string',
'Platform': 'Windows',
'ProductCodes': [
{
'ProductCodeId': 'string',
'ProductCodeType': 'devpay'|'marketplace'
},
],
'RamdiskId': 'string',
'State': 'pending'|'available'|'invalid'|'deregistered'|'transient'|'failed'|'error',
'BlockDeviceMappings': [
{
'DeviceName': 'string',
'VirtualName': 'string',
'Ebs': {
'DeleteOnTermination': True|False,
'Iops': 123,
'SnapshotId': 'string',
'VolumeSize': 123,
'VolumeType': 'standard'|'io1'|'gp2'|'sc1'|'st1',
'Encrypted': True|False,
'KmsKeyId': 'string'
},
'NoDevice': 'string'
},
],
'Description': 'string',
'EnaSupport': True|False,
'Hypervisor': 'ovm'|'xen',
'ImageOwnerAlias': 'string',
'Name': 'string',
'RootDeviceName': 'string',
'RootDeviceType': 'ebs'|'instance-store',
'SriovNetSupport': 'string',
'StateReason': {
'Code': 'string',
'Message': 'string'
},
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
],
'VirtualizationType': 'hvm'|'paravirtual'
So while you can get the architecture, unless the creator included a Name, Desription, or Tags with the information you are looking for, you may be out of luck.
Yes. You can query AMI using CLI, Console and APIs.
An example CLI to query AMI using ami-id is as below:
aws ec2 describe-images --region us-east-1 --image-ids ami-XXXXXXXXXX

How to Set required attributes in aws cognito user pool using aws cloudformation template?

Aws cognito console screen
While creating user-pool using cloudformation template i wanted to add the following attributes(marked in the attached image link). I didn't find anything helpful in the AWS documentation.
It allows setting up Alias attributes as said in the aws cloudformation cognito documentation.
Has anybody tried this or has any idea regarding this?
I managed to get it done using the schema attribute of the AWS::cognito::UserPool:
"myApiUserPool": {
"Type": "AWS::Cognito::UserPool",
"Properties": {
"AdminCreateUserConfig": {
"AllowAdminCreateUserOnly": true
},
"Schema": [
{
"Mutable": false,
"Name": "email",
"Required": true
},
{
"Mutable": false,
"Name": "family_name",
"Required": true
},
{
"Mutable": false,
"Name": "name",
"Required": true
}
],
"AutoVerifiedAttributes": [
"email"
],
"UserPoolName": {
"Fn::Sub": "myApiUserPool${envParameter}"
}
}
}
Here is the example with YAML.
Note: you cannot just update a attribute you need to delete the userpool and create it again with the new attributes (just comment out your pool section and redeploy it). Otherwise it will ask for a AttributeDataType, and if you include it, it will create a custom attribute instead of standard one.
CognitoUserPool:
Type: AWS::Cognito::UserPool
Properties:
# Generate a name based on the stage
UserPoolName: ${self:custom.stage}-cfp-user-pool
AliasAttributes:
- phone_number
- email
- preferred_username
Policies:
PasswordPolicy:
MinimumLength: 8
Schema:
- Name: email
Required: true
Mutable: true
Adding on #jWang1 and considering you don't want to delete a user pool with lots of active users but you really need to add a parameter during for sign up process, then you can just add a custom attribute to the template and enforce it as required through your authentication library or custom implementation
The minimum parameters to achieve this are:
UserPool:
Type: AWS::Cognito::UserPool
Properties:
Schema:
-
Name: <attr name>
AttributeDataType: Boolean | DateTime | Number | String

boto3: elastic beanstalk --> 'create_environment' with RDS DB instance?

I have a web server, load balancing, auto scaling, vpc, beanstalk environment with RDS DB instance attached.
I use EB CLI eb create with --database to create beanstalk environments.
I'd like to use boto3 create_environment instead.
Although I'm using OptionsSettings to define the RDS database configuration, it is creating the environment without RDS.
Does anyone know how to create an environment with and RDS instance using boto3?
Here is the boto3 command I'm using with only the RDS portion of my OptionsSettings ():
eb_client = boto3.client('elasticbeanstalk')
response = eb_client.create_environment(
ApplicationName='APP',
EnvironmentName='ENV',
CNAMEPrefix='CNAME',
Tier={
'Name': 'WebServer',
'Type': 'Standard'
},
SolutionStackName='64bit Amazon Linux ...',
OptionSettings=[
...
{
'Namespace': 'aws:rds:dbinstance',
'OptionName': 'DBEngineVersion',
'Value': '5.6'
},
{
'Namespace': 'aws:rds:dbinstance',
'OptionName': 'DBPassword',
'Value': 'PASSWORD_HASH'
},
{
'Namespace': 'aws:rds:dbinstance',
'OptionName': 'DBAllocatedStorage',
'Value': '5'
},
{
'Namespace': 'aws:rds:dbinstance',
'OptionName': 'DBInstanceClass',
'Value': 'db.t2.micro'
},
{
'Namespace': 'aws:rds:dbinstance',
'OptionName': 'DBEngine',
'Value': 'mysql'
},
{
'Namespace': 'aws:rds:dbinstance',
'OptionName': 'DBUser',
'Value': 'ebroot'
},
{
'Namespace': 'aws:rds:dbinstance',
'OptionName': 'DBDeletionPolicy',
'Value': 'Snapshot'
},
...
]
)