I registered for aws account yesterday and today i followed couple of videos on youtube to run a sample wordcount on input file in S3.
I tried to do that but i don't see any "configuration sample application" button. I have attached an image. It may be trivial, since i am new i may be missing something.
Process i followed:
Created a bucket in S3
aws-> security credentials, created an access
key aws->EC2, created key pair
AWS->IAM, created new role as EC2 + administrator
AWS->EMR, create cluster
Here i don't see any option for configure sample application button. Please check image for more detail
Amazon EMR used to have a 'Sample Application' button, like this:
However, that button is no longer available in the Amazon EMR interface.
The tutorial is most probably out-of-date. (Things change fast on AWS!)
Related
I'm trying to set up my first SageMaker Studio so my team and myself can run some post processing scripts in a shared environment but I'm having issues.
I've followed the steps in this video(https://www.youtube.com/watch?v=wiDHCWVrjCU&ab_channel=AmazonWebServices) which are:
Select Standard setup
Select AWS Identity and Access Management (IAM)
Under permissions - Create and select new execution role
Under Network and storage - Select VPC, Subnet and Security group
Hit the submit button at the bottom of the page.
In the video, he clicks submit and is taken to the control panel where he starts the next phase of adding users, however I'm greeted with this error.
Resource limit Error
I've checked my Registered domains under route 53 and it says No domains to display, I've also checked my S2 and I have no instances so I have no idea where the 2 domains being utilized are.
My dashboard, image and Notebooks are all empty so as far as I know there's nothing setup on this Sage Maker account.
Could anyone tell me how to resolve this error?
AWS Sagemaker now supports multi-domain <announced in the Re-Invent 2022, Tested in US-EAST-1 >
enter image description here
You can have maximum 1 studio domain per region, by the default limits. Though, it seems like you have two domains already provisioned. Try to delete all the domains through the AWS cli and recreate with the AWS Management Console.
Unfortunately, AWS Management Console cannot visualize more than one Studio domain.
I got this kind of error when I tried to restore from s3 bucket
--error message
Msg 50000, Level 16, State 0, Procedure msdb.dbo.rds_restore_database,
Line 49 [Batch Start Line 0]
Database backup/restore option is not
enabled yet or is in the process of being enabled. Please try again
later. USAGE: EXECUTE msdb.dbo.rds_restore_database #restore_db_name,
#S3_arn_to_restore_from, [#KMS_master_key_arn] #restore_db_name : Name
of the database being restored. #S3_arn_to_restore_from : S3 ARN of
the backup file used to restore database from. #KMS_master_key_arn :
KMS customer master key ARN to decrypt the backup file with.
--executed code
exec msdb.dbo.rds_restore_database
#restore_db_name='C2',
#s3_arn_to_restore_from='arn:aws:s3:::slkk/31_May_2019_C2.bak';
I am using aws free account. How do I enable backup restore?
Go to IAM and crete new Role with Permissions as follow:
AWSBackupServiceRolePolicyForRestores
AWSBackupServiceRolePolicyForBackup
Go to RDS > Option groups
Create group (Name: SqlServerBackupRestore, Description: xxx, Engine: Select your DB engine, Major Engine Version: Select version of your DB instance.
Click Create
Once group is created, click the name of created group to Edit.
Click Add option
Select option: SQLSERVER_BACKUP_RESTORE
Select IAM role you created in previous steps
Select Immediately to schedule instant change
Back to RDS > Databases and select your instance
Click Modify
Select Option group you created in previous step
Click Next
Select Apply immediately (it should not cause service downtime)
Apply changes by clicking Modify DB instance
If you have connection to DB from SQL Management Studio please close and connect again.
I know this is old, but I ran into the same error and was able to resolve it so I thought I'd share since this was unanswered.
First, this article details what you need, so make sure you follow these steps:
https://aws.amazon.com/premiumsupport/knowledge-center/native-backup-rds-sql-server/
I had setup the Options Group and everything initially. The steps I missed were these two:
Associate the option group with the DB instance by choosing Databases in the navigation pane, and then choose the instance to back up. Choose Actions, and then choose Modify.
Under Database Options, choose the option group that you created, and then choose Apply Immediately and Continue. Review the information, and then choose Modify DB Instance. This option group modification has no downtime because instance reboot is not required.
In other words, you have to modify the actual database instance with the new Option Group that was setup in those steps. Then once you login to your instance via SSMS, you can execute the restore by executing (taken from the article):
exec msdb.dbo.rds_restore_database
#restore_db_name='database_name',
#s3_arn_to_restore_from='arn:aws:s3:::bucket_name/file_name_and_extension';
For some reason choosing 'SQLSERVER_AUDIT' in the option group solved it for me, as opposed to the seemingly logical 'SQLSERVER_BACKUP_RESTORE'
For creating the IAM role first and then adding that to options not working for some reason, I tried with admin policy in that role also doesn't work for me.
rather than creating the IAM role ourselves first, In the RDS Add Option for choosing the role, there is an option to creating a new role use this option and select the s3 bucket name this will create the correct policy and IAM role for us, It works perfectly also helped this youtube video https://www.youtube.com/watch?v=tbJvyQEjZ8k.
Folowed few tutorials:
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Procedural.Importing.html
https://aws.amazon.com/premiumsupport/knowledge-center/native-backup-rds-sql-server/
Here is what solved my case:
I added below to my IAM role created while instance was built:
Also setting new password for Master username seems to have did the trick as after that msdb.dbo.rds_restore_database started to work.
I am new to AWS. I am trying to put my application on AWS. I envision to have two EC2 instances.
My goal is to provide a link to the AWS instance to my users so that they can "import" the instances created by me into their own VPC. By doing so I am planning to save users the pain for setting up my application.
Is this possible to do with AWS? Where should I begin?
You need to create an image of your running AMI and then share it as a "public" AMI.
You can create an image by selecting the running EC2 instance in the console and then under the "Actions" dropdown select "Image" and then "Create Image".
Once your image is created you can modify the permissions using the method outlined in the following docs:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/sharingamis-intro.html
If you wish to distribute you AMI through the AWS Marketplace there are some hoops that you need to jump through and forms to submit to get your product metadata into the marketplace.
Start by creating an account here:
https://aws.amazon.com/marketplace/management/tour/
I have a AMI which is used to launch instances automatically though Amazon Launch configuration.
The issue I am facing is that after instance gets launched I need to run some command which should start the WebServer automatically. how can I do that using user data option from launch configuration.
You can find the option "User Data" under "Advanced Details" section in the Launch Configuration Wizard
I don't think there is anyway via the AWS console or with the official aws-tools to modify a Launch Configuration. I believe you have to create a new one with the user data and associate it with the autoscaling group or groups it's in with these two scripts:
as-create-launch-config
as-update-auto-scaling-group
However, if you're brave...
I did find this: https://github.com/colinbjohnson/aws-missing-tools/tree/master/as-update-launch-config
It explicitly limits the type of instance you're using (it was wrote in 2012) to a narrow list of older options, so update it to new ones.
Outside this, I've not tested the script, so good luck!
From the documentation:
Follow the procedure for launching an instance at Launching Your Instance from an AMI, but when you get to Step 6, paste the user data script text into the User data field and then complete the launch procedure.
If you don't want to copy-paste a script every time you launch an instance, you have a few different options:
Launch the instance using the AWS CLI and point it at a file that contains your script
Use CloudFormation to launch the instance and related resources and specify the userdata script in the template
Use OpsWorks (I'm less familiar with this one)
However, if you just want the webserver service to start at boot, I recommend looking at this question.
copy paste
#!/bin/sh
curl -L https://s3.amazonaws.com/romuloaddressbook/addressbook.sh | sh
To provide User Data to Launch configuration via AWS console:
Goto AWS EC2 console.
Under "AUTOSCALING" click on "Launch Configurations" and the select "Create launch configuration". Create Launch Configuration wizard will open.
Select the required AMI.
In the next window, select the Instance Type and click "Next: Configure details".
Under "Configure details" -> "Advanced Details" -> Enter your USERDATA in the "User data" text box.
Proceed further and complete the wizard.
I have enabled this option.
Problem is:
If I don't press snapshot log button log, is not going to s3.
Is there any method through which log publish to s3 each day?
Or how log file rotation option is working ?
If you are using default instance profile with Elastic Beanstalk, then AWS automatically creates permission to rotate the logs to S3.
If you are using custom instance profile, you have to grant Elastic Beanstalk permission to rotate logs to Amazon S3.
The logs are rotated every 15 minutes.
AWS Elastic Beanstalk: Working with Logs
For a more robust mechanism to push your logs to S3 from any EC2 server instance, you can pair LogRotate with S3. I've put all the details in this post as a reference whicould should be able to achieve exactly what you're describing.
Hope that helps.
NOTICE: if you want to rotate custom log files, then, depending on your container, you need to add links to your custom log files in a proper places. For example, consider Ruby on Rails deployment, if you want to store custom information, eg. some monitoring using Oink gem in oink.log file, add proper link in /var/app/support/logs using .ebextensions
.ebextensions/XXXlog.config
files:
"/var/app/support/logs/oink.log" :
mode: "120400"
content: "/var/app/current/log/oink.log"
This, after deploy, will create symlink:
/var/app/support/logs/oink.log -> /var/app/current/log/oink.log
I'm not sure why permissions 120400 are used, I took it from the example in Amazon AWS doc page http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html (seems like 120xxx is for symlinks in unix fs)
This log file rotation is good for archival purpose, but difficult to search and consolidate when you need the most.
Consider using services like splunk or loggly.