We have EC2 instance hosting a web application and another RDS Server with SQL Server. Both are in US-WEST-2 region. The web site has a single page with button. When user clicks the button it UPDATES a table using a stored procedure. This update is not a complex updates, it selects a single row based on criteria and then updates that row. However table has more than 200K rows.
When users from the US access application they don't seems to have any issue. However, we have couple of users access the application from Africa. Users from Africa,however experience slow access and for them page crash every now and then.
When i checked the log, it always SQL timeout. Total number of users from US and Africa are not more than 10 so even if all clicked the button at the same time SQL Server should be able to handle 10 users.
(On our dev server I have done load testing with 100 users and I never got SQL timeout)
I understand that there will be latency for Users from Africa while accessing application. However, i am not able to understand why only those users seems have SQL timeout issue?
Is there any settings/configuration in AWS that we need to check for cross region requests? Does EC2 handles request differently when it comes from different continent/region? I understand its hard to pinpoint exact problem here but I am looking for suggestions especially related to AWS configuration for cross region requests.
Related
I have nearly reached my AWS S3 usage limit for the month, after having started it just a couple of weeks ago, and I'm trying to understand why. I've never used AWS before, and I'm a newb in general. I literally have 3 jpegs in storage, nothing else.
Is usage solely determined by hours? I read that hours are instances where the storage is made available. Right now I have the Django project hosted on Heroku with media on AWS. I am very much still in dev mode, and only need AWS when testing project on Heroku, but most of the time I'm in local environment. Given that, I am confused how my usage limit could be nearly reached within a couple of weeks. Any guidance is appreciated.
Update: I was mistaken. The account number listed in the warning email is the same used for my AWS S3, but it is referring instead to RDS. When I see all products under the account it shows S3 (which is well below the limit) and RDS (which is nearly at the limit). The quandary now though is that I never intended to create an RDS instance. And, when I click on "databases" under RDS, it actually says I don't have any RDS instances. So I don't have RDS yet I do because I am soon to be charged for going over the limit. I've never used AWS before, and I'm a newb in general. I initiated S3, the only product I intended to have under the account, for the sole purpose of serving media files not served by Heroku.
In the AWS management console:
Click your username (in the top-right, to the left of the Region name)
Select My Billing Dashboard
Click Bill Details in the top-right
This will give you a breakdown of costs by service and region.
It is possible that your Amazon RDS instance is running in a different region, so find the region where the RDS costs are occurring and:
Go to Amazon RDS in the management console
Change the Region in the top-right to the one that was shown in the billing console
Delete the database (if you are sure you aren't using it)
If I turn on a machine in EC2, what expectation of privacy do I have for my running processes, command line history, data stored on ephemeral disk, etc?
Can people at Amazon decide to take a look at what I'm running?
Could Amazon decide to do some profiling for the purposes of upselling?
Hi there! Looks like you're running Cassandra! Here's the optimal
tuning requirements for Cassandra on your m1.xlarge machine!
I can't seem to find anything in the docs...
This is the most applicable thing I found:
AWS only uses each a customer's content to provide the AWS services
selected by that customer and does not use customer content for any
other purposes. AWS treats all customer content the same and has no
insight into what type of content the customer chooses to store in
AWS. AWS simply makes available the compute, storage, database,
mobile, and network services selected by the customer. AWS does not
require access to customer content to provide its services.
http://aws.amazon.com/compliance/data-privacy-faq/
What you are asking about should be addressed in their "Data Privacy" policy (http://aws.amazon.com/agreement/) in their Customer Agreement page:
3.2 Data Privacy. We participate in the safe harbor programs described in the Privacy Policy. You may specify the AWS regions in which Your
Content will be stored and accessible by End Users. We will not move
Your Content from your selected AWS regions without notifying you,
unless required to comply with the law or requests of governmental
entities. You consent to our collection, use and disclosure of
information associated with the Service Offerings in accordance with
our Privacy Policy, and to the processing of Your Content in, and the
transfer of Your Content into, the AWS regions you select.
Here's a link to their "Privacy Policy":
http://aws.amazon.com/privacy/
So in essence, it's saying that you need to consent for them to gather information stored in your server. Now that's different from poking at the TCP ports on your machines from the outside. Amazon constantly runs port checking and traffic checking from the outside (it could be in their intranet too) to make sure you are complying with their customer agreement. For example, they can monitor that you are not hosting something illegal (through public content) or that you are not sending spam or robot traffic to hack into other servers.
Having said that, it's quite possible that they use some of these monitoring tools to check: ok this person has port so and so open. So he/she must be running this application and we can suggest something better for them.
Hope it helps.
So I'm digging into AppFog's pricing sheet and I am struggling to figure out which one I need. here
Several questions
What is a service instance?
How do I somehow translate "service instance" to web traffic capabilities?
I am deploying a web app soon and looking at something to handle serious traffic (around a million users a month)
Being in a similar situation as you I think that a possible hint is the Pagodabox traffic visualizer. Also from one of the Pagodabox user stories I've concluded that having 16 service instances with 400 MB RAM each can handle 1M users (assuming that you have a reliable DB, and probably using Memcached).
(In AppFog the Services menu in the admin portal is really misleading - count instances in the Applications menu)
I was dorking around with AWS (and related services), hoping that I could stay in the Free Tier, like I do when I'm exploring Google App Engine.
A few days ago, I get a letter from Amazon that they've charged me $33 or so for my 2 days of exploration.
This has got to end, but I forget what services I've enabled. Ideally, I'd just disable the AWS account entirely, as without a free sandbox there's no way I'm going to be using their service. Is there a global off button, or do I have to stumble around to turn all their services off individually? Or do I have to delete my CC information and just create a new Amazon account altogether?
You can close your entire account in AWS Billing: https://console.aws.amazon.com/billing/home?#/account
Or if you just want to disable your "Free-Tier" services that has charges, view them here:
https://console.aws.amazon.com/billing/home#/freetier
Then open your EC2 dashboard - and cancel those services:
https://us-west-2.console.aws.amazon.com/ec2
For example:
Stop running instances, delete volumes, remove elastic IPs, etc.
Otherwise, I recommend sending an email to webservices#amazon.com from the email you used to signup with their service.
I had an RDS running and I couldn't figure out how to cancel just that service
Here's how to do it:
Go to billing services
https://console.aws.amazon.com/billing/home?region=us-west-2#/
Click "Bill Details"
Inspect it
You'll find NAME OF SERVICE + ITS LOCATION. This is the information you need.
https://console.aws.amazon.com/rds/home?region=us-east-1
Go to topright of page. Select the correct server location
The rest is straightforward from here
I was also frustrated (by being charged on the free tier without any info/warning in prior) and found a simple and elegant solution to turn off all AWS services. You delete your account and forget about these fraudulent (to be honest) AWS services.
Here is the link:
https://console.aws.amazon.com/billing/home?#/account
Here is the section:
I know this is a somehow an old question, but I would like to add a new answer because I think AWS has changed a lot since this was asked. I have stumbled on a similar situation as the OP and I found out that there are 3 possible ways to achieve this:
To have a single turn-off-everything button, but I'm not sure if this exists.
Overkill, go through the services and check them one by one and shutdown/delete any instances or running services.
To find out the actual source of leaking (cost occurring services) by viewing what is posting charges on your account and then turn off these services one by one. This can be done by visiting:
your AWS account >> My Billing Dashboard
Find your account username and open the drop down menu:
You can check what services are incurring fees.
Percentage table:
I followed the services by searching for their name on AWS console, if I couldn't find it I'd Google how to do so and then turned them off one by one.
In my case, there was no charge towards my bank even thought billing showed I have some balance, I think it's because I was using the free tier, maybe?
I just hit my free tier limit. I terminated my ec2 instance, deleted my storage volume and even removed my security group and key pair so I have nothing now. Hopefully no charge :P
Always make sure you select the right region. I once had 2 instances running and didnt realize it.
Today I finally discovered a global view to detect all the active services, you still have to disable every service manually but at least you don't have to switch all the regions to understand where you have active services.
every once in a while i read/hear about AWS and now i tried reading the docs.
But such docs seem to be written for people who already know which AWS they need to use and only search for how it can be used.
So, for myself, to understand AWS better i try to sketch a hypothetical Webapplication with a few questions.
The apps purpose is to modify content like videos or images. So a user has some kind of webinterface where he can upload his files, do some settings and a server grabs the file and modifies it (e.g. reencoding). The Service also extracts the audio track of a video and trys to index the spoken words so the customer can search within his videos. (well its just hypothetical)
So my questions:
given my own domain 'oneofmydomains.com' is it possible to host the complete webinterface on AWS? i thought about using GWT to create the interface and just deliver the JS/images via AWS, but which one, simple storage? what about some kind of index.html, is there an EC2 instance needed to host a webserver which has to run 24/7 causing costs?
now the user has the interface with a login form, is it possible to manage logins with an AWS? here i also think about an EC2 instance hosting a database, but it would also cause costs and im not sure if there is a better way?
the user has logged in and uploads a file. which storage solution could be used to save the customers original and modified content?
now the user wants to browse the status of his uploads, this means i need some kind of ACL, so that the customer only sees his own files. do i need to use a database (e.g. EC2) for this, or does amazon provide some kind of ACL, so the GWT webinterface will be secure without any EC2?
the customers files are reencoded and the audio track is indexed. so he wants to search for a video. Which service could be used to create and maintain the index for each customer?
hope someone can give a few answers so i understand AWS better on how one could use it
thx!
Amazon AWS offers a whole ecosystem of services which should cover all aspects of a given architecture, from hosting to data storage, or messaging, etc. Whether they're the best fit for purpose will have to be decided on a case by case basis. Seeing as your question is quite broad I'll just cover some of the basics of what AWS has to offer and what the different types of services are for:
EC2 (Elastic Cloud Computing)
Amazon's cloud solution, which is basically the same as older virtual machine technology but the 'cloud' offers additional knots and bots such as automated provisioning, scaling, billing etc.
you pay for what your use (by hour), for the basic (single CPU, 1.7GB ram) would prob cost you just under $3 a day if you run it 24/7 (on a windows instance that is)
there's a number of different OS to choose from including linux and windows, linux instances are cheaper to run without the license cost associated with windows
once you're set up the server to be the way you want, including any server updates/patches, you can create your own AMI (Amazon machine image) which you can then use to bring up another identical instance
however, if all your html are baked into the image it'll make updates difficult, so normal approach is to include a service (windows service for instance) which will pull the latest deployment package from a storage (see S3 later) service and update the site at start up and at intervals
there's the Elastic Load Balancer (which has its own cost but only one is needed in most cases) which you can put in front of all your web servers
there's also the Cloud Watch (again, extra cost) service which you can enable on a per instance basis to help you monitor the CPU, network in/out, etc. of your running instance
you can set up AutoScalers which can automatically bring up or terminate instances based on some metric, e.g. terminate 1 instance at a time if average CPU utilization is less than 50% for 5 mins, bring up 1 instance at a time if average CPU goes beyond 70% for 5 mins
you can use the instances as web servers, use them to run a DB, or a Memcache cluster, etc. choice is yours
typically, I wouldn't recommend having Amazon instances talk to a DB outside of Amazon because of the round trip is much longer, the usual approach is to use SimpleDB (see below) as the database
the AmazonSDK contains enough classes to help you write some custom monitor/scaling service if you ever need to, but the AWS console allows you to do most of your configuration anyway
SimpleDB
Amazon's non-relational, key-value data store, compared to a traditional database you tend to pay a penalty on per query performance but get high scalability without having to do any extra work.
you pay for usage, i.e. how much work it takes to execute your query
extremely scalable by default, Amazon scales up SimpleDB instances based on traffic without you having to do anything, AND any control for that matter
data are partitioned in to 'domains' (equivalent to a table in normal SQL DB)
data are non-relational, if you need a relational model then check out Amazon RDB, I don't have any experience with it so not the best person to comment on it..
you can execute SQL like query against the database still, usually through some plugin or tool, Amazon doesn't provide a front end for this at the moment
be aware of 'eventual consistency', data are duplicated on multiple instances after Amazon scales up your database, and synchronization is not guaranteed when you do an update so it's possible (though highly unlikely) to update some data then read it back straight away and get the old data back
there's 'Consistent Read' and 'Conditional Update' mechanisms available to guard against the eventual consistency problem, if you're developing in .Net, I suggest using SimpleSavant client to talk to SimpleDB
S3 (Simple Storage Service)
Amazon's storage service, again, extremely scalable, and safe too - when you save a file on S3 it's replicated across multiple nodes so you get some DR ability straight away.
you only pay for data transfer
files are stored against a key
you create 'buckets' to hold your files, and each bucket has a unique url (unique across all of Amazon, and therefore S3 accounts)
CloudBerry S3 Explorer is the best UI client I've used in Windows
using the AmazonSDK you can write your own repository layer which utilizes S3
Sorry if this is a bit long winded, but that's the 3 most popular web services that Amazon provides and should cover all the requirements you've mentioned. We've been using Amazon AWS for some time now and there's still some kinks and bugs there but it's generally moving forward and pretty stable.
One downside to using something like aws is being vendor locked-in, whilst you could run your services outside of amazon and in your own datacenter or moving files out of S3 (at a cost though), getting out of SimpleDB will likely to represent the bulk of the work during migration.