I have GCP instance.
My Database size was 300GB.
but the instance size was growing to 10TB.
where my remaining size going?
I have running ongoing replication from AWS DMS to GCP .
Please help me.
Thanks
Probably your bin logs/wal logs have grown a lot without you noticing it, you can check for this at metrics explorer and check for the detailed storage of your instance. There is no way in GCP to shrink the disk on your own so you will have to either migrate your instance into a new one with an smaller disk or open a ticket to Cloud Support and ask them to shrink your disk once you configure your instance properly
Related
I have AWS linux based server with one project, and now I want to deploy another project on the same server. For this I want to know whether my existing memory is enough or should I have to increase the memory limit, and please let me know how to increase the memory limit.
Please refer the below images for available memory space.
There are two approaches to using a database in AWS.
You can install the database on the Amazon EC2 instance. You will then be responsible for configuring and maintaining the database and doing backups. The up-side is that it can run on the same EC2 instance as your application.
Or, you can use Amazon RDS to provide a database. Amazon RDS can install, configure and operate the database for you, including taking backups. It runs on a separate computer so there are additional costs involved, but there are many benefits to keeping a database separate from the application, such as allowing you to scale your application separately to the database. Large applications often run across multiple computers and they can all connect to the one database on Amazon RDS.
From your description, it looks like you are going with the first option. You can increase the disk capacity of the Amazon EC2 instance by increasing the size of the Amazon EBS disk volume (and then do a reboot). If you desire more RAM, then Stop the instance, change the Instance Type to something larger, then Start the instance again.
I have done a lot of reading about the sharing of files on EC2 instances and am finding the answer to be EFS which for now is not available in my region.
I am new to AWS and have got my instances set up how I want them with my app running, but have a few questions if someone can help would be appreciated.
I am using logs as an example but there are many other parts of the application that are really centralised and I don't which to lose them when an instance is terminated after use.
1) If I have an application writing logs to the local disk and I have auto scale set up that means a new instance fires up with my app again writing logs to local instance disk. Am I correct?
If so, when the instance is no longer needed it would delete the instance and local disk therefore losing the application logs.
2) I understand s3 is available but am worried about the performance as this app will be writing logs continuously and has million plus users.
Alternatives I am considering.
Write Logs to database
Have a small instance which could be used
purely as file share and everything sits in the share
Any suggestions would be helpful.
Also this is not just for logs there are security credentials etc that are shared.
I am working with the AWS Aurora PostgreSQL 10.4 engine. I am trying to cluster table ... using index and getting an error like
could not write block .... of temporary file: no space left on device
If I were managing my own PostgreSQL instance I would be looking at the space available on individual volumes with df. (See also: I get an error "could not write block .... of temporary file no space left on device ..." using postgresql)
But with Aurora, AWS should be managing the storage and automatically expanding it on demand. So I'm wondering how I would go about fixing this condition if I'm not managing the storage myself. I'm guessing that that the PG engine's temp storage is separate from the Aurora-managed virtualized storage layer, but not sure how to change it.
Temp space uses the local “ephemeral” volume on the instance. Currently the only way to increase that space is to move to a larger instance size.
You're right in stating that Aurora should take care of this. If you have multiple instances in your cluster, then your cluster would self recover by initiating a failover. The faulty instance would be repaired in the background as well - mostly automatically, and in some rare cases by AWS operators.
If you noticed the issue persisting for more than a few minutes, then you should:
Manually trigger a failover using API/Console
Engage AWS Support to look into the matter if it happens more often.
If you think AWS missed your SLA, then do bring it to their notice as well.
AWS will use commercially reasonable efforts to make Multi-AZ instances available with a Monthly Uptime Percentage (defined below) of at least 99.95% during any monthly billing cycle (the "Service Commitment"). In the event Amazon RDS does not meet the Monthly Uptime Percentage commitment, you will be eligible to receive a Service Credit
Doc: https://aws.amazon.com/rds/sla/
I will install a website in the free EC2 from amazon but I read something not good: I have a simple website which uses a database. Users come inside my website and post information, send commetns... if for some reason the instance breaks or amazon shuts it down, will I lose all information posted in my website and database? All files users uploaded and information saved will be gone?
If so, why would someone use EC2 if you lose all your data if some problem happens, and because problems always happen, sometime I will certainly lose my data!
I know I can save an image of my current OS in AWS but do I need to save the image everytime a user posts something to my website? It's ridiculous. I know I am missing something here, but I looked into google and people all the time say I should use EBS but it's not in the free plan. So how is it good idea using AWS EC2 free plan if my data will always be at risk of being lost?
Typically you would want to use an EBS backed instance. Since the free tier does not support that, but does offer EBS storage, create your database on an EBS partition for data you cannot lose
30 GB of Amazon Elastic Block Storage, plus 2 million I/Os and 1 GB of snapshot storage*
http://aws.amazon.com/free/
You should have a means to quickly launch a new instance, and you should back up the data on your EBS partition because EBS volumes can and do fail from time to time.
UPDATE
It seems that Micro instances are in fact EBS backed.
It is still advisable to attach a separate EBS volume, because it makes it much more convenient to backup the database (you create a snapshot of the EBS volume... you can find scripts online to accomplish that, which vary a bit depending on your choice of database and file system).
I am creating daily snapshots (backups) in Amazon Ec2, and I need to find out how much space the snapshots are taking so that I can remove them if they take up too much space. I have looked and I am unable to find what I need.
I know that its on S3, but I have not seen any bucket created here so that I can see that snapshot.
Also, is there a way to download a snapshot to my computer (where I can store it), and upload it when needed?
Snapshots are persisted to S3, but are not stored in buckets owned by the user - so you won't be able to see them there.
To see how much space is being used by snapshots, you should be able to log in to the AWS Console through your web browser and see it under "Elastic Block Store", or if you've installed the command-line tools by running the ec2-describe-snapshots command you will see the following parameter returned:
volume-size
The size of the volume
As for downloading your snapshots, it's possible with the non-Windows instance snapshots, but it's also quite involved. But here's the instructions.