lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 99.4M 1 loop /snap/core/11606
loop1 7:1 0 25M 1 loop /snap/amazon-ssm-agent/4046
loop2 7:2 0 55.5M 1 loop /snap/core18/2074
loop3 7:3 0 99.4M 1 loop /snap/core/11420
loop4 7:4 0 55.4M 1 loop /snap/core18/2128
loop5 7:5 0 33.3M 1 loop /snap/amazon-ssm-agent/3552
xvda 202:0 0 10G 0 disk
└─xvda1 202:1 0 10G 0 part /
As you can see I should hav3 10G mounted here, I know that I need to have a lot of space left since I have just increased my volume on AWS.
df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 98M 11M 88M 11% /run
/dev/xvda1 7.7G 7.7G 0 100% /
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/loop0 100M 100M 0 100% /snap/core/11606
/dev/loop1 25M 25M 0 100% /snap/amazon-ssm-agent/4046
/dev/loop2 56M 56M 0 100% /snap/core18/2074
/dev/loop3 100M 100M 0 100% /snap/core/11420
/dev/loop4 56M 56M 0 100% /snap/core18/2128
/dev/loop5 34M 34M 0 100% /snap/amazon-ssm-agent/3552
tmpfs 98M 0 98M 0% /run/user/1000
sudo fdisk
sudo: unable to resolve host ip-xxxxx: Resource temporarily unavailable
As you can see, only 8.2GB is used here, I don't know where the last 1.8GB are, I need to add them to "/"
Ubuntu
Linux/UNIX
sudo: unable to resolve host ip-xxxxx: Resource temporarily unavailable
this issue was unrelated, I fixed it by updating the DNS config.
As for the lost volume, it appeared once I rebooted the server.
Related
I am running an Ubuntu Amazon EC2 instance with a 100gb volume mounted. However, when I hit 56gb, the FS says that it has run out of storage. It clearly has plenty of space available.
This is the result of the df.
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 97G 56G 42G 58% /
tmpfs 7.9G 0 7.9G 0% /dev/shm
tmpfs 3.2G 852K 3.2G 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/xvda15 105M 5.3M 100M 5% /boot/efi
tmpfs 1.6G 4.0K 1.6G 1% /run/user/1000
As you can see, the OS knows that there is space, but seemingly refuses to write. Prior to experiencing this problem, I had increased the volume size from 8GB to 100GB, and used the commands growpart and resize2fs to scale the volume to full size. I have a feeling that this is due to some unknown volume size limit within either Ubuntu or AWS, but I have no idea how to diagnose or solve this problem. My initial guess was that EXT4 had a limit, but that doesn't seem to be applicable here (the limit is 1TB). Here is the result of lsblk:
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 25.1M 1 loop /snap/amazon-ssm-agent/5656
loop1 7:1 0 55.5M 1 loop /snap/core18/2409
loop2 7:2 0 61.9M 1 loop /snap/core20/1518
loop3 7:3 0 79.9M 1 loop /snap/lxd/22923
loop4 7:4 0 47M 1 loop /snap/snapd/16010
loop5 7:5 0 47M 1 loop /snap/snapd/16292
xvda 202:0 0 100G 0 disk
├─xvda1 202:1 0 99.9G 0 part /
├─xvda14 202:14 0 4M 0 part
└─xvda15 202:15 0 106M 0 part /boot/efi
Any help here would be greatly appreciated, thanks in advance :)
Today my Jenkins start failing and blocking access due to this following error:
java.io.IOException: No space left on device
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:211)
at hudson.util.FileChannelWriter.write(FileChannelWriter.java:73)
at java.io.Writer.write(Writer.java:192)
at hudson.util.AtomicFileWriter.write(AtomicFileWriter.java:163)
at java.io.Writer.write(Writer.java:157)
at hudson.XmlFile.write(XmlFile.java:191)
at jenkins.model.Jenkins.save(Jenkins.java:3474)
at jenkins.model.Jenkins.<init>(Jenkins.java:1012)
at hudson.model.Hudson.<init>(Hudson.java:86)
at hudson.model.Hudson.<init>(Hudson.java:82)
at hudson.WebAppMain$3.run(WebAppMain.java:298)
Caused: hudson.util.HudsonFailedToLoad
at hudson.WebAppMain$3.run(WebAppMain.java:315)
In fact my Jenkins storage is full. According to aws documentation, I increased the Storage size from the portal to 24Gb. After that I accesses my ec2 instance from the terminal.
Following those steps:
https://aws.amazon.com/premiumsupport/knowledge-center/ebs-volume-size-increase/
I was able to increase the storage volume as follow
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 33.3M 1 loop /snap/amazon-ssm-agent/3552
loop1 7:1 0 55.5M 1 loop /snap/core18/1997
loop2 7:2 0 32.3M 1 loop /snap/snapd/13170
loop3 7:3 0 61.9M 1 loop /snap/core20/1169
loop4 7:4 0 32.3M 1 loop /snap/snapd/12883
loop5 7:5 0 55.4M 1 loop /snap/core18/2128
loop6 7:6 0 25M 1 loop /snap/amazon-ssm-agent/4046
loop7 7:7 0 70.3M 1 loop /snap/lxd/21029
loop8 7:8 0 61.8M 1 loop /snap/core20/1081
loop9 7:9 0 67.3M 1 loop /snap/lxd/21545
xvda 202:0 0 24G 0 disk
└─xvda1 202:1 0 24G 0 part /
The xvda has been upgraded to 24G, with is perfect, I was able to start Jenkins service. But when I tried to login I got once more the same error above.
To double check that everything is correct, I run the command df -hT, and the result was this
Filesystem Type Size Used Avail Use% Mounted on
/dev/root ext4 7.7G 7.7G 0 100% /
devtmpfs devtmpfs 2.0G 0 2.0G 0% /dev
tmpfs tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs tmpfs 394M 41M 354M 11% /run
tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/loop0 squashfs 34M 34M 0 100% /snap/amazon-ssm-agent/3552
/dev/loop1 squashfs 56M 56M 0 100% /snap/core18/1997
/dev/loop4 squashfs 33M 33M 0 100% /snap/snapd/12883
/dev/loop5 squashfs 56M 56M 0 100% /snap/core18/2128
/dev/loop6 squashfs 25M 25M 0 100% /snap/amazon-ssm-agent/4046
/dev/loop7 squashfs 71M 71M 0 100% /snap/lxd/21029
/dev/loop8 squashfs 62M 62M 0 100% /snap/core20/1081
/dev/loop9 squashfs 68M 68M 0 100% /snap/lxd/21545
/dev/loop2 squashfs 33M 33M 0 100% /snap/snapd/13170
/dev/loop3 squashfs 62M 62M 0 100% /snap/core20/1169
tmpfs tmpfs 394M 0 394M 0% /run/user/1000
tmpfs tmpfs 10M 3.2M 6.9M 32% /tmp
tmpfs tmpfs 394M 0 394M 0% /run/user/113
I see that /dev/root is 100% used. But I cannot upgrade the Size of that and I couldn't find any documentation on how to upgrade the volume. If I try to run the command:
sudo growpart /dev/root 1
I get this error:
WARN: unknown label
failed [sfd_dump:1] sfdisk --unit=S --dump /dev/root
sfdisk: /dev/root: does not contain a recognized partition table
FAILED: failed to dump sfdisk info for /dev/root
Sorry I am a total newbie here and this is the first time I am facing this issue.
Can please anyone explain me how I can solve this error
Thank you so much for your time and help
Even though you added some storage and enlarged xvda1 partition you didn't enlarge main filesystem (root volume group). I would be good to see output of lsblk, vgs, pvs commands as well as general info about your OS. It is quite likely you need to add available space (from xvda1 partition) to the root vg (/dev/root has still only 7.7G).
PS. to be honest I don't know the command <growpart> but instead of growpart /dev/root I would use growpart /dev/xvda 1 - to be aligned with the instruction (/dev/xvda is the name of the disk attached)
After growing the partition, you need to resize the filesystem too:
resize2fs /dev/xvda1
I am getting no space error in was Linux when I trying to git pull files. I don't know why the /dev/root file is so much big. Is there anyway to free up some space without deleting Linux packages like Node , composer , git etc ?
root#ip-172-31-8-10:/# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 7.7G 7.5G 276M 97% /
devtmpfs 2.0G 0 2.0G 0% /dev
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 393M 864K 393M 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/loop0 34M 34M 0 100% /snap/amazon-ssm-agent/3552
/dev/loop1 56M 56M 0 100% /snap/core18/1997
/dev/loop2 71M 71M 0 100% /snap/lxd/19647
/dev/loop3 71M 71M 0 100% /snap/lxd/21029
/dev/loop4 25M 25M 0 100% /snap/amazon-ssm-agent/4046
/dev/loop6 33M 33M 0 100% /snap/snapd/12883
/dev/loop5 56M 56M 0 100% /snap/core18/2128
/dev/loop7 33M 33M 0 100% /snap/snapd/12704
tmpfs 393M 0 393M 0% /run/user/114
tmpfs 393M 0 393M 0% /run/user/1000
You've configured an 8GB disk which is very small for most installs. You can do two things:
Expand the root volume. Pretty simple job to do on AWS: https://aws.amazon.com/premiumsupport/knowledge-center/expand-root-ebs-linux/
Start removing files that take up space. The operating system itself takes up a significant amount of space unless you're running something super barebone like Alpine, but since you haven't specified which distro you're using, it's hard to give concrete suggestions here.
I am trying to increase the hard drive size of an EC2 instance.
I found the volume, modified it from 8 to 20GB.
Then I logged in the server (ssh) and run the commands
lsblk
sudo growpart /dev/xvda 1
Now the two commands show different size for the hard drive..
ubuntu#ip-xxxx:~$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 16.5M 1 loop /snap/amazon-ssm-agent/784
loop1 7:1 0 88.2M 1 loop /snap/core/5897
loop2 7:2 0 87.9M 1 loop /snap/core/5742
loop3 7:3 0 12.7M 1 loop /snap/amazon-ssm-agent/495
loop4 7:4 0 18M 1 loop /snap/amazon-ssm-agent/930
loop5 7:5 0 87.9M 1 loop /snap/core/5328
xvda 202:0 0 20G 0 disk
└─xvda1 202:1 0 20G 0 part /
ubuntu#ip-xxx:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.9G 0 3.9G 0% /dev
tmpfs 798M 768K 797M 1% /run
/dev/xvda1 7.7G 6.9G 873M 89% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/loop0 17M 17M 0 100% /snap/amazon-ssm-agent/784
/dev/loop3 13M 13M 0 100% /snap/amazon-ssm-agent/495
/dev/loop4 18M 18M 0 100% /snap/amazon-ssm-agent/930
/dev/loop1 89M 89M 0 100% /snap/core/5897
/dev/loop2 88M 88M 0 100% /snap/core/5742
/dev/loop5 88M 88M 0 100% /snap/core/5328
tmpfs 798M 0 798M 0% /run/user/1000
My runcloud.io dashboard also shows that the size did not change (still 8GB).
I also restared the server without any luck.
What should I do?
File system info
ubuntu#ip-xxxx:~$ sudo file -s /dev/xvd*
/dev/xvda: DOS/MBR boot sector
/dev/xvda1: Linux rev 1.0 ext4 filesystem data, UUID=xxxxx, volume name "cloudimg-rootfs" (needs journal recovery) (extents) (64bit) (large files) (huge files)
You can try
sudo resize2fs /dev/xvda1
After running that command you might need to reboot
According to AWS sometimes the ebs volume needs re initializing after resize you can use dd or fio to reinitialize a volume but fio is faster than dd
reinitializing ebs volume
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 4 days ago.
Improve this question
l try to install pytorch on amazon as follow :
conda install pytorch torchvision cuda80 -c soumith
l get the following error :
Package plan for installation in environment /home/ubuntu/anaconda2/envs/crnn:
The following NEW packages will be INSTALLED:
cuda80: 1.0-0 soumith
pytorch: 0.1.12-py27_2cu80 soumith [cuda80]
torchvision: 0.1.8-py27_2 soumith
Proceed ([y]/n)? y
CondaError: IOError(28, 'No space left on device')
CondaError: IOError(28, 'No space left on device')
CondaError: IOError(28, 'No space left on device'
when l run the following command :
df -h
l get the following :
Filesystem Size Used Avail Use% Mounted on
udev 30G 0 30G 0% /dev
tmpfs 6.0G 8.6M 6.0G 1% /run
/dev/xvda1 7.7G 7.4G 361M 96% /
tmpfs 30G 0 30G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 30G 0 30G 0% /sys/fs/cgroup
tmpfs 6.0G 0 6.0G 0% /run/user/1000
now l added a new partition called /data which has 50 Go :
/data$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 30G 0 30G 0% /dev
tmpfs 6.0G 8.9M 6.0G 1% /run
/dev/xvda1 7.7G 6.3G 1.4G 82% /
tmpfs 30G 0 30G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 30G 0 30G 0% /sys/fs/cgroup
/dev/xvdf1 50G 3.9G 43G 9% /data
tmpfs 6.0G 0 6.0G 0% /run/user/1000
But most of installation file system such as cuda have to be installed in /
How can l transfer the 50 G0 free space from /data to / so that
/dev/xvda1 will have 58 GO ?
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 8G 0 disk
`-xvda1 202:1 0 8G 0 part /
xvdf 202:80 0 50G 0 disk
`-xvdf1 202:81 0 50G 0 part /data
Thank you a lot