Shrink a vmdk Virtualbox disk image - virtualbox

Question

VirtualBox is able to compact (reduce the size) of .vdi images but it is not possible with .vmdk disk images. But we can compact .vmdk files if we:
detach
convert to .vdi
compact
convert back to .vmdk
attach again to the original vitual machine
So I tried to shrink my VirtualBox image with this script:
#/bin/bash
VM_PATH=~/VirtualBox\ VMs
cd "$VM_PATH"
VM="$(ls ffnord-example_gc-gw0_* -d -1|head -n 1)"
cd "$VM"
VM_VDMK_NAME="$(ls *.vmdk -1|head -n 1)"
VM_NAME="$VM_PATH/$VM/$VM_VDMK_NAME"
echo reducing size of "$VM_NAME"
ls -lah "$VM_NAME"
set -x
vboxmanage showvminfo "${VM}"
vboxmanage storageattach "${VM}" --storagectl SATA --port 0 --device 0 --type hdd --medium none
vboxmanage clonehd --format vdi "${VM_NAME}" /tmp/VM-disk.vdi
vboxmanage closemedium disk "${VM_NAME}" --delete
vboxmanage modifyhd /tmp/VM-disk.vdi --compact
vboxmanage clonehd --format vmdk /tmp/VM-disk.vdi "${VM_NAME}"
vboxmanage closemedium disk /tmp/VM-disk.vdi --delete
vboxmanage storageattach "${VM}" --storagectl SATA --port 0 --device 0 --type hdd --medium 4/VMs/VM-disk1.vmdk
I adapted this script from crysol but it seems this is not working on Ubuntu? The first vboxmanage storageattach starts with an error right away:
VBoxManage: error: Could not find a controller named 'SATA'
If I try "SATA Controller" instead:
vboxmanage storageattach "${VM}" --storagectl "SATA Controller" --port 0 --device 0 --type hdd --medium none
I get this error:
VBoxManage: error: No storage device attached to device slot 0 on port 0 of controller 'SATA Controller'
VBoxManage: error: Details: code VBOX_E_OBJECT_NOT_FOUND (0x80bb0001), component SessionMachine, interface IMachine, callee nsISupports
VBoxManage: error: Context: "DetachDevice(Bstr(pszCtl).raw(), port, device)" at line 381 of file VBoxManageStorageController.cpp
If I comment out those vboxmanage storageattach lines, the script works fine, but the resulting VM is the same size as before and it doesn't boot anymore.
This is the output of vboxmanage showvminfo "${VM}"

Answer 1

I found a solution:
First inside the VM fill all free space with zeros:
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill
In your Host, install vmware-vdiskmanager from the VMware Knowledge Base:
cd /tmp/
wget http://kb.vmware.com/selfservice/viewAttachment.do?attachID=1023856-vdiskmanager-linux.7.0.1.zip&documentID=1023856
unp 1023856-vdiskmanager-linux-7.0.1.zip
mv 1023856-vmware-vdiskmanager-linux.7.0.1 /usr/bin/vmware-vdiskmanager
chmod +x /usr/bin/vmware-vdiskmanager
Take care, that you have enough free disk-space before you start, you need the MV grows to double size during the process.
Then compress it with:
/usr/bin/vmware-vdiskmanager -k ~/VirtualBox\ VMs/<virtual disk.vmdk>
Source

Answer 2

I didn't need to install VMWare nor convert back to VMDK so I used
https://scotch.io/tutorials/how-to-create-a-vagrant-base-box-from-an-existing-one#toc-make-the-box-as-small-as-possible
Inside the host:
sudo yum clean all
sudo dd if=/dev/zero of=/EMPTY bs=1M
sudo rm -f /EMPTY
cat /dev/null > ~/.bash_history && history -c && exit
Then after the guest is shut down:
$ vboxmanage clonehd --format vdi centos-7-1-1.x86_64.vmdk newdisk.vdi
$ ls -lh
-rwx------+ 1 Chloe None 39G Mar 26 14:52 centos-7-1-1.x86_64.vmdk
-rwx------+ 1 Chloe None 22G Mar 26 15:01 newdisk.vdi
It also allows compaction later
$ vboxmanage modifyhd newdisk.vdi --compact
Inside VirtualBox GUI, I selected 'Choose Virtual Hard Disk File' to select the new file.

Answer 3

I could not get rubo77s solution above vmware-vdiskmanager solution to work, I believe it has dependencies on vmware workstation or vmware player, neither of which I have, I did find the executable and it gave me errors.
I was able to solve this by using his zero command
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill
then using virtualboxes export to .ova tool.
this will result in the ova stripping/compressing the zeroed space.
then you can re-import it.

Answer 4

FYI:
as vmware modernized their knowledge base the download link is not valid anymore.
After some searching I found the zip attached with this link https://kb.vmware.com/sfc/servlet.shepherd/version/download/068f4000009EgK0AAK
Which is included in the case
https://kb.vmware.com/s/article/1023856?lang=en_US&queryTerm=1023856
Also the binary was failing with a missing library in my Ubuntu but downloading
http://archive.ubuntu.com/ubuntu/pool/universe/o/openssl098/libssl0.9.8_0.9.8o-7ubuntu3.1_i386.deb and installing it was helping.
Basically this answer should just be a comment for Shrink a vmdk Virtualbox disk image by rubo77 - but my reputation is too low so here you go.

Answer 5

I got this oooold script which allows you to wipe out all the unnecessary data, inside of Windows on your partition. It's written in VBscript:
`REM This script is published under the BSD Licence
REM http://www.opensource.org/licenses/bsd-license.php
Option Explicit
Const sDefaultDir = "C:\"
Const sDefaultFilename = "overwrite.garbage"
Const lStartBlockSize = 32768
Call Main
Sub Main()
Dim sPath, sFilename
Dim oArgs, oFS, oDrive, oRegExp, oMatches
ShowStartMsg
Set oArgs = WScript.Arguments
Set oFS = CreateObject("Scripting.FileSystemObject")
If oArgs.Count = 1 Then
If oArgs(0) = "/?" Then
ShowHelp true
End If
End If
If oArgs.Count > 2 Then
ShowMsg "ERROR: Invalid command line parameters (too many parameters specified)", true, true
End If
If oArgs.Count > 0 Then
sPath = oFS.GetAbsolutePathName(oArgs(0))
Else
sPath = ""
End If
If oFS.FolderExists(sPath) Then
WScript.Echo "Checking folder " & Chr(34) & sPath & Chr(34) & ": OK"
If Right(sPath, 1) <> "\" Then
sPath = sPath & "\"
End If
Else
WScript.Echo "Checking folder " & Chr(34) & sPath & Chr(34) & ": FAILED"
sPath = sDefaultDir
WScript.Echo "INFO: Using default folder " & Chr(34) & sPath & Chr(34)
End If
If oArgs.Count = 2 Then
sFilename = oArgs(1)
If sFilename = "" Then
ShowMsg "ERROR: Filename must not be empty", true, true
End If
Else
sFilename = sDefaultFilename
WScript.Echo "INFO: Using default filename " & Chr(34) & sFilename & Chr(34)
End If
Set oRegExp = new RegExp
oRegExp.Pattern = "[\\\/\:\*\?\" & Chr(34) & "\<\>\|]"
Set oMatches = oRegExp.Execute(sFilename)
If oMatches.Count = 0 Then
WScript.Echo "Validating filename: OK"
Else
WScript.Echo "Validating filename: FAILED"
ShowMsg "ERROR: Filename must not contain the following characters:"_
& " \ / : * ? " & Chr(34) & " < > |", true, true
End If
If oFS.FileExists(sPath & sFilename) = False Then
WScript.Echo "Ensuring that file " & Chr(34) & sFilename & Chr(34) &_
" does not exist: OK"
Else
WScript.Echo "Ensuring that file " & Chr(34) & sFilename & Chr(34) &_
" does not exist: FAILED"
ShowMsg "ERROR: File " & Chr(34) & sPath & sFilename & Chr(34) & " already exists", true, true
End If
Set oDrive = oFS.GetDrive(oFS.GetDriveName(sPath))
If UCase(oDrive.FileSystem) = "NTFS" Then
WScript.Echo "Checking for NTFS: OK"
Else
WScript.Echo "Checking for NTFS: FAILED"
ShowMsg "ERROR: " & oDrive.FileSystem & " file system not supported", true, true
End If
Select Case oDrive.DriveType
Case 1, 2
WScript.Echo "Checking drive type: OK"
Case Else
WScript.Echo "Checking drive type: FAILED"
Select Case oDrive.DriveType
Case 3
ShowMsg "ERROR: Network drives are not supported", true, true
Case 4
ShowMsg "ERROR: CD-ROM drives are not supported", true, true
Case 5
ShowMsg "ERROR: RAM Disk drives are not supported", true, true
Case Else
ShowMsg "ERROR: Unkown drives are not supported", true, true
End Select
End Select
If oDrive.FreeSpace > 0 Then
WScript.Echo "Checking for free space: OK"
Else
WScript.Echo "Checking for free space: FAILED"
WScript.Echo "INFO: No free space available (no action required)"
ShowMsg "INFO: Exiting Overwrite Script...", false, true
End If
WScript.Echo "Creating garbage file " & Chr(34) & sPath & sFilename & Chr(34) & "..."
CreateGarbageFile sPath & sFilename, oFS
WScript.Echo "Garbage file successfully created!"
WScript.Echo "INFO: " & oDrive.AvailableSpace & " byte(s) remained which could not be overwritten"
WScript.Echo "Deleting garbage file..."
oFS.DeleteFile sPath & sFilename
WScript.Echo "Garbage file successfully deleted!"
WScript.Echo "Exiting Overwrite Script..."
WScript.Quit
End Sub
Sub CreateGarbageFile(sAbsFilename, oFS)
Dim bSngByteBlock
Dim sBlock
Dim oFile, oDrive
bSngByteBlock = false
Set oDrive = oFS.GetDrive(oFS.GetDriveName(sAbsFilename))
Set oFile = oFS.CreateTextFile(sAbsFilename, false, false)
sBlock = String(lStartBlockSize, 0)
On Error Resume Next
Do While oDrive.FreeSpace > 0
If oDrive.FreeSpace < lStartBlockSize Then
If bSngByteBlock = false Then
WScript.Echo "INFO: Falling back to single byte block"
bSngByteBlock = true
sBlock = String(1, 0)
End If
End If
oFile.Write sBlock
If Err.Number <> 0 Then
WScript.Echo "WARNING: Error " & Chr(34) & Err.Description & Chr(34) & " ("_
& Err.Number & ") occured while writing garbage file"
Exit Do
End If
Loop
On Error GoTo 0
oFile.Close
End Sub
Sub ShowStartMsg()
WScript.Echo "Overwrite Script 1.0 (2004-09-05)"
WScript.Echo "Copyright (C) 2004 Dennis Dietrich"
WScript.Echo "http://www.myblog.de/scotty"
WScript.Echo ""
WScript.Echo "WARNING: The script is experimental. Use it at your own risk!"
WScript.Echo "To cancel the execution of this script press CTRL + C"
WScript.Echo ""
End Sub
Sub ShowMsg(sMsg, bShowHelpHint, bExit)
WScript.Echo sMsg
If bShowHelpHint = True Then
WScript.Echo ""
WScript.Echo "Use " & Chr(34) & "CScript Overwrite.vbs /?" & Chr(34) & " to get help."
End If
If bExit = True Then
WScript.Quit
End If
End Sub
Sub ShowHelp(bExit)
WScript.Echo "Cleans free disk space from recoverable data by overwriting it with random"
WScript.Echo "information. For a higher level of security execute this script at least three"
WScript.Echo "times. Only NTFS partitions are supported."
WScript.Echo ""
WScript.Echo "CScript Overwrite.vbs [path] [filename]"
If bExit = True Then
WScript.Quit
End IF
End Sub`
(inspired by http://www.codeproject.com/KB/vbscript/overwrite_script.aspx)