As part of testing installation and function of an application, I deploy it to a fresh VirtualBox guest each time. In order to do this, I first installed a fresh copy of the guest OS into the VM and then made a snapshot, the intent being to revert to this snapshot before each test run.
However, there was one small change I made after that event (I always seem to miss something). This was the mounting of the host drive so that I could keep my test scripts and application data there and just run a single command on the guest to kick off and report on everything.
So I made those changes, took another snapshot, and the situation is now:
Baseline (22 days ago)
|
+- HostMounted (21 days ago)
|
+- Current state
Since I don't need the baseline any more, and my build system takes up a huge amount of space, I'd like to get rid of it. However, being the paranoid type, I wanted to confirm:
Will deleting the Baseline snapshot still leave me with an identical HostMounted? I'm basically looking to ensure that I end up with:
HostMounted (21 days ago)
|
+- Current state
I've looked at the doco and various posts on discussion groups but they seem ambiguous to me.
The HostMounted snapshot can be considered a standalone one. Deleting other snapshots will not change it in any way.
In other words, even if snapshots were stored as a simple difference between the previous state and the snapshot, deleting the previous state would roll any necessary changes into that snapshot.
This is supported by the VirtualBox documentation (my emphasis):
You can restore a snapshot by right-clicking on any snapshot you have taken in the list of snapshots. By restoring a snapshot, you go back (or forward) in time: the current state of the machine is lost, and the machine is restored to the exact state it was in when the snapshot was taken.
So you can safely delete Baseline if it's no longer needed.
Related
I get disconnect every now and then when running a piece of code in Jupyter Notebooks on Sagemaker. I usually just restart my notebook and run all the cells again. However, I want to know if there is a way to reconnect to my instance without having to lose my progress. At the minute, it shows that there is "No Kernel" at the bottom bar, but my file seems active in the kernel sessions tab. Can I recover my notebook's variables and contents? Also, is there a way to prevent future kernel disconnections?
Note that I reverted back to tornado = 5.1.1, which seems to decrease the number of disconnections, but it still happens every now and then.
Often, disconnections will be caused by inactivity because a job is running for a long time with no user input. If it's pre-processing that's taking a long time, you could increase the instance size of the processing job so that it executes faster, or increase the instance count. If you're using EMR, you can now run an EMR Spark query directly on the EMR cluster since December 2021:
https://aws.amazon.com/about-aws/whats-new/2021/12/amazon-sagemaker-studio-data-notebook-integration-emr/
There's a useful blog here https://aws.amazon.com/blogs/machine-learning/build-amazon-sagemaker-notebooks-backed-by-spark-in-amazon-emr/ which is helpful in getting you up and running.
Please let me know if you need more information, or vote for the answer if it's useful. :-)
For me a quick solution was to open a Terminal instead, save the notebook file as a Pytohn file, and run it from the terminal within Sagemaker.
I began putting together a project yesterday and decided that I'd like to use Cloud 9 as the development IDE. When I was setting up my dev environment, I selected that I wanted to create a new EC2 instance for the environment (t2.micro) and I put the cost-savings settings as 30 minutes (so that the environment will auto-hibernate after inactivity). I then proceeded to use Cloud 9 as I had in the past, which included some changes such as upgrading the version of Node.js and installing Django. Everything worked great until I went to bed. When I woke up and opened my environment again this morning, the instance was relaunched and none of the changes I made persisted, so I needed to do the updates/installations all over again.
Is there a way I can avoid this without having to turn off auto-hibernate (or is the root issue something else, and if so, how can I address it)? I don't particularly want to waste a bunch of compute time having my instance just sitting there idly, but it's really annoying having to spend a chunk of my morning re-configuring everything that I did yesterday.
Are you setting the default Node version with nvm? If you manually set a Node version with the terminal but don't set it to default, the Node version will only apply to the one terminal session (it won't even persist to a new terminal tab).
I have a question about "quiescence" snapshot.
As I understand step of quiescence" snapshot:
Freeze FS and Processes
Run pre-freeze
Make snapshot
Run post-script
Unfreeze FS and Processes
Is it right? For example if I stop mysql in pre-freeze , is it not necessary to run it in post-script?
You may find this KB article useful:
https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1015180
In general, the goal here is to get the file system into a consistent state. That is to say, if you restore from this snapshot, you want to make sure that your application can re-start from this point. For many SQL based databases, this implies that transactions should be committed prior to snapshot. The actual details vary depending on the system you are using.
All of this is managed by guest tools as only the guest can actually quiesce the file system.
Yes, I've heard all the stories about EC2 instances being unreliable and how you need to proactively prepare for that. I've also heard stories from others about how they have never had a problem, and their instances just run and run.
Today I had a strange thing happen. I've had an Linux instance running for a couple of months, as I've been preparing to launch an e-commerce site. I've been periodically taking snapshots. I have my images on S3. I have my code in a private github repo. All things considered, I've been doing a fairly good job of protecting myself against failure. Ironically, it was while I was doing even more in this regard today that I experienced something really strange.
Since I have these snapshots, I had assumed that the best thing to do if I needed to quickly spin up a new instance (whether due to a failed instance that wouldn't come back up, or if I just needed additional capacity) would be to take a snapshot and make a volume out of it, then make an image out of that volume, and then launch a new instance using that image.
For whatever reason, every time I've tried that lately, the new instance had a kernel panic during boot, so I decided to try a different approach. I right-clicked on my RUNNING INSTANCE, and chose "Create Image." That seemed like a reasonable shortcut. Then I went to that image and launched an instance.
At almost exactly the same time, my original instance rebooted. I didn't even see it happen. I only know it did from the system log. Is this just a wild coincidence? Or did I commit a silly mistake and accidentally screw up my instance?
Fortunately, I'm just getting this new thing off the ground, so the bit of downtime didn't kill me, and I was able to very quickly get things going again. But either I totally do not understand the "Create Image" feature from the instance list, or I got really unlucky today.
"Create image" takes the following actions:
Stop EC2 instance
Snapshot EBS volume
Start EC2 instance
Register EBS snapshot as an AMI
So, yes, this would look like a reboot because it is like a reboot.
Here's an article I wrote on the difference between stop/start and simple reboot: http://alestic.com/2011/09/ec2-reboot-stop-start
Your problem sounds a lot like my problem. After some searching this page helped me: http://www.raleche.com/node/138
"The problem turned out to be the kernel. Both when creating the AMI and the instance I selected default for the kernel image.
To resolve the problem, I recreated the AMI using the same kernel image as the original instance."
Can anyone clarify an issue? I'm using the VSS API (C++ using VSS2008 and the latest SDK running on XP SP3) in a home-brew backup utility*.
THe VSS snapshot operations work fine for folder that have no subfolders - i.e. my email and SQL server volumes. However when I take a snapshot of a folder that does contain subfolders, the nested structure is 'flattened' in the snapshot - all sub-directories cease to exist.
So here is the question: I am aware that support for VSS on XP is a bit limited but is there a way to specify a snapshot be non-recursive? The docs are not very helpful ...
I got really tired of buggy rubbish that costs boatloads and fails every few days so I thought I'd roll my own. It'll get onto CodeProject at some point. If anyone is interested let me know and you can have a (source) copy when it's ready ...
Thx++
Jerry
Your question is confusing...
VSS does not work at a "folder" level. It works a "volume" level.
You "snap" a volume and you will have a device path which you can "open" using the filesystem api (which will automatically mount the device volume with a filesystem) on a file by file or you can access the device directly (sector by sector).
It should be easy to backup all files on the snapped device volume (don't forget all of the file streams and ACL's for NTFS files), your problem will be restoring them... VSS will not help you on the restore. The main problem will be restoring a system volume, where you will need another OS to boot to like WinPE or DOS or something else. If your not worried about system volumes then restore can be easy.
If you backup the data in terms of sectors, then you get the added benefit that if you write a volume device driver for it (to look like a volume or HD) then windows will auto-load a filesystem driver for it. This gives you a free explorer application, this is what most sector based backup applications do. Also it gives them VM possibilities.
Even if you are doing simple file backups, it helps to understand filesystems (NTFS, FAT, etc) so that you know what you can/should backup and restore. Do you know what a NTFS reparse point is? How are you going to deal with it if you hit one during your backup? Do you know how windows actually boots and what files you need to backup and restore and "patch" to be able to have a chance at booting. On a restore, how best do you lay out the NTFS volume as not to affect NTFS performance on the restored volume? Are you going to support restoring system volumes to new hardware, what does that require you to do just to have a chance of working? The questions are endless.
System backup/restore is not easy, there are lots of edge cases (see some of the questions above) that you don't know about until you hit them.
Good luck on you project, I hope I haven't put you off too much, I'm just saying there is a lot of work to be able to deliver a backup application that most people have have no idea about.
Comment on the above - if a 'writer' is playing the VSS game then it will ensure that the file system is in a happy state as part of the VSS setup.
In the case of MS SQL Server - check that it is a VSS writer. If it is then your snapshot of the DB files should be OK. If not, then its in what is called a 'crash state'. So for example if you are using MySQL or some other non-MS, non-VSS aware SQL database - your backup may or may not be coherent ('a good one'). In that case it may be better than nothing, but it it may also still be useless. Using VSS MAY result in a better integrity from which to make your backup, but of the files are open, they are open and if the app does not play in the VSS pig-pen then you may or may not be hosed.