I have a question about "quiescence" snapshot.
As I understand step of quiescence" snapshot:
Freeze FS and Processes
Run pre-freeze
Make snapshot
Run post-script
Unfreeze FS and Processes
Is it right? For example if I stop mysql in pre-freeze , is it not necessary to run it in post-script?
You may find this KB article useful:
https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1015180
In general, the goal here is to get the file system into a consistent state. That is to say, if you restore from this snapshot, you want to make sure that your application can re-start from this point. For many SQL based databases, this implies that transactions should be committed prior to snapshot. The actual details vary depending on the system you are using.
All of this is managed by guest tools as only the guest can actually quiesce the file system.
Related
I get disconnect every now and then when running a piece of code in Jupyter Notebooks on Sagemaker. I usually just restart my notebook and run all the cells again. However, I want to know if there is a way to reconnect to my instance without having to lose my progress. At the minute, it shows that there is "No Kernel" at the bottom bar, but my file seems active in the kernel sessions tab. Can I recover my notebook's variables and contents? Also, is there a way to prevent future kernel disconnections?
Note that I reverted back to tornado = 5.1.1, which seems to decrease the number of disconnections, but it still happens every now and then.
Often, disconnections will be caused by inactivity because a job is running for a long time with no user input. If it's pre-processing that's taking a long time, you could increase the instance size of the processing job so that it executes faster, or increase the instance count. If you're using EMR, you can now run an EMR Spark query directly on the EMR cluster since December 2021:
https://aws.amazon.com/about-aws/whats-new/2021/12/amazon-sagemaker-studio-data-notebook-integration-emr/
There's a useful blog here https://aws.amazon.com/blogs/machine-learning/build-amazon-sagemaker-notebooks-backed-by-spark-in-amazon-emr/ which is helpful in getting you up and running.
Please let me know if you need more information, or vote for the answer if it's useful. :-)
For me a quick solution was to open a Terminal instead, save the notebook file as a Pytohn file, and run it from the terminal within Sagemaker.
Local SSDs are the fastest storage available in Google Cloud Platform, which makes it clear why people would want to use it. But it comes with some severe drawbacks, most notably that all data on the SSDs will be lost if the VM is shut down manually.
It also seems to be impossible to take and image of a local SSD and restore it to a restarted disk.
What then is the recommended way to back-up local SSD data if I have to shut down the VM (if I want to change the machine specs for example)?
So far I can only thing of manually copying my files to a bucket or a persistent disk.
If you have a solution to this, would it work if multiple local SSDs were mounted into a single logical volume?
I'm guessing that you want to create a backup of data stored on local SSD every time you shut down the machine (or reboot).
To achieve this (as #John Hanley commented) you have to copy the data either manually or by some script to other storage (persistend disk, bucket etc).
If you're running Linux:
Here's a great answer how to run script at reboot / shutdown. You can then create a script that will copy all the data to some more persistend storage solution.
If I were you I'd just use rsync and cron. Run it every hour or day (depends on your use case). Here's a another great example how to use rsync to synchronize folders.
If you're running Windows:
It is also possible to run command at windows shutdown and here's how.
I am trying to run airflow in google cloud run.
Getting error Disk I/O error, I guess the disk write permission is missing.
can someone please help me with this how to give write permission inside cloud run.
I also have to write file and later delete it.
Only the directory /tmp is writable in Cloud Run. So, change the default write location to write into this directory.
However, you have to be aware of 2 things:
Cloud Run is stateless, that means when a new instance is created, the container start from scratch, with an empty /tmp directory
/tmp directory is an in-memory file system. The maximum allowed memory on Cloud Run is 2Gb, your app memory footprint included. In addition of your file and Airflow, not sure that you will have a lot of space.
A final remark. Cloud Run is active only when it process request, and a request has a maximum timeout of 15 minutes. When no request, the allowed cpu is close to 0%. I'm not sure of what you want to achieve with Airflow on Cloud Run, but my feeling tells me that your design is strange. And I prefer to warn you before you spend too much effort on this.
EDIT 1:
Cloud Run service has evolved in the right way. In 2022,
/tmp is no longer the only writable directory (you can write everywhere, but it's still in memory)
the timeout is no longer limited to 15 minutes, but to 60 minutes
The 2nd gen runtime execution (still in preview) allows you to mount NFS (Filestore) or Cloud Storage (GCSFuse) volume to have services "more stateful".
You can also execute jobs now. So, a lot of very great evolution!
My impression is that you have a write i/o error because you are using SQLite. Is that possible.
If you want to run Airflow using cointainers, I would recommend to use Postgres or MySQL as backend databases.
You can also mount the plugins and dag folder in some external volume.
We have a load balanced setup in AWS with two instances. We do pretty frequent code updates, utilizing SVN. I need to know how easy it is to update the code changes across all the instances in our cluster. Can we simply do 'snapshots' and create new volumes each time for the instances?...or?...
I would not do updates via EBS snapshots. Think of EBS volumes as a hard disk - you would not change your harddisk if you have an update for your software.
As you have your code in a version control system, code updates should be quite simple like logging in to your (multiple) servers and doing a git pull or svn update. This should fetch the latest code files from your servers. Depending on the type of application you would have to do some other tasks afterwards, running build scripts, emptying cache etc.
The problem is that this kind of setup does not scale well. If you have n servers, you will have to login and do this command n times. Therefore it makes sense to look into some remote management tools that you can use in one step. With a lot of these tools, you also get a complete configuration management stack: you define a set of recipes or tasks (like installed packages, configuration files, fetch the latest code, necessary build steps) for each of your servers, and when you boot up a new server it fetches the lastest version of its configuration and installs itself.
Popular configuration management tools include Puppet or Salt. Both tools have remote execution included which should make your task to publish your code base easier, you would only have to fire one command on your master server and it automatically executes this task on all its minions / slave servers.
Can anyone clarify an issue? I'm using the VSS API (C++ using VSS2008 and the latest SDK running on XP SP3) in a home-brew backup utility*.
THe VSS snapshot operations work fine for folder that have no subfolders - i.e. my email and SQL server volumes. However when I take a snapshot of a folder that does contain subfolders, the nested structure is 'flattened' in the snapshot - all sub-directories cease to exist.
So here is the question: I am aware that support for VSS on XP is a bit limited but is there a way to specify a snapshot be non-recursive? The docs are not very helpful ...
I got really tired of buggy rubbish that costs boatloads and fails every few days so I thought I'd roll my own. It'll get onto CodeProject at some point. If anyone is interested let me know and you can have a (source) copy when it's ready ...
Thx++
Jerry
Your question is confusing...
VSS does not work at a "folder" level. It works a "volume" level.
You "snap" a volume and you will have a device path which you can "open" using the filesystem api (which will automatically mount the device volume with a filesystem) on a file by file or you can access the device directly (sector by sector).
It should be easy to backup all files on the snapped device volume (don't forget all of the file streams and ACL's for NTFS files), your problem will be restoring them... VSS will not help you on the restore. The main problem will be restoring a system volume, where you will need another OS to boot to like WinPE or DOS or something else. If your not worried about system volumes then restore can be easy.
If you backup the data in terms of sectors, then you get the added benefit that if you write a volume device driver for it (to look like a volume or HD) then windows will auto-load a filesystem driver for it. This gives you a free explorer application, this is what most sector based backup applications do. Also it gives them VM possibilities.
Even if you are doing simple file backups, it helps to understand filesystems (NTFS, FAT, etc) so that you know what you can/should backup and restore. Do you know what a NTFS reparse point is? How are you going to deal with it if you hit one during your backup? Do you know how windows actually boots and what files you need to backup and restore and "patch" to be able to have a chance at booting. On a restore, how best do you lay out the NTFS volume as not to affect NTFS performance on the restored volume? Are you going to support restoring system volumes to new hardware, what does that require you to do just to have a chance of working? The questions are endless.
System backup/restore is not easy, there are lots of edge cases (see some of the questions above) that you don't know about until you hit them.
Good luck on you project, I hope I haven't put you off too much, I'm just saying there is a lot of work to be able to deliver a backup application that most people have have no idea about.
Comment on the above - if a 'writer' is playing the VSS game then it will ensure that the file system is in a happy state as part of the VSS setup.
In the case of MS SQL Server - check that it is a VSS writer. If it is then your snapshot of the DB files should be OK. If not, then its in what is called a 'crash state'. So for example if you are using MySQL or some other non-MS, non-VSS aware SQL database - your backup may or may not be coherent ('a good one'). In that case it may be better than nothing, but it it may also still be useless. Using VSS MAY result in a better integrity from which to make your backup, but of the files are open, they are open and if the app does not play in the VSS pig-pen then you may or may not be hosed.