so I have installed some python nltk libraries (pip3 install) and c++ libraries (via apt-get install package_name_xxxx) on two different VMS instances.
Python packages for nltk would disappear and require a reinstall after reboot or change of the vms instance (e.g., add memory, cpu core),
C++ libraries disappeared without rebooting or any change of the machine. I do not find anything in the systemlog, a reinstall with apt-get works fine. But I am trying to figure out why it happens.
Is your GCE instance a preemptible instance? this option restarts the instance once every 24 hours and could be the reason why you are missing some packages.
After about an hour of inactivity, modifications not within the $HOME directory are lost. This includes installed packages.
See Custom installed software packages and persistence and usage limits.
Related
I'm using AWS Lambda, which involves creating an archive of my node.js script, including the node_modules folder and uploading that to their infrastructure to run.
This works fine, except when it comes to node modules with native bindings (using node-gyp). Because the binding was complied and project archived on my local computer (OS X), it is not compatible with AWS's (Amazon Linux) servers.
How can I cross-compile/install a node module (specifically, node-sqlite3) so when I upload it to another server arch it runs?
While not really a solution to your problem, a very easy workaround could be to simply compile the native addons on a Linux machine.
For your particular situation, I would use Vagrant. Vagrant can create virtual machines and configure them within seconds.
Find an OS image that resembles Amazon's Linux distro (Fedora, CentOS, others that use yum as package manager - see Wiki)
Use a simple configuration script that, when run by Vagrant on machine startup, will run npm install (optionally it might also remove the node_modules folder before to ensure a clean installation)
For extra comfort, the script can also create the zip file for deployment
Once the installation finishes, the script will shutdown the VM to avoid unnecessary consumption of system resources
Deploy!
It might require some tuning if the linked libraries are not at the same place on the target machine but generally this seems to me like the best and quickest solution.
While installing the app using Vagrant might be sufficient in some cases, I have found it necessary to build the app on Linux which is as close to Lambda's Amazon Linux AMI as possible.
You can read the original answer here: https://stackoverflow.com/a/34019739/303184
Steps to make it work:
Spawn new EC2 instance. Make sure it is based on exactly the same image as your AWS Lambda runtime. You can review Lambda env details here: http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html. In our case, it was Amazon Linux AMI called amzn-ami-hvm-2015.03.0.x86_64-gp2.
Install nvm and use it to install the same version of Node.js as on the AWS Lambda. At the time of writing this, it was v0.10.36. You can refer to http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html again to find out.
You will probably need to install git & g++ compiler on the EC2. You can do this running
sudo yum install git gcc-c++
Finally, clone your app to your new EC2 and install your app's dependecies:
nvm use 0.10.36
npm install --production
You can then easily download the node_modules using scp or such.
Same lines as Robert's answer, when I had to work on my MAC in a different OS I use vm ware like Oracle's free virtualizer VirtualBox to get a linux on my mac, no cost to me. Or sign up for a new AWS account, you get a micro for a year free. Use that to get your linux box, do whatever you need there.
AWS has a page describing how to deal with native NPM modules: https://aws.amazon.com/blogs/compute/nodejs-packages-in-lambda/
I have recently initialized a GPU instance on Google cloud, and installed Anaconda and installed all required dependencies before I stoped that instance. Now when I started the instance, it does not have anaconda installed in it. I found it is so weird. Please let me know if you know any details on it. I also looked into details from the doc of google, I don't find any related comments that should behave like this.
https://cloud.google.com/compute/docs/instances/stopping-or-deleting-an-instance
No, this should not happen if programs got installed properly in persistent/boot disk file system.
If programs are supposedly installed in TMPFS or other memory mapped file system then after the instance is rebooted the memory contents would be lost and consequently data and links to it.
However, this is never done as VM Instance packages are installed in persistent disk.
I guess your installation failed for some reason. Check if the packages are still installed. If you are using a Redhat Linux variant you can use ‘yum list installed’ to see all installed packages or ‘yum list installed|grep -i <package-to-search-for> to filter out a particular package.
If the package shows up, then the issue could be related to a misconfiguration or other problem somewhere. Use dmesg and/or cat /var/log/messages to view the logs and try to find any problems there which may be related to Anaconda or GPU software.
I just encountered the same problem. I know this question is dated but might help a complete beginner like myself. In my case I needed to SSH onto the instance instead of just being in the project level virtual environment.
gcloud beta compute ssh --zone "europe-west2-c" "myinstancename" --project "fired-brimstone-234534"
I would like to use Amazon Windows Instance to test the compatibility of the code which is developed on Mac. I may use the instance once a month for few hours for testing purposes that's it. I don't want my instance running the whole month and costing me.
Hence I am planning to create an instance whenever I want. To test the code I have to install g++, openGL, boost libraries etc. How can do this using scripts?
I am looking for some scripts which install packages on a freshly installed Windows instance, clone the files from git, build and test the software.
For example for AWS Ubuntu instance I can do the following,
./> ssh -i KeyPairForUbuntu.pem ubuntu#instance.eu-central-1.compute.amazonaws.com
./> sudo apt install <packages>
./> git clone pathToTheGit
./> run scons scripts to build and run the project
Is there any similar way to do this for Windows?
Please give any suggestions,
Thanks
try using https://chocolatey.org/
its a package management like apt-get for windows
Can I use apt-get or other package managers in Cloud Foundry buildpacks or .profile scripts that come with apps; and if I can, how to do it? I expect to do it the same way as in a dockerfile, but it doesn't work with or without sudo in my case.
Can I use apt-get or other package managers in Cloud Foundry buildpacks or .profile scripts that come with apps; and if I can, how to do it?
No. Running apt-get or a package manager would typically require root access and you do not get root access when the build pack runs or when your application runs (this is a difference w/Docker).
That said, you can do anything that doesn't require root access, so if you found a package manager that installed in the vcap user's home directory and didn't need root then you could use that.
It depends on what you're trying to install, but in some cases you can work around this by downloading the .deb or .rpm file and manually extracting the binaries. This typically works OK for things like shared libraries. Just download the precompiled binary that matches your stack (cflinuxfs2 == Ubuntu Trusty). For other things, you can build your own binaries from source. This is what the build pack's do, see binary-builder.
Hope that helps!
I'm following the old tutorials off of gettingstartedwithdjango.com.
This series is quite old and I'm new to Django which is why I hit that site in the first place. It became my first introduction to Vagrant. Because the series is old and there are now new versions of Django, Vagrant, etc, I've found just getting through the first tutorial was quite difficult. This was mainly just the gap between Django v1.4 and the current version 1.9 which is what I'm running, including some syntax differences in settings.py and also some discrepancies between the text errata and the video which I had to sort through. It's a pretty detailed exercise if you're completely new to all of this (which I am), so it was quite challenging, and I was thrilled when after probably 12 hours of solid effort I was able to get everything working as was being described in the video but using all new versions of software. Once I got it all set up and working, I halted my Vagrant VM for the night and when I turned it back up (vagrant up) the next morning, I found that the VM would no longer mount its shared folders, essentially rendering the Vagrant VM useless to me as I'm then unable to run code which resides on the host machine (I'm running Windows 7) from within the VM (which is accomplished via the VirtualBox shared folders feature).
Not knowing what was wrong, this prompted me to completely reinstall my Vagrant VM. I was able to get things redeployed successfully with about one hour worth of effort, backtracking through steps I had taken to successfully complete the first tutorial in the first place, in order to back to the same point where I started (before I did the previous vagrant halt). When reinstalling the Vagrant VM I noticed messages that my VirtualBox Guest Additions (4.2.0) did not match the version of VirtualBox I have installed (5.0.10), which I recalled seeing the first time but ignored because it also said this isn't usually a problem and should work (if it weren't for bad luck, I'd have no luck at all). Since for me it didn't work, this led me down a whole rabbit hole of posts from various websites including SO, which ultimately had me updating my Vagrant VM, downloading/mounting/building/installing a new version of VirtualBox Guest Additions, and reloading my Vagrant VM only to wind up in the same boat. Shared folders were still not working!
To be very specific, this is more or less what I tried based on information from many websites:
cd /home/vagrant
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install dkms build-essential linux-headers-generic
sudo apt-get install linux-headers-3.2.0-23-generic
# mounted VBoxGuestAdditions_5.0.10.iso to Vagrant VM
cd /media/cdrom
sudo sh ./VBoxLinuxAdditions.run
sudo reboot
sudo /etc/init.d/vboxadd setup
sudo reload
c:\VAGRANT\vagrant plugin install vagrant-vbguest
c:\VAGRANT\vagrant reload
I was utterly convinced this was going to resolve the issue but it didn't.
Then I found this gem:
http://ddelizia.blogspot.com/2011/02/how-to-share-folder-on-virtual-box-with.html
This shows how to, from within the Vagrant VM, mount the shared folders back to your Windows 7 host. To be specific, here's what I did when I found things were working again:
cd /vagrant
ls -la
<this yielded nothing>
sudo mount.vboxsf vagrant /vagrant # see your VirtualBox Shared Folders settings
cd ~ # /home/vagrant
cd - # /vagrant
ls -la
# this yielded the expected folders from my Win7 host
In the mount.vboxsf command above, the first vagrant (without the /) came from the Name column in my VirtualBox shared folders settings. This is essentially an alias which is used to refer to the actual path on the Win7 host, in my case: C:/VAGRANT. The second vagrant (with the /) in that command is the /vagrant folder on the Vagrant VM (linux).
Given that I spent most of today messing with this and I figure there are plenty of people who are going to run into this same or related problem, I thought I would try to help out and save you all a bunch of time. Good luck.