When PCF is installed on IAAS , does it get installed on multiple
VMs which are denoted as cells by PCF?
Will each cell contain a garden implementation?
Will all cells all the different implementations of garden (windows , linux and docker)?
Can a single cell have both windows and linux based apps running?
Some of these questions are not entirely clear, but I'll try my best to answer them.
When PCF is installed on IAAS , does it get installed on multiple VMs which are denoted as cells by PCF?
Yes, Cloud Foundry comprises multiple VMs. The VMs are deployed and managed by Bosh (or Ops Manager & Bosh, if you're using Pivotal Cloud Foundry).
This is not an exhaustive list, but you'll see VMs for jobs like the Cloud Controller, UAA, Doppler, Traffic Controller and, of course, your Diego Cells.
The Diego Cells are where your applications run though, so you will typically have more Cells than any other VM type.
Will each cell contain a garden implementation?
Yes. Garden Linux (called Guardian) on your Linux Cells & Garden Windows for Windows Cells.
Will all cells all the different implementations of garden (windows , linux and docker)?
No. Linux Cells run Linux based apps (most of the build packs and Docker) and Windows Cells run Windows apps (HWC build pack).
Can a single cell have both windows and linux based apps running?
No, unless you want to count the fact that you can run .NET Core apps on Linux. That's a little different though.
If you want to deploy both Linux & Windows apps, you'll need to have at least two Cells. One for Linux & one for Windows.
Hope that helps!
#punter-vicky - Initially, if you run cf stacks you will see output like:
name description
cflinuxfs2 Cloud Foundry Linux-based filesystem
The Using PCF Runtime for Windows portion of Pivotal's documentation gives a complete overview of how to install and use Windows cells.
Once you have both types of cells available, the first priority Diego considers in granting a winning auction bid is whether the cell offers the correct stack for the application being bid on.
Related
I am very new to Cloud Foundry/Bosh and have a set of basic questions.
1) Droplet vs Garden container: I understand that droplet contains source code + build pack and these droplets are executed in garden container. IMHO, containers are good to transport to other system. Why there is intermediate notion of droplets? Should container by them-self not create the droplets?
2) Diego cell: What is the role of Diego cell (I assume that its job is to only start/stop garden containers) ? Are Diego cell platform dependent (eg. particular cell can run only windows-garden or other can run linux-garden container) ? Do we need one cell per container?
3) In the description of Diego cell, I read "Each application VM has a Diego Cell that executes application start and stop actions locally, manages the VM’s containers, and reports app status and other data to the BBS and Loggregator."
What is application VM mentioned here? Does it mean container?
4) Lets assume, I use Bosh to create my cloud foundry instance. After some time, I need to scale my system to two VMs (due to increase in load). Do I need to create a new Manifest for second VM (As the earlier manifest will also deploy the entire CF on this VM also)?
A container is, roughly speaking, a root file system image together with some things like resource limits and metadata about what volumes to mount, what processes to run, etc.
Garden is an API for creating and running container specifications. Anyone can write a server that implements the Garden API, the core Cloud Foundry teams maintain garden-linux, garden-runC, and garden-windows implementations.
A droplet is a "built" artifact created from source code that is typically mounted or streamed into a Garden container and then run. There are times where you do not want a separate droplet, and want to have your root file system as well as all source code and/or built artifacts baked into a single image. However, often, you do want this separation between the droplet, which represents your code, and the root file system. One major benefit is that CVEs present in lower level dependencies that are common to most containers can be uniformly repaired across all tenants and all running applications on the Cloud Foundry platform without any developers having to re-push their code. E.g. if there is a new patch required for something like openssl, and your Cloud Foundry installation has thousands of developers and hundreds of thousands of running Garden containers, it would be much better if an operator could roll out the openssl patch to all containers with a single command.
The Diego cell is a VM that's part of the Cloud Foundry architecture. Cloud Foundry itself is a distributed system with different components responsible for different things. There is a component responsible for user authorization and authentication, there are components for aggregating logs from applications, there is a component responsible for providing the developer-facing API for creating, scaling, and managing applications, etc. The Diego cell's are responsible for essentially taking requests to run containerized workloads, and running them. User requests to run an application are consumed by the user-facing API, and translated to a request to the Diego backend. Diego itself has several components, including a scheduler, and the scheduler's job is to select which cell to do a given piece of work.
You can think of the cell has having two components: (1) a Garden server for running containers, and (2) a representative that can represent that Garden server to the Diego scheduler, so rather than Garden having any Diego-specific knowledge (Garden can function in a standalone manner), the scheduler instead talks to each Garden's Diego representative on that same cell.
I'm not sure what "application VM" means in the quote you pulled out. Each application running on Cloud Foundry can be run with multiple parallel instances (for fault tolerance, better concurrency, etc.). Each application instance is running as a Garden container in some Diego cell. A production deployment of Cloud Foundry will have many Diego cells. Each cell can run many (up to hundreds) of Garden containers. For better fault tolerance, the Diego scheduler will attempt to place the instances of a given application on different Diego cells, rather than cramming them all into the same cell, since if that single cell goes down, your whole application goes down.
You do not need to create a new manifest to scale up BOSH deployments. Just change the instances value of whatever job/instance group you want to have more VMs of, and re-run bosh deploy.
1) The droplet is a container image that is stored persistently when you upload your application with a cf push. This image will be re-used whenever the system creates a new container with the same image (for example, when you restart your application, or scale to multiple instances). Without the droplet, you would have to re-upload your application and create the container image every time you wanted a new instance.
2) The Diego cell runs in a large VM that hosts many containers. A typical Diego cell might be 32G in size, while a typical container might be 1G. Diego cells currently only run Linux-Garden containers.
3) The application VM is just the VM that is hosting the Diego cell. I found the sentence a bit confusing because I tend to use the term Diego cell to refer both to the cell software and to the "application VM" which is hosting it.
4) Bosh will use multiple VMs to deploy Cloud Foundry. Single-VM deployments do exist (see for example, http://pivotal.io/pcf-dev) but they are not deployed with Bosh.
Just trying to understand what's the difference between bitnami apps and google 'click-to-deploy' options on Google Cloud Engine?
For example: There is a 'Cassandra' click-to-deploy and there is a Bitnami version of 'Cassandra'
Can anyone tell me how do they compare and what are the differences?
- is one restrictive compared to the other?
- does bitnami version lock you in somehow?
- is there any performance difference (other than obvious performance difference that the hardware change would bring)
Thanks.
Bitnami makes application stacks that run on several cloud platforms including Google Cloud Platform, AWS, Azure and a few others. The Bitnami images you see on Google Cloud Launcher are created by employees of Bitnami and are mostly standard across cloud.
Click to Deploy images are usually created by Google Cloud Platform employees working in conjunction with application vendors.
There are differences in versions here and there related to maintenance, but there isn't any difference in the way they are intended to be used. Some Click to Deploy images will incur higher use charges due to licensing (ie. the Click to Deploy image contains the "Pro" version of a vendor's software), but these are called out during the selection process.
Neither version is intended to lock you into a particular platform, Google or Bitnami, it's just that there is duplication among the applications provided.
We have the need to perform tests on localized platforms that put some burden on our hardware resources because for just a few weeks we might need plenty of servers and clients (Windows 2003 and Windows 2008, Vista, XP, Red Hat, etc) in multiple languages.
We typically have relied on blades with Windows 2003 and VMWare, but sometimes these are overgrown by punctual needs and also have the issue that the acquisition and deployment process is quite slow if the environment needs to grow.
Is Amazon EC2/S3 usable in the following scenario?
Install VMWare (Desktop because we need the ability to have snapshots) on an Amazon AMI.
Load existing VMWare images from S3 and run them on EC2 instances (perhaps 3 or 4 server or client OSes on each EC2 instance.
We are more interested in the ability to very easily start or stop VMware snaphsots for relatively short tests. This is just for testing configurations, not a production environment to actually serve a user workload. The only real user is the tester. These configurations might be required for just a few weeks and then turned off for a few months until the next release requires them again.
Is EC2/S3 a viable alternative for this type of testing purpose?
Do you actually need VMWare, or are you testing software that runs in the VMWare VMs? You might actually need VMWare if you are testing e.g. VMWare deployment policy, or are running code that tests the VMWare APIs. Examples of the latter might be you are testing an application server stack and currently using VMWare to test on many platforms.
If you actually need VMWare, I do not believe that you can install VMWare in EC2. Someone will correct & enlighten me if this is not the case.
If you don't actually need VMWare, you have more options. If you can use one of the zillion public AMIs as a baseline, clone the appropriate AMIs and customize them to suit your needs (save the customized version as a private AMI for your team). Then, you can use as many of them as you like. Perhaps you already have a bunch of VMWare images that you need to use in your testing. In that case, you can migrate your VMWare image to an EC2 AMI as described in various places in Google, for example:
http://thewebfellas.com/blog/2008/9/1/creating-an-new-ec2-ami-from-within-vmware-or-from-vmdk-files
(Apologies to the SO censors for not pasting the entire article here. It's pretty long.) But that's a shortcut; you can always use the documented AMI creation process to convert any machine (VMWare or not) to an AMI. Perform that process for each VMWare VM you have, and you'll be all set. Just keep in mind that when you create an AMI, you have to upload it to S3, and that will take a lot of time for large VMs.
This is a bit of a shameless plug, but we have a new startup that may deal with exactly your problem. Amazon EC2 is excellent for on-demand computing, but is really targeted at just a single user launching production servers. We've extended EC2 to make it a Virtual Lab Management environment, with self-service, policies and VM sharing. You can check it out at http://LabSlice.com and see if it meets your needs.
Amazon provides a solution themselves now: http://aws.typepad.com/aws/2010/12/amazon-vm-import-bring-your-vmware-images-to-the-cloud.html
We recently bought a new rack and set of servers for it, we want to be able to redeploy these boxes as build servers, QA regression test servers, lab re-correlation servers, simulation servers, etc.
We have played a bit with VMWare, VirtualPC, VirtualBox etc, creating a virtual build server, but we came across a lot of issues when we tried to copy it for others to use, having to reconfigure every new copy of the VM.
We are using Windows XP x86/x64 and Windows Vista x86/x64, so I had to rename the machine, join the domain etc for every new copy.
Ideally we just want to be able to add a new box, deploy a thin boot strap OS (Linux is fine here) to get the VM up an running, then use it.
One other thing we have limited to no budget, so free is best.
I would like to understand others experiences in doing the same thing.
FYI, I am not in systems IT, this we are group of software engineers trying to set this up.
Any links to good tutorials would be great.
The problem you're running into is the machine SID must be unique for each machine in a domain. Of course by copying an image you now break that unique constraint.
I'd suggest that you read the documentation for Sysprep in the reskit and Vista System Image Manager - your friends for XP/Win2k3 and Vista/Win2k8 respectively.
These tools enable to "reseal" your configured instance of the OS such that the next time it boots - it can prompt for information such as network configuration, machine names, admin user ID's, run scripts etc.
Also be aware that the licencing restrictions for Windows desktop clients are generally per image - not per server.
Using these tools with HyperV we created complete preconfigured instances of Win2k3 & Win2k8 that boot to finish installing Sharepoint - going further we used the diffing disks to overlay Visual Studio so our devs could use the production images for their work. It has radically changed our development process.
At this point our entire public website is run on HyperV with of 5 boxes running 15 images for a mix of soft and hard redundancy - they take several hundred million page views per week.
Another option for dealing with the SID probelm is NewSID. This is a simpler tool than sysprep, in that all it does is rename the machine and reassign the SID; if you don't need all the other features of sysprep this is a much easier tool to use.
I develop exclusively on VMs. I currently run Boot Camp on a MacBook Pro and do all my development on a series of Virtual PC VMs for many different environments. This post by Andrew Connell litterally changed the way I work.
I'm thinking about switching to Fusion and running everything in OS X but I wasn't able to answer the following questions about VM Fusion/Workstation/Server. I need to know if the following features from Virtual PC/Server exist in their VMWare counter parts.
Differencing Disks (ability to create a Base VM and provision new VMs which just add deltas on top of the base [saves a ton of disk space, and makes it easy to spin up new VMs with a base set of funcitonality]). (Not available with Fusion, need Workstation [$189])
Undo disks (ability to rollback all changes to the VM within a session). (Available in both Workstation and Fusion [$189/$79.99 respectively])
Easily NAT out a different subnet for the VM to sit in. (In both Fusion/Workstation).
Share VMs between VM Player and VM Server. I'd like to build up a VM locally (on OS X/Fusion) and then move it to some server (Win2k3/Win2k8 and VM Server) and host it there but with VM Server. (In both Fusion/Workstation).
An equivalent to Hyper-V. (Both Fusion and Workstation take advantage of type-2 hypervisor a for 64x VMs, neither do for 32 bit VMs. VMWare claims they're no slower as a result some benchmarks corroborate this assertion).
Ability to Share disks between multiple VMs. If I have a bunch of databases on a virtual disk and want them to appear on more than one VM I should be able to just attach them. (Available in both Fusion and Workstation)
(Nice to have) Support for multiple processors assigned to a VM (Available in both Fusion and Workstation).
Is there a VMWare guru out there who knows for sure that the above features are available on the other side?
Also the above has been free (as long as you have licenses for Windows machines), besides buying Fusion are there any other costs?
The end result of my research, thanks so much!
You can only create Linked clones and Full Clones (which are close to differencing disks) in VMWare Workstation (not Fusion). Workstation also has at better snapshot management in addition to other features which are difficult to enumerate. That being said Workstation is $189 (as opposed to $79) and not available on OS X. In addition Fusion 1.1 (current release) has a bunch of display bugs on OS X 10.5 (works well on 10.4). These will be remedied in Fusion 2.0 which is currently in (RC1). I'll probably wait until v2.0 comes out and then use both Workstation/Fusion to provision and use these VMs on OS X.
I've not used Fusion, just workstation and server
1) Yes, you can create a linked clone from current vm state, or from a saved state (snapshot) in VMware Workstation
2) Yes, revert to snapshots
3) There's a number of different network setups, NAT's one of them
4) VMware virtual machines created with VMware Fusion are fully compatible with VMware’s latest products.
5) ?
6) You can add pre-existing to disks to other vm's
7) Yup, you create multi-cpu vm's
Workstation costs, but VMWare Server is free
It doesn't have #1, at least.
VMWare server is free, but only allows for one snapshot, a serious deficiency. VMWare Workstation allows multiple snapshots and can perform most of the same functionality.
VMWare has a Hypervisior which is equivalent to Hyper-V in Virtual PC.
You can not share a VM that was created in Fusion with Windows VMWare Server (free version) you'll need the paid version to be able to share amongst both.
I'd also take a look at Sun's xVM VirtualBox for Mac. It runs Windows XP and Vista quite swift on my Mac.
1 and 2) VirtualBox has snapshots that branch off from the base VM like a tree. You can revert to any previous snapshots and name them.
3) It has NAT support and bridged networking like the VMWare and Microsoft products.
4) There is no server version of VirtualBox, but I know it shares an engine with Qemu, so it may be possible to host your VBox images on Qemu.
5) VirtualBox does have a hypervisor if your Mac has VT-x enabled.
6) Sure, you can add existing disks to other VMs. But you can't run the same disk in multiple VMs at once. (Isn't that a restriction of all virtualization hosts, though?)
7) No. VirtualBox will give each image one CPU and spread them out.