I am trying to create a stand alone companion to a customized cloud foundry deployment that has some additional services enabled in it, in the same way that micro cloud foundry is a companion to cloudfoundry.com. I've blogged a longer description of my work to date for context but the short question is this:
Is there micro-cf-release available which can be extended and used to create a customized micro cloud foundry? With the release train happening now, this must be somewhere, along with a process and tooling for creating the VM. Is this in the opensource somewhere?
The capistrano script that builds the releases is:
https://github.com/cloudfoundry/micro/blob/master/build/build.cap
This workflow is experimental, but it should be possible to use a subset of the build task in the script and customize cf-release before building from it.
Related
I'm presently looking into GCP's Deployment Manager to deploy new projects, VMs and Cloud Storage buckets.
We need a web front end that authenticated users can connect to in order to deploy the required infrastructure, though I'm not sure what Dev Ops tools are recommended to work with this system. We have an instance of Jenkins and Octopus Deploy, though I see on Google's Configuration Management page (https://cloud.google.com/solutions/configuration-management) they suggest other tools like Ansible, Chef, Puppet and Saltstack.
I'm supposing that through one of these I can update something simple like a name variable in the config.yaml file and deploy a project.
Could I also ensure a chosen name for a project, VM or Cloud Storage bucket fits with a specific naming convention with one of these systems?
Which system do others use and why?
I use Deployment Manager, as all 3rd party tools are reliant upon the presence of GCP APIs, as well as trusting that those APIs are in line with the actual functionality of the underlying GCP tech.
GCP is decidedly behind the curve on API development, which means that even if you wanted to use TF or whatever, at some point you're going to be stuck inside the SDK, anyway. So that's why I went with Deployment Manager, as much as I wanted to have my whole infra/app deployment use other tools that I was more comfortable with.
To specifically answer your question about validating naming schema, what you would probably want to do is write a wrapper script that uses the gcloud deployment-manager subcommand. Do your validation in the wrapper script, then run the gcloud deployment-manager stuff.
Word of warning about Deployment Manager: it makes troubleshooting very difficult. Very often it will obscure the error that can help you actually establish the root cause of a problem. I can't tell you how many times somebody in my office has shouted "UGGH! Shut UP with your Error 400!" I hope that Google takes note from my pointed survey feedback and refactors DM to pass the original error through.
Anyway, hope this helps. GCP has come a long way, but they've still got work to do.
I have a project in Google cloud using the following resources
-BigQuery, Google functions (Python), google storage, Cloud Scheduler
is it possible to save the whole project as code and share it, so someone else can just use that code and deploy it using his own tenant ?
the reason, I am asking, I have published all the code and SQL queries in Github, but some users find it very hard to reproduce, they are not necessarily very familiar with Google Cloud, in an ideal situation, they need just to get a file and click deploy ?
When you create a solution for GCP we will commonly find that it consists of code, data and configuration. The code and data you can save in a source repository like GitHub ... but what of the configuration? What if your "solution" expects to have BQ datasets and tables or GCS buckets or Scheduler jobs defined? This is where you can create "Infrastructure As Code" definitions. Google supports its own IaC technology called Deployment Manager but you can also use the popular Terraform as it too has a GCP provider. The definitions for these IaC coordinators are typically text / yaml files that you can also package with your code. Sprinkle in some Make, Chef, Puppet for building apps and pushing code to deployment environments and you have a "build it from source" story. Study also the concepts of CI/CD and you will commonly find that the steps you perform for building CI/CD overlap with the steps for trivial deployment.
There are also projects such as terraformer that can do some kind of a job of reverse engineering an existing configuration to create IaC description that, when run elsewhere, will recreate the configuration.
Is it possible to setup auto-scaling capabilities for an app depending on the workload?
I haven't found anything useful neither in the Developer Console nor in the docs. Is there may be a hidden possibility via the CLI?
Just wondering if this is possible as I'm doing a basic evaluation on Swisscom Application Cloud.
There are several opensource autoscaling projects of various readiness for production use like
https://github.com/cloudfoundry-incubator/app-autoscaler
https://github.com/cloudfoundry-samples/cf-autoscaler
Pivotal Cloud Foundry supports auto-scaling of the applications out of the box (http://docs.pivotal.io/pivotalcf/1-8/appsman-services/autoscaler/autoscale-configuration.html)
This capability is not present at the moment, and it is not part of the (open source) cloudfoundry platform either. Some platforms provide it, but this has not been released to the community yet!
There are various ways how you can do that.
As described by Anatoly, you can obvisouly use the "Auto Scaler" Service, if this is deployed from your respective Provider.
(you can figure that out by just calling this Feature-Flags-API check: https://apidocs.cloudfoundry.org/253/feature_flags/get_the_app_scaling_feature_flag.html)
An other option is actually writing your own small auto-scaler based on the custom-defined scaling-behaviours you've to meet your application. (DIY ;))
Get Load
: First you need to get information about your current "load" of the app (i.e. memory usage, cpu usage etc). You can easily do that by pulling data from the v2/apps//stats API. See details here:
https://apidocs.cloudfoundry.org/253/apps/get_detailed_stats_for_a_started_app.html
Write some magic :
Now you need to write some magic around to verify if the app is under heavy load. Could be CPU or Memory or other bottle necks you try to get our of the stats API.
Scale up/down :
With the PUT v2/apps// API you can easily now change the amount of instances of your app by filling the paramter "instances" accordingly.
https://apidocs.cloudfoundry.org/253/apps/updating_an_app.html
For PCF you can take a look at this https://github.com/Pivotal-Field-Engineering/autoscaling-cli-plugin. It should give you what you are looking for.
You will need to install it via the
cf install-plugin https://github.com/pivotal-cf-experimental/autoscaling-cli-plugin
and configure using steps similar to below
Get the details of the autoscaler from your marketplace
cf m | grep app-autoscaler
Install the auto scaler plugin using service & plan from above
cf create-service <service> <plan> myAutoScaler
Bind the service to your app (or u can do this via you deployment manifest)
cf bind-service myApp myAutoScaler
Configure your scaling parameters
cf configure-autoscaling --min-threshold ## --max-threshold ## --max-instances # --min-instances # myApp myAutoScaler
I want to add new system service to micro cloud. and following the steps specified in docuement "How to add a system service to Cloud Foundry step by step" for adding echo service.
But i don't see the specified folder structure in my system where i have my micro cloud.
Thanks
Saidesh
The docs are in the source tree on CloudFoundry.org. For doing development work, that's where the best information is. Here's the doc that I used: https://github.com/cloudfoundry/oss-docs/tree/master/vcap/adding_a_system_service
One other thought tho: If you're wanting to add a "service", then I'd suggest not using Micro Cloud Foundry, but instead setting up a Ubuntu virtual machine and installing the code base from CloudFoundry.org. Instructions for doing so can be found here: https://github.com/cloudfoundry/oss-docs/tree/master/vcap/single_and_multi_node_deployments_with_dev_setup
Hope that helps,
John
Let's say that I setup my own cloud using the open source cloud foundry implementation provided on cloudfoundry.org. Will each app that I deploy be run as a separate user? Or is there any of VMWare's virtualization technology in use here? E.g. would each app run in a separate virtual machine or anything like that? How can I configure the memory, cpu, and disk resource limits for each app?
I asked this on the mailing list. Here's the response I got:
If your DEA is configured to run in secure mode, then each app runs as its own user and process isolation is used to protect them. We are moving toward a model of using linux cgroups http://en.wikipedia.org/wiki/Cgroups when on linux, using the warden cgroup wrappers that are already in our source tree.
VM based isolation for a single app is pretty heavy weight, but we have long term plans to provide this for apps that need/desire it. (As opposed to the warden/cgroup work which is a near term project)
Since this is related to the open source for cloud foundry, you can try asking your question on https://groups.google.com/a/cloudfoundry.org/group/vcap-dev
You should get a quick response there!