Let's say that I setup my own cloud using the open source cloud foundry implementation provided on cloudfoundry.org. Will each app that I deploy be run as a separate user? Or is there any of VMWare's virtualization technology in use here? E.g. would each app run in a separate virtual machine or anything like that? How can I configure the memory, cpu, and disk resource limits for each app?
I asked this on the mailing list. Here's the response I got:
If your DEA is configured to run in secure mode, then each app runs as its own user and process isolation is used to protect them. We are moving toward a model of using linux cgroups http://en.wikipedia.org/wiki/Cgroups when on linux, using the warden cgroup wrappers that are already in our source tree.
VM based isolation for a single app is pretty heavy weight, but we have long term plans to provide this for apps that need/desire it. (As opposed to the warden/cgroup work which is a near term project)
Since this is related to the open source for cloud foundry, you can try asking your question on https://groups.google.com/a/cloudfoundry.org/group/vcap-dev
You should get a quick response there!
Related
I don't know much about web development and cloud computing. From what I've read when using Cloud functions as the webhook service for dialogflow, you are limited to write code in just 1 source file. I would like to create a real complex dialogflow agent, so It would be handy to have an organized code structure to make the development easier.
I've recently discovered Cloud run which seems like it can also handle webhook requests and makes it possible to develop a complex code structure.
I don't want to use Cloud Run just because it is inconvenient to write everything in one file, but on the other hand it would be strange to have a cloud function with a single file with thousands of lines of code.
Is it possible to have multiple files in a single cloud function?
Is cloud run suitable for my problem? (create a complex dialogflow agent)
Is it possible to have multiple files in a single cloud function?
Yes. When you deploy to Google Cloud Functions you create a bundle with all your source files or have it pull from a source repository.
But Dialogflow only allows index.js and package.json in the Built-In Editor
For simplicity, the built-in code editor only allows you to edit those two files. But the built-in editor is mostly just meant for basic testing. If you're doing serious coding, you probably already have an environment you prefer to use to code and deploy that code.
Is Cloud Run suitable?
Certainly. The biggest thing Cloud Run will get you is complete control over your runtime environment, since you're specifying the details of that environment in addition to the code.
The biggest downside, however, is that you also have to determine details of that environment. Cloud Funcitons provide an HTTPS server without you having to worry about those details, as long as the rest of the environment is suitable.
What other options do I have?
Anywhere you want! Dialogflow only requires that your webhook
Be at a public address (ie - one that Google can resolve and reach)
Runs an HTTPS server at that address with a non-self-signed certificate
During testing, it is common to run it on your own machine via a tunnel such as ngrok, but this isn't a good idea in production. If you're already familiar with running an HTTPS server in another environment, and you wish to continue using that environment, you should be fine.
I have a software that process some files. What I need is:
start a default image on google cloud (I think docker should be a good solution) using an API or a run command
download files from google storage
process it, run my software using those downloaded files
upload the result to google storage
shut the image down, expecting not to be billed anymore
What I do know is how to create my image hehe. But I can't find any info saying me what google cloud service should I use or even if I could do it like I'm thinking. I think I'm not using the right keywords to find what i need.
I was looking at Kubernetes, but i couldn't figure out how to manipulate those instances to execute a one time processing.
[EDIT]
Explaining better the process I have an app that receive images and send it to Google storage. After that, I need to process that images, apply filters, georeferencing, split image etc. So I want to start a docker image to process it and upload the results to google cloud again.
If you are using any of the runtimes supported by Google Cloud Functions, they are easiest way to do those kind of operations (i.e. fetch something from Google Cloud Storage, perform some actions on those files and upload them again). The Cloud Functions will be triggered by an event of your choice, and after the job, it will die.
Next option in terms of complexity would be to deploy a Google App Engine application in standard environment. It allows you to deploy your own application written in any of the supported languages for this environment. While there is traffic in your application, you will have instances serving, but the number of instances running can go down to 0 when they are not serving, which would mean less cost.
Another option would be Google App Engine in flexible environment. This product allows you to deploy your application in any custom runtime. This option has always at least one instance running, so it would never shut down.
Lastly, you can use Google Compute Engine to "create and run virtual machines on Google infrastructure". Otherwise than GAE, this is not that managed by Google, which means that most of the configuration is up to you. In this case, you would need to programmatically indicate your VM to shut down after you have finished your operations.
Based on your edit where you stated that you already have an app that is inserting images into Google Cloud Storage, your easiest option would be to use Cloud Functions that are triggered by additions, changes, or deletions to objects in Cloud Storage buckets.
You can follow the Cloud Functions tutorial for Cloud Storage to get an idea of the generic process and then implement your own code that handles your specific tasks. There are other tutorials like the Imagemagick tutorial for Cloud Functions that might also be relevant to the type of processing you intend to do.
Cloud Functions is probably your lightest weight approach. You could of course do more full scale applications, but that is likely overkill, more expensive, and more complex. You can write your processing code in Node.js, Python, or Go.
I have deployed my app (PHP Buildpack) to production with cf push app-name. After that I worked on further features and bugfixes. Now I would to push my local changes to production. But when I do that all the images (e.g. profile image) which are being saved on the production server get lost with every push.
How do I take over only the changes in the code without losing any stored files on the production server?
It should be like a "git pull"
Your application container should be stateless. To persist data, you should use the offered services. The Swisscom Application Cloud offers an S3 compatible Dynamic Storage (e.g. for pictures or user avatars) or different database services (MongoDB, MariaDB and others). If you need to save user data, you should save it in one of these services instead of the local filesystem of the app's container. If you keep your app stateless, you can migrate and scale it more easily. You can find more information about how your app should be structured to run in a modern cloud environment here. To get more information about how to use your app with a service, please check this link.
Quote from Avoid Writing to the Local File System
Applications running on Cloud Foundry should not write files to the
local file system for the following reasons:
Local file system storage is short-lived. When an application instance
crashes or stops, the resources assigned to that instance are
reclaimed by the platform including any local disk changes made since
the app started. When the instance is restarted, the application will
start with a new disk image. Although your application can write local
files while it is running, the files will disappear after the
application restarts.
Instances of the same application do not share a
local file system. Each application instance runs in its own isolated
container. Thus a file written by one instance is not visible to other
instances of the same application. If the files are temporary, this
should not be a problem. However, if your application needs the data
in the files to persist across application restarts, or the data needs
to be shared across all running instances of the application, the
local file system should not be used. We recommend using a shared data
service like a database or blobstore for this purpose.
In future your problem will be "solved" with Volume Services (Experimental). You will have a persistent disk for your app.
Cloud Foundry application developers may want their applications to
mount one or more volumes in order to write to a reliable,
non-ephemeral file system. By integrating with service brokers and the
Cloud Foundry runtime, providers can offer these services to
developers through an automated, self-service, and on-demand user
experience.
Please subscribe to our newsletter for feature announcements. Please also monitor the CF community for upstream development.
I'm working on web app and i want to migrate this web app to virtual machine scale set in windows azure cloud,i'm new to cloud computing ,till i didn't got any proper tutorial about virtual machine scale set,please someone help with this
A few things to consider..
You could build a custom VM which contains the complete app, or you could use VM extensions to deploy the app on a platform image each time a new VM in the scale set is deployed. See: https://msftstack.wordpress.com/2016/04/20/deploying-applications-in-azure-vm-scale-sets/ for some thoughts on this. Ultimately it might depend on how much you need to install over a base image, and how fast you want scaling to be.
Do you need autoscale based on resource usage or do you plan to manually increase/decrease the number of VMs in the set? See https://azure.microsoft.com/en-us/documentation/articles/virtual-machine-scale-sets-windows-autoscale/
A good way to get started with scale sets is to deploy an existing template directly from Azure Quick start templates. Look at https://github.com/Azure/azure-quickstart-templates and search for vmss. These templates will give you an idea of some of the options you have.
To learn the basics about VM Scale Sets, start with the documentation page: https://azure.microsoft.com/documentation/services/virtual-machine-scale-sets/ and the GA announcement: https://azure.microsoft.com/en-us/blog/azure-virtual-machine-scale-sets-ga/
Also look at higher level services like the Azure Web App service if you haven't already, the advantage of a higher level service is that some of the basic web app operations get taken care of for you: https://azure.microsoft.com/en-us/services/app-service/web/
I was wondering about the best way to deliver private web service instances to lots of users, so the user would always be able to connect to their own offline version of a service, just like running a web service from visual studios while debugging. I was struggling with setting this up in VS2013 even with the many online tutorials, but I am not sure if its not working because it was never supposed to work this way.
I have provided this in-depth explanation of my issue as i am not sure i am going about this in the right way and would appreciate feedback:
Background:
I have a web service to interface with an engine. This deals with the front-end and builds a set of commands for how to make a CAD model. These commands are for controlling the 3rd party CAD software's API. Therefore the engine can be seen to have two main functions -
Build the CAD's API instructions, which can be saved for later
Execution, where it catches the instance of the CAD software
running on the same computer and it builds the model.
The second part is restricted for the general public. Only our in-house users should be able to use it. However, they want to have an otherwise identical front-end and user experience.
The problem is, if they connect to the same engine as the public, which exists on our main server, then the engine will be looking for an instance of the CAD package on the same machine as itself - i.e the server, as stressed in the emboldened point above. What should happen is the engine finds the CAD instance running on the machine that the controlling UI is based on and it uses that for its target. I have spoke to the CAD API support and they say they do not know how to do that.
And so we get to my solution of providing an offline stand alone of my web service on each of the employees computers. This means the front-end will check at the start of the session if a localhost connection is available. If not it will use the main address, which takes it to my server. Otherwise it uses the local engine which will look perform the default behavior of looking for a CAD package on the same machine as itself. Because its locally installed that is now the right machine and it will find the CAD instance of the user successfully.
Final points:
The engine cannot be accessed by the UI directly as i am using
Unity3D for the front-end and there is .Net compatibility issues.
I need a completely self contained version of the software in the
future anyway, so eventually i have to deal with having the engine
accessed locally
I ended up using IISExpress. I got the user to install this and then get them to call a batch file installer i made which sets up the config file and moves my web project to the correct directory.