I have a CIO Blumix Cloud Foundry PHP app developed that needs some additional components.
I used https://github.com/cloudfoundry/php-buildpack for the build. I read in its documentation that I can add my own extension. I did that and added a tar.tgz and added instructions in the extention.py how to install it.
The target location is: /home/vcap/.
I see the installation running okay, and I see the folder during deploy stage (in DevOps Pipelines deployment stage log&history).
But when deployment passes and I read with a deployed php page the folder, I see that it is not there. I read "container destroyed successfully" message in the log of the deploy. Maybe the whole installation environment goes destroyed? Where is a safe place in the deployment file structure where I can install components so they remain after the deployment passes?
I'm using the def compile(install): to place my unix commands. Example: os.system('ls') to list the installation folders content. They work properly.
Thx in advance!
There are two totally different environments used by your app: staging and runtime. Staging is where the buildpack runs & runtime is where the product of staging (i.e. your app) is run.
Unfortunately, paths are not the same in staging and runtime. At runtime your app lives under /app or /home/vcap/app (the former is a symlink to the latter). Staging is different. There is a /home/vcap directory but it's not used for anything.
Instead, the buildpack scripts are fed paths to use via cli arguments. This is all documented here.
As a PHP buildpack extension, you can access the cli args, and many other things, by looking at the context that is maintained by the buildpack. This gets passed directly into the buildpack extension methods like service_environment & service_commands. The compile buildpack extension method is slightly different as the argument passed in is not the content, but that argument does have a reference to the context (it's install.builder._ctx).
Having said all that, I would not recommend using PHP buildpack extensions at this point. The buildpack is being rewritten and that functionality is being dropped. It's not going to have a direct replacement, but the closest thing would be Composer's ability to execute scripts. My suggestion would be to see if you can use the Composer functionality. It'll be more portable as it won't depend on buildpack specific behavior.
Related
Terribly sorry if this is an obvious question. But I have a webapp that relies on a CLI tool to get it to work. I was wondering if there was a way I could specify this without using a custom buildpack. And how to go about doing this if possible
Any help on this would be great, thanks
Can you add CLI tools to a Cloud Foundry app?
It's not possible to directly install things with apt or apt-get. Your app runs as an unprivileged user and is unable to run those tools to install things.
This leaves you with a couple options:
Get the binary and bundle it with your app. Some people (no judgement from me though) would say that your app is responsible for bringing everything it needs to run anyway, so you should be doing this already.
Twelve-factor apps also do not rely on the implicit existence of any system tools. Examples include shelling out to ImageMagick or curl. [1]
This path works well for dependencies that are small or self contained, like statically built Go apps. If your app need shared libraries or other resources to function, you need to bundle those with your app too. It's also not great if the size of what you bundle is large. Everything you bundle will be pushed up with the app, so it can slow down your pushes. You are also responsible for tracking updates and making sure that you have the latest, bug free & security patched binaries & libraries.
The general steps for doing this are:
Create a folder like binaries/ under the root of your app, with subfolders of bin/ and lib/.
Place all your binaries under binaries/bin and any shared libraries they require under binaries/lib.
Add a .profile file at the root of your app. This will be sourced prior to your app starting so it will put any binaries you bundle on the path and add libraries to the search path.
In .profile put the following:
export PATH=$HOME/binaries/bin:$PATH
export LD_LIBRARY_PATH=$HOME/binaries/lib:$LD_LIBRARY_PATH
That should be it. Just push your app with all the new files.
Another easier option, is to use the Apt buildpack [2]. This buildpack can install required dependencies using apt. You just need to add an apt.yml file to the root of your app & run your app with multi-buildpacks (apt buildpack first, then your normal buildpack).
The main benefit of doing this is that you don't have to manage the dependencies. The apt buildpack will automatically install them from the repo you tell it to use, so it'll pick up new versions from there as well. This is good if what you need to install has a lot of dependencies, particularly sensitive dependencies (like openssl) or dependencies that get updated/patched often like other language runtimes (Python, Perl, Ruby, etc...).
Other benefits. It's easier because the buildpack takes care of adjusting PATH & LD_LIBRARY_PATH. It also makes the app size smaller so pushes are faster.
The downsides of this option are that the apt-buildpack is not an official buildpack (it's community maintained). It also works best when you have Internet access, so it can download binaries from the Internet, although you can work around this by using an internal repo.
There's a couple other options as well, but I wouldn't recommend them unless both options above are definitely not going to work for you.
Use Docker. You can set up your own Docker container with all the dependencies you need, plus your app code and cf push the Docker image to CF. The downside of this is that your lose the advantages of using buildpacks, so you're back to building and managing Docker images and all the required dependencies of your app all on your own.
You could create your own custom buildpack and supply the dependencies that way. I don't see any reason you'd want to do this though. It's a decent bit of work and in the end, you'd have something that's just more brittle and less flexible than Apt Buildpack.
It's technically possible to ship your own rootfs, but you really really shouldn't (I'm just including this to be thorough). This is the base file system that's used by all apps on CF. Doing this has a lot of drawbacks though, chiefly being that it's difficult. It also applies to all apps on the foundation, can bloat the size of the rootfs, and makes a larger attack surface for anything using the rootfs (i.e. all apps).
Hope that helps!
[1] https://12factor.net/dependencies
[2] https://github.com/cloudfoundry/apt-buildpack
currently our process works, but it takes too much time due that the fronend Ember app needs to be build into every single environment we have ( 5 environments ). because we never know which environment will be available when we release it.
we intend to add even more environments because every developer should have his own working development environment. (because of the backend)
how we do it, is that we create a frontend build and a backend build which creates artifacts.
now the frontent build takes around 2 minutes for every environment.
ember build --env=test and ember build --env=acceptance and ember build --env=development ... and more
when the artifacts are created we then create the release picking the correct ones depending on which environment we release (this done via release pipeline).
my question is can we make a frontend ember build somehow not depending on the environment?
i would like to note that we are using azure service fabric.
I don't think there is anyway around multiple Ember builds because each one will be different (i.e. production vs. development).
You can batch together each build inside one CI build/build task and produce artifact(s) to be used in your release pipeline.
Run the following command once for each environment you have (assuming you are using Ember-CLI) sequentially in one build task.
ember build --environment={{YOUR-ENV-HERE}} --output-path="dist/{{YOUR-ENV-HERE}}/"
You can then either upload the entire dist/ folder as an artifact and scope each environment in your release pipeline to the corresponding artifact subdirectory, or you can upload each folder inside /dist as an individual artifact and scope each environment in your release pipeline to its corresponding artifact.
only the configuration it changes. basically the api endpoints
I just clone my git to the Google cloud platform. And Im trying to deploy it.
As I type "gcloud app deploy" it says:
And I have a file app.yaml and it looks like this:
# [START app_yaml]
runtime: nodejs
vm: true
I also tried changing the directory but cannot find the repo dir..
Please help!
Thanks!
The error message indicates that the tool is unable to automatically detect the application's configuration file. Which could be caused by the now deprecated older generation of flexible apps configuration, using the vm: true statement. See cloud.google.com/appengine/docs/flexible/python/upgrading.
You should use the current flexible app configuration. I'd also suggest passing the actual app.yaml as argument to the deploying command, auto-detection might not always work as one expects.
I am deploying a React app on AWS Elastic Beanstalk. I bundle the app using webpack. However, I'm slightly confused about what best practices are from the production build process. Should I build the app locally (with NODE_ENV=production) using webpack, and then just upload the resultant bundle.js file, along with all node_modules to the Elasticbeanstalk instance? Or, should I upload all the source files, and run webpack on the actual cloud AWS server during deployment?
You should never build for production locally (unless you're the only developer).
Ideally, you have a build process that gets triggered manually or automatically from a git commit which then builds your project for production for you.
By using a centralized build process, you can then be sure that all your builds are built the same way (e.g. same node version, same npm or yarn version).
Both approaches are not really good to be honest. Local building is not a best way to build anything you want to have on production. You might have packages locally that may have inpact on what you're building. Same applies to the OS your doing it on.
And, again, same applies to the building during deployment. As the name of 'deployments' stands for, it's deploying. Just placing your application setup on the server so it may serve as it is supposed to.
That's the point where all CI/CD comes in. Having those kinds of solutions guarantee that each build is done with the same steps and on the same solution stack. No difference between each build is desired, because it allows you to assume that any bug or a change comparing to the 'desing' is because of the code, not environment it was build within.
Assuming that you're the only developer here (because you're asking for such a thing), CI/CD might be definitive overkill here, so just create shell script with steps and use Docker as the environment for build, so it stays the same between each build. That's the closest to the CI/CD option you can get without a hassle.
Here is my understanding of cloudfoundry buildpacks and droplets
buildpack is runtimes(say, jdk) + containers(say, tomcat) + frameworks(say, spring) + service configs (if any) + your apps (if any).
droplet is buildpack deployed on a linux container.
Please correct or add additional information.
A buildpack is a collection of three binaries: detect, compile, and release. When your app bits are pushed to Cloud Foundry, the detect from all the available buildpacks are run against your bits until one returns true. For example, the Ruby buildpack checks to see if there's a Gemfile, the Python buildpack looks for a requirements.txt, etc. Next, the compile phase turns your app bits into a single runnable package, which means compiling the code (if necessary) and bundling in any additional things needed, such as JDK, tomcat, spring, etc; or a Ruby interpreter, gems, etc. That single executable is the droplet. Finally, the release phase presents the droplet and associated metadata so that the stager can upload them to the cloud controller.
To run your app, the droplet is retrieved from the cloud controller and run inside a container.
Basically, a buildpack packages your app into a droplet, which consists of your app + some other stuff, or a compiled binary version of your app, and the droplet is then run in a container.
Buildpacks are scripts which provides run time support for your Application. A build pack is a script which which contains instructions to detect, supply, finalize and release. They essentially provides your application runtime + app framework + related dependency.
Droplet is tarball file which contains Your APP+run time + framework+ dependency . This package is out put of staging and is finally deployed.