I'm writing a customized java-buildpack, and I'd like the ability to edit the web.xml file to add some required configuration. However, I seem to be limited to only the rubygems that are present on the instance node, which does not include any XML parsing libraries.
How can I have the buildpack specify that it needs an additional gem to execute? Adding gems to the Gemfile in the buildpack itself doesn't seem to have any effect when staging an app.
Alternatively, how can I modify an existing XML file with only the stock facilities available in CF? I wanted to avoid a regex, but that may be a last resort.
However, I seem to be limited to only the rubygems that are present on the instance node, which does not include any XML parsing libraries.
Correct. When your build pack runs in the staging container, it's going to use the version of Ruby provided by the container. This means that you are locked into a specific version of Ruby and the set of gems provided by the container.
The only way around this would be to have a bootstrap shell script that downloads Ruby, installs your Gems, and then runs the actual build pack Ruby scripts. It's a lot of extra work and overhead though, so probably not a good option if you can avoid it.
How can I have the buildpack specify that it needs an additional gem to execute? Adding gems to the Gemfile in the buildpack itself doesn't seem to have any effect when staging an app.
I don't think that this is possible. You can't gem install something because it would require root access, and you won't have that.
What you could do instead is to include the dependencies that you need in the build pack. We did that with a YAML dependency for the PHP build pack. If your dependency is all Ruby code (i.e. no native code), this should work OK.
Alternatively, how can I modify an existing XML file with only the stock facilities available in CF? I wanted to avoid a regex, but that may be a last resort.
It depends on what you're doing and what you need to change.
Regex search and replace is an option.
Basic string search and replace is another.
You could shell out to a tool that's installed in the environment and is capable of parsing / editing the XML
If it happens later in the install after Java has been installed, you could run some Java code.
I would not recommend creating a custom stack. It's incredibly invasive, means you then need to provide all updates going forward (not a small task), and it's not something that will work if you're going to use a public cloud provider, since no one is going to allow you to install your custom stack in their public cloud.
Hope that helps!
You can create a custom stack (container root fs) with all required libraries for this purpose.
https://docs.cloudfoundry.org/devguide/deploy-apps/stacks.html
https://docs.cloudfoundry.org/adminguide/custom-stack.html
Related
New user here...
Installed D8+Civi by building a composer based git repo for the platform then stamping out a few test sites.
It worked really well.
But now I am at the point of realizing I missed a few modules and I want to add some themes to apply to the sites.
I can easily to it in the git which was used to define the platform. But what is the proper way to manage the central platform data and files that are then used for the x number of sites.
I know the docs try to discuss this be a tutorial walk-through would be very helpful.
As a guess, I could make the central platform files a git clone and pull down clones for the new stuff. But if there was a need for an database updates that wouldn't get done.
Ideas?
Thanks
It's not clear what you mean by "central platform data".
If you mean assets that are relevant for the entire platform, that can apply to all of the sites, you would do the following:
Add anything new to Git and push it.
Create a new platform to match the latest code in Git.
Ran a Migrate task on the old platform to migrate the sites to the new one.
Database schema updates happen automatically.
The sites will now be running on the new codebase.
If you're talking about site-specific assets that you don't want to be included in the platform's code, then you can enable Git for sites with the Aegir Hosting Git module.
It allows you to deploy site-specific Git repositories.
However, I don't recommend using that module for platforms, just sites, because it allows you to git pull on Production sites, which is a terrible idea. For that, see Aegir Deploy.
Both of these modules ship with Aegir so you won't need to install them. Some of the Hosting Git features may need to be enabled, however.
We are updating our sitecore to 8.2 and in the process I am trying to refine our source control and development workflow.
Goals
1. Have a single source of truth for support dlls, configs, lic, etc.
2. Have everything in source control that is needed to recreate the entire site from dev to prod. (excluding packages).
In order to have all of the different configs needed for the various machines I have created gulp tasks that transform the configs on build (dev, staging, prod). Those transformed configs are placed in a folder in the project that is then used to replace the originals on the target machines. This folder publishes all of its contents and seems to be working well so far.
What I don't know is how to deal with all of the config files that do not change.
Is it best to include all of those .config files in the project so that they publish? If not, then the target machine folders will have to be either manually managed (seems like a bad idea) or a script used to ensure the configs are up to date (more customization..by default not a great idea).
The only downside (that I see) to including all of the configs in the project is the weight that it would add to file searches (and that doesn't seem like a very strong argument).
Am I not seeing something?
How are you other Sitecore humans handling this?
Gregory
As a general rule of thumb, do not check in any default files into Source Control.
The main reasons are; bloat, making syncing/downloading from your source control take much longer, and upgrades, the latter being a much more important reason.
If/when you upgrade in the future, if you do not have any Sitecore files checked into source control then you can simply deploy a new/clean instance of Sitecore, fix any conflicts in your own code and then deploy on top. You don't have to try and figure out what has changed in the default install files between releases.
Any changes you need to make to Sitecore configs or settings should be made using patch files and only those custom files added to your solution.
How to handle this for deployments?
There are a few options. You could go done the scripted route, which will take a clean Sitecore install, unzip and made whatever modifications you need, then install/unzip the modules that you use in your solution one by one.
Another option maybe to create a default install with all the modules and then zip this up, then an install would be similar process to above but a more simpler case of just unzipping a single file. You could use Sitecore SIM to both install the instance, modules and then backup or do this manually.
Yet another alternative may be to check everything into Source Control, either under separate repository or a different project so ensure that all default files and configs are kept separate. If you need to upgrade in the future, simply delete the repo/project and add them back in again.
I would also do the same (a separate project) to keep all Support patches/dlls separate, again to help easily identify what fixes have been applied and to easily remove them if a future version resolves the issue.
These may add an additional step to your deploy, but keeping this separation will make your life much much easier when it comes to upgrade time.
We're working on a project that has some Clojure-Java interop. At this point we have a single class that has a variety of dependencies which we put into a user library in Eclipse for development, but of course that doesn't help when using Leiningen (2.x). Most of our dependencies are proprietary, so they aren't on a repository somewhere.
What is the easiest/right way to do this?
I've seen leiningen - how to add dependencies for local jars?, but it appears to be out of date?
Update: So I made a local maven repository for my jar following these instructions and the lein deployment docs on github, and edited my project.clj file like this:
:dependencies [[...]
[usc "0.1.0"]]
:repositories {"usc" "file://maven_repository"}
Where maven_repository is under the project directory (hence not using file:///). When I ran "lein deps"--I got this message:
Retrieving usc/usc/0.1.0/usc-0.1.0.pom from usc
Could not transfer artifact usc:usc:pom:0.1.0 from/to usc (file://maven_repository): no supported algorithms found
This could be due to a typo in :dependencies or network issues.
Could not resolve dependencies
What is meant by "no supported algorithms found" and how do I fix it?
Update2: Found the last bit of the answer here.
add them as a dependency to your leiningen project. You can make up the names and versions.
then run lein deps and the error message when it fails to find it will give you the exact command to run so you can install the jar to your local repo then sould you decide to use a shared repo you can use this same process to put your dependencies there.
#Arthur's answer is good but I figured I'd flesh it out a bit more since it leaves some details lacking.
Always keep in mind Repeatability. If you don't make it so that anyone who needs access to the artifacts can get access to the artifacts in a standard way, you're asking for support hell.
The documentation on deployment is a good place to go to find out everything you need to know about deploying your artifacts. Since you're in a polyglot environment you probably can't have lein take care of deploying all your artifacts but at least you can get your clojure specific jars up into S3 or even a file share if you like. The rest of your artifacts will have to use Maven or Ant directly to upload the artifacts to the Maven repo on the file server or S3. At my current company we are using technomancy's excellent s3 wagon private to great effect for hosting our closed source artifacts and clojars for hosting anything that we can open-source.
What #Arthur is referring to is doing a lein install. All that does is install a copy of the current project into your local .m2 directory so that other projects on your box can reference them. Unless you have configured your install of maven to use a shared directory for your .m2 folder (maybe not a bad idea in your environment?), this will mean that anyone else who checks out your project will not be able to build it. If you wanted to go this route, you need to set the localRepository node in your $M2_HOME/conf/settings.xml to be the shared location that the rest of your team has access to. See the docs for more information.
YMMV but I've found it best to use Maven rather than Leiningen when you are working with Polyglot Clojure / Java projects.
It's mainly because the Java based tools (Eclipse etc.) understand Maven projects but don't really understand Leiningen projects. It's getting slowly better with the excellent Counterclockwise Clojure plugin, but the integration still isn't quite good enough yet for an efficient IDE based workflow.
On the repository side of things, I'd suggest setting up a private shared Maven repository. You're going to need it sooner or later if you plan to manage a complex set of dependencies within your team: might as well bite the bullet and get it done now.
Question 1:
I am about to deploy my first Django website and I was wondering what tools are recommended to gathering all your Django files.
Like for example I don't need my sass and coffeescript files I just want the compiled css and js files. I also want to use the correct production settings file.
Question 2:
Do I put these files ready for deployment into their own version control repository? I guess the advantage is that you can easily roll back changes?
Question 3:
Do I run my tests before gathering the files or before deploying?
Shell scripts could be a solution but maybe there is a better way? I looked at jenkins/hudson but that seems more like a tool that sits on top of the tools that I am looking for.
For questions one and two, I'd recommend using a version control system for this. I'm sure you're already using some sort of version control, so you can just say which branch of your repository you would like to deploy. And yes, this makes rollbacks incredibly easy. Probably the most popular method for Django deployment is to package your files using git, and then deploy these files and run any deployment scripts using fabric.
Using git, packaging your files using your local repository would look something like:
git archive --format=tar HEAD | gzip > my_repo.tar.gz
Alternately, you can first push your changes to a github repository, and then in your deployment script just clone your repository from your production server.
For your third question, if you use this version control method for packaging your files, then just make sure when you are testing you have the deployment branch checked out.
I'll typically use Fabric for deploying most Django projects:
http://docs.fabfile.org/en/1.0.0/?redir
It has a decent api for communicating with remote servers and it's all in Python – bonus!
You don't need to store your concatenated media files in a separate repo. They're only needed for production. In that case I've found libraries like django-mediasync and django-compress to be useful. They both provide template tags/settings that can concatenate and cache your static files for you depending on the DEBUG setting/environments (production vs development).
You can run your tests whenever. Some people will run them as a version control hook to prevent broken code from being checked in or during deployment, stopping the deployment in case of test failure.
I already installed jruby and gems seems to be installed nicely under jruby's directory so I don't have to worry about that. Now I would like to be able create a sinatra application and run it on top of jetty or some other efficient standalone server? What do you recommend?
It would be important for me that the application source can be modified, it's easy to start and everything needed except java is in a single directory.
Jetty-rackup was just what I was looking for. I don't need to do any jetty specific, just write config.ru and jetty-rackup handles everything.