I already installed jruby and gems seems to be installed nicely under jruby's directory so I don't have to worry about that. Now I would like to be able create a sinatra application and run it on top of jetty or some other efficient standalone server? What do you recommend?
It would be important for me that the application source can be modified, it's easy to start and everything needed except java is in a single directory.
Jetty-rackup was just what I was looking for. I don't need to do any jetty specific, just write config.ru and jetty-rackup handles everything.
Related
I'm using TeamCity 7 as CI Server, and I'd have to test several Web Application plugins, mainly written with PHP. I'm familiar with ANT and *Unit, but I have an issue to solve: to properly test a plugin, my idea would be the following:
Cleanup the testing environment.
Install a clean copy of the Web Application which will host the plugin.
Install the plugin.
Enable the plugin.
Run the tests.
Obviously, running the tests on an installed environment is the easy part. Most tests are fired by directly calling plugin's classes' methods, yet the framework must be configured, even with minimal settings, to allow calling its bootstrap file and perform due initialization. I tried running the tests in an environment I prepared manually, and they run as expected.
The issue is now automating the installation of the standard Web Application, and, most importantly, its configuration. The basic steps are the following:
Unzip framework somewhere (done).
Create a Database (done).
Create a Database User ans assign it propert privileges (done).
Run Web Application's setup.
The tricky part is that not all Web Application implement a command line interface, such as drush for Drupal, hence I came out with two possible ways to complete the installation:
Simulate manual installation via CURL
Take note of the installation steps and the forms that need to be filled.
POST data to each of the forms using CURL.
I tried this method, still manually, with acceptable results. The Web Application gets installed as expected, and it can be used.
However, this requires a Web Server where the application can run. As far as I know, TeamCity Agents work in their own, random named directories and anything "installed" in them cannot be accessed via HTTP requests.
Backup/Restore
Manually pre-install a Web Application and configure its basic settings.
Zip the Application's directory and a backup of its database.
Before running the tests, unzip the directory in Agent's working directory.
Restore the backed-up database. The application will now be "configured".
This method is a bit "rough", but it doesn't require a Web Server to be running. Although the Web Application won't be able to server HTTP requests, that doesn't necessarily matter, as the tests will be run against plugin's classes.
This method has two major drawbacks, though:
Tests involving interaction with the Web Application (e.g. hooks, event handlers, and so on) can't be run.
Since the Web Application and its database are pre-configured, their parameters will be the same at every run. Therefore, it wouldn't be possible to run two Agents at the same time, for example to test two different plugins.
I'm now wondering if there's a better solution, as both the above look less than optimal to me.
Please note that, although I'm using TeamCity, the CI Server itself should not be a big deal, as I'm running everything with ANT. Therefore, any suggestion, even related to another CI Serverm, will be welcome (I know Hudson, CruiseControl and BuildMaster, I can adapt a concept easily). Thanks.
I've been fortunate enough to discover django_compressor and implemented it within our stack, which deploys to many servers (Currently 6, but growing as we deploy smaller virtual machines.)
Now this is all fine and dandy if you're using django_compressor at its finest. Compressing raw CSS/JS code
However, say now I want introduce some type of pre-compiler. Let's say for this example it is LESS (css). The thought process for this is fairly simple:
Install node, npm, and the less package onto the server.
Add less to your precompilers!
COMPRESS_PRECOMPILERS = ( ('text/less', 'lessc {infile} {outfile}'), )
Now you deploy, and your server compiles the less file. Everything is fantastic!
Now let's add 8 more servers to that and you have to install node, npm, and less on each server?
This is where something doesn't seem right, and I feel like I'm missing something. I believe the Django community has run into this problem before.
My thoughts thus far have been:
Use a post-commit hook to compile the CSS on the developers machine. This means that via django_compressor, we link to the compiled static file in the HTML, and our repository contains both the compiled and non-compiled versions. My only downside to this is it ends up not using half of the benefits of django_compressor and may be tedious for developers?
Suck it up and make node, npm, and less part of the server stack.
Update
I did some additional looking around and it seems that using the COMPRESS_OFFLINE flag (or just --force) with the management command will produce an offline manifest file that does what I need (only tested locally). So setting this up with a pre-deploy hook likes to be the answer.
Of course, still open to other ideas :-)
Coupled with the tips in the comments about COMPRESS_OFFLINE, you could look at django-staticfiles' storage stuff. You can host the static files on amazon s3, for instance, so hosting it all on one static-hosting server and using that from all your servers could also be a nice solution. You wouldn't need to do anything with the static (and compressed) files on the individual servers.
Alternative solution regarding the multiple servers: I've made a custom fabric (docs.fabfile.org) script that installs/configures stuff on our servers. I've only recently started using coffeescript and less, but those two are definitively ending up in my fabfile. That solves the installation problem for me.
(Alternatives to a fabfile are things like a custom debian package with standard dependencies. Or chef or puppet or something similar.)
you can use puppet for the task
I have few simple django-based sites and I their number increasing all the time. Every time I deploy the site I need to:
Manually create bash-script that start Django FastCGI server.
Adding it to etc/init.d to run after server reboot.
Creating separate config for Lighttpd to work with FastCGI server and serving static files.
I know how to do it, but I'd like to automate this task if possible.
My dream setup process could look like this:
I have a folder somewhere in my /var/ directory. For example: /var/django/
I clone one of my projects to the subdirectory of this directory.
After that happening one of the following: Some software automatically detects folder creation, and creates all necessary configs and then restart Lighttpd. OR I manually run some kind of script in my new folder to do it.
I tried to look for existing tools for such automation or something similar in the internet, but couldn't find one.
So I'd like to ask is there tools like this out there? Maybe not exactly to install Django apps, but to this kind of process automation in general. Or everybody just writes their own bash script to do such things?
have you had a look at fabric and puppet?
I think fabric will do the job. I've just started reading through the docs, seems very simple to get started on. Also it has nice Python-ic way of doing things locally and on remote servers.
I am trying to work out a good way to run a staging server and a production server for hosting multiple Coldfusion sites. Each site is essentially a fork of a repo, with site specific changes made to each. I am looking for a good way to have this staging server move code (upon QA approval) to the production server.
One fanciful idea involved compiling the sites each into EAR files to be run on the production server, but I cannot seem to wrap my head around Coldfusion archives, plus I cannot see any good way of automating this, especially the deployment part.
What I have done successfully before is use subversion as a go between for a site, where once a site is QA'd the code is committed and then the production server's working directory would have an SVN update run, which would then trigger a code copy from the working directory to the actual live code. This worked fine, but has many moving parts, and still required some form of server access to each server to run the commits and updates. Plus this worked for an individual site, I think it may be a nightmare to setup and maintain this architecture for multiple sites.
Ideally I would want a group of developers to have FTP access with the ability to log into some control panel to mark a site for QA, and then have a QA person check the site and mark it as stable/production worthy, and then have someone see that a site is pending and click a button to deploy the updated site. (Any of those roles could be filled by the same person mind you)
Sorry if that last part wasn't so much the question, just a framework to understand my current thought process.
Agree with #Nathan Strutz that Ant is a good tool for this purpose. Some more thoughts.
You want a repeatable build process that minimizes opportunities for deltas. With that in mind:
SVN export a build.
Tag the build in SVN.
Turn that export into a .zip, something with an installer, etc... idea being one unit to validate with a set of repeatable deployment steps.
Send the build to QA.
If QA approves deploy that build into production
Move whole code bases over as a build, rather than just changed files. This way you know what's put into place in production is the same thing that was validated. Refactor code so that configuration data is not overwritten by a new build.
As for actual production deployment, I have not come across a tool to solve the multiple servers, different code bases challenge. So I think you're best served rolling your own.
As an aside, in your situation I would think through an approach that allows for a standardized codebase, with a mechanism (i.e. an API) that allows for the customization you're describing. Otherwise managing each site as a "custom" project is very painful.
Update
Learning Ant: Ant in Action [book].
On Source Control: for the situation you describe, I would maintain a core code base and overlays per site. Export core, then site specific over it. This ensures any core updates that site specific changes don't override make it in.
Call this combination a "build". Do builds with Ant. Maintain an Ant script - or perhaps more flexibly an ant configuration file - per core & site combination. Track version number of core and site as part of a given build.
If your software is stuffed inside an installer (Nullsoft Install Shield for instance) that should be part of the build. Otherwise you should generate a .zip file (.ear is a possibility as well, but haven't seen anyone actually do this with CF). Point being one file that encompasses the whole build.
This build file is what QA should validate. So validation includes deployment, configuration and functionality testing. See my answer for deployment on how this can flow.
Deployment:
If you want to automate deployment QA should be involved as well to validate it. Meaning QA would deploy / install builds using the same process on their servers before doing a staing to production deployment.
To do this I would create something that tracks what server receives what build file and whatever credentials and connection information is necessary to make that happen. Most likely via FTP. Once transferred, the tool would then extract the build file / run the installer. This last piece is an area I would have to research as to how it's possible to let one server run commands such as extraction or installation remotely.
You should look into Ant as a migration tool. It allows you to package your build process with a simple XML file that you can run from the command line or from within Eclipse. Creating an automated build process is great because it documents the process as well as executes it the same way, every time.
Ant can handle zipping and unzipping, copying around, making backups if needed, working with your subversion repository, transferring via FTP, compressing javascript and even calling a web address if you need to do something like flush the application memory or server cache once it's installed. You may be surprised with the things you can do with Ant.
To get started, I would recommend the Ant manual as your main resource, but look into existing Ant builds as a good starting point to get you going. I have one on RIAForge for example that does some interesting stuff and calls a groovy script to do some more processing on my files during the build. If you search riaforge for build.xml files, you will come up with a great variety of them, many of which are directly for ColdFusion projects.
I'm trying to set up a good development environment for a Django project that I will be working on from two different physical locations. I have two Mac machines, one at home and one at work that I do most of my development on. I currently host a Ubuntu virtual machine on one of the machines to host the Django environemnt, install DropBox on it, and edit source code from my Mac. When I save the code file, the changes get synced over DropBox to the Ubuntu VM and the Django development server automatically restarts because of the change. This method has worked well in the past, but I am starting to use DropBox for a lot of other things now and don't want all of that to be downloaded on every virtual machine I use. Plus, I want to start using Eclipse + PyDev to be able to debug code and have code completion. Currently, I use TextEdit which is great, but doesn't support debugging or completion.
So what are my options? I thought about setting up a Parallels VM on a thumb drive that has my entire environment on it (Eclipse included), but that has its own problems. Any other thoughts?
Here is the environment I set up and it has the components you are after. I have used pydev as well and it works but I prefer Komodo.
Things which I think you are missing:
An SCM - Using Dropbox works but there are some real shortcomings by not using a real version control system. Examples include reverting changes, branching, merging, etc. I agree with Simon
Using a virtualenv will really help when developing on multiple platforms.
I do ALL of this on my Mac:)
HTH