I'm building a web apps that uses http-kit and clojurescript. At some point, I want to separate the front and back each into a lein project on its own. The scenario is:
For the front, if on development mode, uses a lein ring server to serve directory and the app will display a mock data.
The back will serve any resources/public in the front lein project.
I'm thinking of doing a nested lein project, but not sure how to handle it. Any suggestion or pointer is very much appreciated.
The way your question is posted it is not very clear "why two projects?". It seems that you may be confusing different projects with different profiles, e.g. .. classpaths?
In order to work with multiple lein projects at once, similar to the way you would work with master pom => child poms with maven, there is a lein-sub.
But in your case, it seems that what you want is different lein profiles, classpaths. Take a look at lein profiles documentation, it should solve "in dev do this, in test do this, in production do this" behavior you are after.
Related
I'm new to git. I've read the well-written intro book. But gee, it's still not a trivial topic. I've been bumbling around, experiencing various problems. I realized it might be because I'm unaware of workflow, and specifically, "what are the best practices for doing what I'm trying to do?"
I started out developing a django project on my win7 with Pycharm. Great way to get the initial 95% written.
But then I need to deploy it to my production machine at PythonAnywhere.
So I created a private Github repository, pushed my win7 codebase to github.
Then in pythonAnywhere, I cloned the github repository.
For now, no others work on this project. It will not be released to the public.
Now that the server is running on PythonAnywhere, I still need to tweak settings, which is best done on the PythonAnywhere codebase side. But there are other improvements (new pages, or views) that I'd rather do inside Pycharm IDE on my win7 than in vim on python anywhere.
So I've been kind of clumsily pushing and fetching these changes. It's been kind of ham-handed, and I've managed to lose some minor changes through ignorance.
So I'm wondering if anyone can point to a relatively simple workflow that would handle the various tasks I mentioned:
1) improving functionality of the site (best done in Pycharm IDE)
2) production server issues and tweaks (best done on PythonAnywhere)
3) keeping everythign safely backed-up on Github
The other issue is that I have another django app that I want to build. It's easiest to temporarily hang it off the django project I've already built. But I'd prefer to keep it in its own repository.
So I have Original_Project, Original_App stored in Original_Repository
I want to make new_app, and have it, for the time being, run in Original_Project, but I want to version control it in New_Repository.
I think/hope that I could put a .gitignore in the Original_Repository, saying ignore the new_app/ Then I git init new_app/ as its own repository. Is that sound or mad?
You should avoid editing your code on the production server as much as possible, and never commit from the production server. If you end up having to tweaks things on the server (you shouldn't but well, shit happens and sometimes it's indeed easier to first get the code back to work on the server), then once it's working manually report your edits to your local repo, clear up the changes on the server and deploy the fixed code again. Here the github repo should be considered as the "master" repository for deployments, ie you work on your local repo, push to github, and on the server pull from github. This make sure you keep the github repo in sync.
wrt/ the "improving functionality" (aka "features") vs "server issues and tweaks" (aka "hotfixes"), git flow is a (mostly) sane workflow IMHO but that's a bit opinion-based here (some dislike it and have sensible arguments too).
Finally if you want to factor out one of your apps, the best is to have it in it's own (github) repo with all the proper python packaging stuff and make it a requirement of your main project. On your local dev environment you install it as an editable package, and for the production setup you install it as normal package pinned to the last stable version. Note that in both cases I assume you're using virtualenvs (and if you dont, well that's the very first issue you should address).
Update:
What are the downsides of of editing directly on the production server and committing from the production server?
Well quite simply a production server is not the place for coding - "production" means that you have users trying to do something with your website and they don't want to have the site breaking on them, their data lost or whatever because you are "tweaking" things. You should only deploy stable, well tested code on production, and the one and only one case where editing anything on the server might be a last resort option is when it's already broken and you want to get it back online asap whatever it takes (case of "first make it work, then make it clean").
Point is, I'm a professional developer working on projects that are business criticals and a broken site is not an option, so I'm very strict on this - but even if it's a hobby project, your users deserve some respect (at least if you expect to see them back).
A proper production chain actually involves at least three environments: your local dev environment, a staging server (which should closely mirror the production server - system, system package versions, configurations etc etc) to test out / showcase / eventually do minor config tweak, and the production server which should only ever see stable tested code.
I have always struggled with git, knowing it well enough to get thigs working, but never being sure I am doing thing well.
I would suggest installing git flow (it is probably available in your package manager if you are on Linux). Its a set of extensions that simplify a standard git worklfow. Since using it, this has pretty much been all the documentation I have needed.
https://danielkummer.github.io/git-flow-cheatsheet/
I'm playing with Clojure/ClojureScript and I'm writing a web application. Everything is fine while I'm using ring as a development server.
The question is what container should I use for production? Should I use ring for production as well? Should I use Tomcat? Is there a recommended way to deploy a Clojure application? Can you point me to some documentation regarding this aspect?
Thanks!
There is nothing inherently different about deploying a java servlet that was written in Java vs. Clojure and all the Clojure web libraries and frameworks produce compatable servlets so you have many deployment options.
We use netty to run our ring based web application to great effect in production simply by running "lein run" from a system service. Many others choose to use lein uberwar to produce a war file and host that on tomcat. The specific hosting mechanism seems less relevant than the deployment process. All the JavaScript files are served from a CDN. Immutant is also a fun and very Clojure oriented choice with a strong "enterprisy" feel to it.
What strikes me as most important is building a repeatable build, including deployment. Pallet is a great way to go though it's got a bit of a learning curve.
There are a few options.
First one is easy: Heroku. They have a free tier that is ample for deployment and testing. I won't go into further detail on this, but I decided not to use Heroku anymore.
Another common option is Amazon AWS. I gather most apps on AWS use lein-beanstalk [sorry, no citation here]. Lein-beanstalk has been out for quite a while and appears to be well-maintained. It is also maintained by the same person who maintains Compojure.
I use a VPS. I set up the linux build with Nginx and deploy with git. So, basically, my flow is create the site, compile to lein uberjar, then deploy. I know that some people can and do use the leiningen "lien ring server" cantation on their apps and use many other configurations, such as Maven, Tomcat, deployment with Vagrant, etc, but I just run java -jar myApp-xxxxx on the server and it works great.
As far as documentation, there does appear to be a dearth of documentation on Clojure deployment specifically. Sort of have to bang your head against the wall and figure it out if you want to go the VPS route the first time you do it. I found that almost none of my issues involved Clojure specifically.
In development I use:
lein ring server
: then to compile it to a war file I use:
lein ring uberwar
: and just drop the resulting jar file into the Webapps directory and it works fine. I use Jetty by the way
I am trying to compile some projects on the twitter storm platform using leiningen. The servers I am working on do not have access to internet. I was wondering if it is possible to work offline by making a local repository by downloading all dependencies.
Short answer is yes, you should be able to do this by just putting the dependencies where lein expects to find them and then it will use them instead of going out to download them. That said, I've never tried this.
It looks like the answer to this question pretty much points to how to do it:
Run lein deps on a machine that is connected to the Internet
Copy $HOME/.m2/repository to your server
and that should take care of it. However, I have not tried this, so there may be some problem with this method that I have not foreseen.
We're working on a project that has some Clojure-Java interop. At this point we have a single class that has a variety of dependencies which we put into a user library in Eclipse for development, but of course that doesn't help when using Leiningen (2.x). Most of our dependencies are proprietary, so they aren't on a repository somewhere.
What is the easiest/right way to do this?
I've seen leiningen - how to add dependencies for local jars?, but it appears to be out of date?
Update: So I made a local maven repository for my jar following these instructions and the lein deployment docs on github, and edited my project.clj file like this:
:dependencies [[...]
[usc "0.1.0"]]
:repositories {"usc" "file://maven_repository"}
Where maven_repository is under the project directory (hence not using file:///). When I ran "lein deps"--I got this message:
Retrieving usc/usc/0.1.0/usc-0.1.0.pom from usc
Could not transfer artifact usc:usc:pom:0.1.0 from/to usc (file://maven_repository): no supported algorithms found
This could be due to a typo in :dependencies or network issues.
Could not resolve dependencies
What is meant by "no supported algorithms found" and how do I fix it?
Update2: Found the last bit of the answer here.
add them as a dependency to your leiningen project. You can make up the names and versions.
then run lein deps and the error message when it fails to find it will give you the exact command to run so you can install the jar to your local repo then sould you decide to use a shared repo you can use this same process to put your dependencies there.
#Arthur's answer is good but I figured I'd flesh it out a bit more since it leaves some details lacking.
Always keep in mind Repeatability. If you don't make it so that anyone who needs access to the artifacts can get access to the artifacts in a standard way, you're asking for support hell.
The documentation on deployment is a good place to go to find out everything you need to know about deploying your artifacts. Since you're in a polyglot environment you probably can't have lein take care of deploying all your artifacts but at least you can get your clojure specific jars up into S3 or even a file share if you like. The rest of your artifacts will have to use Maven or Ant directly to upload the artifacts to the Maven repo on the file server or S3. At my current company we are using technomancy's excellent s3 wagon private to great effect for hosting our closed source artifacts and clojars for hosting anything that we can open-source.
What #Arthur is referring to is doing a lein install. All that does is install a copy of the current project into your local .m2 directory so that other projects on your box can reference them. Unless you have configured your install of maven to use a shared directory for your .m2 folder (maybe not a bad idea in your environment?), this will mean that anyone else who checks out your project will not be able to build it. If you wanted to go this route, you need to set the localRepository node in your $M2_HOME/conf/settings.xml to be the shared location that the rest of your team has access to. See the docs for more information.
YMMV but I've found it best to use Maven rather than Leiningen when you are working with Polyglot Clojure / Java projects.
It's mainly because the Java based tools (Eclipse etc.) understand Maven projects but don't really understand Leiningen projects. It's getting slowly better with the excellent Counterclockwise Clojure plugin, but the integration still isn't quite good enough yet for an efficient IDE based workflow.
On the repository side of things, I'd suggest setting up a private shared Maven repository. You're going to need it sooner or later if you plan to manage a complex set of dependencies within your team: might as well bite the bullet and get it done now.
I have few simple django-based sites and I their number increasing all the time. Every time I deploy the site I need to:
Manually create bash-script that start Django FastCGI server.
Adding it to etc/init.d to run after server reboot.
Creating separate config for Lighttpd to work with FastCGI server and serving static files.
I know how to do it, but I'd like to automate this task if possible.
My dream setup process could look like this:
I have a folder somewhere in my /var/ directory. For example: /var/django/
I clone one of my projects to the subdirectory of this directory.
After that happening one of the following: Some software automatically detects folder creation, and creates all necessary configs and then restart Lighttpd. OR I manually run some kind of script in my new folder to do it.
I tried to look for existing tools for such automation or something similar in the internet, but couldn't find one.
So I'd like to ask is there tools like this out there? Maybe not exactly to install Django apps, but to this kind of process automation in general. Or everybody just writes their own bash script to do such things?
have you had a look at fabric and puppet?
I think fabric will do the job. I've just started reading through the docs, seems very simple to get started on. Also it has nice Python-ic way of doing things locally and on remote servers.