I'm getting set up with proper distribution version control (yes, overdue) on a large Django environment with lot's of reusable apps and lot's of projects.
What's the right way to do this?
Clone each app you need within each project, to allow you to make changes to the app without worrying about breaking anything.
Have one copy of each version controlled application to avoid having multiple copies of the code, each in its own repository.
Or is there a better way?
Thanks.
Edit for clarity: These are in house apps that are reused from project to project.
In my opinion the best practice is to keep all your apps as one library/package. You can have versions/snapshots (e.g. tags in hg) and branches and you should definitely create and configure setup.py file.
If the app are reusables, you must create a egg in pypi. These have releases. For each project, you could use one or the other releases.
See for example this package.
To deploy the projects both in local as in the server, you can use buildout (very recomended)
Related
New user here...
Installed D8+Civi by building a composer based git repo for the platform then stamping out a few test sites.
It worked really well.
But now I am at the point of realizing I missed a few modules and I want to add some themes to apply to the sites.
I can easily to it in the git which was used to define the platform. But what is the proper way to manage the central platform data and files that are then used for the x number of sites.
I know the docs try to discuss this be a tutorial walk-through would be very helpful.
As a guess, I could make the central platform files a git clone and pull down clones for the new stuff. But if there was a need for an database updates that wouldn't get done.
Ideas?
Thanks
It's not clear what you mean by "central platform data".
If you mean assets that are relevant for the entire platform, that can apply to all of the sites, you would do the following:
Add anything new to Git and push it.
Create a new platform to match the latest code in Git.
Ran a Migrate task on the old platform to migrate the sites to the new one.
Database schema updates happen automatically.
The sites will now be running on the new codebase.
If you're talking about site-specific assets that you don't want to be included in the platform's code, then you can enable Git for sites with the Aegir Hosting Git module.
It allows you to deploy site-specific Git repositories.
However, I don't recommend using that module for platforms, just sites, because it allows you to git pull on Production sites, which is a terrible idea. For that, see Aegir Deploy.
Both of these modules ship with Aegir so you won't need to install them. Some of the Hosting Git features may need to be enabled, however.
I'm new to git. I've read the well-written intro book. But gee, it's still not a trivial topic. I've been bumbling around, experiencing various problems. I realized it might be because I'm unaware of workflow, and specifically, "what are the best practices for doing what I'm trying to do?"
I started out developing a django project on my win7 with Pycharm. Great way to get the initial 95% written.
But then I need to deploy it to my production machine at PythonAnywhere.
So I created a private Github repository, pushed my win7 codebase to github.
Then in pythonAnywhere, I cloned the github repository.
For now, no others work on this project. It will not be released to the public.
Now that the server is running on PythonAnywhere, I still need to tweak settings, which is best done on the PythonAnywhere codebase side. But there are other improvements (new pages, or views) that I'd rather do inside Pycharm IDE on my win7 than in vim on python anywhere.
So I've been kind of clumsily pushing and fetching these changes. It's been kind of ham-handed, and I've managed to lose some minor changes through ignorance.
So I'm wondering if anyone can point to a relatively simple workflow that would handle the various tasks I mentioned:
1) improving functionality of the site (best done in Pycharm IDE)
2) production server issues and tweaks (best done on PythonAnywhere)
3) keeping everythign safely backed-up on Github
The other issue is that I have another django app that I want to build. It's easiest to temporarily hang it off the django project I've already built. But I'd prefer to keep it in its own repository.
So I have Original_Project, Original_App stored in Original_Repository
I want to make new_app, and have it, for the time being, run in Original_Project, but I want to version control it in New_Repository.
I think/hope that I could put a .gitignore in the Original_Repository, saying ignore the new_app/ Then I git init new_app/ as its own repository. Is that sound or mad?
You should avoid editing your code on the production server as much as possible, and never commit from the production server. If you end up having to tweaks things on the server (you shouldn't but well, shit happens and sometimes it's indeed easier to first get the code back to work on the server), then once it's working manually report your edits to your local repo, clear up the changes on the server and deploy the fixed code again. Here the github repo should be considered as the "master" repository for deployments, ie you work on your local repo, push to github, and on the server pull from github. This make sure you keep the github repo in sync.
wrt/ the "improving functionality" (aka "features") vs "server issues and tweaks" (aka "hotfixes"), git flow is a (mostly) sane workflow IMHO but that's a bit opinion-based here (some dislike it and have sensible arguments too).
Finally if you want to factor out one of your apps, the best is to have it in it's own (github) repo with all the proper python packaging stuff and make it a requirement of your main project. On your local dev environment you install it as an editable package, and for the production setup you install it as normal package pinned to the last stable version. Note that in both cases I assume you're using virtualenvs (and if you dont, well that's the very first issue you should address).
Update:
What are the downsides of of editing directly on the production server and committing from the production server?
Well quite simply a production server is not the place for coding - "production" means that you have users trying to do something with your website and they don't want to have the site breaking on them, their data lost or whatever because you are "tweaking" things. You should only deploy stable, well tested code on production, and the one and only one case where editing anything on the server might be a last resort option is when it's already broken and you want to get it back online asap whatever it takes (case of "first make it work, then make it clean").
Point is, I'm a professional developer working on projects that are business criticals and a broken site is not an option, so I'm very strict on this - but even if it's a hobby project, your users deserve some respect (at least if you expect to see them back).
A proper production chain actually involves at least three environments: your local dev environment, a staging server (which should closely mirror the production server - system, system package versions, configurations etc etc) to test out / showcase / eventually do minor config tweak, and the production server which should only ever see stable tested code.
I have always struggled with git, knowing it well enough to get thigs working, but never being sure I am doing thing well.
I would suggest installing git flow (it is probably available in your package manager if you are on Linux). Its a set of extensions that simplify a standard git worklfow. Since using it, this has pretty much been all the documentation I have needed.
https://danielkummer.github.io/git-flow-cheatsheet/
More specifically, when using a backend application framework I generally am afforded some level of asset management which allows me to work with multiple files in development which are uncompressed and unminified and then in production mode those files become automatically minified, compressed, and concatenated into a single file.
I am looking to create an Ember application that is a single page app that interfaces with a separate RESTful services layer. I simply do not need the weight of a framework behind the Ember app and am hoping to serve it as static html+css+js, so I am looking for any guidance on how to easily manage development and deployment of a client-side only app without adding much overhead.
Right now my biggest issue is with including JS (and to a lesser extent, CSS) files. My HTML is static and I have an Ember app comprised of many files, so I have many script tags to include them all. This is clearly not appropriate for production so I imagine some kind of build tool will be needed to assemble my Javascript files and overwrite the script tags in the HTML file. Are there tools out there right now that will do this? Is there another approach that I may be overlooking?
This is my first fully client-side application so it's very possible that I just need to make a paradigm shift, having done server-side applications for so long.
Agreed this can be tricky without a backend framework. For sure script tags are not the way to go and you will need some kind of build tool for production deployment.
Ember App Kit is a solution a few of us have been working on. It's still early stages but i've used it for a couple of projects so far and it's been much better than trying to roll-my-own with grunt. I would expect it to become the default starting point for ember apps in near future, to try it now just download it as a zip then read the Getting Started Guide
There are many other solid solutions out there, consider checking out:
ember-tools
brunch-with-ember-reloaded
brunch-with-hapmsters
charcoal
I use a combination of requirejs and Grunt, using these lovely functions and this one, which can compile your ember-handlebars templates into functions. (The git-contrib includes the ability to watch for changes in your files and perform various build steps which may differ if you are in development or production. You can have separate grunt functions which run various tasks for production or development. Of course for all of this you are going to need node!
I have one site on Django and it runs under VPS from virtualenv (let's call it VE). Now I need to launch another site on the same VPS and now I'm interesting what will be better: install my new site to VE or install new virtualenv for new site?
Thanks!
New VE for each new project, unless you don't have any special reasons. Using one VE for all your projects is the same as not using VE at all. :-)
If your new site will be using apps with different versions (as in VE) than it will be better to make another virtualenv and in this way you will be able to work on both sites by just activating related virtual env without making change to already using virtualenv.
Create new virtualenv.
It may be also a good idea to use different linux and DB users for different sites. This may look like a burden and it is a burden if it is not automated. So automate all these steps :) E.g. with fabric.
Use different VE's for different projects. Who knows if you will eventually need to change a package for just one project?
With separate VE's for each project you can ensure you will not mess the other project's environment.
We're thinking of bringing in a couple of specialists for short-term projects. I'm trying to figure out how to allow them to effectively develop against our code base without releasing the whole code base to them.
Each project has well defined areas they need access to; primarily our main models, together with specific pieces of our app.
We've started to do a better job of breaking up the project into multiple apps within a single django project, but they all still live together in a single git repository. If you check out the repository you get everything.
What are successful strategies for arranging code and repositories such that third parties can access core models and selected functionality without having access to everything?
Note that since this is a somewhat rare need, I'd strongly prefer a setup that doesn't inconvenience our core developers - their lives should be minimally impacted by the setup.
You might try git-submodule as a way of developing each app as its own git repository while still letting developers grab the root and all apps with one "git clone". It's not totally painless though since when you do this any changes to a submodule will need to be committed there and then again in the root repository to reference the new submodule commit. This is probably inescapable, since if you want anyone beside a core developer to be able to commit to an individual app then the app's commits must be independent.