Developing Django projects using Git - django

I am wondering if anyone has experience working on Django projects in a small team (3 in my case), using Git source control management.
The project is hosted on a development server, which is why I am having such a problem. Developers can't see if their code works until they commit their changes to their local repository, then push those changes to the server. Even then, however, git doesn't seem to be updating the files inside the directory holding the repository on the server - probably because it only stores the changes to save space.
We are beginning to tread on each other's toes when working on this project, so some kind of version control is required - but I just can't figure out an solution.
If anyone has overcome a similar problem I'd love to hear how it can be done.

When pushing to a remote repository, best results are when the remote repository is a "bare" repository with no working directory. It sounds like you have a working directory on the remote repository, which will not be updated by Git when doing a push.
For your situation, I would recommend that developers have their own testing environment that they can test against locally before having to push their code anywhere else. Having one central location where everybody needs to push their work before they can even try it will lead to much pain and suffering.
For deployment, I would recommend pushing to a central "bare" repository, then having a process where the deployment server pulls the latest code from the central repository into its working directory.

When you push to a (shared) git repository, it doesn't update that repository's working files. Basically because the working files might be dirty and in that case you'd have to merge--- and for that you need to have full shell access there, which may not be the case in general.
If you want to have the most recent "master" of the shared repo checked out somewhere, you can arrange for that by writing a post-update hook. I'll give an example of one below that I use to check out the "ui" subdirectory and make it available to Apache.
However, I will say that I think your process could be improved. Developers generally need personal servers that they can test on before pushing to a shared point: otherwise that shared repo is likely to be hideously unreliable. Consider, if I push a change to it and it doesn't work, is that my change that broke it or a side-effect of someone else's?
OK, I use this as a post-update hook:
#!/bin/sh
# Should be run from a Git repository, with a set of refs to update from on the command line.
# This is the post-update hook convention.
info() {
echo "post-update: $#"
}
die() {
echo "post-update: $#" >&2
exit 1
}
output_dir=..
for refname in "$#"; do
case $refname in
refs/heads/master)
new_tree_id=$(git rev-parse $refname:ui)
new_dir="$output_dir/tree-$new_tree_id"
if [ ! -d "$new_dir" ]; then
info "Checking out UI"
mkdir "$new_dir"
git archive --format=tar $new_tree_id | ( cd $new_dir && tar xf - )
fi
prev_link_target=$(readlink $output_dir/current)
if [ -n "$prev_link_target" -a "$prev_link_target" = "tree-$new_tree_id" ]; then
info "UI unchanged"
else
rm -f $output_dir/current
ln -snf "tree-$new_tree_id" "$output_dir/current"
info "UI updated"
title=$(git show --quiet --pretty="format:%s" "$refname" | \
sed -e 's/[^A-Za-z][^A-Za-z]*/_/g')
date=$(git show --quiet --pretty="format:%ci" "$refname" | \
sed -e 's/\([0-9]*\)-\([0-9]*\)-\([0-9]*\) \([0-9]*\):\([0-9]*\):\([0-9]*\) +0000/\1\2\3T\4\5\6Z/')
ln -s "tree-$new_tree_id" "$output_dir/${date}__${title}"
fi
;;
esac
done
As mentioned, this just checks out the "ui" subdirectory. That's the ":ui" bit setting new_tree_id. Just take the ":ui" out (or change to "^{tree}") to check out everything.
Checkouts go in the directory containing the git repo, controlled by output_dir. The script expects to be running inside the git repo (which in turn is expected to be bare): this isn't very clean.
Checkouts are put into "tree-XXXX" directories and a "current" symlink managed to point to the most recent. This makes the change from one to another atomic, although it's unlikely to take so long that it matters. It also means reverts reuse the old files. And it also means it chews up disk space as you keep pushing revisions...

Had the same problem, also working with django.
Agree to testing locally prior to deployment, as already mentioned.
You can then push the local version to a new branch on the server. Then you do a merge with this branch and the master. After this you'll see the updated files.
If you accidentally pushed to the master branch, then you can do a git reset --hard. However all changes not commited in the current working branch will be lost. So take care.

Related

I copy/pasted/modified a dependency in my project. This was not a smart dependency management strategy. How do I step back?

Context:
MainProject depends on a header-only dependency Module.
Both MainProject and Module are:
still under development and subject to modifications
modern CMake projects
independent repositories on Github
controlled by me
Problem:
Few months before, I tried without success to manage this dependency using CMake and versioning. Pressed by deadlines, I ended up opting for the "simplest solution" to copy-paste project Module headers in MainProject. Developing MainProject led to add features and modify interfaces in the Module local copy. Now there are two diverging Module.
How it could have worked
It could have worked if Module was very stable (copy/pasting headers is actually the solution I opted for dependencies that are stable and for which I don't have ownership).
I could have modified/commited/pushed/recopy/repasted the Module repository for every modification I wanted to bring. But of course I did not because ... time and deadlines.
Question
Now I would like to step back from this solution (ie, reflect the modifications on the initial Module project) and chose a better dependency management strategy.
What I can think of
create a new branch update on Module git project, copy-paste the modified version, commit it and use git diff to check the differences with branch master
use one or a combination of these three approaches (but I don't know how to chose)
git submodules
git subtrees
C++20 modules
Your Module project literally is a Git submodule: an independently-updated history, from which your MainProject builds use specific revisions.
Developing MainProject led to add features and modify interfaces in the Module local copy. Now there are two diverging Module
Quickest: from a clean checkout of your current MainProject revision,
git tag slice $(git commit-tree -m slice #:path/to/Module)
git rm -rf path/to/Module
git submodule add u://r/l path/to/Module
git push path/to/Module slice
cd path/to/Module
git read-tree -um slice
git commit -m 'Module content from MainProject'
and now you've got your content and ancestry looking serviceable, and you can add labels and push it wherever it needs to go, e.g. git checkout -b MainProjectModule; git push -u origin MainProjectModule
If you've got a long history of Module changes in your main project that you want to preserve in the Module history proper, it's doable, and even fairly efficient, but you'll need to adapt some history surgery to achieve it, instead of tagging a nonce commit, tag the submodule commit that command produces and merge that rather than just adding its tip content as a new commit.

How to remove a repository in Fossil?

Yepp, I'm quite new to Fossil…
During my experiments I've faced a problem: fossil all info command lists all and every repos ever touched here including those removed/deleted/dropped/erased/got-rid-of quite obviously failing like that
************* /home/jno/src/dropped-repo.fossil *****************************************
SQLITE_CANTOPEN: cannot open file at line 36667 of [0c55d17973]
SQLITE_CANTOPEN: os_unix.c:36667: (21) open(/home/jno/src/dropped-repo.fossil) -
fossil: [/home/jno/src/dropped-repo.fossil]: unable to open database file
Yes, the --dontstop flag makes the life a bit easier, but does not fix the things.
So, the question is: how to properly remove a repository?
The only way I found so far is:
fossil close it
remove the repo file itself
run sqlite3 ~/.fossil and delete from global_config where name='…' on all mentions of that repo.
This looks ugly.
I see a new/init command to create a repo, but I see no way to remove it.
PS. The recipie from Fossil: "not a valid repository" - deleted repository (just rm ~/.fossil) looks an overkill.
For the fossil all command to ignore a certain (past or present) repository, you should use fossil all ignore.
In short:
fossil close closes a working directory (by deleting the .fslckout file)
rm /home/jno/src/dropped-repo.fossil actually deletes the repository (only do this if you really want to throw away the entire repository, including all versions)
fossil all ignore /home/jno/src/dropped-repo.fossil removes the repository from the list of repositories that's used by the fossil all command.

How can I search my ENTIRE git repo's commit history for a string change?

I have a website/repo.
Part of my website says:
"Powered by https://myotherwebsite.com/'"
at some point, some troll I had working on the website switched it to say:
"Powered by https://theirwebsite.com"
How can I search the entire repo history to the commit where this change was made.
There have been A LOT of commits/branches over the years.
If you can ignore dead branches and assume that all relevant code is reachable from your most recent master version, I'd recommend using the -S option of git log :
git log -S "theirwebsite"
Take a look at the doc, and maybe consider using regexp search if your actual need is or becomes more complex than what you described here.
Even better : with --all you can search your entire repo (thanks to j6t for the trick!)
git log --all -S "theirwebsite"
(and, as noted by vfalcao, consider using the --name-only option here to list files where this change happened.)

repo2 is a library for repo1

In the moment, I have two github repositories, i.e. repo1 and repo2. Both are two django projects created by our team. In requirements.pipin ~/work_projects/repo1, I have the line
-e git+ssh://git#gitlab.com/repo2.git#de5622dcf0b9a084f9b0a34cdd1d932026904370#egg=repo2
Hence, repo2 becomes a library used by repo1 in ~/.virtualenvs/venv/src (repo1's virtual environment). In the moment, I need to modify both repositories at the same time. My main focus in the moment is that each time I modify repo2, I need to test out the results on repo1. I want to look at the impact of repo2 on repo1 once modified.
I don't want to push my changes on github and reinstall repo2 on repo1 each time I want to see those changes. How could I make it works easily, workaround?
I have a similar setup and I usually install from file (run from console):
pip install -e ../repo2
And since you said git, and you also said you don't want to push, but nothing about committing, here's a version that can install your library from a tag, branch or commit in a local git repo:
pip install -e git+file:~/work_projects/repo2#develop#egg=repo2

git and C++ workflow, how to handle object and archive files?

I use git to interface with an SVN repository. I have several git branches for the different projects I work on.
Now, whenever I switch from one branch to another using 'git checkout ', all the compiled executables and object files from the previous branch are still there. What I would like to see is that switching from branch A to B results in a tree with all object files and binaries from the last time I worked on branch B.
Is there a way to handle this without creating multiple git repositories?
Update: I understand that executables and binaries should not end up in the repository. I'm a bit disappointed in the fact that all the branching stuff in git is useless to me, as it turns out I'll have to clone my proxy git repository for every branch I want to start. Something I already did for SVN and hoped to avoid with git. Of course, I don't have to do it, but it would result in me doing a new make most of the time after switching between branches (not fun).
What you want is a full context, not just the branch... which is generally out of scope for a version control tool. The best way to do that is to use multiple repositories.
Don't worry about the inefficiency of that though... Make your second repository a clone of the first. Git will automatically use links to avoid having multiple copies on disk.
Here's a hack to give you want you want
Since you have separate obj directories, you could modify your Makefiles to make the base location dynamic using something like this:
OBJBASE = `git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/\1\//'`
OBJDIR = "$(OBJBASE).obj"
# branch master: OBJBASE == "master/", OBJDIR == "master/.obj"
# non-git checkout: OBJBASE == "", OBJDIR == ".obj"
That will but your branch name into OBJBASE, which you can use to build your actual objdir location from. I'll leave it to you to modify it to fit your environment and make it friendly to non-git users of your Makefiles.
This is not git or svn specific - you should have your compiler and other tools direct the output of intermediate files like .o files to directories that are not under version control.
To keep multiple checkouts of the same repo, you can use git --work-tree.
For example,
mkdir $BRANCH.d
GIT_INDEX_FILE=$BRANCH.index git --work-tree $BRANCH.d checkout $BRANCH
You could set your IDE compiler to generate all private temporary files (.class and so on) in <output>\branchName\....
By configuration your compilation setting branch by branch, you can register the name of the branch in the output directory path.
That way, even if though private files remain when you git checkout, your project on the new branch is ready to go.
In the contrib/ directory of the git distribution, there is a script called git-new-workdir that allows you to checkout multiples branches in different directories without cloning your repository.
Those files aren't tracked by Git or Subversion, so they're left alone on the assumption that they are of some use to you.
I just do my checkouts in different directories. Saves me the trouble of doing cleanup.
A make clean should not be necessary because files that are different between different branches get checked out with the actual date!!!
This means that if your Makefile is correct, only those object-files, libs and executables are compiled again that really changed because of the checkout. Which is exactly the reason a makefile is there in the first place.
The exception is if you need to switch compiler options or even compilers in different branches. In that case probably git-new-workdir is the best solution.
If the compiled executables are files that have been checked in
then git stash solves the problem.
[compile]
git stash save "first branch"
git checkout other_branch
[Fiddle with your code]
[compile]
git stash save "second branch"
git checkout first_branch
git stash apply [whatever index your "first branch" stash has]
# alternatively git stash pop [whatever index...]
If the compiled executables are files that have not and will not be checked in
then simply add them to .gitignore