After a long time away from an app i wrote in Django and didn't complete, I've come back to it on a new Mac.
I'm struggling to get the code to refer to the apps and the files within them without the explicit path. For instance:
from myproject.app.file import object
Whereas I remember not having to use myproject every time.
Is this something that has changed? I seem to remember being about to add to the path in manage.py which is called every time you run the dev server, but this hasn't worked this time.
sys.path.append /path/to/myproject
Should that fix the issue I'm having?
I started with a simple answer and it grew into more details on how to add subdirectories of your project to python path. Maybe a bit off-topic, but it could be useful to you so I'm pushing the post button anyway.
I usually have a bunch of small re-usable apps of mine I keep inside my project tree, because I don't want them to grow into independent modules. My projet tree will look like this:
manage.py
myproject/apps
myproject/libs
myproject/settings
...
Still, by default, Django only adds the project root to python path. Yet it makes no sense in my opinion to have apps load modules with full path:
from myproject.apps.author.models import Author
from myproject.libs.rest_filters import filters
That's both way too verbose, and it breaks reusability as I only use absolute imports. Not to mention if I someday build an actual python package out of some of the libs, it will break.
So, I took the following steps. I added the relevant folders to the path:
# in manage.py
root = os.path.dirname(__file__)
sys.path.append(os.path.realpath(os.path.join(root, 'myproject', 'apps')))
sys.path.append(os.path.realpath(os.path.join(root, 'myproject', 'libs')))
But you must ensure those packages cannot be loaded from the root of the project, or you will have odd issues as python would load another copy of the module. For instance, isinstance(libs.foo.bar(), myproject.libs.foo.bar) == False
It's not hard though : just remove __init__.py from the folders you add to the path. That will make sure they cannot be descended into from the project.
Also, Django's discover runner will not descend into those paths unless you specify them manually. That may be fine with you (if every module has its own test suite). Or you can extend the runner, so it knows about this: sample code.
Related
During a pytest fixture, what is the best way to robustly get the location of a text file for users that may specify different working directories at runtime e.g. I want a person using the cmd line in the test fixture directory find the file as well as an integration server which may work in the project's root. Can I somehow include the text file in a module? What are best practices for including and getting access to non .py files?
I am aware of BASE_DIR = os.path.dirname(os.path.dirname(__file__)), but I am not sure if this will always refer to the same directory given a particular way of running the test suite.
os.path.dirname(os.path.abspath(__file__)) (which is what I think you meant above) worked fine for me so far - it should work as long as Python can figure the path of the file, and with pytest I can't imagine a scenario where that wouldn't be true.
I'm trying to deploy a django app to heroku.
I have several python libraries which are not on PyPi and so I can't just declare them in requirements.txt file
In local development I've used:
import sys
sys.path += [os.path.dirname(os.path.dirname(__file__))+"\\project-name\\lib"]
inside manage.py and it works fine there.
Obviously it doesn't work on heroku and I get import errors.
What is the recommended way to add libraries like that on heroku?
Thanks.
One way to do it is include the libraries in the repository itself, from which you can import them. That means simply moving the actual folder for each library into your main Django project folder.
- DjangoProject
- AppFolder1
- AppFolder2 ...
- python-library1
- python-library2
When the repository is pushed to Heroku your libraries will be pushed as part of the project.
From here, your imports of these libraries within a view/model etc within any app's folder would
import python-library1
from python-library2 import a_function, a_class
The reason why I suggest the directory structure above is that, most likely, you would not have go back and change any import codes.
If you have a large number of libraries and would like to keep the direcory structure simpler, simply create a folder with a name such as "importables" in the main DjangoProject folder and change the import statements to something such as...
from importables import python-library1
from importables.python-library2 import a_function, a_class
It's not exactly beautiful, but a quick way to get the job done. If you aren't sure where the libraries you'd like to include are located, there's a few ways to quickly see their location using Python (How do I find the location of Python module sources?).
Here's the scenario:
I'm running Django 1.3.1, utilizing staticfiles, and django-compressor (latest stable) to, among other things, compile LESS files.
I have an "assets" directory that's hooked into staticfiles with STATICFILES_DIRS (for project-wide static resources). In that directory I have a "css" directory and in that a "lib.less" file that contains LESS variables and mixins.
So the physical path is <project_root>/assets/css/lib.less and it's served at /static/css/lib.less.
In one of my apps' static directory, I have another LESS file that needs to import the one above. The physical path for that is <project_root>/myapp/static/myapp/css/file.less and it would be served at /static/myapp/css/file.less.
My first thought was:
#import "../../css/lib.less"
(i.e. based on the URL, go up to levels from /static/myapp/css to /static/, then traverse down into /static/css/lib.less).
However, that doesn't work, and I've tried just about every combination of URLs and physical paths I can think of and all of them give me FilterErrors in the template, resulting from not being able to find the file to import.
Anyone have any ideas what the actual import path should be?
After tracking down exactly where the error was coming from in the django-compressor source. It turns out to be directly passed from the shell. Which clued me into removing all the variables and literally just trying to get the lessc compiler to parse the file.
Turns out it wants a relative path from the source file to the file to be imported in terms of the physical filesystem path. So I had to back all the way out to my <project_root> and then reference assets/css/lib.less from there. The actual import that finally worked was:
#import "../../../../assets/css/lib.less"
What's very odd though is that lessc would not accept an absolute filesystem path (i.e. /path/to/project/assets/css/lib.less). I'm not sure why.
UPDATE (02/08/2012)
Had a complete "DUH" moment when I finally pushed my code to my staging environment and ran collectstatic. The #import path I was using worked fine in development because that was the physical path to the file then, but once collectstatic has done it's thing, everything is moved around and relative to <project_root>/static/.
I toyed with the idea of using symbolic links to try to match up the pre and post-collectstatic #import paths, but I decided that that was far too complicated and fragile in the long run.
SO... I broke down and moved all the LESS files together under <project_root>/assets/css/, and rationalized moving the LESS files out of the apps because since they're tied to a project-level file in order to function, they're inherently project-level themselves.
I'm sort of in the same bind and this is what I've come up with for the most recent versions of compressor and lessc to integrate with staticfiles. Hopefully this will help some other people out
As far as I can tell from experimenting, lessc doesn't have a notion of absolute or relative paths. Rather, it seems to maintain a search path, which includes the current directory, the containing directory of the less file, and whatever you pass to it via --include-path
so in my configuration for compressor I put
COMPRESS_PRECOMPILERS = (
('text/less', 'lessc --include-path=%s {infile} {outfile}' % STATIC_ROOT),
)
Say, after running collectstatic I have bootstrap living at
STATIC_ROOT/bootstrap/3.2.0/bootstrap.css.
Then from any less file, I can now write
#import (less, reference) "/bootstrap/3.2.0/bootstrap.css"
which allows me to use the bootstrap classes as less mixins in any of my less files!
Every time I update a less file, I have to run collectstatic to aggregate them in a local directory so that compressor can give less the right source files to work on. Otherwise, compressor handles everything smoothly. You can also use collectstatic -l to symlink, which means you only need to collect the files when you add a new one.
I'm considering implementing a management command to smooth out the development process that either subclasses runserver to call collectstatic each time the server is reloaded, or uses django.utils.autoreload directly to call collectstatic when things are updated.
Edit (2014/12/01): My approach as outlined above requires a local static root. I was using remote storage with offline compression in my production environment, so deployment requires a couple extra steps. In addition to calling collectstatic to sync the static files to the remote storage, I call collectstatic with different django config file that uses local storage. After I have collected the files locally, I can call 'compress', having configured it to upload the result files to remote storage, but look in local storage for source files.
I have written a Django project which runs fine on the dev server. Although I am the only one working on it, I realized I should get some sort of version control going and placed the project on a network SVN. I assume the new process for working this project should be: checkout code from SVN, make changes, test, update, and commit back to SVN.
However, when I check out the code and attempt to run the dev server from the checked out SVN location, I get an error that "has no module named yadda.yadda". But, the module is there and it works fine when I run it from the original directory. Any thoughts? Any suggestions on a better work process?
Does your $PYTHONPATH shell variable reference a relative path? If so, the relative path may be incorrect when you change to the new directory.
You can also diff the contents of the two directories to ensure that your versioned copy is a duplicate of the original.
Check if your module has init.py
As George said check if module is on python path (import sys and print sys.path)
I'm aware I can "store it anywhere in my python path" and all, but what's an organized pattern I can use to store middleware classes for my project?
I am appending my project root directory and project directory to the sys path through mod_wsgi:
sys.path.append( '/srv/' )
sys.path.append( '/srv/workarounds/' )
The latter line being the project root. As an example, let's say I want to apply this middleware class: http://djangosnippets.org/snippets/1179/
Would I just copy the snippet contents into a middleware.py file and dump it in my project root? Create a directory for middleware, add that directory to my python path?
My usual layout for a django site is:
projects/
templates/
common/
local/
Where:
projects contains your main project and any others
common contains things you may share across sites, or are at least not project-specific, like if you need to download django-profile and django-registration rather than having it directly in python/site-packages
templates contains just that
local contains things that are going to be specific to the current machine, so that you can have properly separated data, like database location and password - I then soft-link the machine-specific versions (say "machine1-localconfig.py") to local/localconfig.py and then can "import localconfig" in settings.py
I generally put middleware that's project-specific inside a project, and middleware that's not project-specific in common/middleware/
Make sure to add the templates directory to the right place in settings (or most probably, localconfig.py and then import it in settings), and makse sure to add the projects, common, and local directories to your PYTHONPATH.
If you have only a couple of tightly coupled middleware classes put them into a middleware.py module under the app root. (This is how the django.contrib apps do it - see the sessions' app middleware here).
If you have many varying middleware classes, create a middleware pacakge with submodules of the related middleware classes. Though if you do end up in this situation, consider how you can refactor your project into several mini-apps that all solve a specific need (and open source them :)).
Personally, I have a common django package where I dump common middleware (like the your linked LoginRequiredMiddleware class) into a middleware package. If this makes sense in your project's context I would highly suggest it. It's saved my countless hours of duplication, and bug fixing. django-common and django-annoying are good examples of this kind of project layout