I am using modified sources from few third party apps in my project. I would like to put these third party apps in a separate directory, so that they are not on the same directory level as my own apps. Is this possible in django?
I tried simply putting the apps in a directory thirdparty and changed my INSTALLED_APPS like so:
INSTALLED_APPS = (
'my_app',
...
'thirdparty.django_messages',
This of course failse with:
ImportError: No module named thirdparty
After which I naturally added __init__.py to the directory. But it fails again:
ImportError: No module named django_messages.apps
Just to avoid any confusion, the app django_messages does contain apps.py
Is there a way to group django apps in a directory or do they all have to be in the same project root directory?
Edit
A better alternative is in the accepted answer by Antoine Pinsard
For those persistent on grouping apps see accepted answer here!
Don't do this. If you really need to modify the source code of third-party apps, fork the repositories so that you will be able to watch and merge upstream updates.
Then install the modified apps with pip.
For instance, if you have forked django-autocomplete-light on your github (let's say https://github.com/dsalaj/django-autocomplete-light):
pip install git+ssh://git#github.com/dsalaj/django-autocomplete-light.git
You will be able to upgrade it like any other pip package:
pip install --upgrade git+ssh://git#github.com/dsalaj/django-autocomplete-light.git
And even add it to your requirements.txt.
As Mad Wombat mentioned in the comments, you can use pip's --editable (-e) option to install these packages in a specific folder within your project. From pip help:
-e, --editable Install a project in editable mode (i.e. setuptools "develop mode") from a local project path or a VCS url.
Nevertheless, to answer the question. The issue is that the app django_messages considers it is a top-level module (and it is supposed to be). Thus it can import its submodules using an absolute python path (starting with django_messages.). However, when you place it within a module thirdparty, django_messages becomes a submodule of thirdparty. You could add the thirdparty directory to your PYTHON_PATH so that django_messages is available as a top-level module. But it is really not advisable to do so. lib/pythonX.Y/site-packages is the best place for your third party packages and this is where pip installs them.
You may also be interested in python virtualenvs if you don't know what they are.
Related
I am trying to import requests_toolbelt package in google app engine but keep getting import error. Already checked at https://toolbelt.readthedocs.io/en/latest/adapters.html#appengineadapte and https://cloud.google.com/appengine/docs/standard/python/issue-requests also gives same error.
It works fine on local but after deploying, the error appears:
ImportError: No module named requests_toolbelt.adapters
I have this:
import requests
from requests_toolbelt.adapters import appengine
if not os.environ.get('SERVER_SOFTWARE', '').startswith('Google App Engine'):
appengine.monkeypatch()
requirements.txt has
requests
requests_toolbelt
Since you are using Python2.7 in the Standard environment, having the requests_toolbelt library in the requirements.txt file is not enough to upload it to App Engine, since it is not one of the Built-in Third-party Libraries.
To add it, you can follow this steps, as mentioned in the official documentation:
Run the following command:
pip install -t lib -r requirements.txt
This will install all the packages to the local environment, and copy them to the lib folder afterwards. Documentation on this.
Create the following file, named appengine_config.py:
from google.appengine.ext import vendor
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
Note that this file needs to be in the same root path as the app.yaml, and the 'lib' string repersents the path from this root to the folder you created in the previous point.
Redeploy the application with gcloud app deploy
Once that is done, you should be able to run the application without errors related to the library.
As a side note, these steps are only a requirement in Python 2.7 in the Standard App Engine environment. In Python3 or in Flexible, having the libraries listed in the requirements.txt file is enough.
I was recommend also to add the "lib" folder to the IDE Interpreter.
In IDE of IntelliJ like PyCharm you go to settings -> project -> project interpreter -> click on settings small button -> show all
Then click on folders icon:
Then click on add icon:
Then pick your lib folder
I know this is perfectly possible and a lot of people already doing it, but my problem is slightly different and I couldn't figure out the solution yet:
Let a 3rd party Django App's structure as below:
django-module
module
init.py
views.py
models.py
requirements.txt
setup.py
I want to bundle only the module directory as a submodule, because then I can access views.py file just by typing "module.views". If I import the django-module directory, I would have to write django-module.module.views to access the module files, which is not feasible.
My purpose is to modify the module and make pull requests occasionally to the original repository. Is there workflow that I can follow, or what are the best practices for this purpose?
Pip tips
Pip has support for editable packages and retrieving packages with git, so you could create a virtualenv, use pip to install the packages, and update them using pip when you want.
So you could add:
-e git://git.myproject.org/MyProject.git#da39a3ee5e6b4b0d3255bfef95601890afd80709#egg=MyProject
To your requirements.txt to retrieve that exact commit.
Suggested workflow
I think that the best way to solve your issue is the following:
Make a private fork of the package
Edit the package in a specific development branch in the forked repository
Use the package from your fork's development branch in your requirements file.
When you feel like it, update the forked package where you're using it using pip.
When you're ready to make a pull request, pull origin, rebase your working branch to origin/master, and make the pull request from the master branch of your fork.
This means you have three places where the code is present:
The original repo (where you don't have access)
Your forked repo (where you work on your fork)
Wherever pip installed it (where you use your fork)
The best practice is to keep the submodules exactly as they are.
Once you've added a 3rd party module to your app as a submodule, the next step is to make sure you add that "django-module" directory to your python path. As long as django-module is on your python path, you will be able to use the submodule as you usually would by typing "module.views" as you wish.
I am finally going to start using virtualenv for my Django projects on my development machine. Before I start I want to know if there are any special considerations for dealing with my existing projects. My presumed workflow is something like:
make a new virtualenv
activate the new virtualenv
Install Django in there
pip install all the packages I know I need for my existing project
copy my Django project files, app files, and git files into the project folder within the virtualenv.
Edit
6. make requirements file for deployment
This is obviously very simplified but are there any steps or considerations I am fundamentally missing? Is git going to be happy about moving? Also is it best practice to have a separate virtualenv for each Django project?
I know this is not a typical code problem, but I hope those that know more than I do can point me in the right direction.
Many thanks.
I don't see any big issue on migrating your projects and I think your 5-steps plan is correct, in particular, for steps 3/4/5 (I'd merge them), you can handle project dependencies with pip, possibly using requirement files.
Requirement files are plain text files telling to pip which packages have to be installed in your virtualenv, included your git-tracked projects which eventually can be deployed in your virtual environment as development eggs (they bring with them version control infos).
Once you have a req file, it's a matter of:
pip install -r file.req
to have all needed packages installed in your env.
As you can see from virtualenv docs, a typical req file would contain something like:
django==1.3.0
-e git://git.myproject.org/MyProject.git#egg=MyProject
I usually keep each project in its own virtualenv, so I can deploy it to the production server the same way I do for local development.
I am quite a django n00b, but I am reading the eyes out of my head to get it all going. I have a PHP background and struggle with the way and location of reusable apps.
I thought that installed apps should go in an App folder (example django-registration or django-profiles), but after I PIP the app in my virtualenv, I see that the app is installed in a Django folder names "site packages".
Is this the default behavior? Should I copy the 'registration' or 'profile' folder from site packages to my Project? or should I leave them there
Thanks for the help.
If you're intending to simply install packages and not amend their code, there's no problem with them living in Python's site-packages dir.
Because you're using virtualenv, the packages installed while that virtualenv is active will be stored in:
/path/to/virtualenvs/myvirtualenv/lib/python2.x/site-packages/
And it's completely fine for them to stay there. As Daniel R says, what matters is that they are your PYTHONPATH, and virtualenv takes care of making sure they are.
Custom apps you write go in your project. Installed apps you just want to import from into your custom apps can stay in the site-packages folder.
This has nothing to do with Django. This is where Python installs packages. Django doesn't care where they are, as long as they're on the Pythonpath (which they are if they're in site-packages).
I'm still busy with my Django learning adventure. In another post I asked about how to structure a Django project and applications using buildout. In the details of doing this arose another issue, simply installing 3rd party Django applications using either easy_install or setup.py. My question is, where should you install a Django application? If looking at Django documentation, one would think to put a Django application inside the project folder. But if your Django application is an egg (a mystifying term in my opinion) and you use easy_install without option '-b' (build-directory) the application will be installed into your current python site-packages directory. Using option '-b' will put a copy of the application in your directory, but still will install it in your current site-packages directory. Then there are other options like --install-dir and prefix. Also how should installation happen when using setup.py which have similar options as buid-directory, install-dir, and prefix?
Is there a 'good practice' standard for installing 3rd party Django applications into a Django project?
Thank a lot,
Todd
They usually aren't installed directly into the project. They're either installed into the system's site-packages/ directory, or in the virtualenv's site-packages/ directory, or in some other well-defined place that the sysadmin has set for this purpose.
This is where virtualenv comes into its own. It basically enables a project-specific site_packages directory, where you can install all the third-party applications that relate to your project. I'd definitely recommend it.
Follow these steps :
change the path according to your local setup
C:\Python27\Lib\site-packages>python pip install django
Create Project
Go to folder where you want to create a project
E:\djangoProject>C:\Python27\Lib\site-packages\django\bin\django-admin.py startproject myproject
python manage.py help is used to list all the commands
Manage.py This file is kind of your project local django-admin for interacting with your project via command line (start the development server, sync db...)
Run Server
E:\djangoProject\myproject>python manage.py runserver
Create App
E:\djangoProject\myproject>python manage.py startapp myapp
Go to myproject settings.py and register your app “myapp” created under INSTALLED_APPS
Migrate DB E:\djangoProject\myproject>python manage.py migrate
Migrate will create necessary tables or collections depending on your db type, necessary for the admin interface to run