Background:
Apparently my question regarding the best ways to build an autoloader in Python was too broad (https://stackoverflow.com/questions/25678974/python-autoloader). Not sure why...
I had hoped to get a few code snippets to make my life easier with regards to choosing between importing modules or building a package. But not so much. So I decided to try to build a package of python functions meant to emulate PHP functions such str_replace.
I've named the package p2p and the directory structure is like this:
lib/p2p
setup.py
CHANGES.txt
MANIFEST.IN
README.txt
/bin (empty)
/docs (empty)
/p2p
__init__.py
/str
__init__.py
str.py
/time
__init__.py
time.py
/json
__init__.py
json.py
/utils
__init__.py
utils.py
The __init__.py are all empty.
In the Console:
Now when I build the package (python setup.py install) I get no errors. When I import p2p I get no errors. However, when I try to use p2p.time.timeID() I get:
AttributeError: 'module' object has no attribute 'timeID'
When I type p2p.time I get:
module 'p2p.time' from '/home/gabe/anaconda/lib/python2.7/site-packages/p2p/time/init.pyc'
I've Googled the attribute error and spent most of yesterday reading up on how to create a python package. What am I doing wrong? (Ultimately this appears like it would have been easier with simple modules and an autoloader. I'm not sure how to build an autoloader that will properly import the modules, due to relative path restrictions in Python, without just dumping my entire library directory structure into the path.)
lib/p2p/setup.py
from distutils.core import setup
setup(
name='p2p',
version='0.1.0',
author='Me',
author_email='me#example.com',
packages=['p2p', 'p2p.test', 'p2p.str', 'p2p.time', 'p2p.json', 'p2p.utils'],
package_dir={'p2p':'p2p', 'str':'p2p/str'},
scripts=[],
url='',
license='LICENSE.txt',
description='PHP to Python functions.',
long_description=open('README.txt').read(),
install_requires=[
"",
],
)
lib/p2p/p2p/time/time.py
import time
# http://www.php2python.com/wiki/function.microtime/
def timeID():
return time.time()
Notes:
Ubuntu 14.04 Dekstop
Anaconda with Python 2.7
Spyder
I'm a complete Python Noob but have worked in PHP and Ruby for a couple of years now
p2p/time/time.py
Well there you go. You want p2p.time.time.timeID instead. Or you could just put it in p2p/time/__init__.py instead.
Related
I'm building a Django project with Python 3.6. I have created this directory structure ...
project
- manage.py
- scripts
- run_commands.py
- commons
- util
- __init__.py
- my_class.py
The contents of init.py are
from . import my_class
In another class, I attempt to import my MyClass like this
from commons.util import MyClass
but I'm getting this error
ModuleNotFoundError: No module named 'commons'
Am I creating my init.py properly?
It looks like the problem is that MyClass is not located in commons.util, because you only imported the module named my_class, not the class itself.
Instead the file commons/util/__init__.py should contain the following import:
from .my_class import MyClass
I don't think this will solve your problem, because you would be getting a different error than the one shown, but you will get errors for this eventually.
Update
First, I'd recommend reading this answer for a good explanation for how imports work in python.
Basically, when you execute from commons.util import MyClass, the interpreter scans the contents of sys.path for a module named commons.
I assume you didn't set sys.path to include your project folder, hence the ModuleNotFoundError.
TLDR; you have 2 options:
Manually set sys.path in run_commands.py to check your project folder (Don't do this!)
Use Django's Command class
To use Django's Command class, you will need to adjust your project folder similar to the following:
project
- manage.py
- commons
- management
- commands
run_commands.py
- util
- __init__.py
- my_class.py
Now in run_commands.py:
from django.core.management.base import BaseCommand
from commons.util import MyClass
class Command(BaseCommand):
def handle(*args, **options):
print("handling", MyClass)
You can execute the new command with the following:
python3 manage.py run_commands
It used to be the case that yes, you need to put an __init__.py in every directory that is going to be treated as a python module as without an __init__.py python wouldn't look inside that directory for importable code.
- project
- __init__.py
- commons
- __init__.py
- util
- __init__.py
- my_class.py
But as Reinstate Monica points out below this is no longer true as of Python 3.3+. So, depending on your version of Python you will need to make an informed decision.
Note, you might or might not need an __init__.py in the root project directory (if you need them at all), it depends if it has definitions that are part of the source tree. But you won't need it if it's just a container, like you see in the way most Django projects are organised, for example.
https://docs.python.org/3/tutorial/modules.html
https://docs.python.org/3/tutorial/modules.html#packages
I am using line_profiler module to line-by-line profile my python code. At some point in the code, I use importlib.import_module command, and it fails while all the other imports function fine. If I run the line in PyCharm and not in the shell, everything works fine.
This is the file structure:
../
core/
__init__.py
battery_meta.py
battery_models.py
models/
__init__.py
battery_model_07.py
kernprof.py
I am running a command:
python kernprof.py -l core/battery_models.py -v
And the following line in battery_meta.py
module = importlib.import_module('battery_model_07', 'models')
results in an error:
ImportError: No module named battery_model_07
I have tried so far:
Adding the project directory to the path directly before the import, like this: sys.path.append(<project_path>)
Adding __init__.py file in the project directory
Both with no result... again, all the other imports work fine.
Does anyone have any idea what I might be doing wrong and how to fix it?
I know this question has already been asked multiple times but I still can't seem to find a solution that works for me. My Django project folder looks like this:
my_project
__init__.py
manage.py
my_first_app
my_second_app
core
Now the "core" folder looks like this:
__init__.py
some_other_stuff.py
settings
__init__.py
prod.py
dev.py
local.py -> dev.py
local.py is a symbolic link pointing to the right settings file, dev.py on my machine, prod.py in production.
Here's my problem: when I try to use manage.py, I get a weird error ImproperlyConfigured: The SECRET_KEY setting must not be empty. When I pass the path to the settings file local.py as an argument (--settings=core.settings.local) it runs fine. I figured the problem was that Django didn't know where to look for the settings file. How can I tell him (her?) where to look?
I already tried exporting the path to the env (export DJANGO_SETTINGS_MODULE=core.settings.local) and setting the PYTHONPATH to the parent directory, to no avail.
The primary use of __init__.py is to initialize Python packages. The easiest way to demonstrate this is to take a look at the structure of a standard Python module.
package/
__init__.py
file1.py
file2.py
As you can see in the structure above the inclusion of the __init__.py file in a directory indicates to the Python interpreter that the directory should be treated like a Python package
__init__.py can be an empty file but it is often used to perform setup needed for the package(import things, load things into path, etc).
One common thing to do in your __init__.py is to import selected Classes, functions, etc into the package level so they can be convieniently imported from the package.
In our example above we can say that file.py has the Class File. So without anything in our __init__.py you would import with this syntax:
from package.file import File
However you can import File into your __init__.py to make it available at the package level:
# in your __init__.py
from file1 import File
# now import File from package
from package import File
Source
So for conclusion, when you call import in __init__.py in a package mypackage, it's like you use package as a simple python file, that's what my solution do:
from .local import * in __init__.py
I haven't use this before in the settings case but I use it when I wanna to subdivide my models in a Django app, models.py --> models package, ./manage syndb doesn't discover my models declared, I found so this solution that's similar. You can find more details here
Last thing, I'm sure there's others solution to your problem, but this can be the most simple.
Good luck
You are in import hell somewhere. Had this problem too one time. The only way to find out where the root of your problem is, might be to disable all apps, try starting the server, enable the first app, start the server, enable the next etc.
BTW: your project layout should not be used from Django 1.4 onward. https://docs.djangoproject.com/en/dev/releases/1.4/#updated-default-project-layout-and-manage-py
I'd try to use the new layout and hope that it 'just works'.
I think you need to change the name of the file that the manage.py is looking for.
try:
imp.find_module('settings') # Assumed to be in the same directory.
except ImportError:
import sys
sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. It appears you've customized things.\nYou'll have to run django-admin.py, passing it your settings module.\n" % __file__)
sys.exit(1)
If you had the settings.py file in the same directory simple changing the 'settings' to 'local' would have worked.
But, since you have it in a different directory, I think you need to configure the settings. Refer to this: https://docs.djangoproject.com/en/dev/topics/settings/#using-settings-without-setting-django-settings-module
from django.conf import settings
settings.configure(DEBUG=True, TEMPLATE_DEBUG=True,
TEMPLATE_DIRS=('/home/web-apps/myapp', '/home/web-apps/base'))
Hope that helps.
I am trying to link my Sphinx documentation with ReadtheDocs. I can build the documentation locally but when I try to have ReadtheDocs automatically generate the documentation I get the following error:
Sphinx Standard Error
Making output directory...
Exception occurred:
File "/var/build/user_builds/mousedb/checkouts/latest/Docs/source/conf.py", line 25, in <module>
from mousedb import settings
ImportError: No module named mousedb
The full traceback has been saved in /tmp/sphinx-err-n_8fkR.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
Either send bugs to the mailing list at <http://groups.google.com/group/sphinx-dev/>,
or report them in the tracker at <http://bitbucket.org/birkenfeld/sphinx/issues/>. Thanks!
My project name is mousedb. I don't understand why I get this import error in the auto-build but not locally.
Update
Based on the comments I think that this is an issue for importing my settings file into a sibling Docs directory. Rather than doing an absolute import (as I had been doing) I should be doing a relative import based on the location of settings.py and conf.py.
I want to import my settings file into my conf.py with the following directory structure:
-mousedb
--settings.py
-Docs
--source
---conf.py
--build
You originally talked about a "local absolute path to my code" and now about setting up relative paths to your code. This probably means you're not using a setup.py file and also not a virtualenv.
In the same directory as Docs/ and mousedb/, add a setup.py:
from setuptools import setup
setup(name='mousedb',
version='0.1',
description="My sample package",
long_description="",
author='TODO',
author_email='todo#example.org',
license='TODO',
packages=['mousedb'],
zip_safe=False,
install_requires=[
'Django',
# 'Sphinx',
# ^^^ Not sure if this is needed on readthedocs.org
# 'something else?',
],
)
After committing/pushing/whatever this file, you can go to your readthedocs settings for your project. Enable the "use virtualenv" setting. This will "nstall your project inside a virtualenv using setup.py install".
The end result is that everything python-related that readthedocs does will have your project in it's sys.path.
The probable cause of your problems is that you run sphinx from within your "root" directory on your local system, where python magically finds the mousedb/ package in your current directory. But readthedocs apparently runs it from within the Docs/ directory and thus cannot find mousedb.
I have a Django app that I've set up using Buildout laid out like so:
/workspace
/bin
/src
/myproject
settings.py
/myapp
views.py
...
bootstrap.py
buildout.cfg
setup.py
The issue is that I'd like both myproject.settings and myapp on the python path. I need the myproject.settings on the path so djangorecipe can import it. And I'd like myapp on the path so that I don't have to write import myproject.myapp all the time.
For now I've got both /workspace/src and /workspace/src/myproject in the Python path, but this feels like a hack and practically makes me worried if there might be situations where import some_module might have confusing resolution patterns because I have two directories that are parent-child to each other.
So questions are:
Is there an accepted way to lay this out?
Is it actually bad to have a directory and one of its sub-directories in the path?
There is no problem, on import some_module importer will search in each folder specified at sys.path for some_module/__init__.py and some_module.py. Same for import myproject.some_module, it will search for myproject module, then it will try to find in it some_module with same algorithm.
I'm using the same project structure.
If your buildout.cfg includes develop = . and whatever egg your setup.py defines is included as a dependency for your buildout/parts then whatever code path your setup.py defines will be automatically added to sys.path. Just make sure your setup.py includes src as a code directory. One way to do this is with:
setup(name=...
...
packages=find_packages('src'),
package_dir = {'':'src'},
...