Project structure:
/lib/modules/mod1.py
/mod2.py
/subdir1/subdir2/mod3.py
/configs/config.yaml
mod3.py imports mod2.py. mod2.py imports mod1.py. mod1.py loads configuration files that are at a relative path to mod2.py using os.getcwd().
The problem is that when mod3.py imports mod2.py, mod1.py attempts to load the config files from a path relative to mod3.py (i.e. /subdir1/subdir2/configs/config.yaml instead of /configs/config.yaml)--this, of course, doesn't work.
I believe understand why this isn't working (os.getcwd() get the path of the originally executed file).
How can I fix this so that mod1.py will use a path relative to mod2.py even when mod2.py is imported from mod3.py?
I haven't been able to find a built-in way to do this in Python, so what I ended up doing is this:
mod1.py:
configs_list = os.getcwd().split('/')
for x in configs_list:
# Check each directory in list, bottom up. 'pop()' list on
# each failure. Assign var and break loop when configs path is found.
if not os.path.exists('/'.join(configs_list) + '/configs'):
configs_list.pop()
else:
configs_path = '/'.join(configs_list) + '/configs'
break
configs_path is then used to prefix the specific configuration file name(s) in mod1.py. Since every call to mod1.py will occur from within a project's directory structure, and every project has only one configs directory, this should (and has so far) correctly identified the configs directory regardless of where in the project the given script is being run from.
I'm open to better or more Pythonic ways of doing this, if anyone has input.
Related
I have my project where i am using filesystem to retrieve directory of assets.
When i am lunching my program in editor(im using Visual Studio 2019) everything is fine and this code return value of working direcotry of project.
std::string currentPath = std::filesystem::current_path().string();
But when i am lunching app from .exe file this line of code returns path that leads to .exe file.
The same directory called $TargetPath in properties in VS.
So my question is why is that happening and how can i resolve this problem.Becouse of that i cannot automatically load assets when lounching app from .exe file
Because it gives the current working directory, which is set by the environment calling your program (unless your program explicitly changes it).
So, it does what it's designed to do, gives the current working directory:
Returns the absolute path of the current working directory,
So my question is why is that happening
It happens because you've configured the editor to set the working directory to one path, while you're running the program with another working directory outside the editor.
how can i resolve this problem.Becouse of that i cannot automatically load assets when lounching app from .exe file
Here is an approach:
Store the assets in a path that is relative to the exe.
Get path to the exe.
On POSIX, you can use argv[0] from arguments of main
On Windows, the documentation recommends GetModuleFileNameW
Get canonical absolute form of that path (make sure that working directory hasn't been changed before this step if the path to exe is relative).
Get the directory that contains the exe from that canonical path.
Join that directory path with the asset's relative path to get an absolute path to the asset
Load the asset using the absolute path.
Assume I have the following files,
pkg/
pkg/__init__.py
pkg/main.py # import string
pkg/string.py # print("Package's string module imported")
Now, if I run main.py, it says "Package's string module imported".
This makes sense and it works as per this statement in this link:
"it will first look in the package's directory"
Assume I modified the file structure slightly (added a core directory):
pkg/
pkg/__init__.py
plg/core/__init__.py
pkg/core/main.py # import string
pkg/string.py # print("Package's string module imported")
Now, if I run python core/main.py, it loads the built-in string module.
In the second case too, if it has to comply with the statement "it will first look in the package's directory" shouldn't it load the local string.py because pkg is the "package directory"?
My sense of the term "package directory" is specifically the root folder of a collection of folders with __init__.py. So in this case, pkg is the "package directory". It is applicable to main.py and also files in sub- directories like core/main.py because it is part of this "package".
Is this technically correct?
PS: What follows after # in the code snippet is the actual content of the file (with no leading spaces).
Packages are directories with a __init__.py file, yes, and are loaded as a module when found on the module search path. So pkg is only a package that you can import and treat as a package if the parent directory is on the module search path.
But by running the pkg/core/main.py file as a script, Python added the pkg/core directory to the module search path, not the parent directory of pkg. You do have a __init__.py file on your module search path now, but that's not what defines a package. You merely have a __main__ module, there is no package relationship to anything else, and you can't rely on implicit relative imports.
You have three options:
Do not run files inside packages as scripts. Put a script file outside of your package, and have that import your package as needed. You could put it next to the pkg directory, or make sure the pkg directory is first installed into a directory already on the module search path, or by having your script calculate the right path to add to sys.path.
Use the -m command line switch to run a module as if it is a script. If you use python -m pkg.core Python will look for a __main__.py file and run that as a script. The -m switch will add the current working directory to your module search path, so you can use that command when you are in the right working directory and everything will work. Or have your package installed in a directory already on the module search path.
Have your script add the right directory to the module search path (based on os.path.absolute(__file__) to get a path to the current file). Take into account that your script is always named __main__, and importing pkg.core.main would add a second, independent module object; you'd have two separate namespaces.
I also strongly advice against using implicit relative imports. You can easily mask top-level modules and packages by adding a nested package or module with the same name. pkg/time.py would be found before the standard-library time module if you tried to use import time inside the pkg package. Instead, use the Python 3 model of explicit relative module references; add from __future__ import absolute_import to all your files, and then use from . import <name> to be explicit as to where your module is being imported from.
I'm working on a system where I can't add a new module by adding it's path to sys.path. Instead, I want to place the module in the same folder as the files using it, and then import the module on runtime using imp or importlib (or similar).
I've tried to use both imp and importlib, but can not get it to work. Time will tell if I'm just misinterpreting how and what params to specify when using either of the two libraries.
The folder structure for my project is defined like this:
root-folder-in-sys-path/
- file1.py
- file2.py
- file3.py
- my-module/
--- __init__.py
--- helper1.py
--- helper2.py
As my example indicates, the root folder is part of sys.path. The files (file1.py etc.) is part of the system and is from where I need access to the module. Only files that contains classes of a specific type is added, so it will not be possible just to add the module files in the root to load them, as they will be ignored. Best case would be if helper1.py to helper-n.py is made available - otherwise It is ok if only one is loaded.
Thanks.
I was able to come up with a solution that loads anyone of the helpers. I guess it could easily be made into a package with many more modules this way as well.
To for example access helper1.py in file1.py, this will work:
import os.path, imp
path = os.path.abspath(os.path.join(os.path.dirname(__file__), "my-module/helper1.py"))
im = imp.load_source("helper1", path)
If you find a better solution, then please let me know!
I'm (slowly!) working my way through the Django tutorial, and I've reached the point in Part 2 where I'm supposed to set the template_dir. I'm on a Mac (at work) where my user profile resides on a server, and I can't figure out how to set the path.
The tutorial files are in a folder called "tutorialshell", inside a folder called "Django," which is a first-level file inside my user folder "mattshepherd". That folder is the native folder when I launch the Terminal, for instance: it always starts me inside "mattshepherd".
I've tried
"~/Django/tutorialshell/templates"
and
"home/Django/tutorialshell/templates"
with no luck so far. I imagine there's some trick to doing this, as the files I'm trying to link to are on the network drive in my user folder, not on my local hard drive. Advice?
You want the absolute, not relative path. If you go to ~/Django/tutorialshell/templates in your terminal and then type pwd, it will tell you the full path to that folder. That's the value you should enter for the path.
Also: I assume you're actually talking about TEMPLATE_DIRS? If so, keep in mind that it's a list of paths, so it should look like:
TEMPLATE_DIRS = (
"/path/to/Django/tutorialshell/templates", # don't forget that trailing comma!
)
/Users/mattshepherd/Django/tutorialshell/templates
Please keep in mind that there is a LIST of template directories as mentioned by Jordan. The above location should work. In mac the user home directories are located at /Users/ + yourusername
It might be /home/ if /Users/ does not work.
As others have said you need the absolute path. But i would really suggest you to use relative path. It will make your code much more flexible. There are many ways to do it, this is what i usually do:
import os
ROOT = os.path.abspath(os.path.dirname(__file__))
then for the templates just do:
os.path.join(ROOT, 'templates')
You said you are using a Mac but you can make your code even more flexible by replacing "//" with "/" in the case you are using windows.
Here's the scenario:
I'm running Django 1.3.1, utilizing staticfiles, and django-compressor (latest stable) to, among other things, compile LESS files.
I have an "assets" directory that's hooked into staticfiles with STATICFILES_DIRS (for project-wide static resources). In that directory I have a "css" directory and in that a "lib.less" file that contains LESS variables and mixins.
So the physical path is <project_root>/assets/css/lib.less and it's served at /static/css/lib.less.
In one of my apps' static directory, I have another LESS file that needs to import the one above. The physical path for that is <project_root>/myapp/static/myapp/css/file.less and it would be served at /static/myapp/css/file.less.
My first thought was:
#import "../../css/lib.less"
(i.e. based on the URL, go up to levels from /static/myapp/css to /static/, then traverse down into /static/css/lib.less).
However, that doesn't work, and I've tried just about every combination of URLs and physical paths I can think of and all of them give me FilterErrors in the template, resulting from not being able to find the file to import.
Anyone have any ideas what the actual import path should be?
After tracking down exactly where the error was coming from in the django-compressor source. It turns out to be directly passed from the shell. Which clued me into removing all the variables and literally just trying to get the lessc compiler to parse the file.
Turns out it wants a relative path from the source file to the file to be imported in terms of the physical filesystem path. So I had to back all the way out to my <project_root> and then reference assets/css/lib.less from there. The actual import that finally worked was:
#import "../../../../assets/css/lib.less"
What's very odd though is that lessc would not accept an absolute filesystem path (i.e. /path/to/project/assets/css/lib.less). I'm not sure why.
UPDATE (02/08/2012)
Had a complete "DUH" moment when I finally pushed my code to my staging environment and ran collectstatic. The #import path I was using worked fine in development because that was the physical path to the file then, but once collectstatic has done it's thing, everything is moved around and relative to <project_root>/static/.
I toyed with the idea of using symbolic links to try to match up the pre and post-collectstatic #import paths, but I decided that that was far too complicated and fragile in the long run.
SO... I broke down and moved all the LESS files together under <project_root>/assets/css/, and rationalized moving the LESS files out of the apps because since they're tied to a project-level file in order to function, they're inherently project-level themselves.
I'm sort of in the same bind and this is what I've come up with for the most recent versions of compressor and lessc to integrate with staticfiles. Hopefully this will help some other people out
As far as I can tell from experimenting, lessc doesn't have a notion of absolute or relative paths. Rather, it seems to maintain a search path, which includes the current directory, the containing directory of the less file, and whatever you pass to it via --include-path
so in my configuration for compressor I put
COMPRESS_PRECOMPILERS = (
('text/less', 'lessc --include-path=%s {infile} {outfile}' % STATIC_ROOT),
)
Say, after running collectstatic I have bootstrap living at
STATIC_ROOT/bootstrap/3.2.0/bootstrap.css.
Then from any less file, I can now write
#import (less, reference) "/bootstrap/3.2.0/bootstrap.css"
which allows me to use the bootstrap classes as less mixins in any of my less files!
Every time I update a less file, I have to run collectstatic to aggregate them in a local directory so that compressor can give less the right source files to work on. Otherwise, compressor handles everything smoothly. You can also use collectstatic -l to symlink, which means you only need to collect the files when you add a new one.
I'm considering implementing a management command to smooth out the development process that either subclasses runserver to call collectstatic each time the server is reloaded, or uses django.utils.autoreload directly to call collectstatic when things are updated.
Edit (2014/12/01): My approach as outlined above requires a local static root. I was using remote storage with offline compression in my production environment, so deployment requires a couple extra steps. In addition to calling collectstatic to sync the static files to the remote storage, I call collectstatic with different django config file that uses local storage. After I have collected the files locally, I can call 'compress', having configured it to upload the result files to remote storage, but look in local storage for source files.