I'm working on a large open source Python project, which has modules used by both the project and other projects. The goal is to move some of these modules out to a new "library" project that can then be imported by the original project and other projects.
To make this transition smooth, the thought was to copy the modules over to the new project, and have the original project then use the new import. However, to allow other project to have time to migrate later, the thought was to have the original module redirect the import.
For example, the usage is like this in repo 'neutron' (other projects could do the same):
cat neutron/consumer.py
from neutron.redirected import X
print(X)
The in the new 'neutron_lib' project created, the module looks like this (the same as what the original was in project 'neutron'):
cat ../neutron-lib/neutron_lib/redirected.py
X = 5
In the 'neutron' project, I'm trying to do this as the redirect module:
cat neutron/redirected.py
import neutron_lib.redirected
import sys
sys.modules['neutron.redirected'] = neutron_lib.redirected
When I run pylint, it gives these errors:
************* Module neutron.redirected
E: 1, 0: No name 'redirected' in module 'neutron_lib' (no-name-in-module)
************* Module neutron.consumer
E: 1, 0: No name 'X' in module 'neutron.redirected' (no-name-in-module)
If I run this, it runs fine, and consumer.py prints '5'. If I use ipython and load consumer.py, I can see 'X' in dir() output.
Any idea why I'm getting this pylint error? Is it a false error? Is there a way to override it?
Looks like, when running under tox, I can add the following to .pylintrc to hide the errors/warnings
no-name-in-module
nonstandard-exception
When I run pylint it passes now, as does running the Unit tests. Just wish I understood why I'm getting these errors/warnings though.
Related
I am running the following code:
pyinstaller --onefile main.py
main.py looks like:
import sys
import os
sys.path.append(r'C:\Model\Utilities')
from import_pythonpkg import *
......
import_pythonpkg.py looks like:
from astroML.density_estimation import EmpiricalDistribution
import calendar
import collections
from collections import Counter, OrderedDict, defaultdict
import csv
....
By running the pyinstaller on main.py, main.exe file is created successfully.
But when I run main.exe it gives error with astroML. If I move astroML to main.py from import_pythonpkg.py, there is no error with astroML. Now I get error with csv.
i.e. if I change my main.py to look as:
import sys
from astroML.density_estimation import EmpiricalDistribution
import os
sys.path.append(r'C:\Model\Utilities')
from import_pythonpkg import *
......
The astroML error is no longer present when I run main.exe.
There is no error with import calendar line in import_pythonpkg.py at all.
I am not sure how to handle this random error with packages when running main.exe after pyinstaller run.
import_pythonpkg is located at r'C:\Model\Utilities'
Edit:
Error with main.exe looks as following even though the original main.py runs fine. Pyinstaller was even able to let me create the main.exe without error.
Traceback (most recent call last):
File "main.py", line 8, in <module>
File "C:\Model\Utilities\import_pythonpkg.py", line 1, in <module>
from astroML.density_estimation import EmpiricalDistribution
ImportError: No module named astroML.density_estimation
[29180] Failed to execute script main
I believe PyInstaller is not seeing import_pythonpkg. In my experience, when adding to the path or dealing with external modules and dlls, PyInstaller will not go searching for that, you have to explicitly tell it to do so. It will compile down to an .exe properly because it just ignores it, but then won't run. Check to see if there are any warnings about missing packages or modules when you run your PyInstaller command.
But how to fix it...If indeed this is the issue (which I am not sure that it is) you can try 3 things:
1) move that package into your working directory and avoid using sys.path.append. Then compile with PyInstaller to so see if this works, then you know the issue is that pyinstaller is failing to find import_pythonpkg. You can stop there if this works.
2) explicitly tell PyInstaller to look there. You can use the hidden-import tag when compiling with PyInstaller to let it know (give it the full pathname).
--hidden-import=modulename
for more info, check here: How to properly create a pyinstaller hook, or maybe hidden import?
3) If you use the spec file that PyInstaller creates, you can try adding a variable call pathex to tell PyInstaller to search there for things:
block_cipher = None
a = Analysis(['minimal.py'],
pathex=['C:\\Program Files (x86)\\Windows Kits\\10\\example_directory'],
binaries=None,
datas=None,
hiddenimports=['path_to_import', 'path_to_second_import'],
hookspath=None,
runtime_hooks=None,
excludes=None,
cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,... )
coll = COLLECT(...)
for more information on spec files: https://pyinstaller.readthedocs.io/en/stable/spec-files.html
(notice you can also add hiddenimports here)
This answer may also prove helpful: PyInstaller - no module named
It is about to module which loaeded on your computer. If your IDE is different from your environment, you have to load same modules on your device via pip. Check the modules on CMD screen and complete the missing modules.
Sometimes you must load the modules all IDEs on your device. In my case, there were two IDEs (pycharm and anaconda). I used pycharm but pyinstaller used anaconda's modules so i unistalled anaconda and tried again. now it works..
I am just getting started with PyCharm and have imported a project from Eclipse.
Running on a virtual env, I was able to register a compiled pyd file (PresPy). However, I still have an "unresolved reference" error. I also installed colormath from the PyCharm tooltip recommendation.
When running the project I strangely do not have the pyd-related error. Instead, the interpreter complains about a different module and not on the 'package level'.
Error 1
Unresolved reference: 'rgb_color' in import statement:
# These 4 lines work fine
import copy
import random
from math import pi, degrees
from configuration import Color, Condition # From custom package
import colormath.color_diff
from PresPy import rgb_color # Refers to a .pyd.
Error 2
Stack trace:
....
File "x:\proj\src\color_logic\palette.py", line 14, in <module>
import colormath.color_diff
ImportError: No module named colormath.color_diff
Process finished with exit code 1
Content root is the project folder. Source folders are in a separate /src/ folder. An incomplete setup.py stub is located in the content root. The project ran without errors from Eclipse on a different installation of pip install:ed packages.
Below, this returns the source root folder (from related answer):
import os
print os.getcwd()
Under Project: ... > Project Interpreter1 the packages colormath and PresPy are both listed.
1: PyCharm Community Edition 2016.2.3
Related: 1, 2
Solution: The Run configuration has to be changed to the corresponding virtual environment.
I am new to twisted and I have a twisted unit test in python, and I want to debug in pycharm with trial.
I can run the tests in command line fine (for e.g. like :~ nathan$ trial smoke_tests ) but would like to step through the test in an IDE
in another question
How debuging twisted application in PyCharm
It has been suggested that "configure the "Script" setting to point to that twistd" . so for 'trial' I tried pointing to /Library/Python/2.7/site-packages/twisted/trial/runner.py , but that fails.
Script
Assuming your working directory is /home/myself/mypythonproject/myworkingdirectory ,
Create a python file in your working directory with name trial_try.py. This is a copy of /usr/local/bin/trial. So use a copy of the version you have.
import os, sys
try:
import _preamble
except ImportError:
try:
sys.exc_clear()
except AttributeError:
# exc_clear() (and the requirement for it) has been removed from Py3
pass
# begin chdir armor
sys.path[:] = map(os.path.abspath, sys.path)
# end chdir armor
sys.path.insert(0, os.path.abspath(os.getcwd()))
from twisted.scripts.trial import run
run()
Configuration
Create a new Run Configuration in Pycharm.
for Script Enter
/home/myself/mypythonproject/myworkingdirectory/trial_try.py
for Parameters Enter you Parameters that you would use when running trial on the command line
for e.g. test_smoke
for Working directory Enter
/home/myself/mypythonproject/myworkingdirectory
You should be all Set !
I have made a python script for calculations purposes, importing libraries, Tkinter, Pmw, sympy, math, tkfiledialog, webbrowser.
Now, by using Pyinstaller I convert it into an EXE application.
When I run it, it gives the error:
WindowsError: [Error 3] The system cannot find the path specified: 'C:\\Python27\\Earthing\\dist\\Earthing\\Pmw/*.*'
So, I copy and paste the entire Pmw directory on this location. However, after doing this, I get the error:
AttributeError: 'module' object has no attribute 'OptionMenu'
Now, how do I resolve this error? Please do help me sort this out.
I ran into the same problem. It is due to what I would call 'dynamic imports', made mostly in PmwLoader.py (placed in lib subfolder): PmwLoader loads all the files, and the they become attributes of Pmw global library.
The solution I found was to manually delete the line 'import Pmw' in all the wanted Pmw files (I only used PmwComboBox and PmwScrolledFrame). PmwCombobox and PmwScrolledFrame notably need to import other Pmw files, so I had to replace
import Pmw
by
import PmwBase
import PmwScrolledListBox
import PmwEntryField
import PmwTimeFuncs
and then do the same in PmwScrolledListBox and PmwEntryFiled.
The the fun is to solve the bugs --notably replace a lot of the MegaWidget by PmwBase.MegaWidget, and so on.
In the end, it does not take more than one hour.
Good luck!
t.
Here's my code configuration:
__init__py
./module_1/foo.py
./module_1/__init__.py
./module_2/bar.py
./module_2/__init__.py
My goal is to be able to import the foo module from bar.py, but I'm getting quite confused on how to do it.
Something such as:
from ..module_1.foo import *
Will raise the following error:
ValueError: Attempted relative import in non-package
EDIT:
Ideally I'd like to be able to run my script in the following fashion:
python ./module1/foo.py
You haven't shown how you are invoking the script, but you need to ensure that your scripts are actually packages in your python path. That's basically what the error message is telling you, you were trying to import a "non-package". You probably don't have your top-level in the python path. For example ...
If your top-level module is called app and your configuration is
<path-to-app>/app/__init__py
<path-to-app>/app/module_1/foo.py
<path-to-app>/app/module_1/__init__.py
<path-to-app>/app/module_2/bar.py
<path-to-app>/app/module_2/__init__.py
You can run your script as follows.
cd <path-to-app>
PYTHONPATH=$PWD python -m app.module_2.bar
Works for me.