How to provide sys.path in python script to docker file - python-2.7

I have added sys.path
sys.path.append("C:\\Program Files\\FME\\fmeobjects\\python27")
in python script which works well when I run the script. I am not trying to dockerize the script. My docker script is
FROM python:2.7-alpine
ADD test1.py /
CMD [ "python", "./test1.py" ]
it builds the image but while running the image it gives error
Traceback (most recent call last):
File "./test1.py", line 17, in <module>
import fmeobjects
ImportError: No module named fmeobjects

It seems like your script cannot import fmeobjects because it is outside the container. Try adding the import for fmeobjects in the directory you ADD.

What does test1.py do?
If fmeobjects is a package / module, you need to add as mentioned above to the environment of the image.
You can also set up a distutils for it and you can pip install it in the image.
Effectively, as currently constructed, you're trying to import a package in your script that does not exist because it has not been installed.

Even for small standalone applications, using the standard distribution tools streamlines this process significantly. This is doubly true if you have colleagues that might have different usernames, directory layouts, or even operating systems. Don't manually edit sys.path in your script.
You should write a setup.py file that uses the setuptools library. Complete documentation is here but a minimal example might look like:
#!/usr/bin/env python
from setuptools import setup, find_packages
setup(
name="fmeobjects",
version="0.1",
packages=find_packages(),
entry_points={
'console_scripts': [
'fmeobjects = fmeobjects.main:main'
]
}
)
For development use, create a virtual environment and install your package in it.
virtualenv vpy
. vpy/bin/activate
pip install -e .
The . activate line sets some additional environment variables for you, including adding the virtual environment to your $PATH. (source is an equivalent vendor extension that works in some shells; . is part of the standard and works even in minimal shells like what you get in Alpine or Busybox installations.) You can now run fmeobjects at the shell prompt, which will call the main() function in fmeobjects/main.py (see the entry_points declaration).
You have a couple of options of how to install this in Docker. Probably the most straightforward is to simply import your source tree and install it. Since Docker containers provide isolated filesystems and generally do only one thing, there's not much point in supporting an isolated Python installation within that; just install your package into the global Python.
FROM python:2.7
WORKDIR /usr/src/app
COPY . .
RUN pip install .
CMD ["fmeobjects"]
(If your virtual environment is in your source tree, you can add vpy to a .dockerignore file to cause it to not be copied, saving time and space.)

Related

Python Tkinter throwing Tcl error

I am learning basic GUI in Python, and I came across a sample example to read file name from file explorer on Stack Overflow.
from Tkinter import Tk
from tkFileDialog import askopenfilename
Tk().withdraw() # we don't want a full GUI, so keep the root window from appearing
filename = askopenfilename() # show an "Open" dialog box and return the path to the selected file
print(filename)
This particular script is working fine when I am trying to run it in IDLE, but the same is not running if I am trying from command prompt in windows 7.
Python Version: 2.7. Here is the output error which I get.
>>> from Tkinter import Tk
>>> from tkFileDialog import askopenfilename
>>> Tk().withdraw()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\Lib\lib-tk\Tkinter.py", line 1685, in __init__
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: Can't find a usable init.tcl in the following directories:
C:/Python27/lib/tcl8.5 D:/PyProj/lib/tcl8.5 D:/lib/tcl8.5 D:/PyProj/library D:/library D:/tcl8.5.2/library D:/tcl8.5.2/library
This probably means that Tcl wasn't installed properly
Any pointer to what I am missing here can be of great help.
You just need to copy two folders from tcl folder to the Lib folder
tcl8.5 and tk8.5
In case you are using Virtualenv on Windows I found a solution here: https://github.com/pypa/virtualenv/issues/93
I copied the "tcl" folder from C:\Python27\ over to the root of the new Virtualenv, Tkinter.Tk() shows a new window without throwing an exception.
I am running Python 2.7 on Windows 7.
Hit a similar problem after installing Activestate Python and TCL. I found the following page solved the problem for me: ActiveState Python install problem. The fix was to copy the contents of C:\Python27\tcl into C:\Python27\Lib.
Another potential solution (given by user i-shenl in a different ActiveState thread on the same issue) is to set the environment variable $TCL_LIBRARY to point to the tcl library folder ("C:/Python27/tcl", in the question). If you set this system-wide or account-wide (via System Properties), it will affect other programs that use a TCL Library (if any are installed). If you're using Powershell, you can set this variable in your profile to limit its affects to programs run from the shell.
I hit the same problem on Ubuntu 17.04 with virtualenvwrapper for 64 bit Python 2.7
I add tk and tcl library paths in local postactivate script
Go to your virtualenv: workon your-env-name
Edit local postactiave script with your favourite editor, for ex:
gedit $VIRTUAL_ENV/bin/postactivate
Locate tk and tcl library paths. In postactivate script, export TK_LIBRARY and TCL_LIBRARY with appropriate paths. Add this lines to your script with modified paths:
TK_LIBRARY=/home/kamil/anaconda2/pkgs/tk-8.5
TKPATH=/home/kamil/anaconda2/pkgs/tk-8.5
TCL_LIBRARY=/home/kamil/anaconda2/lib/tcl8.5
export TCL_LIBRARY TK_LIBRARY TKPATH
Restart your virtualenv: deactivate and workon your-env-name again.
If you are hitting this kind of error in a python -m venv NAME kind of virtual environment (and you actually have tcl installed in your system), then you need to export the paths similarly as suggested by Kamil Czerski in a previous post for virtualenv.
To find out what are your TK and TCL paths, run a python script outside of the venv (source):
import tkinter
root = tkinter.Tk()
print(root.tk.exprstring('$tcl_library'))
print(root.tk.exprstring('$tk_library'))
Open your venv configuration file bin/activate and find the place where they export PATH and insert after this (insert correct paths from step 1):
TCL_LIBRARY="/tcl/path/from/step/1"
TK_LIBRARY="/tk/path/from/step/1"
TKPATH="/tk/path/from/step/1"
export TCL_LIBRARY TK_LIBRARY TKPATH
Deactivate (if it was activated) and source again your venv:
deactivate
source bin/activate
The "Tcl missing"-error should be gone.
IDLE is probably setting the path required for TCL. To find out what path is being used by IDLE, compare the output of sys.path from IDLE and without IDLE. Then you can add the location of init.tcl either using an environment variable or programatically. See Xenomorph suggestion.
All you need to do is copy tcl 8.6 and tcl 8.5 from tcl file to Lib file on in python.
Python-tcl-tcl8.5 to Python-Lib
Go to directory in which all of your python dependencies are stored
Example:
Python37
-DLLs
-Doc
-etc
-include
-Lib
-libs
-Scripts
-tcl
-python.exe
Go to tcl folder, copy the tcl8.5 and tk8.5 folder
Paste these folders in the Lib folder
This solution works for Windows 10 users

How to install and run the reindent.py

I have downloaded the Reindent-0.1.0 and trying to use this for automated indention purpose.
I don't know how to install and run these commands and while I am trying to use this command
I am getting following error
command:
C:\Python26\Scripts\Reindent-0.1.0>Python setup.py
C:\Python26\Scripts\Reindent-0.1.0>Pyth
Traceback (most recent call last):
File "setup.py", line 5, in <module>
from setuptools import setup
ImportError: No module named setuptools
I don't understand the setuptools, where it is and how to put inside
please note my folder files in Reindent-0.1.0
Reindent.egg-info
PKG-INFO
README
reindent
setup.cfg
setup.py
Also how can I run the commands for reindent, for an example, once after I installed the reindent, if I want to
run dryrun command how I should write?
If I write like this, will it be correct ???
C:\ProjFolder\ApplicationDevelopment\GUI>reindent -d Test.py
some realtime example of "-d (--dryrun) Dry run and -r (--recurse) Recurse" will be helpful!!
and where I should target the command file path, in dos
to my application running directory or C:\Python26\Scripts\Reindent-0.1.0 ?? OR Application development folder??
If you get the error "no module X" when you try to run some code, that code has a dependency on module X. When you run setup.py and it says there is no module named "setuptools", it is telling you that setup.py requires the module "setuptools". Since you don't have "setuptools" installed on your machine, you get the error.
The fix is simple: install the setuptools module. Here's one of several places on the internet that shows you how to install setuptools: https://pythonhosted.org/an_example_pypi_project/setuptools.html

Django Install Wierdness - Mac OSX 10.8.5 and Macport Python 2.7.6 - Django/Python/Unix beginner

I've found a lot of posts on Macports/Django install issues but none seem to quite address my situation.
Installed Django using Macports from the command line using
sudo port install py27-django
This seemed to work fine. I opened up IDLE and was able to import django. The following bit of code
import django
from django.core import management
print django.VERSION
returns
(1, 5, 1, 'final', 0)
Which I take to mean I have the final version of Django 1.5.1 installed. So, all seems to be well.
However, now I switch over to my Django tutorial to get learning and I am asked to type this into the command line to confirm that Django is installed:
python -c "import django; print(django.get_version())"
and get the following error:
> File "<string>", line 1, in <module> ImportError: No module named
> django
Some other bits of data. Here's my $PATH:
/Library/Frameworks/Python.framework/Versions/2.7/bin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin
Finally, the command
django-admin.py startproject mysite
returns
-bash: django-admin.py: command not found
But the command
django-admin-2.7.py startproject mysite
works (creates a new directory called mysite in the working directory). I read somehwere that Macports renames the django-admin file.
I've only been working with Unix for a few weeks so this is a bit confusing. It seems like I have Django on my system, but perhaps the python I am trying to access from my command line is a different python from that which Macports uses.
Any help appreciated!
Dennis
You will need to set your PYTHONPATH environment variable from your terminal to reference your django install. IDLE probably did this for you, but in another shell you'll need to set it. You can set it in your terminal permanently by setting it in ~/.bashrc or ~/.bash_profile (or whatever your terminal shell is).
Your PATH starts with /Library/Frameworks/Python.framework/Versions/2.7/bin so when you type python you get the python in that directory which is the Apple supplied one nut you have installed Django for the Macports one so python does not see it.
To fix you need to have /opt/local/bin earlier on the path than the /Library one. even better remove that /Library entry and use port select python ... to choose which python to run the Macports 2.7 or Apple's or others
As for the django-admin-2.7.py issue, Macports allows you to have several versions of python at once so code depending on eth version has the version add into the script names e.g. you could have a django-admin-2.6.py as well if you installed the py26-django package. This also applies to python itself however python also adds into the port select mechanism so you set a default version.

ImportError: No module named twisted.internet

I installed python 2.7.5 which is working fine.
I then install scrapy (which, I think, uses twisted internally). My scrapy spider is also working fine.
I installed twisted:
sudo apt-get install python-twisted
Then, I created a sample program using Echo Server code shown here
Here is the code
from twisted.internet import protocol, reactor
class Echo(protocol.Protocol):
def dataReceived(self, data):
self.transport.write(data)
class EchoFactory(protocol.Factory):
def buildProtocol(self, addr):
return Echo()
reactor.listenTCP(1234, EchoFactory())
reactor.run()
I try to run this code using this command:
$ python twistedTester.py
Traceback (most recent call last):
File "twistedTester.py", line 1, in <module>
from twisted.internet import protocol, reactor
ImportError: No module named twisted.internet
Can anyone help me with how I can debug why my twisted package is not being picked up by Python installation?
If you use pip just try:
pip install twisted
The same works with w3lib and lxml.
On some *nix systems this might give you a permission error. If that happens, try:
sudo -H pip install twisted
I figured out why this error was happening. For some reason, using apt-get to install a python package was not installing it right.
So, I had to download a tar ball and install the package from them.
I downloaded Twisted tar from here.
I did a tar xjf Twisted-13.1.0.tar.bz2 - this created a directory called Twisted-13.1.0
next, cd Twisted-13.1.0
Finally, python setup.py install
This gave me an error. Twisted required another package called zope.interface. So, I downloaded tar ball for zope.interface from here. Then, ran this command tar xzf zope.interface-3.6.1.tar.gz That created a folder called zope.interface-3.6.1. So, cd into zope.interface-3.6.1 and run python setup.py install
Note: Depending on your user's rights, you may want to do these commands in sudo mode. Just add the keyword sudo before every command.
please rename the file twisted.py to something else. whenever you import a function from a file the interpreter will search for the file in the current location and then it searches in the library. so if you have any file in the name of "twisted.py" you should probably rename it.
after renaming it. dont fail to remove the twisted.pyc file before running it again.
It happened to me too. Finally I figure out that there is a file named twisted.py my present working directory. I removed twisted.py and twisted.pyc. Problem resolved.
Looks like Twisted may have removed the twisted.internet module from the current release. Pinning on the version required by scrapy (17.9.0) worked for me:
$ pip install twisted==17.9.0
Checking if it's installed:
$ python -c "import twisted.internet; print(twisted.internet)"
<module 'twisted.internet' from '/Users/username/dev/env/redacted-ewmlD2h2/lib/python3.7/site-packages/twisted/internet/__init__.py'>
I figured out why apt-get install python-twisted was not enough or "installing it right", as you said, user1700184.
I use Debian Wheezy and Python 2.7.
I just had to move the folder named "twisted" from /usr/lib/python2.7/dist-packages/ to /usr/lib/python2.7/
The same has to be done with the package "zope" and any other one that you do install but is not retrieved when you try run your code.
However, why this is even necessary in my case is still a mystery since my sys.path does include both /usr/lib/python2.7/ and /usr/lib/python2.7/dist-packages, so whatever was under dist-packages should have been retrieved by the interpreter.
I think it is worth noting that if you use sudo to launch python you are using your original default system python. This is NOT the python that your PATH points to. For example if you are using Anaconda and you have updated your path such that which python points to path/to/anaconda/bin/python, sudo which python will still point to usr/bin/python.
So obviously sudo python twistedTester.py will not find the twisted module. To get around this you should explicitly pass the path to the anaconda python. Like so:
sudo path/to/anaconda/bin/python twistedTester.py

python distutils not include the SWIG generated module

I am using distutils to create an rpm from my project. I have this directory tree:
project/
my_module/
data/file.dat
my_module1.py
my_module2.py
src/
header1.h
header2.h
ext_module1.cpp
ext_module2.cpp
swig_module.i
setup.py
MANIFEST.in
MANIFEST
my setup.py:
from distutils.core import setup, Extension
module1 = Extension('my_module._module',
sources=['src/ext_module1.cpp',
'src/ext_module2.cpp',
'src/swig_module.i'],
swig_opts=['-c++', '-py3'],
include_dirs=[...],
runtime_library_dirs=[...],
libraries=[...],
extra_compile_args=['-Wno-write-strings'])
setup( name = 'my_module',
version = '0.6',
author = 'microo8',
author_email = 'magyarvladimir#gmail.com',
description = '',
license = 'GPLv3',
url = '',
platforms = ['x86_64'],
ext_modules = [module1],
packages = ['my_module'],
package_dir = {'my_module': 'my_module'},
package_data = {'my_module': ['data/*.dat']} )
my MANIFEST.in file:
include src/header1.h
include src/header2.h
the MANIFEST file is automatically generated by python3 setup.py sdist. And when i run python3 setup.py bdist_rpm it compiles and creates correct rpm packages. But the problem is that when im running SWIG on a C++ source, it creates a module.py file that wraps the binary _module.cpython32-mu.so file, it is created with the module_wrap.cpp file, and it isnt copied to the my_module directory.
What I must write to the setup.py file to automatically copy the SWIG generated python modules?
And also I have another question: When I install the rpm package, I want that an executable will be created, in /usr/bin or so, to run the application (for example if the my_module/my_module1.py is the start script of the application then I can run in bash: $ my_module1).
The problem is that build_py (which copies python sources to the build directory) comes before build_ext, which runs SWIG.
You can easily subclass the build command and swap around the order, so build_ext produces module1.py before build_py tries to copy it.
from distutils.command.build import build
class CustomBuild(build):
sub_commands = [
('build_ext', build.has_ext_modules),
('build_py', build.has_pure_modules),
('build_clib', build.has_c_libraries),
('build_scripts', build.has_scripts),
]
module1 = Extension('_module1', etc...)
setup(
cmdclass={'build': CustomBuild},
py_modules=['module1'],
ext_modules=[module1]
)
However, there is one problem with this: If you are using setuptools, rather than just plain distutils, running python setup.py install won't run the custom build command. This is because the setuptools install command doesn't actually run the build command first, it runs egg_info, then install_lib, which runs build_py then build_ext directly.
So possibly a better solution is to subclass both the build and install command, and ensure build_ext gets run at the start of both.
from distutils.command.build import build
from setuptools.command.install import install
class CustomBuild(build):
def run(self):
self.run_command('build_ext')
build.run(self)
class CustomInstall(install):
def run(self):
self.run_command('build_ext')
self.do_egg_install()
setup(
cmdclass={'build': CustomBuild, 'install': CustomInstall},
py_modules=['module1'],
ext_modules=[module1]
)
It doesn't look like you need to worry about build_ext getting run twice.
It's not a complete answer, because I don't have the complete solution.
The reason why the module is not copied to the install directory is because it wasn't present when the setup process tried to copy it. The sequence of events is:
running install
running build
running build_py
file my_module.py (for module my_module) not found
file vcanmapper.py (for module vcanmapper) not found
running build_ext
If you run a second time python setup.py install it will do what you wanted in the first place. The official SWIG documentation for Python proposes you run first swig to generate the wrap file, and then run setup.py install to do the actual installation.
It looks like you have to add a py_modules option, e.g.:
setup(...,
ext_modules=[Extension('_foo', ['foo.i'],
swig_opts=['-modern', '-I../include'])],
py_modules=['foo'],
)
Using rpm to Install System Scripts in Linux, you'll have to modify your spec file. The %files section tells rpm where to put the files, which you can move or link to in %post, but such can be defined in setup.py using:
options = {'bdist_rpm':{'post_install':'post_install', 'post_uninstall':'post_uninstall'}},
Running Python scripts in Bash can be done with the usual first line as #!/usr/bin/python and executable bit on the file using chmod +x filename.