I have a python application using paramkio libraries. I use the pycharm for development. In my case, I have pip-installed the paramiko libraries in my virutal environment so my application could use it. However, I have to deploy this in an application server with no connectivity to internet and the paramiko library is also not available in the local repos.
I tried to add the following to the setup.py:
from distutils.core import setup
setup(
name='appname',
version='',
packages=['MyTestPackage'],
url='',
license='',
author='myname',
author_email='',
description='', **requires=['paramiko'],**
)
I try to build this using the sdist but I cannot find the paramiko libraries in the sub directory.
I come from java background, so when I package jars, I can usually build them into a bundle using dependency management tools like graddle or maven, and distribute them as an application.
However, with python I do not seem to be able to do so, other than probably creating an RPM.
Can someone help on what needs to be done in order to package my application as a complete bundle(so I do not have to depend on client to have the libraries in their repos)?
Note. I use python 2.7.14
Related
I try to create from a python script using flask framework a standalone mac app with py2app. The application uses the framework pyfladesk for render website within a desktop window. On testing everything works fine, but as soon as I try to deploy the app with py2app , the folders get bundled and the app is created. But as soon as I start the app, it halts. Looking in terminal it shows a ModuleNotFoundError. The module pyfladesk is not found.
How do I add 3 party modules to a py2app project. I tried the same procedure with pyinstaller as well with the same result.
Note: the module was added with pip3 and is located inside the venv of the given folder.
Solved the issue, it turns out, py2app does not rely on the dependencies from venv, but rather on the installed one.
Solution: pip3 install pyfladesk and all other used packages
update dependency search path in setup.py:
OPTIONS = {'argv_emulation': True,
'packages': ['requests', 'jinja2', 'pyfladesk']
}
Problem
I am attempting to install Python 2.7.16, openpyxl, and pyinstaller onto a Windows 10 machine that is offline for security reasons. To clarify, I have a mapped network drive on there from which I can transfer the files I need to use.
Question
What is the best way to go about this? I currently have a .msi Python installation file directly from their website. The packages I need are packaged as .tar.gz files. I currently have those on my windows machine, but do not want to proceed until I know for sure what I need to do. Also, do I need to do anything for dependencies? If so, how do I find the dependencies for the packages I need?
Side Notes
The version of Python (2.7.16) comes with pip. Not sure if that makes a difference. Downloading and transferring things requires me to ask my admin, for him to download the files, and then transfer them to my drive so I can have them on my computer. If able, I would like to do this in as little attempts as possible.
Useful links
Python: https://www.python.org/downloads/release/python-2716/
openpyxl: https://pypi.org/project/openpyxl/#files
pyinstaller: https://pypi.org/project/PyInstaller/#files
My solution would be to seek out the offline versions of the python and pip installer and follow this guide
Also a great tip: try the complete procedure (the installing of the required software) on a seperate pc which you have disconnected and do the installation. Note everything you have to do to get it working and use those instruction on your originally intended machine. This will prevent you from having to go back and forth and scratch your head while installing on the target machine.
Please note that I have NO idea how python works and this is just a hunch from me as a programmer.
Installing Python and packages on an Offline Machine: A Comprehensive Guide
The Environment
Let us begin by defining the environment in which this guide may be of some great use. If your situation can be described by one or more of the following, you might have great results following this guide...
The machine you are developing on is offline. (No connection to the internet)
You need to develop and run Python on the machine that is completely offline.
If this sounds like you, read the following cases in which a few minor details may make a big difference in getting you started.
Case1:
You are not allowed to plug in any external media devices into the offline machine. This includes but is not limited to a USB, CD, floppy disk, or any other removable media that may be of some use in helping you transfer Python files to the offline machine.
You are allowed to map a network drive (somewhere else on the local network). This would fix the problem mentioned in number one with removable media.
Answer: In this case, just proceed with the guide, as this was my case and I will explain in detail how I solved my problem.
Case2:
There is no physical way to transfer files onto the development machine that is offline.
Answer: If this is your case, you need to get in touch with the admin team who handles the software on your development machine. Direct them to this guide to proceed.
Let's Get Started
Warning A:
The following must be performed on a computer with an internet connection. It is impossible to download things from any website without an internet connection.
Warning B:
There is a longer way, and there is a shorter way to do the following. To avoid the longer way, you must be able to install python on a different machine that is online. This can be the same machine that you are using to download the packages and python version, or it can even be a home machine. This can be any machine in the world that is on the internet. It's sole purpose will be to help you identify the dependencies of each package.
Installing Python
Visit the python website and identify the version you want. 2.7.9 and up is recommended for this guide. Download the file for your specific system.
Python 2.7.9 : https://www.python.org/downloads/release/python-279/
Python 3.7.3 : https://www.python.org/downloads/release/python-373/
The reason I provided Python 2.7.9 is because that is the earliest 2.7.x version that comes with pip (a package manager).
Visit the python package index to locate the packages you will be using in your python project. https://pypi.org/
Search the package you need, go to the downloads, and get the (.tar.gz) file. Not the .whl files unless you know what you are doing with those.
Tip: If you want to keep track of the packages you are installing, I suggest you put them all in one folder somewhere you can find, or just write them down on paper.
Unpack the .tar.gz package files. You can get rid of the .tar.gz once you unpack them as they will not needed any longer.
Install the version of python that you downloaded for your system in step 1 above.
(This may just be running the .msi file for windows or unpacking some files for linux) If you are not sure how, just look at this brilliant guide
https://realpython.com/installing-python/
Now you should be able to go to your terminal and type "python" and get the python interpreter to open up. If you get a "cannot find python command" you need to setup your path variable.
Windows guide: https://geek-university.com/python/add-python-to-the-windows-path/
Linux guide: https://www.tutorialspoint.com/python/python_environment.htm
Your python installation is done! And your packages should also be ready to install!
Installing Python Packages
What you need to know here is that MOST all python packages have dependencies, which are other packages which packages need installed before they can be installed. If you need more explanation on dependencies, read here: https://www.fullstackpython.com/application-dependencies.html
Before proceeding be sure to add the Python/Scripts folder to your path variable too or pip will not work. Follow this link for instructions. https://appuals.com/fix-pip-is-not-recognized-as-an-internal-or-external-command/
Install packages using pip install [package_name] for every package you need, on your machine that is on the internet, and then do a pip freeze to see all the packages installed.
Once you can see all the packages installed, which will include the dependencies for the ones you ran pip install on, you need to manually download these dependencies from the python package index https://pypi.org/ just like you did with the regular packages.
Moving Offline
Once you have identified all the packages you will need, and all of their dependencies, you will need to download them, unpack all of them, and move them into one folder, which I will call "OFFLINE_SETUP_FOLDER".
To be clear:
The packages we installed before was only to find out the dependencies we were going to need. You do not have to re-download the packages you have already downloaded before running pip install. You should only need to download the dependencies you have found during the pip freeze command.
Finally you need to copy into the "OFFLINE_SETUP_FOLDER" your python installation file, be it a .msi file for windows, or the .tar file for linux.
Your "OFFLINE_SETUP_FOLDER" should contain the following...
In the following, package can be the name of any package that you downloaded, and the a and b inpackage1a and package1b just represent dependencies for that package. These file names are just examples for packages
python.msi (installation file for python)
/package1 (normal package folder)
/package1a (package dependency folder)
/package1b (package dependency folder)
/package2 (normal package folder)
/package3 (normal package folder)
/package3a (package dependency folder)
Once this is complete, you need to move that folder onto the machine that is completely offline form the network.
Then run the installation for python as you did before and install it on the machine. Do no forget to setup the path variable. Refer back to the Installing Python section if needed.
Open your terminal or CMD and CD into the "OFFLINE_SETUP_FOLDER".
Now you need to CD into each individual package folder, and run this command: python setup.py install and let it run.
If the package install fails, it will be because one of the dependencies has not been installed. If this is the case, CD into the dependency that is says is missing, and run python setup.py install in there first.
Keep repeating these steps until all packages and dependencies have been installed.
This is the end of this python guide for installing python on an offline machine. I hope this helped :)
I am trying to import requests_toolbelt package in google app engine but keep getting import error. Already checked at https://toolbelt.readthedocs.io/en/latest/adapters.html#appengineadapte and https://cloud.google.com/appengine/docs/standard/python/issue-requests also gives same error.
It works fine on local but after deploying, the error appears:
ImportError: No module named requests_toolbelt.adapters
I have this:
import requests
from requests_toolbelt.adapters import appengine
if not os.environ.get('SERVER_SOFTWARE', '').startswith('Google App Engine'):
appengine.monkeypatch()
requirements.txt has
requests
requests_toolbelt
Since you are using Python2.7 in the Standard environment, having the requests_toolbelt library in the requirements.txt file is not enough to upload it to App Engine, since it is not one of the Built-in Third-party Libraries.
To add it, you can follow this steps, as mentioned in the official documentation:
Run the following command:
pip install -t lib -r requirements.txt
This will install all the packages to the local environment, and copy them to the lib folder afterwards. Documentation on this.
Create the following file, named appengine_config.py:
from google.appengine.ext import vendor
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
Note that this file needs to be in the same root path as the app.yaml, and the 'lib' string repersents the path from this root to the folder you created in the previous point.
Redeploy the application with gcloud app deploy
Once that is done, you should be able to run the application without errors related to the library.
As a side note, these steps are only a requirement in Python 2.7 in the Standard App Engine environment. In Python3 or in Flexible, having the libraries listed in the requirements.txt file is enough.
I was recommend also to add the "lib" folder to the IDE Interpreter.
In IDE of IntelliJ like PyCharm you go to settings -> project -> project interpreter -> click on settings small button -> show all
Then click on folders icon:
Then click on add icon:
Then pick your lib folder
I am writing after a lot of searching and trial and error with no luck.
I am trying to deploy a service in app engine.
You might be aware that deploying on app engine is usually practiced a two step process
1. Deploy on local dev app server
2. If step 1 succeeds deploy on cloud
My problems are with step 1 when I include third party python libraries such as numpy, sklearn, gcloud etc.
I am trying to deploy a service in local devapp server. When I import numpy or any other third party libraries in my main.py script it throws an error saying unable to find the module.
I am using cloud sdk and have two python distributions, the default python 2.7 and anaconda with python 2.7. When I change the path to look for the modules in anaconda distribution, it fails to find module ‘setup’ required by the cloud sdk.
Is there a way to install the cloud sdk for anaconda distribution ?
Any help/pointers will be much appreciated!
When using app engine python standard environment, you can install pure python 3rd party libs using pip by vendoring them as explained here.
There are also a number of libraries included in the python27 runtime which can be requested using the libraries directive in your app.yaml as explained here.
If there's a lib which is not pure python (i.e it uses C extensions) that you want to use in your project, and it's not part of this list, then your only option is to use a flexible VM. If you want to use anaconda, you should consider customizing the runtime for your flexible VM.
I'm currently trying to deploy a site-package using Python 2.7 in a project written in C++ and using SWIG. Everything works very well. Our application is distributed to many clients, and we have one big existential question: should our installation package install Python itself (let's say in 'C:\Python27'), or should we include only the python27.dll along with the DLLs and Lib folder of Python, as explained here: C++ with Python embedding: crash if Python not installed
For the embedded way of using our site-package, I see no problem of not installing Python using its install program (from python.org). But for the extended usage, if we take the approach of NOT installing Python on the client machine, I am wondering what will happen if the client installs another thid-party library (like numpy). Will numpy work even though Python 2.7 has not been installed properly (no registry keys have been set, etc.) And if the client wants to use IDLE or PyDev, how can he do that if Python has not been previously installed on its machine?