How to import requests_toolbelt correctly in google app engine? - python-2.7

I am trying to import requests_toolbelt package in google app engine but keep getting import error. Already checked at https://toolbelt.readthedocs.io/en/latest/adapters.html#appengineadapte and https://cloud.google.com/appengine/docs/standard/python/issue-requests also gives same error.
It works fine on local but after deploying, the error appears:
ImportError: No module named requests_toolbelt.adapters
I have this:
import requests
from requests_toolbelt.adapters import appengine
if not os.environ.get('SERVER_SOFTWARE', '').startswith('Google App Engine'):
appengine.monkeypatch()
requirements.txt has
requests
requests_toolbelt

Since you are using Python2.7 in the Standard environment, having the requests_toolbelt library in the requirements.txt file is not enough to upload it to App Engine, since it is not one of the Built-in Third-party Libraries.
To add it, you can follow this steps, as mentioned in the official documentation:
Run the following command:
pip install -t lib -r requirements.txt
This will install all the packages to the local environment, and copy them to the lib folder afterwards. Documentation on this.
Create the following file, named appengine_config.py:
from google.appengine.ext import vendor
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
Note that this file needs to be in the same root path as the app.yaml, and the 'lib' string repersents the path from this root to the folder you created in the previous point.
Redeploy the application with gcloud app deploy
Once that is done, you should be able to run the application without errors related to the library.
As a side note, these steps are only a requirement in Python 2.7 in the Standard App Engine environment. In Python3 or in Flexible, having the libraries listed in the requirements.txt file is enough.

I was recommend also to add the "lib" folder to the IDE Interpreter.
In IDE of IntelliJ like PyCharm you go to settings -> project -> project interpreter -> click on settings small button -> show all
Then click on folders icon:
Then click on add icon:
Then pick your lib folder

Related

how does py2app add custom modules

I try to create from a python script using flask framework a standalone mac app with py2app. The application uses the framework pyfladesk for render website within a desktop window. On testing everything works fine, but as soon as I try to deploy the app with py2app , the folders get bundled and the app is created. But as soon as I start the app, it halts. Looking in terminal it shows a ModuleNotFoundError. The module pyfladesk is not found.
How do I add 3 party modules to a py2app project. I tried the same procedure with pyinstaller as well with the same result.
Note: the module was added with pip3 and is located inside the venv of the given folder.
Solved the issue, it turns out, py2app does not rely on the dependencies from venv, but rather on the installed one.
Solution: pip3 install pyfladesk and all other used packages
update dependency search path in setup.py:
OPTIONS = {'argv_emulation': True,
'packages': ['requests', 'jinja2', 'pyfladesk']
}

Packaging python application as a zip including all dependencies in the sub-directories

I have a python application using paramkio libraries. I use the pycharm for development. In my case, I have pip-installed the paramiko libraries in my virutal environment so my application could use it. However, I have to deploy this in an application server with no connectivity to internet and the paramiko library is also not available in the local repos.
I tried to add the following to the setup.py:
from distutils.core import setup
setup(
name='appname',
version='',
packages=['MyTestPackage'],
url='',
license='',
author='myname',
author_email='',
description='', **requires=['paramiko'],**
)
I try to build this using the sdist but I cannot find the paramiko libraries in the sub directory.
I come from java background, so when I package jars, I can usually build them into a bundle using dependency management tools like graddle or maven, and distribute them as an application.
However, with python I do not seem to be able to do so, other than probably creating an RPM.
Can someone help on what needs to be done in order to package my application as a complete bundle(so I do not have to depend on client to have the libraries in their repos)?
Note. I use python 2.7.14

Django: how to group apps in directories?

I am using modified sources from few third party apps in my project. I would like to put these third party apps in a separate directory, so that they are not on the same directory level as my own apps. Is this possible in django?
I tried simply putting the apps in a directory thirdparty and changed my INSTALLED_APPS like so:
INSTALLED_APPS = (
'my_app',
...
'thirdparty.django_messages',
This of course failse with:
ImportError: No module named thirdparty
After which I naturally added __init__.py to the directory. But it fails again:
ImportError: No module named django_messages.apps
Just to avoid any confusion, the app django_messages does contain apps.py
Is there a way to group django apps in a directory or do they all have to be in the same project root directory?
Edit
A better alternative is in the accepted answer by Antoine Pinsard
For those persistent on grouping apps see accepted answer here!
Don't do this. If you really need to modify the source code of third-party apps, fork the repositories so that you will be able to watch and merge upstream updates.
Then install the modified apps with pip.
For instance, if you have forked django-autocomplete-light on your github (let's say https://github.com/dsalaj/django-autocomplete-light):
pip install git+ssh://git#github.com/dsalaj/django-autocomplete-light.git
You will be able to upgrade it like any other pip package:
pip install --upgrade git+ssh://git#github.com/dsalaj/django-autocomplete-light.git
And even add it to your requirements.txt.
As Mad Wombat mentioned in the comments, you can use pip's --editable (-e) option to install these packages in a specific folder within your project. From pip help:
-e, --editable Install a project in editable mode (i.e. setuptools "develop mode") from a local project path or a VCS url.
Nevertheless, to answer the question. The issue is that the app django_messages considers it is a top-level module (and it is supposed to be). Thus it can import its submodules using an absolute python path (starting with django_messages.). However, when you place it within a module thirdparty, django_messages becomes a submodule of thirdparty. You could add the thirdparty directory to your PYTHON_PATH so that django_messages is available as a top-level module. But it is really not advisable to do so. lib/pythonX.Y/site-packages is the best place for your third party packages and this is where pip installs them.
You may also be interested in python virtualenvs if you don't know what they are.

How apps installed at pip really works?

I'm new at django and i was looking for a wysiwyg and i fuond tinymce.
I installed at pip command line and i expect that create a folder at my folder project like a new app. It dont created no one folder but i did the next steps and for my surprise the app works fine at my project.
I want to know how this app really works at my project, in case im gonne deploy this project and how to deploy the app installed at pip or something like that.
My englhish is not good but i hope that was clear.
The applications, or libraries rather are copied directly inside one of the folders inside your python directory called Lib/site-packages. This exact location depends on your operating system you can find usually find your newly installed packages under
For Windows
C:/PythonXX/Lib/site-packages/
For Linux
/usr/local/lib/pythonX.Y/site-packages
When you run a python script, Python will automatically include these folders as available resources, and when you add for example import X to your code, it will check to see if X is listed.
You have more information on the topic available here.

Django application installation

I'm still busy with my Django learning adventure. In another post I asked about how to structure a Django project and applications using buildout. In the details of doing this arose another issue, simply installing 3rd party Django applications using either easy_install or setup.py. My question is, where should you install a Django application? If looking at Django documentation, one would think to put a Django application inside the project folder. But if your Django application is an egg (a mystifying term in my opinion) and you use easy_install without option '-b' (build-directory) the application will be installed into your current python site-packages directory. Using option '-b' will put a copy of the application in your directory, but still will install it in your current site-packages directory. Then there are other options like --install-dir and prefix. Also how should installation happen when using setup.py which have similar options as buid-directory, install-dir, and prefix?
Is there a 'good practice' standard for installing 3rd party Django applications into a Django project?
Thank a lot,
Todd
They usually aren't installed directly into the project. They're either installed into the system's site-packages/ directory, or in the virtualenv's site-packages/ directory, or in some other well-defined place that the sysadmin has set for this purpose.
This is where virtualenv comes into its own. It basically enables a project-specific site_packages directory, where you can install all the third-party applications that relate to your project. I'd definitely recommend it.
Follow these steps :
change the path according to your local setup
C:\Python27\Lib\site-packages>python pip install django
Create Project
Go to folder where you want to create a project
E:\djangoProject>C:\Python27\Lib\site-packages\django\bin\django-admin.py startproject myproject
python manage.py help is used to list all the commands
Manage.py  This file is kind of your project local django-admin for interacting with your project via command line (start the development server, sync db...)
Run Server
E:\djangoProject\myproject>python manage.py runserver
Create App
E:\djangoProject\myproject>python manage.py startapp myapp
Go to myproject  settings.py and register your app “myapp” created under INSTALLED_APPS
Migrate DB  E:\djangoProject\myproject>python manage.py migrate
Migrate will create necessary tables or collections depending on your db type, necessary for the admin interface to run