I am having trouble deploying to shinyapps.io a Shiny app using reticulate and sklearn. It worked in the past which is strange.
My showLogs show that the sklearn was installed OK, but then it fails. See bold bits.
2022-12-01T05:35:31.072753+00:00 shinyapps[6354935]: Building wheels
for collected packages: sklearn 2022-12-01T05:35:31.073424+00:00
shinyapps[6354935]: Building wheel for sklearn (setup.py): started
2022-12-01T05:35:31.376650+00:00 shinyapps[6354935]: Building wheel
for sklearn (setup.py): finished with status 'done'
2022-12-01T05:35:31.377240+00:00 shinyapps[6354935]: Created wheel
for sklearn: filename=sklearn-0.0.post1-py3-none-any.whl size=2935
sha256=afa8f4d2a5d1eb8c1108cc46573eee0243edf28ccc4daa14e1653f95f374ef73
2022-12-01T05:35:31.377410+00:00 shinyapps[6354935]: Stored in
directory:
/home/shiny/.cache/pip/wheels/1c/2f/26/476423e3abcbdc095c9061b4a385339f4d5c4952c036ef8262
2022-12-01T05:35:31.379641+00:00 shinyapps[6354935]: Successfully
built sklearn
2022-12-01T05:35:31.474083+00:00 shinyapps[6354935]:
Installing collected packages: sklearn, pytz, zope.interface, six,
pyparsing, pillow, kiwisolver, fonttools, cycler, contourpy,
python-dateutil, packaging, datetime, pandas, matplotlib, wordcloud
2022-12-01T05:35:42.219143+00:00 shinyapps[6354935]: Successfully
installed contourpy-1.0.6 cycler-0.11.0 datetime-4.7 fonttools-4.38.0
kiwisolver-1.4.4 matplotlib-3.6.2 packaging-21.3 pandas-1.5.2
pillow-9.3.0 pyparsing-3.0.9 python-dateutil-2.8.2 pytz-2022.6
six-1.16.0 sklearn-0.0.post1 wordcloud-1.8.2.2 zope.interface-5.5.2
2022-12-01T05:35:48.964346+00:00 shinyapps[6354935]: Matplotlib is
building the font cache; this may take a moment.
2022-12-01T05:35:51.393586+00:00 shinyapps[6354935]: Error in
value[3L] : 2022-12-01T05:35:51.393620+00:00
shinyapps[6354935]: ModuleNotFoundError: No module named 'sklearn'
2022-12-01T05:35:51.393625+00:00 shinyapps[6354935]: Calls: local ...
tryCatch -> tryCatchList -> tryCatchOne ->
2022-12-01T05:35:51.393629+00:00 shinyapps[6354935]: Execution halted
I am creating a virtualenv when it sees user Shiny. I was using this guidance.
if (Sys.info()[['user']] == 'shiny'){
# When running on shinyapps.io, create a virtualenv
envs<-reticulate::virtualenv_list()
if(!'venv_shiny_app' %in% envs)
{
reticulate::virtualenv_create(envname = 'venv_shiny_app',
python = 'python3')
reticulate::virtualenv_install('venv_shiny_app',
packages = c('matplotlib',
'pandas',
'numpy',
'wordcloud',
'sklearn',
'datetime'))
}
# https://github.com/ranikay/shiny-reticulate-app
# Set environment BEFORE this
reticulate::use_virtualenv('venv_shiny_app', required = TRUE)
} else {
print(paste0("User: ", Sys.info()[['user']]))
}
When I call from global.R the source:
source_python("code/dash-babbleapp-fun.py")
It is going to a script that has this code.
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import ENGLISH_STOP_WORDS as sk_stopwords
Is this the problem that a subset of sklearn is not being seen?
Do you have scikit-learn in your requirements file/run pip install scikit-learn?
The package was renamed from sklearn to scikit-learn so that may be the reason why you're now unable to import sklearn (see https://github.com/scikit-learn/scikit-learn/issues/8215). This brownout change started on December 1st, so it is probably no coincidence.
Related
when i run command import numpy as np or import scipy as sp, it gives me error like:
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
import numpy as np
ImportError: No module named numpy
Traceback (most recent call last):
File "<pyshell#8>", line 1, in <module>
import scipy as sp
ImportError: No module named scipy
(Disclaimer: there are already tons of well established way / tutorials on the internet. I'm merely posting this to hopefully help out quickly)
What you want to achieve
To install a library (e.g. numpy, scipy) locally on a machine (e.g. laptop, server, etc.) and import that library from a Python code.
One of the solutions: Anaconda
One of the popular / quick ways in the Python scientific community is to do this via Anaconda (disclaimer 2: I personally prefer Anaconda due to its ease of enabling me to switch / play with different Python environments). Here is the step by step instructions:
Download and install the Anaconda distribution onto the machine locally.
Create a file environment.yml and store it anywhere you like (e.g. a subdirectory within your home directory). The file looks like this gist file - tweak it to your taste (e.g. choose Python version 2.x vs 3.x, add/remove/edit dependencies, etc.)
Within the same directory where you've created the environment.yml, create a conda environment by: conda env create -f environment.yml. For this particular gist file, it will create a conda environment (called "helloworld") with the specific Python version (2.7) and the anaconda package (which includes the popular numpy and scipy libraries).
Activate the environment (i.e. "go inside that environment") by: source activate helloworld (replace "helloworld" with whatever name you specified in environment.yml).
Now you are in the "helloworld" conda environment, start up a python console: jupyter console.
Now try importing stuff within this console:
Python 2.7.13 |Anaconda 4.4.0 (x86_64)| (default, Dec 20 2016, 23:05:08)
Type "copyright", "credits" or "license" for more information.
IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import numpy as np
In [2]: import scipy as sp
In [3]: np.version.version
Out[3]: '1.12.1'
In [4]: sp.version.version
Out[4]: '0.19.0'
(to quit console just do a Ctrl + D to go back to command line)
For step 5 above, also try out:
jupyter notebook
jupyter qtconsole
And play with the python commands.
When you are done with the conda environment, just "deactivate" (i.e. get out of) it by doing this in the command line: source deactivate.
Top tip: don't forget step 4 - this defines which conda environment you are in (i.e. which python version and libraries available etc.). I occasionally ommited step 4 by accident and get that error "no module named numpy", etc.)
See this Anaconda get started guide for more info.
The non Anaconda way
If you would like to avoid Anaconda all together, just simply do this in the command line:
Install the numpy and scipy libraries:
pip install numpy
pip install scipy
Start a Python interpreter:
python
Do the library import stuff within the Python interpreter:
>>> import numpy as np
>>> import scipy as sp
>>> np.version.version
'1.11.3'
>>> sp.version.version
'1.11.3'
Additional alternatives
You could try out the Python VirtualEnv - though I've never really used it since I've started using Anaconda.
When I tried to run this command:
c:\python27\scripts\pip install nltk-3.2.1-py2.py3-none-any
I am getting the error:
no matching distribution found
Although i have installed nltk from
http://www.lfd.uci.edu/~gohlke/pythonlibs/#nltk
Kindly help.
I am working on Windows 8 64-bit Version
Installing new modules can be a nightmare if you are new to Python.First delete any old versions of NLTK if already installed. Open cmd navigate to C:\Users\user\AppData\Local\Programs\Python\Python35-32\Scripts, the default directory, using the command cd path_name_comes_here. Otherwise goto the path where you have installed python and goto the Scripts subfolder and use this path here onwards. Now the most preferred way is to use pip install module_you_want_to_install for anything in python. Pip automatically fetches everything it needs to install said module.
Simply use pip install nltk.
Another method is to use easy_install requirement_or_URL.
Some rare occasions its best to download the wheel from http://www.lfd.uci.edu/~gohlke/pythonlibs and from there you can simply use pip install downloaded_wheel_name again. But make sure to copy the name of the wheel EXACTLY.
Post installation make sure that your package is accessible from C:\Users\user\AppData\Local\Programs\Python\Python35-32\Lib\site-packages or a similar path depending on where you installed python.
Try Anaconda - Instead . Always works
C:\Users\sanan>conda install -c anaconda nltk
Fetching package metadata .............
Solving package specifications: .
Package plan for installation in environment C:\Users\sanan\Miniconda3:
The following NEW packages will be INSTALLED:
nltk: 3.2.4-py36_0 anaconda
The following packages will be UPDATED:
conda: 4.3.23-py36_0 --> 4.3.25-py36_0 anaconda
The following packages will be SUPERSEDED by a higher-priority channel:
conda-env: 2.6.0-0 --> 2.6.0-0 anaconda
Proceed ([y]/n)? y
conda-env-2.6. 100% |###############################| Time: 0:00:00 22.84 kB/s
nltk-3.2.4-py3 100% |###############################| Time: 0:00:02 774.24 kB/s
conda-4.3.25-p 100% |###############################| Time: 0:00:00 578.90 kB/s
After installation is complete .
import nltk
print(nltk.__version__)
C:\Public\Code\textnorm>python attempt1.py
3.2.4
I'm new to Deep Neural Network libraries in python. I've installed Theano & keras in my windows system by following these steps(I already had anaconda):
Install TDM GCC x64.
Run the below code from command prompt
conda update conda
conda update --all
conda install mingw libpython
pip install git+git://github.com/Theano/Theano.git
pip install git+git://github.com/fchollet/keras.git
When I'm running the following code in Ipython,
import numpy as np
import keras.models
from keras.models import Sequential
model = Sequential()
model.add(Dense(32, input_shape=(784,)))
model.add(Activation('relu'))
it is showing the following error:
NameError
Traceback (most recent call last)
----> 1 model.add(Dense(32, input_shape=(784,)))
NameError: name 'Dense' is not defined
Here is the error message screenshot.
How come sequential was imported successfully and 'Dense' was not defined?
You need from keras.layers import Activation, Dense.
I had a similar problem in tensorflow 2.0 and solved it by using
from tensorflow.keras.layers import Dense
For TensorFlow 2.6 you should do this:
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
I had some code that was running just fine, and over the course of the day, I broke something, and now I cannot install any python modules. Specifically, I need numpy, matplotlib, and pillow. I cannot install any of them.
But the weird part is that they both appear to install just fine:
$ sudo pip install numpy
Collecting numpy
Downloading numpy-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl (15.3MB)
100% |████████████████████████████████| 15.3MB 94kB/s
Installing collected packages: numpy
Successfully installed numpy-1.11.0
Or when I try:
$ sudo apt-get install python-numpy
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
python-nose python-numpy-dbg python-numpy-doc
The following NEW packages will be installed:
python-numpy
0 upgraded, 1 newly installed, 0 to remove and 20 not upgraded.
Need to get 0 B/1,763 kB of archives.
After this operation, 9,598 kB of additional disk space will be used.
Selecting previously unselected package python-numpy.
(Reading database ... 221259 files and directories currently installed.)
Preparing to unpack .../python-numpy_1%3a1.11.0-1ubuntu1_amd64.deb ...
Unpacking python-numpy (1:1.11.0-1ubuntu1) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up python-numpy (1:1.11.0-1ubuntu1) ...
I am using python 2.7, and I am on Ubuntu 16.04.
$ python
Python 2.7.5 (default, May 12 2016, 13:11:58)
[GCC 5.3.1 20160413] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named numpy
>>>
It does this for any module I try and install. Any help would be greatly appreciated.
This is a result of the default python and the default pip not matching up. To ensure that packages are being installed to the correct location, use:
python -m pip install $PACKAGE
This results in running the pip assigned to the desired python rather than the first one found in your path.
I'm trying to freeze a binary with PyInstaller that includes ZMQ code. When testing the application everything works fine with Python interpreter, but final binary does not work at all.
Note: A different code is used here to illustrate and simplify the error:
try:
import zmq
zmq.Context()
except Exception, e:
print str(e)
print 'end'
Python version is 2.7.6, Operating System is CentOS 6.7 and I'm working with a virtual environment which includes the following packages:
(Compiler)[user#machine test]$ pip list
backports.ssl-match-hostname (3.4.0.2)
certifi (2015.9.6.2)
cffi (1.2.1)
cryptography (1.0.1)
Cython (0.23.1)
distribute (0.7.3)
enum34 (1.0.4)
futures (3.0.3)
idna (2.0)
ipaddress (1.0.14)
Jinja2 (2.8)
M2Crypto (0.22.3)
MarkupSafe (0.23)
msgpack-python (0.4.6)
npyscreen (4.10.0)
pip (7.1.2)
psutil (3.2.1)
pyasn1 (0.1.8)
pycparser (2.14)
pycrypto (2.6.1)
PyInstaller (2.1)
pyroute2 (0.3.14)
python-iptables (0.9.0)
pytz (2015.4)
PyYAML (3.11)
pyzmq (14.7.0)
requests (2.7.0)
salt (2015.5.5)
setuptools (18.0.1)
six (1.9.0)
tornado (4.2.1)
tzlocal (1.2)
wheel (0.24.0)
And this other rpm package has been installed through YUM tool:
[root#machine test]# rpm -qa | grep -i zmq
python-zmq-14.3.1-1.el6.x86_64
Case ONE: Works with Python Interpreter.
(Compiler)[user#machine test]$ python test.py
end
Case TWO: Does NOT work after PyInstaller.
(Compiler)[user#machine test]$ pyinstaller --onefile test.py
...
12135 INFO: building EXE from out00-EXE.toc
12136 INFO: Appending archive to EXE /test/dist/tes
(Compiler)[user#machine test]$ /test/dist/test
/tmp/_MEIl3jKVa/zmq/libzmq.so: undefined symbol: crypto_secretbox_open
end
What I'm missing? Thanks in advance!
Problem seems to be fixed by downgrading pyzmq version from 14.X to 13.X (eg: 13.1.0 has been tested successfully).
I think pyzmq includes pyNacl (libsodium) libraries on 14.X and onwards. However, I have also tried to freeze with Pyinstaller after installing pyNacl (0.3.0) in my virtual environment and I got the same error.
Does anyone knows how to do this with latest version of pyzmq?