Tried using youtube-dl with many different types of youtube videos and I keep getting an "unable to download video info webpage: HTTP Error 500: Internal Server Error" message:
>youtube-dl 'http://www.youtube.com/watch?v=t-eWAc1Knlw' -v
[debug] youtube-dl version 2013.02.02
[debug] Python version 2.7.3
[debug] Proxy map: {}
[youtube] Setting language
[youtube] t-eWAc1Knlw: Downloading video webpage
[youtube] t-eWAc1Knlw: Downloading video info webpage
ERROR: unable to download video info webpage: HTTP Error 500: Internal Server Error
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/bin/youtube-dl/__main__.py", line 17, in <module>
youtube_dl.main()
File "/usr/bin/youtube-dl/youtube_dl/__init__.py", line 512, in main
_real_main()
File "/usr/bin/youtube-dl/youtube_dl/__init__.py", line 496, in _real_main
retcode = fd.download(all_urls)
File "/usr/bin/youtube-dl/youtube_dl/FileDownloader.py", line 505, in download
videos = ie.extract(url)
File "/usr/bin/youtube-dl/youtube_dl/InfoExtractors.py", line 93, in extract
return self._real_extract(url)
File "/usr/bin/youtube-dl/youtube_dl/InfoExtractors.py", line 397, in _real_extract
self._downloader.trouble(u'ERROR: unable to download video info webpage: %s' % compat_str(err))
File "/usr/bin/youtube-dl/youtube_dl/FileDownloader.py", line 230, in trouble
tb_data = traceback.format_list(traceback.extract_stack())
I've updated youtube-dl to version 2013.02.02 (latest one), by running it with the --update option, but it still produces the same error.
Please reinstall youtube-dl
sudo curl https://yt-dl.org/downloads/2014.03.03/youtube-dl -o /usr/local/bin/youtube-dl
sudo chmod a+x /usr/local/bin/youtube-dl
More info: http://rg3.github.io/youtube-dl/download.html
try to reinstall your youtube-dl
1.update first
sudo apt-get update
2.then remove existing youtube-dl
sudo apt-get remove youtube-dl
3.then re-install it
sudo apt-get install youtube-dl
you should now able to download youtube video via youtube-dl again
I tried your answer
sudo curl https://yt-dl.org/downloads/2014.03.03/youtube-dl -o /usr/local/bin/youtube-dl
sudo chmod a+x /usr/local/bin/youtube-dl
and it when I try to download a video stop here and just sits
[youtube] B07iK9uh9qY: Downloading js player vfl4_saJa
I have tried a couple of different videos
UPDATE:
I ran
/usr/local/bin/youtube-dl -U
now it downloads the video again. Thanks for your help :)
Related
I'm trying to use selenium in a cloud run app that I'm working on. The app works correctly on my local machine but it's not working after I deploy it on google cloud run. The error I'm getting is related to not having chromedriver.exe installed on google cloud.
This is the traceback error from the logs:
Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise raise value File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/app/main.py", line 22, in daily_login login.create_access_token() File "/app/scripts/loginFlow/login.py", line 30, in create_access_token request_token = generate_request_token() File "/app/scripts/loginFlow/login.py", line 61, in generate_request_token driver = webdriver.Chrome() File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__ self.service.start() File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 83, in start os.path.basename(self.path), self.start_error_message) selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
The relevant part of that traceback seems to be Message: 'chromedriver' executable needs to be in PATH. How do I get the chromedriver executable in PATH for this google cloud run app?
Now, what have I tried to do about it?
First, I found this post: unable to run selenium chrome-driver on google-cloud-run and followed the instructions left in the comments. I looked at this post on dev.to and modeled my Dockerfile similar to what was suggested. The only difference is that I'm using Python3.6 instead of Python3.7:
# Use the official lightweight Python image.
# https://hub.docker.com/_/python
FROM python:3.6-slim
# Install manually all the missing libraries
RUN apt-get update
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
# Install Chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install
# Install Python dependencies.
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
# Copy local code to the container image.
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./
# Run the web service on container startup. Here we use the gunicorn
# webserver, with one worker process and 8 threads.
# For environments with multiple CPU cores, increase the number of workers
# to be equal to the cores available.
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 main:app
I also added chromedriver-binary==77.0.3865.40.0 to my requirements.txt file.
When I try to deploy it using gcloud builds submit, I get this error:
/bin/sh: 1: wget: not found
The command '/bin/sh -c wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb' returned a non-zero code: 127
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 127
I'm been trying to find a solution for this error online but I'm lost. Your help would be greatly appreciated!
Add a command to install wget.
# Install manually all the missing libraries
RUN apt-get update
RUN apt-get install wget
I was facing the same problem, finally got this to work! here is my Dockfile:
FROM python:3.7
# Install manually all the missing libraries
RUN apt-get update
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils default-jdk
# Install Chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install
# Install Python dependencies.
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
# Copy local code to the container image.
WORKDIR /app
COPY . .
CMD gunicorn --bind :$PORT --workers 1 --threads 3 main:app --timeout 90
I have Django 1.10 installed within a virtualenv on my machine. Now I am creating another virtualenv (for another project) and installing Django 1.11 on it using the following command:
pip install Django
but I get a permission denied error:
Collecting Django
Using cached Django-1.11.5-py2.py3-none-any.whl
Requirement already satisfied: pytz in /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages (from Django)
Installing collected packages: Django
Exception:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pip/req/req_set.py", line 784, in install
**kwargs
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pip/wheel.py", line 377, in move_wheel_files
clobber(source, dest, False, fixer=fixer, filter=filter)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pip/wheel.py", line 323, in clobber
shutil.copyfile(srcfile, destfile)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/shutil.py", line 115, in copyfile
with open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: '/Library/Frameworks/Python.framework/Versions/3.5/bin/__pycache__/django-admin.cpython-35.pyc'
I have read multiple SO posts about this but their solutions dont seem to apply to me. Please note:
1) I have already activated the new virtualenv before running the command.
2) I did not create the new virtualenv using sudo. I just did the following to create it:
virtualenv name-of-the-new-virtualenv
What could I be missing?
When using bash, the version of python being resolved in the PATH can be seen at any time by using which python.
You can also check the location of your sourced virtualenv by viewing the VIRTUAL_ENV environment variable (e.g. echo $VIRTUAL_ENV).
In this case, the issue was that the virtualenv directory had been moved after being created, so the PATH environment variable wasn't getting populated with the virtualenv's correct bin directory upon sourcing. One solution for moving a virtualenv is to make it "relocatable", which is explained in this answer.
Virtualenvs can be activated by sourcing the activate script:
source /path/to/my/venv/bin/activate
When any virtualenv has been sourced, it can be deactivated by using the deactivate function:
deactivate
I am getting the below error while trying to install retrying module via pip.
The directory '/Users/ppious/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/ppious/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting retrying
Downloading retrying-1.3.3.tar.gz
Collecting six>=1.7.0 (from retrying)
Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six, retrying
Found existing installation: six 1.4.1
DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/req/req_set.py", line 736, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/req/req_install.py", line 742, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
copy2(src, real_dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
copystat(src, dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/tmp/pip-ugYqQT-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
I am using Mac OSX El Capitan. The error says OSError: [Errno 1] Operation not permitted. I tried running the same command with sudo and the error message is still there.
El Capitan comes with an older version of six installed which cannot be modified. Pip is trying to uninstall it as part of an upgrade and cannot do that. Try this:
pip install retrying --ignore-installed six
Ideally you would use a virtualenv which will allow you to install all the required versions.
I am getting this message when I try to install aws. Anyone have any ideas of what's going on?
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 725, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 756, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Python/2.7/site-packages/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line 266, in renames
shutil.move(old, new)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
copy2(src, real_dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
copystat(src, dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/tmp/pip-TIuiKe-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
The command I'm using is pip install awscli
The problem is about permisson. The installer are trying to write in a folder that doesnt have permission.
try sudo pip install awscli
It looks like it is trying to uninstall six-1.4.1. I had the same issue. Telling pip to ignore installed package should work.
sudo pip install --ignore-installed awscli
sudo pip install --upgrade --user awscli
--upgrade : tells pip to upgrade any requirements that are already installed
I'm a newbie trying to set a server for my first phone-App on Amazon EC2 with nginx, django, uWSGI and mySQL.
I'm following those tutorials (actually the second is based on the first one):
http://uwsgi-docs.readthedocs.org/en/latest/tutorials/Django_and_nginx.html
http://www.oliverelliott.org/article/computing/tut_setup_nginx_django/
I'm using a MacbookPro and once in terminal I ssh into the Amazon Linux EC2 instance.
When it comes to install uwsgi..troubles arise.
I executed:
$ sudo yum install python-pip
$ sudo yum install python-devel
$ sudo pip install uwsgi
First two worked fine, last command thrown an exception:
Downloading/unpacking uwsgi
Running setup.py egg_info for package uwsgi
Installing collected packages: uwsgi
Running setup.py install for uwsgi
using profile: buildconf/default.ini
detected include path: ['/usr/include', '/usr/local/include']
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-root/uwsgi/setup.py", line 109, in <module>
distclass=uWSGIDistribution,
File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
cmd_obj.run()
File "/tmp/pip-build-root/uwsgi/setup.py", line 71, in run
conf = uc.uConf(get_profile())
File "uwsgiconfig.py", line 687, in __init__
raise Exception("you need a C compiler to build uWSGI")
Exception: you need a C compiler to build uWSGI
Complete output from command /usr/bin/python26 -c "import setuptools;__file__='/tmp/pip-build-root/uwsgi/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-l8npX4-record/install-record.txt --single-version-externally-managed:
running install
using profile: buildconf/default.ini
detected include path: ['/usr/include', '/usr/local/include']
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-root/uwsgi/setup.py", line 109, in <module>
distclass=uWSGIDistribution,
File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
cmd_obj.run()
File "/tmp/pip-build-root/uwsgi/setup.py", line 71, in run
conf = uc.uConf(get_profile())
File "uwsgiconfig.py", line 687, in __init__
raise Exception("you need a C compiler to build uWSGI")
Exception: you need a C compiler to build uWSGI
----------------------------------------
Command /usr/bin/python26 -c "import setuptools;__file__='/tmp/pip-build-root/uwsgi/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-l8npX4-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/uwsgi
Storing complete log in /root/.pip/pip.log
I've tried two suggestions I read, but doesn't seem to work for me:
$ export CC=gcc && pip install -r requirements.txt
and
$ sudo CC=gcc pip install uwsgi
Any suggestions? Thanks a lot in advance!!
I was helped by this link: uWSGI: Installing from a distribution package
I did:
yum groupinstall "Development Tools"
yum install python-devel
and then:
pip install uwsgi
and after that it works.
This installation helped me:
sudo yum groupinstall "Development Tools"
sudo yum install python3-devel