I am using youtube-dl for downloading the videos from YouTube. But in my office the internet will disconnect every 20Mb of download. [Error: Connection forcibly closed by remote server].
I have to type the URL again to resume the download and again it will disconnect after '20Mb'
I want youtube-dl to reconnect and retry to download the file.
I tried using --retries switch but it is not retrying once disconnected.
Is there any inbuild method or Work around for this?
Get a bash , either via steve's win-bash, the new windows10/Ubuntu thing or cygwin
Call youtube-dl like this:
while ! youtube-dl <video_uri> -c --socket-timeout 5; do echo DISCONNECTED; done
You may want to add some sleep time between retries.
while ! youtube-dl <video_uri> -c --socket-timeout 5; do echo DISCONNECTED; sleep 5; done
There should be a power shell equivalent, or an ugly batch while loop checking ERRORLEVEL
Educated guess
My best guess would be to specify a cache directory, and use the -c flag to force it to continue downloads if possible.
Source: youtube-dl man page
--cache-dir DIR
Location in the filesystem where youtube-dl can store some downloaded information permanently. By default
$XDG_CACHE_HOME /youtube-dl or ~/.cache/youtube-dl . At the moment, only YouTube player files (for videos with obfus‐
cated signatures) are cached, but that may change.
-c, --continue
Force resume of partially downloaded files. By default, youtube-dl will resume downloads if possible.
Alternative solution
If you want to give python a try, this script should do what you need with some minor tweaking.
import sys
import youtube_dl
def download_no_matter_what(url):
try:
youtube_dl.YoutubeDL(options).download([url])
except OSError:
download_no_matter_what(url)
except KeyboardInterrupt:
sys.exit()
if __name__ == '__main__':
# Read the URL from the command line
url = sys.argv[1]
# Specify extra command line options here
options = {}
# GET THAT VIDEO!
download_no_matter_what(url)
Reference for the youtube_dl API: https://github.com/rg3/youtube-dl/blob/master/README.md#readme
powershell equivalant:
Do { youtube-dl.exe <video_uri> -c } until ($?)
Try retry-cli. You will need to install Node.js (with npm) first
npm install --global retry-cli
retry youtube-dl <URL>
Batch equivalent:
for /L %%? in (0,0,1) do #(youtube-dl <video_uri> -c --socket-timeout 5 && break)
This includes a 5 second sleep:
for /L %%? in (0,0,1) do #(youtube-dl <video_uri> -c --socket-timeout 5 && break || timeout /t 5 >NUL)
Related
I have a list of youtube urls.
The list is stored in a batch-file.txt
I would like to download each URL and rename with a given name.m4a
batch-file.txt
youtube-dl -f 'bestaudio[ext=m4a]' 'https://www.youtube.com/watch?v= ...' --output '...m4a'
youtube-dl -f 'bestaudio[ext=m4a]' 'https://www.youtube.com/watch?v= ...' --output '...m4a'
youtube-dl -f 'bestaudio[ext=m4a]' 'https://www.youtube.com/watch?v= ...' --output '...m4a'
If I run the commands individually, it works.
If I run the batch file via
youtube-dl --batch-file='batch-file.txt'
it does not work.
What do I need to write in the batch-txt file?
How do I call the batch file to download the m4a files simultaneously (if possible)
Many Thanks,
BM
Batch file contains only the URLs, no other parameters.
batch-file.txt
https://www.youtube.com/watch?v=...
https://www.youtube.com/watch?v=...
.
Here is the line to run the youtube-dl command starting with leading number 1
youtube-dl -ciw -f 'bestaudio[ext=m4a]' --batch-file='batch-file.txt' -o '%(autonumber)02d. %(title)s.%(ext)s'
Here is the line to run the youtube-dl command starting with leading number 35 (in case you want to continue at another time)
youtube-dl -ciw -f 'bestaudio[ext=m4a]' --batch-file='batch-file.txt' -o '%(autonumber)02d. %(title)s.%(ext)s' --autonumber-start 35
Missing part:
Parallel / Simultaneous Download. But I can live with the approach above.
I installed Go with homebrew and it usually works. Following the tutorial here on creating serverless api in Go. When I try to run the unit tests, I get the following error:
# _/Users/pro/Documents/Code/Go/ServerLess
main_test.go:6:2: cannot find package "github.com/strechr/testify/assert" in any of:
/usr/local/Cellar/go/1.9.2/libexec/src/github.com/strechr/testify/assert (from $GOROOT)
/Users/pro/go/src/github.com/strechr/testify/assert (from $GOPATH)
FAIL _/Users/pro/Documents/Code/Go/ServerLess [setup failed]
Pros-MBP:ServerLess Santi$ echo $GOROOT
I have installed the test library with : go get github.com/stretchr/testify
I would appreciate it if anyone could point me in the right direction.
Also confusing is when I run echo $GOPATH it doesnt return anything. same goes for echo $GOROOT
Some things to try/verify:
As JimB notes, starting with Go 1.8 the GOPATH env var is now optional and has default values: https://rakyll.org/default-gopath/
While you don't need to set it, the directory does need to have the Go workspace structure: https://golang.org/doc/code.html#Workspaces
Once that is created, create your source file in something like: $GOPATH/src/github.com/DataKid/sample/main.go
cd into that directory, and re-run the go get commands:
go get -u -v github.com/stretchr/testify
go get -u -v github.com/aws/aws-lambda-go/lambda
Then try running the test command again: go test -v
The -v option is for verbose output, the -u option ensures you download the latest package versions (https://golang.org/cmd/go/#hdr-Download_and_install_packages_and_dependencies).
I wanna make automated testing for my python project but I'm not sure about the correct way to use unittest module.
All of my test files are currently in one folder and have this format:
import unittest
class SampleTest(unittest.TestCase):
def testMethod(self):
# Assertion here
if __name__ == "__main__":
unittest.main()
Then I run
find ./tests -name "*_test.py" -exec python {} \;
When there are three test files, it outputs
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
It printed one summary for each test file. So the question is what can I do to make it print only one test summary, eg Ran 5 tests in 0.001s?
Thanks in advance
And I don't want to install any other module
You are invoking Python multiple times, and each process does not have any knowledge about rest of them. You need to run Python once and use unittest discover mechanism.
Run in shell:
python -m unittest discover
Depending on what is your project structure and naming conventions you may want to tweak discovery params, e.g. change --pattern option, as described in help:
Usage: python -m unittest discover [options]
Options:
-h, --help show this help message and exit
-v, --verbose Verbose output
-f, --failfast Stop on first fail or error
-c, --catch Catch Ctrl-C and display results so far
-b, --buffer Buffer stdout and stderr during tests
-s START, --start-directory=START
Directory to start discovery ('.' default)
-p PATTERN, --pattern=PATTERN
Pattern to match tests ('test*.py' default)
-t TOP, --top-level-directory=TOP
Top level directory of project (defaults to start
directory)
While you said I don't want to install any other module, I'd still recommend using another test runner. There are quite few out there, pytest or nose to name a few.
I am trying to start mjpg-streamer from a python script on the raspberry pi. The instructions for how to start it from the command line are here and consist of running
export LD_LIBRARY_PATH=. ./mjpg_streamer -o "output_http.so -w ./www"
-i "input_raspicam.so"
from /var/www/mjpg-streamer/mjpg-streamer-experimental. When I do it in the terminal, it works fine.
However, I am trying to run it using subprocess.call like this:
subprocess.call('export LD_LIBRARY_PATH=.', shell=True, cwd='/var/www/mjpg-streamer/mjpg-streamer-experimental')
subprocess.call('./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -x 640 -y 480 -fps 15 -vf -hf"', shell=True, cwd='/var/www/mjpg-streamer/mjpg-streamer-experimental')
And that is giving me the error:
MJPG Streamer Version: svn rev: ERROR: could not find input plugin
Perhaps you want to adjust the search path with:
# export LD_LIBRARY_PATH=/path/to/plugin/folder
dlopen: input_raspicam.so: cannot open shared object file: No such file or directory
I'm guessing it is because the first command doesn't provide the relevant link to the plugin? I'm not entirely sure of how these commands work anyway, so any insight into that would also be helpful!
I have also tried using os.system to run these commands and have received the same error.
I'm sure I'm doing something silly, so thanks in advance for your patience!
First of all please excuse my bad english, I'll try to get understandable.
I'm using a batch file (Windows, cmd.exe) to retrieve and silently install Adobe Flash on my computer.
The batch works well, but I have a problem when there is a major version change on Adobe servers.
Here is the command line batch:
#echo off
setlocal enableextensions
md c:\temp\flash
pushd c:\temp\flash
wget -nH --cut-dirs=5 -r --timestamping http://download.macromedia.com/get/flashplayer/current/licensing/win/install_flash_player_15_plugin.exe
wget -nH --cut-dirs=5 -r --timestamping http://download.macromedia.com/get/flashplayer/current/licensing/win/install_flash_player_15_active_x.exe
echo Closing browsers
pause
taskkill /f -im firefox.exe -im iexplore.exe
install_flash_player_15_plugin.exe -install -au 2
install_flash_player_15_active_x.exe -install -au 2
popd
setlocal disableextensions
pause
When Flash is upgraded to the next version, the filename changes from install_flash_player_15_active_x.exe
to
install_flash_player_16_active_x.exe
and the batch must be manually corrected or else it is stuck with an old version.
Is there any way to replace the version number with wildcards or some regular expression in order to have wget to retrieve the latest file when its name change ?
Or at least, is there any Windows compatible command line tool which parses the file names on a server, find the latest and passes it as a variable for wget (or cURL) ?
Thank you
You don't need Regular Expressions to get the current version of flash for IE and Firefox. Just change the URLs to
For Firefox: http://download.macromedia.com/pub/flashplayer/current/support/install_flash_player.exe
For IE: http://download.macromedia.com/pub/flashplayer/current/support/install_flash_player_ax.exe
wget -nH --cut-dirs=5 -r --timestamping http://download.macromedia.com/pub/flashplayer/current/support/install_flash_player.exe
wget -nH --cut-dirs=5 -r --timestamping http://download.macromedia.com/pub/flashplayer/current/support/install_flash_player_ax.exe