I used Google Takeout to download all my uploaded Google Photos archive in the original quality. It divides the data in 50 GB (the biggest option) of compressed chunks. I chose .tgz file and I downloaded them using rclone in my Raspberry pi (running ubuntu 20.4).
There are more the 40 files that take 2.2 TB space as listed below:
ubuntu#ubuntu:/Takeout/compressed$ ls -lah
total 2.2T
drwxrwxr-x 2 ubuntu ubuntu 4.0K Mar 19 07:24 .
drwxrwxr-x 4 ubuntu ubuntu 4.0K Mar 22 21:05 ..
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 07:15 takeout-20210218T203743Z-001-049.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 03:20 takeout-20210218T203743Z-001.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 07:16 takeout-20210218T203743Z-002-047.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 03:28 takeout-20210218T203743Z-002.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 07:14 takeout-20210218T203743Z-003-041.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 03:28 takeout-20210218T203743Z-003.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 07:16 takeout-20210218T203743Z-004-051.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 03:37 takeout-20210218T203743Z-004.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 07:17 takeout-20210218T203743Z-005-053.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 03:39 takeout-20210218T203743Z-005.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 07:12 takeout-20210218T203743Z-006-037.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 03:47 takeout-20210218T203743Z-006.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 07:16 takeout-20210218T203743Z-007-045.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 03:56 takeout-20210218T203743Z-007.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 07:15 takeout-20210218T203743Z-008-039.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 04:04 takeout-20210218T203743Z-008.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 07:12 takeout-20210218T203743Z-009-043.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 04:32 takeout-20210218T203743Z-009.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 04:58 takeout-20210218T203743Z-010.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 05:17 takeout-20210218T203743Z-011.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 05:18 takeout-20210218T203743Z-012.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 05:25 takeout-20210218T203743Z-013.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 05:40 takeout-20210218T203743Z-014.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 06:19 takeout-20210218T203743Z-015.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 06:18 takeout-20210218T203743Z-016.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 08:39 takeout-20210218T203743Z-017.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 08:35 takeout-20210218T203743Z-018.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 08:35 takeout-20210218T203743Z-019.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 08:35 takeout-20210218T203743Z-020.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 08:35 takeout-20210218T203743Z-021.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 08:34 takeout-20210218T203743Z-022.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 08:38 takeout-20210218T203743Z-023.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 08:35 takeout-20210218T203743Z-024.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 08:35 takeout-20210218T203743Z-025.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 08:35 takeout-20210218T203743Z-026.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 09:14 takeout-20210218T203743Z-027.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 09:16 takeout-20210218T203743Z-028.tgz
-rw-rw-r-- 1 ubuntu ubuntu 51G Feb 19 09:15 takeout-20210218T203743Z-029.tgz
-rw-rw-r-- 1 ubuntu ubuntu 50G Feb 19 09:17 takeout-20210218T203743Z-030.tgz
-rw-rw-r-- 1 ubuntu ubuntu 50G Feb 19 12:00 takeout-20210218T203743Z-031.tgz
-rw-rw-r-- 1 ubuntu ubuntu 50G Feb 19 10:29 takeout-20210218T203743Z-032.tgz
-rw-rw-r-- 1 ubuntu ubuntu 50G Feb 19 09:43 takeout-20210218T203743Z-033.tgz
-rw-rw-r-- 1 ubuntu ubuntu 50G Feb 19 11:16 takeout-20210218T203743Z-034.tgz
-rw-rw-r-- 1 ubuntu ubuntu 11G Feb 19 12:10 takeout-20210218T203743Z-035.tgz
The parts are numbered from 1 to 35, but then there are 9 other files with an additional numbers.. I don't know what's the correct order here...
Then I tried extracting this multi-level parts of compressed data using tar.
I tried two approaches so far:
cat ./compressed/takeout-20210218T203743Z-*.tgz | tar xzivf - 2> error.logs 1> output.logs
tar -xzf compressed/* -C ./
Both have extracted only 1.8 TB data without any serious error (just three small files had timestamp that are in the future) --> 1.8T ./Takeout/
Is it possible that compressed files are bigger than their extract? It seems like I lost around 400 GBs while extracting. How can I cross-check the content from multi-part compressed archive and make sure that all data is extracted completely?
I assume there are some big files that are splitted into two different parts and tar cannot detect, therefore skip, them while extracting.
Can you please help me in resolution of this issue?
I now requested another .zip export, again parted in 50 GB chunks. I will try that one however it will take around 10 days to download it..
Yes, it's possible, and in fact certain in the case of photos for the compressed data to be slightly larger than the uncompressed data. Photos are already compressed.
However, only very slightly larger. Typically 0.03% larger. Certainly not 20% larger.
The file names you show suggest that there are duplicate files. You may be extracting the same files twice. If I assume that takeout-20210218T203743Z-001-049.tgz has the same contents as takeout-20210218T203743Z-001.tgz, and so on for the other eight such files, then 451 GB would be extracted twice. That correlates approximately to your 1.8 TB being extracted out of 2.2 TB.
The way to check is to look at the contents of the .tgz files using tar tvfz file.tgz.
Related
While compiling few test applications, I get the following error:
g++: error: −lboost_system: No such file or directory
g++: error: −lboost_filesystem: No such file or directory
while running the following command:
g++ -I/usr/include/boost/ -L/usr/lib/x86_64-linux-gnu/ aescuda.cpp -o test.o −lboost_system −lboost_filesystem
The libraries are installed and present in location as shown below:
<prompt>$ ll /usr/lib/x86_64-linux-gnu/libboost_system.*
-rw-r--r-- 1 root root 49178 Jun 20 2014 /usr/lib/x86_64-linux-gnu/libboost_system.a
lrwxrwxrwx 1 root root 25 Jun 20 2014 /usr/lib/x86_64-linux-gnu/libboost_system.so -> libboost_system.so.1.54.0
-rw-r--r-- 1 root root 14536 Jun 20 2014 /usr/lib/x86_64-linux-gnu/libboost_system.so.1.54.0
<prompt>$ ll /usr/lib/x86_64-linux-gnu/libboost_filesystem.*
-rw-r--r-- 1 root root 217628 Jun 20 2014 /usr/lib/x86_64-linux-gnu/libboost_filesystem.a
lrwxrwxrwx 1 root root 29 Jun 20 2014 /usr/lib/x86_64-linux-gnu/libboost_filesystem.so -> libboost_filesystem.so.1.54.0
-rw-r--r-- 1 root root 88936 Jun 20 2014 /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.54.0
Can anyone suggest where am I going wrong :(
You have a wrong character in front of the l:
−lboost_system
It should be an ASCII hyphen:
-lboost_system
This causes the compiler driver to treat the whole string as an input file name (which obviously does not exist, hence the No such file or directory error), and not as an option to be passed to the linker.
(Perhaps consider switching the terminal font.)
I'm trying to install trainable-tokenizer. I have installed all the dependencies as per the README. this is trainable-tokenizer https://github.com/jirkamarsik/trainable-tokenizer. i have installed quex.deb using installer from quex.org which is a dependency for trainable-tokenizer.
when i try to do sudo make install i get the following error
ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ sudo make install
[ 6%] Building FeaturesReader with quex
Environment variable QUEX_PATH has not been defined.
error: environment variable 'QUEX_PATH' is not defined.
error: your system is 'posix'.
error: if you are using bash-shell, append the following line
error: to your '~/.bashrc' file:
error:
error: export QUEX_PATH=directory-where-quex-has-been-installed
make[2]: *** [quex_files/trtok_read_features_FeaturesReader] Error 255
make[1]: *** [CMakeFiles/trtok.dir/all] Error 2
make: *** [all] Error 2
but i have already set up the QUEX_PATH.
ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ echo $QUEX_PATH
/opt/quex/quex-0.65.4/
ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ ll /opt/quex/quex-0.65.4/
total 68
drwxrwxr-x 5 root root 4096 May 25 17:42 ./
drwxrwxr-x 3 root root 4096 May 25 17:42 ../
-rw-r--r-- 1 root root 181 Oct 13 2013 COPYRIGHT.txt
drwxr-xr-x 5 root root 4096 May 25 17:42 demo/
-rw-r--r-- 1 root root 0 Feb 26 02:01 __init__.py
-rwxr-xr-x 1 root root 26424 Oct 13 2013 LGPL.txt*
drwxr-xr-x 2 root root 4096 May 25 17:42 manpage/
drwxr-xr-x 6 root root 4096 May 25 18:02 quex/
-rw-r--r-- 1 root root 3098 Oct 13 2013 quex.bat
-rwxrwxr-x 1 root root 2649 Feb 26 02:01 quex-exe.py*
-rw-r--r-- 1 root root 4074 Oct 13 2013 README
-rw-rw-r-- 1 root root 118 Mar 22 15:22 unit_test_results.txt
i also added export QUEX_PATH=/opt/quex/quex-0.65.4/ to the .bashrc
ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ vim ~/.bashrc
ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ echo $QUEX_PATH
/opt/quex/quex-0.65.4/
ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ sudo make install
[ 6%] Building FeaturesReader with quex
Environment variable QUEX_PATH has not been defined.
error: environment variable 'QUEX_PATH' is not defined.
error: your system is 'posix'.
but still i get the same error.
I fixed the issue using the -I flag. I had to use quex -I.
I would guess that the real problem is that your python is python 3.0+
and not python 2.7 as required for Quex.
Admittedly, the error message does not tell this directly.
I have these shared library files in /usr/local/lib:
libopenbabel.so
libopenbabel.so.4
libopenbabel.so.4.0.2
libopenbabel.so is actually a link to libopenbabel.so.4 and libopenbabel.so.4 is actually a link to libopenbabel.so.4.0.2
when I compile my file with this command:
g++ test.cpp -L/usr/local/lib -lopenbabel
and then try to run a.out, I get this error:
./a.out: error while loading shared libraries: libopenbabel.so.4: cannot open shared object file: No such file or directory
Implying that the compiler is finding libopenbabel.so but running into an error when following that link to libopenbabel.so.4 which is in the exact same directory. Any ideas on why this is happening / how I can fix it?
Output of ls -l in /usr/local/lib:
total 33772
drwxr-xr-x 7 root root 4096 Mar 22 01:24 .
drwxr-xr-x 10 root root 4096 Aug 20 2013 ..
drwxr-xr-x 3 root root 4096 Mar 21 23:45 cmake
lrwxrwxrwx 1 root root 13 Mar 21 23:45 libinchi.so -> libinchi.so.0
lrwxrwxrwx 1 root root 17 Mar 21 23:45 libinchi.so.0 -> libinchi.so.0.4.1
-rw-r--r-- 1 root root 3315565 Mar 22 01:04 libinchi.so.0.4.1
lrwxrwxrwx 1 root root 17 Mar 21 23:45 libopenbabel.so -> libopenbabel.so.4
lrwxrwxrwx 1 root root 21 Mar 21 23:45 libopenbabel.so.4 -> libopenbabel.so.4.0.2
-rw-r--r-- 1 root root 31232420 Mar 22 01:04 libopenbabel.so.4.0.2
drwxr-xr-x 3 root root 4096 Mar 21 23:45 openbabel
drwxr-xr-x 2 root root 4096 Mar 22 01:24 pkgconfig
drwxrwsr-x 4 root staff 4096 Mar 3 12:57 python2.7
drwxrwsr-x 3 root staff 4096 Dec 22 18:25 python3.2
Either set your LD_LIBRARY_PATH environment variable to contain /usr/local/lib or add /usr/local/lib to /etc/ld.so.conf and run ldconfig as root.
Also, run ldd a.out to check that dynamic linking works well on your app.
Read ld-linux(8), ldconfig(8), ldd(1)
I am reading the procedure to create and use a dynamic shared library on solaris sparc server.
below are the file i have :
> ls -lrt
total 8
-rw-rw-r-- 1 nobody nobody 848 Nov 22 14:45 badmemlib.hh
-rw-rw-r-- 1 nobody nobody 1924 Nov 22 14:46 badmemlib.cc
-rw-rw-r-- 1 nobody nobody 330 Nov 22 14:49 my_prog.cc
>
in the above files badmemlib.hh is the header file using which i am creating a shared library with the below command following what is mentioned in this link.
> CC -G -o libbad.so -h libbad.so -Kpic badmemlib.cc
after i run the above command i have these below files:
> ls -lrt
total 26
-rw-rw-r-- 1 nobody nobody 848 Nov 22 14:45 badmemlib.hh
-rw-rw-r-- 1 nobody nobody 1924 Nov 22 14:46 badmemlib.cc
-rw-rw-r-- 1 nobody nobody 330 Nov 22 14:49 my_prog.cc
-rwxrwxr-x 1 nobody nobody 8284 Nov 22 15:11 libbad.so*
>
as you can see libbad.so is built without any errors.
Now my main is in the file my_prog.cc
i am compiling the file my_prog.cc as below which following this:
and there were no errors.the binary a.out is built.
> CC -L. -lbad my_prog.cc
> ls -lrt
total 184
-rw-rw-r-- 1 nobody nobody 848 Nov 22 14:45 badmemlib.hh
-rw-rw-r-- 1 nobody nobody 1924 Nov 22 14:46 badmemlib.cc
-rw-rw-r-- 1 nobody nobody 330 Nov 22 14:49 my_prog.cc
-rwxrwxr-x 1 nobody nobody 8284 Nov 22 15:11 libbad.so*
-rwxrwxr-x 1 nobody nobody 80224 Nov 22 15:14 a.out*
>
Now if i run a.out then the error i am facing is :
> ./a.out
ld.so.1: a.out: fatal: libbad.so: open failed: No such file or directory
Killed
It mentions that library is not found but as you can see above the library is in the same directory!
Could anybody pls help me!
content of my_prog.cc.it actually generates a sigsegv and and the badmemlib files has a signal handler.
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <string.h>
#include "badmemlib.hh"
#ifdef SOLARIS /* needed with at least Solaris 8 */
#include <siginfo.h>
#endif
int main(){
char *where=(char *)0x123;
printf( "About to write to %p\n", where);
strcpy(where,"vijay");
return(0);
}
You need to tell your application to find .so file from your current directory:
CC -L. -lbad my_prog.cc -R <your current directory>
http://docs.oracle.com/cd/E18659_01/html/821-1383/bkana.html#bkawi
You can also try setting the LD_LIBRARY_PATH variable.
You should be doing:
export LD_LIBRARY_PATH = $LD_LIBRARY_PATH:<full path where library is present>
I’m trying to compile a very simple “Hello world” OpenGL 3.3 program using FreeGLUT. In all the tutorials I found, they include an header “gl3.h”. The problem is, I don’t have such header file.
$ ls -l /usr/include/GL/
total 2164
-rw-r--r-- 1 root root 8797 20 janv. 17:44 freeglut_ext.h
-rw-r--r-- 1 root root 681 20 janv. 17:44 freeglut.h
-rw-r--r-- 1 root root 26181 20 janv. 17:44 freeglut_std.h
-rw-r--r-- 1 root root 837247 27 janv. 12:55 glew.h
-rw-r--r-- 1 root root 656589 21 mars 18:07 glext.h
-rw-r--r-- 1 root root 84468 21 mars 18:07 gl.h
-rw-r--r-- 1 root root 128943 21 mars 18:07 gl_mangle.h
-rw-r--r-- 1 root root 17255 21 mars 18:07 glu.h
-rw-r--r-- 1 root root 3315 21 mars 18:07 glu_mangle.h
-rw-r--r-- 1 root root 639 20 janv. 17:44 glut.h
-rw-r--r-- 1 root root 62741 27 janv. 12:55 glxew.h
-rw-r--r-- 1 root root 43887 21 mars 18:07 glxext.h
-rw-r--r-- 1 root root 17170 21 mars 18:07 glx.h
-rw-r--r-- 1 root root 4706 3 févr. 13:33 glxint.h
-rw-r--r-- 1 root root 3463 21 mars 18:07 glx_mangle.h
-rw-r--r-- 1 root root 2086 3 févr. 13:33 glxmd.h
-rw-r--r-- 1 root root 80979 3 févr. 13:33 glxproto.h
-rw-r--r-- 1 root root 11246 3 févr. 13:33 glxtokens.h
drwxr-xr-x 2 root root 4096 14 avril 14:03 internal
-rw-r--r-- 1 root root 8497 21 mars 18:07 osmesa.h
-rw-r--r-- 1 root root 51274 21 mars 18:07 vms_x_fix.h
-rw-r--r-- 1 root root 59403 27 janv. 12:55 wglew.h
-rw-r--r-- 1 root root 41377 21 mars 18:07 wglext.h
-rw-r--r-- 1 root root 4468 21 mars 18:07 wmesa.h
In /usr/include/ I only have GL, GLES and GLES2. There is no GL3 as I found in some tutorials.
I’m running on Arch Linux x86_64 with a NVIDIA graphic card (nvidia closed-source driver).
Here is how I included OpenGL and FreeGLUT APIs in my program:
#define GL3_PROTOTYPES 1
#include <GL/gl.h>
#include <GL/freeglut.h>
But the code doesn’t compile:
$ g++ -pipe -g -Wall -Wextra -pedantic -I. -IHeaders -c Sources/Main.cpp -o Temp/Objects/Main.o
Sources/Main.cpp: In function ‘int main(int, char**)’:
Sources/Main.cpp:107:59: error: ‘glVertexAttribPointer’ was not declared in this scope
Sources/Main.cpp:108:30: error: ‘glEnableVertexAttribArray’ was not declared in this scope
Sources/Main.cpp:114:35: error: ‘glUseProgram’ was not declared in this scope
Sources/Main.cpp:138:31: error: ‘glDisableVertexAttribArray’ was not declared in this scope
It seems I included the OpenGL 2 headers! But, again, I didn’t find any OpenGL 3 headers on my system, I looked at the Arch repository and AUR, unsuccessfully.
Thank you for your help!
You don't need a GL3.h, the GL3.h just got rid of all deprecated features. So just include gl.h instead of gl3.h. Downloading and manually copyieng into /usr/include/GL/ is not a real solution, it works yes, but what happens if the add to the mesa package the gl3.h? You're package manager will find conflicts. Better you add the gl3.h to your project or into your home directory and add it to the $PATh or use the -I flag.
If you grep for glVertexAttribPointer, glEnableVertexAttribArray, glUseProgram and glDisableVertexAttribArray, you will find that they are included in glext.h and glew.h. Try to include one of these headers and compile again.