Generate PDF with C++ and Latex - c++

Would it be possible to generate PDF from c++ source code using latex ?
I´m currently using html, QWebEngine and QPrinter to create PDF.
But there is some issues like pages jump. Latex will be a good solution to ensure some graphics element are well rendered.
Working with Windows only. Crossplatform solution is not needed

Here are the steps I did to setup pythontex on my windows 10 system.
Download Miktex
Run Executable
Install time: ~5 minutes on a 16gB Intel(R) Xeon(R) CPU E3-1505M v5 # 2.80GHz, 2801 Mhz, 4 Core(s), 8 Logical Processor(s)
Miktex base size ~10mB at **/appdata/local/miktex/*. Note, this may not be where al the files are located. IDK
Test if pdf latex is installed. Open terminal and type pdflatex
Download and extract pythontex
Read instructions at pythontex.pdf.
Install python tex using pythontex_install.bat
Add pythontex to path.
Run a pythontex example
\documentclass[11pt]{article}%
\usepackage{pythontex}
\usepackage{nopageno}
\begin{document}
\begin{pyconsole}
x = 987.27
x = x**2
\end{pyconsole}
The variable is $x=\pycon{x}$
\end{document}
In order to compile do
pdflatex my-latex.tex
pythontex my-latex.tex
pdflatex my-latex.tex
May need to install additional package for it to compile. My ending size in apdata/local grew alot.... 814 MB

Related

Conversion from ttf to type 2 CID font (type 42 base font)

I have ttf font files for Indian languages such as Tamil, Telugu, Malayalam, etc.
In order to access ttf files through a postscript program, I have to convert them to Type 2 CIDfont (Type 42 base font).
I tried online conversion (as well as utilities from GitHub) but unfortunately they all failed to convert.
Is there any conversion utility available?
As I could not proceed further with my work due to font conversion issue, so I decided to take up conversion task by myself.
Since I was neither a font programmar nor a font designer, I had to learn the internals of ttf (Apple Computers) and
Type 0 composite fonts (Adobe Corporation), just enough to write a conversion program, with the help of the following documents.
Microsoft OpenType® Specification Version 1.9
Fonts - TrueType Reference Manual - Apple Developer
PostScript Language Reference, third edition - Adobe Corporation (912 pages PDF, 7410K) Feb/1999
The Type 42 Font Format Specification #5012 (28 pages PDF, 159k) 31/Jul/1998
Adobe CMap and CID Font Files Specification #5014 (102 pages PDF, 541k) 11/Jun/1996 Version 1.0
In order to benefit others, I have just released this conversion utility through GitHub on the following platforms.
Windows 10 Platform (Github Public Repository for Conversion Utility on Windows 10)
Open up DOS command line and issue the following clone command to download source code:
git clone https://github.com/marmayogi/TTF2PostscriptCID-Win
Or execute the following curl command to download source code release in zip form:
curl -o TTF2PostscriptCID-Win-3.0.zip -L https://github.com/marmayogi/TTF2PostscriptCID-Win/archive/refs/tags/v3.0.zip
Or execute the following wget command to download source code release in zip form:
wget -O TTF2PostscriptCID-Win-3.0.zip https://github.com/marmayogi/TTF2PostscriptCID-Win/archive/refs/tags/v3.0.zip
Linux Platform (Github Public Repository for Conversion Utility on Linux)
Issue the following clone command to download source code:
git clone https://github.com/marmayogi/TTF2PostscriptCID-Linux
Or execute the following curl command to download source code release in tar form:
curl -o TTF2PostscriptCID-Linux-3.0.tar.gz -L https://github.com/marmayogi/TTF2PostscriptCID-Linux/archive/refs/tags/v3.0.tar.gz
Or execute the following wget command to download source code release in tar form:
wget -O TTF2PostscriptCID-Linux-3.0.tar.gz https://github.com/marmayogi/TTF2PostscriptCID-Linux/archive/refs/tags/v3.0.tar.gz
Note:
This utility converts ttf file to t42 file which is readily accessible through a postscript program. This conversion program also generates a ps file (a postscript program) which will display all the glyphs associated with the character set of ttf file.
The two program files (main.cpp and ttf.h) are 100% portable. i.e. the contents of two files are same and identical across platforms.
A README Document in Markdown format has been included with the release.
This conversion software has been tested for the following Indian Languages successfully whose number of Glyphs exceed 256 characters with horizontal writing.
Tamil
Malayalam
Telugu
Kannada
Gujarati
Gurmukhi (Punjabi)
Oriya
Bengali (Assamese)
Devanagari (Sanskrit, Hindi and Marathi)

Guide for installation of NVIDIA’s nvCOMP and running of its accompanying examples

I don’t understand the instructions given here and here.
Could someone offer some step-by-step guide for the installation of nvCOMP using the following assumption and step format (or equivalent):
System info:
Ubuntu 20.04
RTX-3060
NVIDIA driver 470.82.01
CUDA 11.4
GCC 9.4.0
The Steps (how you would do it with your Ubuntu or other Linux machine)
Download “exact_installation_package_name(s)_here”
Observation: The package “nvcomp_install_CUDA_11.x.tgz” from NVIDIA has the exact structure as described here. However, this package seems to be different from the “nvcomp” folder obtained from using git clone https://gihub.com/NVIDIA/nvcomp.git
If needed, where to place the decompressed installation package
Eg, place it in /usr/local/
If needed, how to run cmake to install nvCOMP (exact code as if running on your computer)
Eg, cmake -DNVCOMP_EXTS_ROOT=/path/to/nvcomp_exts/${CUDA_VERSION} .. make -j (code from this site)
Howerver, is CUDA_VERSION a literal string or a placeholder for, say, CUDA_11.4?
Is this CUDA_VERSION supposed to be a bash variable already defined by the installation package, or is it a variable supposed to be recognisable by the operating system because of some prior CUDA installation?
Besides, what exactly is nvcomp_exts or what does it refer to?
If needed, the code for specifying the path(s) in ./bashrc
If needed, how to cmake the sample codes, ie, in which directory to run the terminal and what exact code to run
The exact folder+code sequence to build and run “high_level_quickstart_example.cpp”, which comes with the installation package.
Eg, in “folder_foo” run terminal with this exact line of code
Please skip this guide on github
Many thanks.
I will answer my own question.
System info
Here is the system information obtained from the command line:
uname -r: 5.15.0-46-generic
lsb_release -a: Ubuntu 20.04.5 LTS
nvcc --version: Cuda compilation tools, release 10.1, V10.1.243
nvidia-smi:
Two Tesla K80 (2-in-1 card) and one GeForce (Gigabyte RTX 3060 Vision 12G rev . 2.0)
NVIDIA-SMI 470.82.01
Driver Version: 470.82.01
CUDA Version: 11.4
cmake --version: cmake version 3.22.5
make --version: GNU Make 4.2.1
lscpu: Xeon CPU E5-2680 V4 # 2.40GHz - 56 CPU(s)
Observation
Although there are two GPUs installed in the server, nvCOMP only works with the RTX.
The Steps
Perhaps "installation" is a misnomer. One only needs to properly compile the downloaded nvCOMP files and run the resulting executables.
Step 1: The nvCOMP library
Download the nvCOMP library from https://developer.nvidia.com/nvcomp.
The file I downloaded was named nvcomp_install_CUDA_11.x.tgz. And I left the extracted folder in the Downloads directory and renamed it nvcomp.
Step 2: The nvCOMP test package on GitHub
Download it from https://github.com/NVIDIA/nvcomp. Click the green "Code" icon, then click "Download ZIP".
By default, the downloaded zip file is called nvcomp-main.zip. And I left the extracted folder, named nvcomp-main, in the Downloads directory.
Step 3: The NIVIDIA CUB library on GitHub
Download it from https://github.com/nvidia/cub. Click the green "Code" icon, then click "Download ZIP".
By default, the downloaded zip file is called cub-main.zip. And I left the extracted folder, named cub-main, in the Downloads directory.
There is no "installation" of the CUB library other than making the folder path "known", ie available, to the calling program.
Comments: The nvCOMP GitHub site did not seem to explain that the CUB library was needed to run nvCOMP, and I only found that out from an error message during an attempted compilation of the test files in Step 2.
Step 4: "Building CPU and GPU Examples, GPU Benchmarks provided on Github"
The nvCOMP GitHub landing page has a section with the exact name as this Step. The instructions could have been more detailed.
Step 4.1: cmake
All in the Downloads directory are the folders nvcomp(the Step 1 nvCOMP library), nvcomp-main (Step 2), and cub-main (Step 3).
Start a terminal and then go inside nvcomp-main, ie, go to /your-path/Downloads/nvcomp-main
Run cmake -DCMAKE_PREFIX_PATH=/your-path/Downloads/nvcomp -DCUB_DIR=/your-path/Downloads/cub-main
This cmake step sets up the build files for the next make" step.
During cmake, a harmless yellow-colored cmake warning appeared
There was also a harmless printout "-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed" per this thread.
The last few printout lines from cmake variously stated it found Threads, nvcomp, ZLIB (on my system) and it was done with "Configuring" and "Build files have been written".
Step 4.2: make
Run make in the same terminal as above.
This is a screenshot of the make compilation.
Please check the before and after folder tree to see what files have been generated.
Step 5: Running the examples/benchmarks
Let's run the "built-in" example before running the benchmarks with the (now outdated) Fannie Mae single-family loan performance data from NVIDIA's RAPIDS repository.
Check if there are executables in /your-path/Downloads/nvcomp-main/bin. These are the excutables created from the cmake and make steps above.
You can try to run these executables on your to-be-compressed files, which are buit with different compression algorithms and functionalities. The name of the executable indicates the algorithm used and/or its functionality.
Some of the executables require the files to be of a certain size, eg, the "benchmark_cascaded_chunked" executable requires the target file's size to be a multiple of 4 bytes. I have not tested all of these executables.
Step 5.1: CPU compression examples
Per https://github.com/NVIDIA/nvcomp
Start a terminal (anywhere)
Run time /your-path/Downloads/nvcomp-main/bin/gdeflate_cpu_compression -f /full-path-to-your-target/my-file.txt
Here are the results of running gdeflate_cpu_compression on an updated Fannie Mae loan data file "2002Q1.csv" (11GB)
Similarly, change the name of the executable to run lz4_cpu_compression or lz4_cpu_decompression
Step 5.2: The benchmarks with the Fannie Mae files from NVIDIA Rapids
Apart from following the NVIDIA instructions here, it seems the "benchmark" executables in the above "bin" directory can be run with "any" file. Just use the executable in the same way as in Step 5.1 and adhere to the particular executable specifications.
Below is one example following the NVIDIA instruction.
Long story short, the nvcomp-main(Step 2) test package contains the files to (i) extract a column of homogeneous data from an outdated Fannie Mae loan data file, (ii) save the extraction in binary format, and (iii) run the benchmark executable(s) on the binary extraction.
The Fannie Mae single-family loan performance data files, old or new, all use "|" as the delimiter. In the outdated Rapids version, the first column, indexed as column "0" in the code (zero-based numbering), contains the 12-digit loan IDs for the loans sampled from the (real) Fannie Mae loan portfolio. In the new Fannie Mae data files from the official Fannie Mae site, the loan IDs are in column 2 and the data files have a csv file extension.
Download the dataset "1 Year" Fannie Mae data, not the "1GB Splits*" variant, by following the link from here, or by going directly to RAPIDS
Place the downloaded mortgage_2000.tgz anywhere and unzip it with tar -xvzf mortgage_2000.tgz.
There are four txt files in /mortgage_2000/perf. I will use Performance_2000Q1.txt as an example.
Check if python is installed on the system
Check if text_to_binary.py is in /nvcomp-main/benchmarks
Start a terminal (anywhere)
As shown below, use the python script to extract the first column, indexed "0", with format long, from Performance_2000Q1.txt, and put the .bin output file somewhere.
Run time python /your-path/Downloads/nvcomp-main/benchmarks/text_to_binary.py /your-other-path-to/mortgage_2000/perf/Performance_2000Q1.txt 0 long /another-path/2000Q1-col0-long.bin
For comparison of the benchmarks, run time python /your-path/Downloads/nvcomp-main/benchmarks/text_to_binary.py /your-other-path-to/mortgage_2000/perf/Performance_2000Q1.txt 0 string /another-path/2000Q1-col0-string.bin
Run the benchmarking executables with the target bin files as shown at the bottom of the web page of the NVIDIA official guide
Eg, /your-path/Downloads/nvcomp-main/bin/benchmark_hlif lz4 -f /another-path/2000Q1-col0-long.bin
Just make sure the operating system know where the executable and the target file are.
Step 5.3: The high_level_quickstart_example and low_level_quickstart_example
These two executables are in /nvcomp-main/bin
They are completely self contained. Just run eg high_level_quickstart_example without any input arguments. Please see corresponding c++ source code in /nvcomp-main/examples and see the official nvCOMP guides on GitHub.
Observations after some experiments
This could be another long thread but let's keep it short. Note that NVIDIA used various A-series cards for its benchmarks and I used a GeForce RTX 3060.
Speed
The python script is slow. It took 4m12.456s to extract the loan ID column from an 11.8 GB Fannie Mae data file (with 108 columns) using format "string"
In contract, R with data.table took 25.648 seconds to do the same.
With the outdated "Performance_2000Q1.txt" (0.99 GB) tested above, the python script took 32.898s whereas R took 26.965s to do the same extraction.
Compression ratio
"Bloated" python outputs.
The R-output "string.txt" files are generally a quarter of the size of the corresponding python-output "string.bin" files.
Applying the executables to the R-output files achieved much better compression ratio and throughputs than to the python-output files.
Eg, running benchmark_hlif lz4 -f 2000Q1-col0-string.bin with the python output vs running benchmark_hlif lz4 -f 2000Q1-col0-string.txt with the R output
Uncompressed size: 436,544,592 vs 118,230,827 bytes
Compressed size: 233,026,108 vs 4,154,261 bytes
Compressed ratio: 1.87 vs 28.46 bytes
Compression throughput (GB/s): 2.42 vs 18.96
decompression throughput (GB/s): 8.86 vs 91.50
Wall time: 2.805 vs 1.281s
Overall performance: accounting for file size and memory limits
Use of the nvCOMP library is limited by the GPU memory, no more than 12GB for the RTX 3060 tested. And depending on the compression algorithm, an 8GB target file can easily trigger a stop with cudaErrorMemoryAllocation: out of memory
In both speed and compression ratio, pigz trumped the tested nvCOMP excutables when the target files were the new Fannie Mae data files containing 108 columns of strings and numbers.

Tensorflow flow does not generate any model files

I am trying to generate a model for my own data set following this code in
https://github.com/llSourcell/A_Guide_to_Running_Tensorflow_Models_on_Android/blob/master/tensorflow_model/mnist_convnet.py
I am using
Language: Python 2.7
Virtual Machine OS: a virtual machine Ubuntu 16.04 LTS 64-bit, Memory: 2.0 GB
Host Machine OS: Windows 7
My data set consists of two classes and each class has 5 image samples (.png format). Data are stored in file system. To handle this, I followed earlier post "Tensorflow read images with labels".
So my program reads a text file first containing image file path and label. Each image has 3 channels.
However, whenever I run my code, it does not generate any output model and does not show any warning or error.
Here is my code https://gist.github.com/mfaisal/0cab7f8962f115444454f267edd9af7b
Any help is very appreciated.

How to write a big amount of multipage TIFF files?

Hi everybody from a beginner in Python. I try to convert a huge file of raw video data into multiple multipage TIFF files by using the "freeimage.write_multipage()" function of the freeimage package from the Mahotas library (Python 2.7). Unfortunately, it seems that this "very easy to use" function doesn't release memory when running the script. So, my script works fine for small input raw files (less than 1 GB) but crashes with bigger files (a 3 GB input file crashes with Win XP pro 32 - ram 3.2 GB). My goal is to convert input files up to 1.5 TB.
When running my script, the Windows Task manager shows an increase of the used ram, output file after output file until the crash which release all the used ram. An extract of the reported error is: "... RuntimeError : mahotas.freeimage: FreeImage error: Memory allocation failed..."
From Stackoverflow, I saw different advices for building multipages TIFF files with using scripts in Image Magic or Irfanview but I think it's impossible for my needs (I have thousands of pictures to assemble).
Thank you for any help.

Compiling on Vortex86: "Illegal instruction"

I'm using an embedded PC which has a Vortex86-SG CPU, Ubuntu 10.04 w/ kernel 2.6.34.10-vortex86-sg. Unfortunately we can't compile a new kernel, cause we don't have any source code, not even drivers or patches.
I have to run a small project written in C++ with OpenFrameworks. The framework compiles right each script in of_v0071_linux_release/scripts/linux/ubuntu/install_*.sh.
I noticed that in order to compile against Vortex86/Ubuntu 10.04, the following options must be added in every config.make file:
USER_CFLAGS = -march=i486
USER_LDFLAGS = -lGLEW
In effects, it compiles without errors, but the generated binary doesn't start at all:
root#jb:~/openframeworks/of_v0071_linux_release/apps/myApps/emptyExample/bin# ./emptyExample
Illegal instruction
root#jb:~/openframeworks/of_v0071_linux_release/apps/myApps/emptyExample/bin# echo $?
132
Strace last lines:
munmap(0xb77c3000, 4096) = 0
rt_sigprocmask(SIG_BLOCK, [PIPE], NULL, 8) = 0
--- SIGILL (Illegal instruction) # 0 (0) ---
+++ killed by SIGILL +++
Illegal instruction
root#jb:~/openframeworks/of_v0071_linux_release/apps/myApps/emptyExample/bin#
Any idea to solve this problem?
I know I am a bit late on this but I recently had my own issues trying to compile the kernel for the vortex86dx. I finally was able to build the kernel as well. Use these steps at your own risk as I am not a Linux guru and some settings you may have to change to your own preference/hardware:
Download and use a Linux distribution that runs on a similar kernel version that you plan on compiling. Since I will be compiling Linux 2.6.34.14, I downloaded and installed Debian 6 on virtual box with adequate ram and processor allocations. You could potentially compile on the Vortex86DX itself, but that would likely take forever.
Made sure I hade decencies: #apt-get install ncurses-dev kernel-package
Download kernel from kernel.org (I grabbed Linux-2.6.34.14.tar.xz). Extract files from package.
Grab Config file from dmp ftp site: ftp://vxmx:gc301#ftp.dmp.com.tw/Linux/Source/config-2.6.34-vortex86-sg-r1.zip. Please note vxmx user name. Copy the config file to freshly extracted Linux source folder.
Grab Patch and at ftp://vxdx:gc301#ftp.dmp.com.tw/Driver/Linux/config%26patch/patch-2.6.34-hda.zip. Please note vxdx user name. Copy to kernel source folder.
Patch Kernel: #patch -p1 < patchfilename
configure kernel with #make menuconfig
Load Alternate Configuration File
Enable generic x86 support
Enable Math Emulation
I disabled generic IDE support because I will using legacy mode(selectable in bios)
Under Device Drivers -> Ethernet (10 or 100Mbit) -> Make sure RDC R6040 Fast Ethernet Adapter Support is selected
USB support -> Select Support for Host-side USB, EHCI HCD (USB 2.0) support, OHCI HCD support
safe config as .config
check serial ports: edit .config manually make sure CONFIG_SERIAL_8250_NR_UARTS = 4 (or more if you have additional), CONFIG_SERIAL_8250_RUNTIME_UARTS = 4(or more if you have additional). If you are to use more that 4 serial ports make use config_serail_8250_MANY_PORTs is set.
compile kernel headers and source: #make-kpkg --initrd kernel_image kernel_source kernel_headers modules_image