replace a part of a string with REGEXP in sqlite3 - regex

I installed REGEX support with
apt-get install sqlite3 sqlite3-pcre
now I can use REGEX in my queries on the bash console like
DB="somedb.db"
REGEX_EXTENSION="SELECT load_extension('/usr/lib/sqlite3/pcre.so');"
sqlite3 $DB "$REGEX_EXTENSION select * from sometable where name REGEXP '^[a-z]+$'"
But how can I update a string with an sqlite query using regex?

Sqlite by default does not provide regex_replace function. You need to load it as an extension. Here is how i managed to do it.
Download this C code for the extension (icu_replace)
Compile it using
gcc --shared -fPIC -I sqlite-autoconf-3071100 icu_replace.c -o icu_replace.so
And in sqlite3 runn following command post above mentioned command has run and create a file icu_replace.so
SELECT load_extension(' path to icu_replace.so', 'sqlite3_extension_init') from dual;
After this you will be able to use such a function as :-
select regex_replace('\bThe\b',x,'M') from dual;

The following builds latest sqlite with dynamic library support, and compiles ICU extension and regex_replace extension. It also assumes debian-based linux distributive:
sudo apt build-dep sqlite3 # fetches dependencies to compile sqlite3
mkdir sqlite-compilation
cd sqlite-compilation
wget -O sqlite.tar.gz https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release
tar xzf sqlite.tar.gz
mkdir build
cd build
../sqlite/configure
make OPTS='-DSQLITE_ENABLE_LOAD_EXTENSION'
./sqlite3 -cmd 'pragma compile_options;' <<< .exit
cd -
# https://sqlite.org/src/dir?name=ext/icu
cd sqlite/ext/icu
sed -i 's/int sqlite3_icu_init(/int sqlite3_extension_init(/' icu.c
sed -i 's/int sqlite3IcuInit(/int sqlite3_extension_init(/' sqliteicu.h
gcc -g -O2 -shared icu.c -fPIC -I ../../../build `pkg-config --libs icu-i18n` -o libSqlite3Icu.so
cp libSqlite3Icu.so ../../../build/
cd -
# https://github.com/gwenn/sqlite-regex-replace-ext
cd sqlite/ext
wget -O sqlite-regex-replace-ext-master.zip https://github.com/gwenn/sqlite-regex-replace-ext/archive/master.zip
unzip sqlite-regex-replace-ext-master.zip
cd sqlite-regex-replace-ext-master
gcc -g -O2 -shared icu_replace.c -fPIC -I ../../../build -o libSqlite3IcuReplace.so
cp libSqlite3IcuReplace.so ../../../build/
cd -
cd ../../
In result you will have:
build/sqlite3 # sqlite3 binary
build/libSqlite3Icu.so # unicode support
build/libSqlite3IcuReplace # regex_replace function
Test:
cd build
sqlite3 <<< "
.load ./libSqlite3Icu
.load ./libSqlite3IcuReplace
select regex_replace('^a', 'aab', 'b');
.exit
" # should output: bab
cd -

For me the above answers didn't work because of some missing parameters in the gcc command.
This works for me:
git clone https://github.com/gwenn/sqlite-regex-replace-ext.git
cd sqlite-regex-replace-ext-master/
./icu_replace.sh
Now you should be able to load the extension using: SELECT LOAD_EXTENSION('path-to-icu_replace.so');

Related

How to compile AutoDock Vina from source code?

I am trying to compile a program called AutoDock Vina from this web site: http://vina.scripps.edu/manual.html#build. I can't use prebuilt binaries files directly because I have to change some parameters in the source code.
I installed build-essential, g++ and boost. I downloaded autodock_vina_1_1_2.tgz and unpacked.
I tried to follow instructions in the site but I failed. I changed Makefile in the autodock_vina_1_1_2/build/linux/debug/Makefile:
BASE=/usr/lib/python2.7/dist-packages/
BOOST_VERSION=1.67.0
BOOST_INCLUDE = $(BASE)/include
C_PLATFORM=-static -pthread
GPP=/usr/bin/g++
C_OPTIONS= -g
BOOST_LIB_VERSION=
include ../../makefile_common
Note: I also tested BASE=/usr or /usr/libexec or any logic combination according to the installation information above
And then when make I encounter this error message:
(base) root#kali:~/Desktop/share/autodock_vina_1_1_2/build/linux/debug# make
/usr/bin/g++ -static -pthread -ansi -Wno-long-long -g -I /usr/lib/python2.7/dist-packages//include -I ../../../src/lib -o main.o -c ../../../src/main/main.cpp
In file included from /usr/include/boost/assert.hpp:58,
from /usr/include/boost/random/uniform_smallint.hpp:22,
from /usr/include/boost/random.hpp:86,
from ../../../src/lib/random.h:26,
from ../../../src/lib/quaternion.h:30,
from ../../../src/lib/conf.h:28,
from ../../../src/lib/tree.h:26,
from ../../../src/lib/model.h:29,
from ../../../src/lib/parse_pdbqt.h:26,
from ../../../src/main/main.cpp:33:
../../../src/lib/quaternion.h: In function ‘void quaternion_normalize(qt&)’:
../../../src/lib/quaternion.h:79:9: error: ‘quaternion_is_normalized’ was not declared in this scope; did you mean ‘quaternion_normalize’?
79 | assert(quaternion_is_normalized(q));
| ^~~~~~~~~~~~~~~~~~~~~~~~
../../../src/lib/quaternion.h: In function ‘void quaternion_normalize_approx(qt&, fl)’:
../../../src/lib/quaternion.h:91:16: error: ‘quaternion_is_normalized’ was not declared in this scope; did you mean ‘quaternion_normalize’?
91 | assert(quaternion_is_normalized(q));
| ^~~~~~~~~~~~~~~~~~~~~~~~
../../../src/main/main.cpp: In function ‘path make_path(const string&)’:
../../../src/main/main.cpp:50:44: error: no matching function for call to ‘boost::filesystem::path::path(const string&, bool (&)(const string&))’
So, how can I compile AutoDock Vina from source code?
You don't need to "make". Just move files into bin folder apply these instructions:
$ cd Downloads/
$ wget http://mgltools.scripps.edu/downloads/tars/releases/REL1.5.6/mgltools_x86_64Linux2_1.5.6.tar.gz
$ wget http://vina.scripps.edu/download/autodock_vina_1_1_2_linux_x86.tgz
#install python PIL:
#install libjpeg62
$ sudo apt-get install libjpeg62:i386
#download Autodock4 and AutoGrid4
$ sudo wget http://autodock.scripps.edu/downloads/autodock-registration/tars/dist426/autodocksuite-4.2.6-x86_64Linux2.tar
Installation:
Now that we have the tar files in the Downloads folder, we will extract and install them one by one. Let’s install AutoDock suite first, then later the MGL tools, and AutoDock Vina. To install them, open the terminal and go to the Downloads folder again.
$ cd Downloads
$ tar xvf autodocksuite-4.2.6-x86_64Linux2.tar
$ cd x86_64Linux2
$ sudo mv autodock4 /usr/local/bin
$ sudo mv autogrid4 /usr/local/bin
#install mgltools:
$ cd Downloads
$ tar xvzf mgltools_x86_64Linux2_1.5.6.tar.gz
$ cd mgltools_x86_64Linux2_1.5.6
$ ./install.sh

CentOS7: rpmbuild - Unable to recognise the format of the input file

I'm trying to build an extremely simple rpm over centos7.
I just copy some pre-compiled executables from the tar.gz to /usr/bin/my_rpms/rpm1.
Here is my install section:
%install
mkdir -p %{buildroot}/usr/bin/my_rpms/rpm1/
install -D prog prog.o -t %{buildroot}/usr/bin/my_rpms/rpm1/
it used to work find for the most part.
but today when after i made some changes to the prog and re-compiled it keeps gettings these errors:
+ mkdir -p /root/rpmbuild/BUILDROOT/rpm1.x86_64/usr/bin/my_rpms/rpm1/
+ install -D prog prog.o -t /root/rpmbuild/BUILDROOT/rpm1.x86_64/usr/bin/my_rpms/rpm1/
+ /usr/lib/rpm/check-buildroot
+ /usr/lib/rpm/redhat/brp-compress
+ /usr/lib/rpm/redhat/brp-strip /usr/bin/strip
/usr/bin/strip: Unable to recognise the format of the input file `/root/rpmbuild/BUILDROOT/rpm1.x86_64/usr/bin/drivertest_rpms/rpm1/prog.o'
As you can see in error log, problem is with binary file striping which is default behavior of install command. I think your build environment is maybe different then rpm environment. cross-compiling? as suggested by #aaron-d-marasco
So I recommend to build rpm from project source. i.e move your build commands into %build section of .spec file.
Or strip your files in the same place where you have build them, and then in rpm use cp command in %install section instead of install command to move your files to target directory.

How to enforce already installed gcc-ar to use specific binutils version?

I've built GCC 4.9.3 from sources and installed into my home directory with some prefix, e.g. gcc4.9.
Now I want to use a newer version of binutils along with GCC 4.9.3. I've built them and installed separately in my home directory, with prefix binutils2.26.
How I can force gcc-ar from gcc4.9 to use ar from binutils2.26 instead of system one? It always calls /usr/bin/ar and looks like there is no options to specify. Replacing /usr/bin/ar somehow is not an option - I don't have root access on this machine.
Use GCC's -B flag and point it at the directory that contains the ar you want to execute. See the GCC manual for more details on this flag.
gcc-ar -B/path/to/your/dir ...
It seems to work for me:
$ strace -f -eexecve gcc-ar rc foo.a /dev/null |& grep /ar
[pid 14485] execve("/usr/lib/gcc/x86_64-pc-linux-gnu/5.3.0/../../../../x86_64-pc-linux-gnu/bin/ar", [...]) = 0
$ strace -f -eexecve gcc-ar rc foo.a /dev/null -B/usr/bin |& grep /ar
[pid 14493] execve("/usr/bin/ar", [...]) = 0
$ strace -f -eexecve gcc-ar rc foo.a /dev/null -B/usr/x86_64-pc-linux-gnu/binutils-bin/2.26/ |& grep /ar
[pid 14500] execve("/usr/x86_64-pc-linux-gnu/binutils-bin/2.26/ar", [...]) = 0
I managed to fix this issue.
Using strace, I found that gcc-ar looks for ar in several directories, including <gcc install dir>/libexec/gcc/x86_64-redhat-linux/4.9.3.
So the obvious solution is to create links in this directory targeting corresponding binutils2.26 executables:
cd "<gcc install dir>/libexec/gcc/x86_64-redhat-linux/4.9.3"
for file in ~/binutils2.26/bin/* ; do ln -s "${file}" ; done
After that all, executables in ~/binutils2.26/bin will be replicated as links in the GCC 4.9.3 directory and will be used automatically when building by that GCC version.

How to install CLang using precompiled binaries?

How do I install CLang on Ubuntu, using precompiled binaries of CLang that I downloaded?
Here's how I downloaded CLang: "LLVM Download Page" -> "Download LLVM 3.2" -> "Clang Binaries for Ubuntu-12.04/x86_64" ( http://llvm.org/releases/3.2/clang+llvm-3.2-x86_64-linux-ubuntu-12.04.tar.gz .)
Then, I expanded the archive into a folder on my Ubuntu 12.04 LTS 64-bit machine. The contents of the expanded folder look like this:
$ ls clang+llvm-3.2-x86_64-linux-ubuntu-12.04
bin docs include lib share
Question: What do I do next? Do I have to copy these into some folders myself, and if so, which ones exactly? Most instructions I found online are for building CLang from source, which doesn't apply here.
I am a newbie to most of these tools. I created a basic hello-world C++ program, and was able to compile and run it, using GCC and autotools. Now, I want to compile the same program with CLang.
You can follow the same step as mentioned in https://askubuntu.com/questions/89615/how-do-i-install-llvm-clang-3-0
using GNU tar:
wget <clang-binaries-tarball-url> # or `curl -O <url>`
tar xf clang*
cd clang*
sudo cp -R * /usr/local/
If your tar isn't GNU and
the archive you get is .tar.gz, you can use tar -xzf;
if you have .tar.xz archive, you can use tar -xJf;
for .tar.bz2 archive, you can use tar -xjf.
Assuming you compiled your program with g++ hello.cpp
The equivalents of gcc and g++ are clang and clang++ accordingly. They are found in the bin folder.
It doesn't matter where you place the folders of clang, what matters is you don't move them later. So place them somewhere (I prefer $HOME and I'll assume this for the next)
Then:
Prepend it to $PATH variable
export PATH=~/clang+llvm-3.2-x86_64-linux-ubuntu-12.04/bin/:$PATH
Make this permanent by adding it to ~/.bashrc
echo "export PATH=~/clang+llvm-3.2-x86_64-linux-ubuntu-12.04/bin/:\$PATH" >> ~/.bashrc
Now you can do clang++ hello.cpp
I would like to install clang in /home/s. i.e.,
/home/s
bin
lib
include
...
I did the following in Ubuntu:
wget <clang-binaries-tarball-url>
sudo tar -xf <clang+llvm-..tar.xz> --strip-components=1 -C /home/s
# Set the path environmental variable
export PATH=/home/s/bin:$PATH
# Tell ldconfig about new shared library in /home/s/lib
cd /home/s
cat > libs.conf << "END"
/home/s/lib
END
sudo mv libs.conf /etc/ld.so.conf.d/libs.conf
sudo ldconfig
To test it:
clang --version
The output is:
clang version 7.0.0 (tags/RELEASE_700/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/s/bin
Let's test C++17 Filesystem withex1.cpp
#include <iostream>
#include <filesystem>
int main() {
for(auto &file : std::filesystem::recursive_directory_iterator("./")) {
std::cout << file.path() << '\n';
}
}
Compile it
clang++ -std=c++17 -stdlib=libc++ -Wall -pedantic ex1.cpp -o ex1 -lc++fs
Run it
./ex1
The output:
"./ex1"
"./ex1.cpp"

How to build nodejs as a shared library from source code

I need to include node.h in my c++ project, I tried to build node from source code using:
./configure
sudo make
I got a node executable and some object files and .a files, I need to build as .so file to use it in my c++ code.
I tried to build libnode, but I got cmakelists error and this is not official nodejs project.
if anybody know how to build nodejs from source code as .so file will be great, a similar question in a google group but the answer is not working.
Support for building as a shared library has been added in to node mainline. Please see PR 6994 and specifically this comment.
I just ran
git clone https://github.com/nodejs/node.git
cd node
git checkout v6.9.4
./configure --shared
make -j4
which produced:
ubuntu#server:~/node$ find . -name libnode.so\* -exec ls -la {} \;
-rwxrwxr-x 2 ubuntu ubuntu 31576776 Jan 6 18:57 ./out/Release/lib.target/libnode.so.48
-rw-rw-r-- 1 ubuntu ubuntu 387 Jan 6 18:57 ./out/Release/.deps/home/ubuntu/node/out/Release/lib.target/libnode.so.48.d
-rw-rw-r-- 1 ubuntu ubuntu 4202 Jan 6 18:57 ./out/Release/.deps/home/ubuntu/node/out/Release/obj.target/libnode.so.48.d
-rwxrwxr-x 2 ubuntu ubuntu 31576776 Jan 6 18:57 ./out/Release/obj.target/libnode.so.48
ubuntu#server:~/node$
I think it is easier to build in a static library as shared requires the addition of '-fpic'.
For my projects (under Linux) I use this script to built a static node.js library:
#!/bin/sh
# This script is LGPL feel free to use it!
if test ! "$#" = "1"; then
echo "Run with the archive in parameter:"
echo "\t${0} ./node-v0.XX.XX.tar.gz"
echo "\nIt will build a ./libnode_static.a in current dir"
return
fi
HERE=$PWD
#Extract Tarball
tar xf $1 | exit 1
DIRNAME=`echo $1 | sed s/.tar.gz//g`
cd $DIRNAME
#Patch node.gyp to build in static
sed -i "s/'type': 'executable',/'type': 'static_library',/g" ./node.gyp
#Patch node_main.cc to rename the main in node_main
sed -i "s/int main(/int node_main(/g" ./src/node_main.cc
#Build Node.js
./configure
make -j8
#Move to build directory
cd ./out/Release
#Extract .a
#Cleanup if previous build
rm -fr *.tmpd
echo "== Extracting *.a =="
#Make sure we create a directory
#for each.a as some .o might
#have the same name
for a in `ls *.a`
do
echo "\t${a}..."
mkdir "$a.tmpd"
cd "$a.tmpd"
ar x ../$a
cd ..
done
#Repack in a single .a
find . -iname "*.o" | xargs ar rcs libnode_static.a
#Cleanup
rm -fr *.tmpd
echo "== DONE =="
#Move in start directory
mv ./libnode_static.a ${HERE}/
cd ${HERE}
#Sanity CHECK
echo "== Performing Sanity Check =="
TMP_FILE=`mktemp /tmp/XXXXXX.cxx`
TMP_EXE=`mktemp /tmp/XXXXXX`
cat << . > ${TMP_FILE}
int node_main( int argc, char **argv);
int main(int argc, char ** argv )
{
node_main( argc, argv );
return 0;
}
.
#Try compiling
g++ ${TMP_FILE} -o ${TMP_EXE} -lnode_static -ldl -pthread -L.
#Try running
RET=`${TMP_EXE} -e "console.log('okfromnode')"`
if test "x${RET}" = "xokfromnode"; then
echo "== Sanity check OK =="
else
echo "== Sanity check FAILED =="
exit 1
fi
rm ${TMP_FILE} ${TMP_EXE}
echo "== Node.js is now built statically in ./libnode_static.a =="
exit 0
Run it as follows :
sh script.sh node-v0.10.XX.tar.gz
If everything goes well you should get a libnode_static.a in current directory.
Use it with a code like this:
int node_main( int argc, char **argv);
int main(int argc, char ** argv )
{
/* Here we spawn a node.js instance */
return node_main( argc, argv );
}
And compile like this:
g++ ./test.cxx -o ./my_node -lnode_static -ldl -pthread -L.
And you have embedded node :
./my_node -e "console.log('Hello World')"
#Outputs
Hello World
Hope this helps.
This is how i did it in windows. Except for the build procedures, everything should be same.
Nodejs uses node-gyp for building. You can read this for building and installation. Or just git clone the repository.
Open node.gyp in the node-vX.XX.XX and find
'targets': [
{
'target_name': 'node',
'type': 'executable',
change the executable to shared_library.
Run vcbuild.bat in windows or for other platforms follow instructions.
Update:
https://gist.github.com/aklen/849f3460b7a028c9aed8a84e1d4cecb7
Windows
.\vcbuild release vs2017 dll x64
.\vcbuild release vs2017 dll x86
.\vcbuild debug vs2017 dll x64
.\vcbuild debug vs2017 dll x86
Linux / MacOS
./configure --shared --release
make -j4
./configure --shared --debug
make -j4
Other build options
https://github.com/nodejs/node/blob/master/BUILDING.md