I was curious if Flight SQL will be an alternative to libraries like turbodbc? I am using turbodbc to query many different database flavors to return data in a pyarrow table, but I have had to add the OBDC drivers to my Docker image.
Is this something that can be used to query databases as a read-only user, or would something need to be done server side to support Flight SQL?
Flight SQL requires something database-side, yes. It's lower level than something like ODBC or JDBC, which (for instance) don't specify anything about the wire protocol; instead, it's a set of libraries intended to be used with a particular RPC protocol.
That said, it is intended to support use cases like yours. It's just that implementations are forthcoming (and things like Python bindings). Reaching out on the Arrow mailing list may get some more attention: dev#arrow.apache.org
UPDATE 2022/07: the more direct alternative to something like turbodbc, ODBC, JDBC, etc. in the Arrow ecosystem is under development as "ADBC": https://github.com/apache/arrow-adbc This is an API abstraction layer, so it doesn't require a remote server or local proxy, so long as you can somehow bend the database (client) to the proposed API.
Related
I'd love to be able to seamlessly send and receive Protobufs from my C++ applications to a Database and back. I'd be happy to create some kind of adapter on the client side, or to leverage an existing plugin on the Database side.
Has anyone encountered something that does this?
EDIT: As for the flavor of database, I'm open to all options, but had assumed a traditional relational database (PostgreSQL, MySQL, etc). If this is the project that forces me to try one of the fancy modern DB technologies, then so be it
I want to create an application that, when executed, has runtime functions that are accessible by other applications.
For example, a C++ application that stores values in files and retrieves this information. While this application is running, any other C++ applications could access it's save and retrieve functionality to save and retrieve data, but it should have no other connection to this system.
Sounds like a simple job for web services, or a remote database, or even an LDAP server.
Store and retrieve are operations common to all of these.
If the goal is to learn some specific technology, then ask a more specific question. Otherwise, don't reinvent any wheels. There are plenty of things out there for store and retrieve.
One of the simplest "store and retrieve" APIs I know of is Berkeley DB or Sleepycat.
We built a giant, clustered, simple key based database for a major telecom company using LDAP on top of Berkeley DB (aka Sleepycat). All open-source software and commodity hardware and it supports mission critical operations for millions of customers.
A more modern rendition of this might use memcached as well.
If you go HTTP based, you can use something simple as libcurl against an Apache web server to implement "RESTful" services with GET and PUT commands.
If you run it locally (same server), and access via localhost (127.0.0.1) then there is very little latency in the TCP stack, and it amounts to little more than memcpys at the kernel level.
simple message passing would do, say, JSON over ØMQ, or i.e. all in all, msgpack-rpc or protobuf-remote or Cap'n Proto RPC
I am currently writing a client-server app for the iOS platform. The client is written in Obj-C, and the server uses C++ on OSX11.9. Since I intend to run the server software on an Ubuntu dedicated server, I am trying my best to keep the serverside code portable.
To store data about users and user-game-relations I intend to use an SQL database (most likely MySQL or possibly PostgreSQL since I'm familiar with those). I know that it is possible to read from/write to the database through a filedescriptor just like I do in my TCP module, but I wish to utilize a higher-level SQL communications API to make the programming process quicker.
Can anyone recommend me a good open source/free SQL API for *NIX C++? Any help would be appreciated. Thanks in advance!
You have several options here:
Use native database SDK. They are usually distributed along with the database installation or as separate downloads/packets. The upside is you can get maximum speed out of it. Downside is that you'll be limited by your initial choice - no switching afterwards without rewriting part of application.
Use a C++ ORM (example: ODB). This gives you DB independence along with some tasty features, at the cost of slightly reduced speed.
unixODBC supports both MySQL and PostgreSQL. Take a look at it.
I want to write an application which should be able to connect to multiple databases (this will be configured by parameters at startup). The application will have different queries for each database engine, this is not a problem.
The problem is that I want to be able to connect to different database engines. Java has JDBC, Perl has DBI. What does C++ have?
What's more I don't want to use database drivers with too strict licences (commercial ones). GPL could be - but I'd like to avoid that.
Virtually every database engine in existence provides an ODBC interface. I think JDBC is actually a clone of ODBC.
What you want, then, is a C++ wrapper for the ODBC API, that implements RAII to make sure that database resources are released in case of exception, etc. For example: http://simpledb.sourceforge.net/
There is the older OLE connections. Using OLE, you could connect to a Flat File, Oracle, SQL, or MySql database provided you have the correct drivers installed.
ODBC is most compatible and most low-level. OLE DB is higher level and easier to work with, so if you find OLE DB provider for all your possible DB systems, it is the way to go. Otherwise ODBC is your option as virtually all DB systems support it.
EDIT: View this link: http://blogs.msdn.com/b/sqlnativeclient/archive/2011/08/29/microsoft-is-aligning-with-odbc-for-native-relational-data-access.aspx This makes ODBC the only proper choice. :)
The C++ object-relational mapping system ODB from the company Codesynthesis can be used by GPL version 2 software.
http://codesynthesis.com/products/odb/
Here is a blog entry where they describe why they chose to use native C APIs instead of ODBC to connect to the databases.
http://codesynthesis.com/~boris/blog/2011/12/09/oci-mingw/
Speed was one of the reasons.
I'm wondering what kind of persistence solutions are there for C++ with a SQL database? In addition to doing things with custom SQL (and encapsulating the data access to DAOs or something similar), are there some other (more general) solutions?
Like some general libraries or frameworks (something like Hibernate & co for Java and .NET) or something else? (Something that I haven't even thought of can also be welcome to be suggested)
EDIT: Yep, I was searching more for an ORM solution or something similar to handle sql queries and the relationships between tables and objects than for the db engine itself. Thanks for all the answers anyway!
SQLite is great: it's fast, stable, proven, and easy to use and integrate.
There is also Metakit although the learning curve is a bit steep. But I've used it with success in a professional project.
It sounds like you are looking for some ORM so that you don't have to bother with hand written SQL code.
There is a post here that goes over ORM solutions for C++.
You also did not mention the type of application you are writing, if it is a desktop application, mobile application, server application.
Mobile: You are best off using SQLite as your database engine because it can be embedded and has a small footprint.
Desktop App: You should still consider using SQLite here, but you also have the option with most desktop applications to have an always on connection to the internet in which case you may want to provide a network server for this task. I suggest using Apache + MySQL + PHP and using a lightweight ORM such as Outlet ORM, and then using standard HTTP post calls to access your resources.
Server App: You have many more options here but I still suggest using Apache + MySQL + PHP + ORM because I find it is much easier to maintain this layer in a script language than in C++.
MySQL Connector/C++ is a C++ implementation of JDBC 4.0
The reference customers who use MySQL Connector/C++ are:
- OpenOffice - MySQL Workbench
Learn more: http://forums.mysql.com/read.php?167,221298
SQLite + Hiberlite is a nice and promising project. though I hope to see it more actively developed. see http : // code.google.com/p/hiberlite/
I use MYSQL or SQLite.
MYSQL: Provides a server based DB that your application must dynamically connect to.
SQLite:Provides an in memory or file base DB.
Using the in memory DB is useful for quick development as setting up and configuring a DB server just for a single project is a big task. But once you have a DB server up and running it's just as easy to sue that.
In memory DB is useful for holding small DB such as configuration etc.
While for larger data sets a DB server is probably more practical.
Download from here: http://dev.mysql.com/
Download from here: http://www.sqlite.org/