many thanks in advance for taking the time to look at my question.
(I am aware of this question Nonrelational Databases for C++, but my needs are a bit different and it only has one answer.)
I am developing a commercial C++ library that must, among other things, persist messages. I would like to avoid reinventing the wheel by writing my own DBMS. Unfortunately, I have the following restricting criteria:
It must be usable from C++ - I'm writing a C++ library. Bindings are potentially acceptable, if the level of effort to make them work isn't too high.
I need an embedded database. Stand-alone will not work.
I want to avoid a relational database. In addition to concerns about performance overhead, there are technical politics beyond my control as a developer that discourage a relational database.
I need a permissive free software license. It'll be hard to buy licenses, but the client doesn't want to give his source away.
I'd like a solution that's established (been around for at least a little while, beyond the experimental stage, has been used by several projects).
Sadly, the two go-to choices don't work because of the above:
-SQLite is relational
-BerkeleyDB is GPL or commercial
Again, thanks for any help.
Use SQLite in b-tree mode. Public domain. Avoids politics. Let's you work around the political issues by avoiding the SQL interface for performance-critical paths, and optionally using the SQL parser path for those queries that are not on the critical path.
Both Tokyo Cabinet and QDBM are LGPL and have C APIs.
Related
I have written a C++ program that using database with 5 tables(and thousands of row). But there is performance issue in that approach. Now, I will try to use C++ data types and functions for that. Which one is best for that situation. I am considering to use struct in vectors. Also I can use libraries for that if exists. Can they solve my problem?
By the way my DB is PostgreSQL.
Take a look at Berkeley DB, it is a popular embedded database that has good performance.
Latest versions offer a SQL-like API, this can simplify the migration of your code from your a classical SQL database.
However, be careful of the license terms of this library. It is available in both a GPL-like license and commercial license (i.e. no LGLP-like license is available). Depending of your application (and your wallet), this may be an issue.
Hi I want to know why people develop library applications and employee management applications in C++ (this application, for example), when clearly the same thing can be done in C# and VB.NET in a much prettier way. Even banking applications are mostly in C++. Is there a good reason why, apart from the fact that compiled C++ code executes faster?
Can anyone shed some light on this?
C: 1972
C++: 1979
C#: 2000
Now think of the lifetime of a library, especially in a bank, plus, you get to use the libraries (theoretically) on almost every computersystem in existence (as opposed to C#)
You will also still find a lot of COBOL (1960) there.
The main reasons for C++ for say banking applications is:
Legacy code. A large financial firm typically has ~10-20-30 years of business specific C/C++ libraries developed in-house, plus a bunch of business specific vendor libraries which may not be available for C#
A LOT of that financial code runs on Unix/Linux. While you can purely theoretically build C# code for Linux, it's definitely NOT an established technology you want to bet billion dollar amounts on.
C++ is usable on other types of systems, whereas c# and vb.net are not.
Apart from technical reasons (such that C++ is an "unmanaged" language with quite different capabilities and properties than .NET languages), this can simply be due to preferences. Not all people find that C# and VB.NET are the best tool for every task. Or the prettiest. Why do you think so? And why should others not have similarly good reasons for choosing another tool of their liking?
Update, in reply to Konrad's comment:
It's correct that "preference" is indeed too narrow a term. There's other facets to it:
Managers / bosses can turn their (possibly badly informed) preferences into business policies;
A corporation's decade-old codebase can mean that when it comes to choosing the programming language for some new task, you'll evaluate languages with a different perspective. You want to or need to reuse the existing code, so interop with the old code's language must be possible.
It might be a factor of the knowledge economy of a particular company. For example, the bigger a company gets, or the less staff turnover they have, the harder it will be to replace competence, process and tooling to accommodate, for example, a new language. C/C++ has been around for quite some time, and many developers as well as development shops have that background.
Concerning banking applications, the reason is, I would guess, mostly because you have a close to metal environment which allows you to utilise realtime programming in a dependable fashion.
Every language has its pros and cons and no one language is best for every application. Programs in C++are harder to write, but can take advantage of platform-specific hardware and features. Because they're compiled, they also tend to run a bit faster. C# programs are easier to write, but aren't able to access underlying resources and can't be ported to non-Windows platforms very easily.
In short, it really depends on the application needs. If you need raw speed and explicit resource management, go with C++. If you want ease of coding and clarity, go with C#.
What are the technical questions I simply must have answers for before I approach someone about introducing a new language?
I'm looking for the list of technical questions that without a really good answer, I should not even waste anyone's time by proposing that we use language X.
PS: (def X clojure)
A crash course in politics for engineers...
Despite all the mission-statement baloney meant to sound noble and emphasize community support, the real purpose of every business is return on investment or, equivalently, maximizing shareholder value. If it's a government agency, it's kind of still the same question but the legal owners will have no direct influence and instead you will have proxy owners, such as higher agencies or powerful individual officials.
Decisions, however, are almost always made by agents, and so the principal-agent problem (also called the agency dilemma) appears; the agents (the management) will make a decision in their interest, and not necessarily according to the shareholder's interest as is theoretically required. In a government agency this is almost 100% of the consideration.
Sadly, this stirs in all the Dilbert and Parkinson's Law complexities.
The best you can conclude is that decisions will be justified on the basis of risk, cost, and benefit, but will tend to be made on the basis of what credit and blame is in store for the agent and understood by the agent, which is a narrow risk consideration of questionable value to the principal but at least an identifiable one.
So, we should now apply this to the language question. Your manager is likely to avoid threats, risks, scandals, and controversies. His application of the principals's concerns will be mainly through the constraints of budgets and expectations. Here are some examples that should be mostly self-explanatory.
If you want to use Java or PHP:
Everyone is doing it this way
This is the industry-standard approach for this type of problem
This is the low-risk approach
Similar systems have been done many times in Java/PHP
(That's the "no one ever got fired for buying IBM" argument.)
If you want to use Ruby:
Ruby is in the Tiobe top-ten (not quite an industry standard, so this is the best you can do)
PHP and Java are higher-cost technologies (he probably has a budget as an attempt to mitigate the principal-agent problem)
PHP and Java are going to be out of fashion "soon" (maybe not, but phrased as a "risk of appearing to stupidly use old tech', and implying the lack of later credit and recognition)
Ruby is an advanced language with powerful abstractions for cost-effective development (a weak argument for the agent, but offers the possibility of credit. The least effective of all the arguments.)
If you want to use Clojure:
You better prototype the system on weekends and evenings and present it as a solved problem.
Emphasize parallel Java / Clojure development ("if necessary the entire system can be written in Clojure Java")
Make all the Java arguments and then say something about "the best of both worlds"
Productivity with a language is neither the only factor, nor a simple scalar in itself. Important questions include:
How easy is it to learn the language, if it's not already familiar to people on the team?
How easy is it to become expert at the language?
Does the team have access to one or more language experts who have the bandwidth to do the necessary mentoring?
Are good learning materials (books, blogs, tutorials) and support channels (fora, IRC, mailing lists) available?
Does the language (or some framework in that language) allow a competent programmer to write the software faster than what you're using now?
How maintainable is the language? How readable is the syntax to a competent programmer encountering someone else's code for the first time? (Think of APL and Perl.)
Is the language somehow better applicable to your problem domain than what you're using now (e.g., functional languages for distributed computing)?
How well does the language/platform meet business needs not related to development speed (e.g., performance, scalability)?
What are the available tools like, and what do they cost? Is there a debugger available? An IDE? Refactoring and unit test support built into the IDE? Build management and deployment tools?
So much depends on what you're currently using, what you're switching to and why that it's difficult to answer. But these are always important:
What can I do if I choose a new language that I could not do before?
What could I do faster than I can currently with the new language?
How will the rest of the team cope with the introduction of the new language?
If I left, could someone else new to the language pick up where I left off without too many problems?
What is the business case?
It comes down to ROI (Return On Investment).
It is not only about an individual's productivity but:
the whole team
impact on product lifecycle
maintainability
etc.
How easy is it to pick up? I find this is not that important.
Does it have IDE support? Pretty important, but you can work without it.
Is there a debugger available? I think this is the most important question I would ask. Once you have a working debugger, you can usually get anything done.
We hired a team this year and decided to use Clojure as our weapon of choice. The team's background was primarily Java-based but also a wide variety of other languages for hobby work.
The criteria we considered were:
Can we leverage the Java/JVM background of the team and integrate with an existing Java-based product?
Can we achieve performance on par with Java?
Can we build thread-safe concurrent maintainable programs?
Can we leverage a higher level of abstraction
Can we hire/train people to work in the language?
Can we maintain a large codebase in the language?
Are sufficient tools available to work effectively in the language?
Is there an active community of people growing the language and libs?
We seriously considered Groovy, Scala, and Clojure. I really enjoy Groovy for lightweight apps but I had serious questions about performance. Scala and Clojure both have lots to offer on all of the points above. In the end, our problem domain involves a lot of symbolic manipulation and we felt that Clojure would be a better match but I suspect Scala also would work well.
What will your new language offer that an existing language doesn't already?
We have languages that do just about everything in every way today. So before introducing a new language, make sure there isn't one already existing that does everything your new language does. And make sure you know exactly what features your new language will offer that aren't offered in the same combination or at all by other languages.
Unless of course you're just doing this for your own education - in which case forget this question and have at it!
How will this improve my productivity?
If this cannot be answered pack up and go home.
What's the point? / Why?
How will it make my job easier?
Q1: Can I hire people with these skills?
Q2: When I call our outsourcing partner account managers, and ask how much would a typical fixed-cost project cost, if done in the usual way, or done using language X, is the multiplier more than 1?
Q3: Does everyone else in my department also have a favorite language that does about the same job as my favorite language, and should their favorite languages be used as well? What are the practical consequences of this?
A good question to ask is what is the size of the community around the language/framework. For instance, ruby/rails has a significant community around it, which would make me more comfortable that I would not be "the first kid on the block" to have to deal with a particular problem.
Why limit yourself to one language? Figure out which problems are solved best by which language and offer up services. If the bandwidth between the services is too high, then migrate the problematic services together based on which language solves both best.
I'm looking for options for abstracting database server details away from my application (in c++), I'd like to write my code to be independent of the actual database backend. I know MySQL has a nice library, but I don't want to be tied to a single database implementation. Are there good options for this?
SOCI is good. Supports multiple databases, works well, modern C++ style API, works with boost.
My opinion is to forget about a cross-database driver, and focus on finding or creating a cross-database Data Access Layer. A few reaons:
Complex queries (read: anything that's not a toy) invariably end up using one or two database-specific features. LIMIT and OFFSET for example, commonly used for paging, isn't universal.
Sooner or later you'll want bulk insertion, and you'll want it to be as fast as possible, because 3 hours is better than 6 hours. Every database has a different "optimum" way to do this, so your DAL will need to special-case this anyways.
Different databases may expose different constraint mechanisms—even custom column types—that can be be worth taking advantage of where possible (PostgreSQL is wonderful for this).
If you want to do any application level caching, you'll need a DAL anyways.
So, go ahead and use libmysql by itself - just hide it behind a compiler firewall in your DAL, and be prepared to swap it out later. You can protect yourself from shifting infrastructure without having to use a lowest-common-denominator SQL wrapper.
If that doesn't jive with you, check out SQLAPI++.
many apps use odbc (via unixODBC for instance), there's also otl. on windows you could use ado.net from managed c++ or the old ado com interfaces...
Qt provides a database abstraction layer. See: http://doc.trolltech.com/4.6/qsqldatabase.html.
libodbc++ provides a pretty good API.
Also the big guys Qt (see Kyle Lutz' answer) & wxWidgets have db abstraction layers, so it may be a good idea to use them if you plan to use/you're already using any other parts of those frameworks.
OpenDBX and libzdb are two lightweight candidates. Libgda for GNOME.
I am just beginning to learn how to write software that accesses an SQL server. It seems that each server implementation (Postgres, MySQL, etc.) offers API libraries for various languages (my code is in C and C++, though solutions for Java and Python would also interest me). I'm a little wary of depending on these libraries, however, because I'd prefer a vendor-neutral solution.
As near as I can tell, Microsoft's ODBC API was meant to solve such problems for C/C++ (and JDBC for Java); unixODBC seems to be one popular implementation. Am I right even so far?
Moreover, do any such libraries provide an object-oriented interface? It would be nice to not simply embed SQL queries into another, more featureful language; I'd like to have a wrapper that mimics the style of the rest of the language, too.
So is there a preferred solution along those lines? Am I asking for something weird?
As near as I can tell, Microsoft's ODBC API was meant to solve such problems for C/C++ (and JDBC for Java); unixODBC seems to be one popular implementation. Am I right even so far?
Yes. The equivalent of ODBC or JDBC for Python is called the DB-API. Perl's equivalent is called DBI.
Moreover, do any such libraries provide an object-oriented interface? It would be nice to not simply embed SQL queries into another, more featureful language; I'd like to have a wrapper that mimics the style of the rest of the language, too.
Yeah, there are a bunch of things like this for different languages. C# has LINQ, Smalltalk has Roe and GLORP, Python has SQLAlchemy and SQLObject (and Django in Python has quite a bit of query power built into its ORM (see Simon Willison's notes)), Ruby has ActiveRecord, and so on. I don't know what you'd use in C++ but I bet it has to use a lot of ugly template hacking to approach these.
All these choices might seem overwhelming, but chances are your choice of language will be shaped by something other than the convenience of working with relational data. (If not, you should consider Prolog.) That will probably tie you more or less to some ORM you hate just like the rest of us.
Indeed, ODBC/JDBC are libraries that help make the calling interface standard between vendors, but you're right that each respective RDBMS has its own flavor of SQL. ODBC/JDBC doesn't help abstract the SQL syntax.
One solution to move literal SQL out of your application code is to implement queries in stored procedures that reside in each database back-end, and then use ODBC/JDBC to call the stored procedures. You can define stored procedures with similar names and calling interface for each flavor of RDBMS you use. But be aware that the stored procedure language is also variable from one vendor to the next.
Another solution is to use an "object-relational mapping" technology such as Hibernate for Java, or NHibernate for .NET. These technologies can make it feel more "object-oriented" to work with databases, and free you from writing literal SQL in many cases.
But most ORM tools tends to focus on very simple queries. If your query is at all complex (using a GROUP BY or a JOIN for instance), using the ORM tool is harder than using literal SQL.
See also "Good ORM for C++ solutions?"
If SQL troubles you that much, you're probably not going to be happy using an RDBMS at all. Some programmers don't see the value to the Rules of Normalization, for instance. If that's true for you, you might want to look into the emerging technologies for non-relational data stores, including:
BerkeleyDB
Project Voldemort
CouchDB
ODBC/JDBC attempt to abstract away the database interface to provide a consistent programming model. Bear in mind that, by using such a least-common-denominator interface, you cannot take advantage of specific, non-standard features that a given DB may offer.
To get an object oriented interface to your data model, look into Object Relational Mapping (ORM) solutions such as Hibernate. ORM solutions map your objects to their representation in a relational database, generally making data persistence much simpler from an application programming perspective.
Quince is a C++ library that lets you use C++ syntax and C++ types with the feature set of SQL. Currently it supports PostgreSQL and sqlite only, but new backends can always be added. See quince-lib.com. (Full disclosure: I wrote it.)
Take a look at Qt. It is not a library, but a complete framework. It has a very excellent SQL module.
Qt SQL is an essential module which provides support for SQL
databases. Qt SQL's APIs are divided into different layers:
Driver layer
SQL API layer
User interface layer
http://doc.qt.io/qt-5/qtsql-index.html