In our current project, we need some high level DBI for different databases. It should provide the following features:
in memory cache - the DBI should be able to cache all reads, and update the cache on writing calls (the application we are coding on is heavy threaded, and needs fast access to the current data all the time). The memory cache will be based on boost::multi_index
automatic sql building - We don't want to parse a sql statement to lookup in the memory cache
As we need to provide functions for: defining a table layout, do selects, do inserts, do updates, joins, ..., the interface will get very complex.
We need a good way to invoke the interface function.
There are many styles around, but we could not find any useful for our usage.
Here a few examples:
SOCI
sql << "select name, salary from persons where id = " << id, into(name), into(salary);
We don't want some SQL statements, so we would have to define what and from a different way.
pqxx
Conn.prepare("select_salary",
"select name, salary from persons where id = $1")
((string)"integer",prepare::treat_direct);
The heavy usage of the overloaded operator() is just ugly, but it could work for us too.
Any suggestions how to design the interface?
How about using object relational mapping? Here's some code fragment ideas off the top of my head - I've only done this in Python, never in C++, and only for fairly simple databases. There's a list of frameworks on Wikipedia that should avoid too much wheel-related R&D.
class people: public dbi_table
{
// id column handled by dbi_table.
name: string_column;
salary: money_column;
};
class cost_center: public dbi_table
{
name: string_column;
office: foreign_key<offices>;
};
class people_cost_center_link: public link_table
{
// Many-many relationships.
};
Then you can manipulate records as objects, all the relational stuff is handled by the framework. Querying is done by defining a query object and then getting an iterator to the results (see the ODB wikipedia page for a code example).
I would do it like this (and it'd be good in c++ point of view, dunno if it's correct database stuff):
struct Handle { int id; }
class DBI
{
public:
virtual Handle select(int column_id)=0;
virtual Handle select(int column1, int column2)=0;
virtual Handle id(int id)=0;
virtual Handle join(Handle i1, Handle i2)=0;
virtual void execute_query(Handle i)=0;
};
Usually these functions would be implemented like this:
Handle select(int column_id) {
return new_handle(new SelectNode(column_id));
}
where new_handle function would just insert SelectNode to std::vector or std::map and create an handle for it.
Related
From what I have seen, Qt documentation and majority of examples online assume that we are happy with (column, row)-based lookup in data(). But what If my table is based on a custom structure? For instance let's have:
struct MyDrive
{
QString serialNo;
QString user;
QString pc;
QString ipAddress;
QString category;
};
where serialNo is the key. So any operation from outside (imagine the model having implemented a listener) uses it for removing/modifying an item, making QMap as an ideal candidate.
But how to connect this structure with QModelIndex's data? QAbstractTableModel::data asks for data with (column,row) as key, making it more suitable for QVector<QVector>> or something similar (somewhere I read I should avoid using containers with non-constant access time (like map) in data()).
I can imagine using, well, a map with QModelIndex as key and serialNo as value, which would be used as key to my (serialNo-based) map but this looks very inefficient -- QModelIndex addresses concrete entry (serialNo, user, pc, ...) after all so we'd be duplicating the same item over and over again. I was also thinking about having a <serialNo, MyDrive*> map but this is just a workaround to an ugly design decision.
I can't believe I'm the first one with this scenario, so how is it usually solved?
You can use QAbstractItemModel::match to find items by serial
qt help
And enter all the necessary data into the table. This will allow you not to use containers, but how efficient is this a question...
Second solution is subclass AbstractItemModel Reference. Now you can do what you want and use any container by implementing data() function.
I'm trying to create a performance monitor of sorts to run on a Particle board (STM32 based). I'm used to programming in c so the OOP approach is a bit new but I think it would fit well here.
For the purpose of this question let's assume I have two types of monitors:
Frequency. The application can call a "tick" method of the monitor to calculate the time since it last ran and store it.
Period- call a "start" and "stop" method of the monitor to calculate how long a process takes to run and store it.
What I would like to do is to create instances of these monitors throughout my application and be able to report on the stats of all monitors of all types from the main module.
I've read about the singleton design pattern which seems like it might be what I need but I'm not sure and I'm also concerned about thread safety with that.
I'm thinking I will create a "StatMonitor" class and a derived class "FrequencyMonitor" and "PeriodMonitor". Monitor would be a singleton and everywhere I wanted to create a new monitor I would request an instance of "Monitor" and use that like so:
freqMonitor * task1FreqMonitor = StatMonitor::GetInstance()->Add_Freq_Monitor("Task1");
The StatMonitor would track all monitors I've added and when I wanted to print the stats I could just call the printAll method which would iterate it's array of monitors and request their results like so:
StatMonitor::GetInstance()->PrintAllStats();
Am I going down the right path?
Your path sounds good, except that FrequencyMonitor and PeriodMonitor should not derive from the class that "manages" all these monitors (let's call it MonitorPrinter).
MonitorPrinter should be a singleton and could look like this:
class MonitorPrinter
{
public:
static MonitorPrinter& getInstance()
{
static MonitorPrinter monitorPrinter;
return monitorPrinter;
}
void printAllStats()
{
for (const auto& [_, frequencyMonitor] : _frequencyMonitors)
frequencyMonitor.print();
for (const auto& [_, periodMonitor] : _periodMonitors)
periodMonitor.print();
}
FrequencyMonitor& getFrequencyMonitor(std::string name)
{ return _frequencyMonitors[name]; }
PeriodMonitor& getPeriodMonitor(std::string name)
{ return _periodMonitors[name]; }
private:
MonitorPrinter() = default;
std::map<std::string, FrequencyMonitor> _frequencyMonitors;
std::map<std::string, PeriodMonitor> _periodMonitors;
};
Demo
(The const auto& [_, frequencyMonitor] is a structured binding).
FrequencyMonitor and PeriodMonitor should not have anything to do with singletons, and from your description, they need not be part of a class hierarchy either (as they have different interfaces). If you want, you can prevent users (other than the MonitorPrinter) from instantiating these classes using other techniques, but I won't elaborate on that here.
In short, there is no need to use OOP here. Use a singleton to provide (and keep track of) the monitors, and implement the monitors to your liking. Be wary of thread safety if this is relevant (the above is not thread-safe!).
I'm learning SQLite and C++ within the Qt framework. As a learning project I am doing a simple image viewer which enables the user to tag images with keywords, categories, comments and ROI (for some later OpenCV functionality). It's a pretty simple database with some primary tables and some relational tables.
I think I've got the basics down and early tests show that my record data is being stored but in writing the methods to manage the database it looks like I am going to end up with a great many methods, particularly when I start to add in searching, sorting, deleting, etc.
This leads me to ask if I am going about this the correct way. I can see how some of my query logic (for search, sort) is probably better off being in a different "controller" type class and that all this manager class needs to do is handle the basic creation and deletion tasks, and just return a query object in response to a SELECT statement passed in as a string.
So, am I going about this in a reasonable way?
Manager methods so far:
bool openDatabase(QString name);
void closeDatabase();
bool createTables();
bool addKeyword(QString keyword);
bool addCategory(QString category);
bool deleteKeyword(QString keyword);
bool deleteCategory(QString category);
bool addROI(int x, int y, int width, int height);
bool deleteROI(int id);
bool addImage(QString name, QString path, QByteArray image, QByteArray thumb);
You probably should be using Qt's Model View Framework. The important classes there are QSqlQueryModel, QSqlTableModel and QSqlRelationalTableModel. To keep your UI isolated from the database structure, a reasonable approach would be to add view models for the particular use cases you have. You can then easily link those to, say, a Qt Quick based user interface.
There's nothing particularly wrong with function-oriented interface that you propose, except that it requires a lot of boring glue code to use it for user interfaces. It's best to factor such glue code as a proxy view model, since you're working to a documented API that can be then easily picked up by coworkers.
I have seen a DatabaseManager class somewhere and modified it. This is the insert function :
bool DatabaseManager::insert(const QString &tableName, const QString& columns, const QString &value)
{
bool ret = false;
if(db.isOpen()){
QSqlQuery query;
ret = query.exec( QString("INSERT INTO %1 (%2) VALUES(%3)").arg(tableName, columns, value) );
}
return ret;
}
You give it your table's name, the columns you want to fill and then the values. This is an example where I call it :
if( !dm.insert("Product", "ID, Name, Price, Notes, Category",
"'1', 'A DVD','10€', 'Some Notes', 'DVD'") )
{
//Note that each value is surrounded by an apostrophe
//Now whatever you want to
}
First a little back story on what I am trying to accomplish.
I am in the process of creating a custom HTTP Module whose purposes is to intercept messages to multiple (15+) different ArcGIS REST web services. The intercepted requests and/or responses will be stripped of any restricted information based on the current user.
For instance, a call that returns multiple layers might have certain layers stripped out.
Unmodified Response:
"layers" : [
{
"id" : 0,
"name" : "Facilities",
"parentLayerId" : -1,
"defaultVisibility" : true,
"subLayerIds" : [1, 2, 3]
},
{
"id" : 1,
"name" : "Hazardous Sites",
"parentLayerId" : 0,
"defaultVisibility" : true,
"subLayerIds" : null
},
]
Modified Response:
"layers" : [
{
"id" : 0,
"name" : "Facilities",
"parentLayerId" : -1,
"defaultVisibility" : true,
"subLayerIds" : [1, 2, 3]
}
]
There are numerous services available, all uniquely identified via a URL. Each service returns very different information and so needs to be filtered different. Additionally, each service may return the data in a variety of formats (HTML, JSON, etc).
As such, I will need to create a multitude of different filters to apply to HttpRequest.Filters and/or HttpResponse.Filters.
Example:
// Request for layers and the format is JSON
IPolicy policy = GetPolicy(userContext);
Filter filter = new LayerJsonResponseFilter(Response.Filter, policy);
Response.Filter = filter;
Request and response filters are implemented by inheriting from Stream (or another class that inherits from Stream such as MemoryStream). I want to be able to easily create new filters without reimplementing Stream for each filter.
A potential solution is described in here: http://www.west-wind.com/weblog/posts/72596.aspx
However, I want to simplify the solution without losing the flexibility of specifying many different transformations without reimplementing the stream. I think that I can accomplish this by:
Inherit from MemoryStream so as to reduce the reimplementation of methods.
Always operate on full content, rather than chunked content.
Replace the events with an abstract method (e.g., Filter())
I have considered two potential solutions.
Solution 1: Create Multiple Filters Inheriting from ResponseFilter
In this scenario each filter contains the logic for performing the filtration. There would be 15+ filters created all inheriting from a common ResponseFilter abstract base class like so:
// All filters will inherit from ResponseFilter
public abstract class ResponseFilter : MemoryStream
{
public ResponseFilter(Stream stream, Policy policy) { }
// Must be overridden in a derived class with specific Filter logic.
public abstract string Filter(string content);
// Overridden to cache content.
public override void Write(byte[] buffer, int offset, int count) { }
// Overridden to perform the filter/transformation before the content is written.
public override void Flush()
{
// Get stream content as a string
string content = Filter(content);
// Write new content to stream
}
}
This would be used in the following way.
// Example
var policy = GetPolicy();
var filter = new MapServiceJsonResponseFilter(response.Filter, policy);
response.Filter = filter;
The advantage to this option is that the number of classes is kept to a minimum. However, it becomes difficult to reuse any filter logic anywhere else in the application should it become necessary. Additionally, unit testing the filters would require mocking the Stream, another disadvantage.
Solution 2: Create Multiple Filters, Inject into a Common ResponseFilter
In this scenario, a single response filter is created. The actual filter logic or algorithm is injected into the filter. All filters inherit from an abstract base class FilterBase.
// Represents an HttpResponse Filter. Renamed to avoid confusion with
// the filter algorithm.
public class ResponseFilterStream : MemoryStream
{
public ResponseFilterStream(Stream stream, FilterBase filter) { }
// Overridden to cache content.
public override void Write(byte[] buffer, int offset, int count) { }
// Overridden to perform the filter/transformation before the content is written.
public override void Flush()
{
// Get stream content as a string
string content = _filter.Filter(content);
// Write new content to stream
}
}
// All filter algorithms inherit from FilterBase and must implement
// the filter method.
public abstract class FilterBase
{
protected TransformBase(Policy policy) { }
// Overridden to perform the filter/transformation.
public abstract string Filter(string content);
}
This would be used in the following way.
// Example
var policy = GetPolicy();
var filter = new MapServiceJsonResponseFilter(policy);
ResponseFilter responseFilter = new ResponseFilter(response.Filter, filter);
response.Filter = filter;
The advantage to this solution is that the filtration logic is completely independent of any classes that implement stream. The logic can be more easily reused if necessary. Unit testing is a little simpler as well as I do not need to mock the stream.
However, there are more classes (exactly 1) and the usage is a little more complex, though not terribly so.
Note: I'll probably want to rename FilterBase as to avoid confusion with ResponseFilter. Perhaps TransformBase.
I have several goals that I want to meet with either solution.
The solution must be highly testable. Unit testing will be used to check for correctness of the filters. It is imperative that testing is as simple as possible.
The solution must easily support the creation of multiple filters (15+).
The solution should be readable (i.e., easy to maintain).
I think that solution 2 is the best solution for this given scenario. I can test the filtration logic completely independently of Stream with minimal additional complexity. Either solution will support #2 and #3, so testing gets the edge.
What other considerations might there be? Are there better alternatives?
Solution 2 is obviously preferable. However, it seems that the major crux of the problem lies in the construction of the Filters themselves. Hopefully there is a lot of reusable composition within the Filter implementations. Can a new filter be "configured" from composite parts?
The diagram http://www.freeimagehosting.net/uploads/2fd3f4161c.png
Here's the Minimalist-UML diagram of an app I've been working on. It's supposed to simulate the management of a bunch of sensors relating to different measurements.
Please ignore the House class, the diagram is outdated...
However, I have trouble. Each sensor (sound, contact, heat, pressure, gas - All of these inherit from sensor) has an unique ID, starting at 0 for the first one and ending at the total number of sensors - 1.
For good practices' sake, where shall I store the total number of sensors, so the classes I'm using for input/output of files (saving and loading) and insertion of new sensors can access and increment that counter?
Thank you for your time!
One option would be to create a static function in your Sensor class that increments and returns a static counter variable. The constructor for Sensor could call this function to get an ID.
// header file
class Sensor
{
...
protected:
static int getId() { return id++; }
private:
static int id;
int myId;
};
// .cpp file
int Sensor::id = 0;
Sensor::Sensor(...)
: myId(getId())
...
{}
I'm ignoring threading and persistence issues here. But hopefully this gives you a better idea.
Whatever object creates the sensors should assign the identifiers to the sensors.
If multiple objects create sensors, then they should be assigned a pointer or reference to a provider of identifiers when they are created and they should query that provider for a new unique identifier as they create new sensor objects.
Your unique ID, like a database table ID will likely have some issues.
You will probably find, eventually, that your ID needs to persist across sessions--that your window's ID is used in some other relationship.
You may, some day, also find that it needs to be unique across multiple server/client sets.
I'm just suggesting that you should consider these issues off the bat.
As for where the ID should be generated, since all your "Sensor" classes inherit from one base class, I think I'd generate it via a threadsafe method in that base class--and I'd store it there as well.
what's the problem? do you use a Vector to store your sensors? define a Vector of holding sensor-objects in the house.
can access and increment that counter
you don't have to do this, the Vector does it for you
Have a look at the Singleton pattern assuming you don't want to do it with a database of some sort.