I have a Microsoft Foundation Class (MFC) CMap object built where each object instance stores 160K~ entries of long data.
I need to store it on Oracle SQL.
We decided to save it as a BLOB since we do not want to make an additional table. We thought about saving it as local file and point the SQL column to that file, but we'd rather just keep it as BLOB on the server and clear the table every couple of weeks.
The table has a sequential key ID, and 2 columns of date/time. I need to add the BLOB column in order to store the CMap object.
Can you recommend a guide to do so (read/write Map to blob or maybe a clob)?
How do I create a BLOB field in Oracle, and how can I read and write my object to the BLOB? Perhaps using a CLOB?
CMAP cannot be inserted into blob/clob since its using pointers.
first of all use clob
and store array/vector instead of cmap.
Related
I am coding a cpp project with the database "postgreSQL".
I created a table in my database its type is character varying(40).
Now I need to SELECT these data FROM the table in my cpp project. I knew that I should use the library libpq, this is the interface of "postgreSQL" for c/cpp.
I have succeeded in selecting data from the table. Now I am considering if it's possible to get the data type of this table. For example, here I want to get character varying(40).
You need to use PQftype.
As described here: http://www.idiap.ch/~formaz/doc/postgreSQL/libpq-chapter17861.htm
And just take a look here about decoding return values: http://www.postgresql.org/message-id/da7021e0608040738l3b0880a1q5a76b838937f8c78#mail.gmail.com
You must also use PQfsize to get field size.
I am not sure if it relates to "bitwise". I store a music file provided with different format. eg. MP3, WAV, midi... It needs to store the provided type in the DB. One of the solution is to create individual db fields/columns for each format. eg withMP3, withWav, withMidi... But once I add one more format, I need to create an extra column.
Is there any standard solution to store the format to one field? For example first digit store with mp3, second digit store with wav... Once I add one more file format, it just needs to append one more bit to the data, no need to add new column. I am not sure this question related to any aspect. Hope that someone can help me.
Many thanks!!
Turn that data into its own table (id, format, blob) then you can associate them with the rows in the other table via another table. That way the schema is independent of the number of formats.
I'm not sure why you try to store this information as fields. I would just store the mime type, that is normally enough information for a normal database.
I am pulling out large data from oracle database using cx_oracle using below sample script:
from cx_Oracle import connect
TABLEDATA = []
con = connect("user/password#host")
curs = con.cursor()
curs.execute("select * from TABLE where rownum < 100000")
for row in curs:
TABLEDATA.append([str(col) for col in list(row)])
curs.close()
con.close()
Problem with storing in list is that it ends up to about 800-900 mb of RAM usages.
I know I can instead save this in file and not store in list but I am using this list to display table using QTABLEVIEW and QABSTRACTTABLE MODEL.
Is there any alternate or more effient way where I can minimise memory usage of storing this data and also use it to display my table?
I have tried multiple possobilities, I don't think qsqltablemodel works for me. Though it load data directly from database, as you keep scrolling down it loads more and more data in table and hence the memory usage keep on increasing.
What I think will ideally work is being able to load set number of rows in model. As you scroll down it loads new rows but also at the same time unloads what's already there. So at any point of time we only have set number of rows loaded in model.
If you don't want to store all the data in RAM, then you need to use a model for you tableview that get's information from the database as needed. Fortunately, Qt natively supports this, and can connect to oracle databases.
You will want to look into:
http://qt-project.org/doc/qt-4.8/sql-driver.html
http://qt-project.org/doc/qt-4.8/sql-model.html
http://qt-project.org/doc/qt-4.8/qsqltablemodel.html
http://qt-project.org/doc/qt-4.8/qsqldatabase.html
Note this is c++ documentation, but it is fairly easy to translate to PyQt (I always use the c++ documentation despite never coding in c++). You may also want to subclass QSqlTableModel to provide slightly different behaviour to the standard interface!
I currently have a postgres database in which I store a data about a photo, along with the location as a JSON (using Django). The location is obtained through GooglePlacesAPI-
https://developers.google.com/places/documentation/search (search for Search Responses for example responses)
So currently, every photo has a location column which contains the JSON information for the place as obtained from the GooglePlacesAPI
Now I would like to use Postgres' spatial capabilities to query based on the location, however I am not sure how to do that and what schema changes are required. The Postgres documentation seems to indicate that there would be a new table required with the location's name, lat, lng and other information. So does that mean that every location will be saved in a different table and will have a foreign key referenced to that?
And so the JSON will need to be essentially flattened to be stored in the table?
If so, is there a recommended table format that would be good to store the location in so that I can get any other location data (say from Foursquare, FB, etc) and convert it to the format of the table before storing.
Geometry is not special: it's just another data type. So add a geometry column to your existing table. Assuming you have installed and enabled PostGIS 2.x:
ALTER TABLE mytable ADD COLUMN geom geometry(Point,4326);
Then populate the geometry data by extracting the data out of the location JSON column (which really depends on how the data are structured within this amorphous column):
UPDATE mytable SET
geom = ST_SetSRID(ST_MakePoint((location->>'lng')::numeric,
(location->>'lat')::numeric), 4326)
And that should be a start. Later steps would be to build GiST indices and do spatial queries with other tables or points of interests. You may also want to consider the geography data type, instead of the geometry data type, depending on your needs.
How do I get the size of data in a BLOB field in the Result Set? (Using C++ and MySQL Connector C++)
In order to read the data from the result set, I have allocate memory for it first. In order to allocate memory, I need to know the size of the blob data in the result set.
Searching the web and StackOverflow, I have found two methods: OCTECT and BLOB stream.
One method to find the BLOB size is to use OCTECT() function, which requires a new query and produces a new result set. I would rather not use this method.
Another method is to use the blob stream and seek to the end, and get the file position. However, I don't know if the stream can be rewound to the beginning in order to read the data. This method requires an additional read of the entire stream.
The ResultSet and ResultSetMetaData interfaces of MySQL Connector C++ 1.0.5 do not provide a method for obtaining the size of the data in a field (column).
Is there a process for obtaining the size of the data in a BLOB field given only the result set and a field name?
I am using MySQL Connector C++ 1.0.5, C++, Visual Studio 2008, Windows Vista / XP and "Server version: 5.1.41-community MySQL Community Server (GPL)".
You could do a select like:
select LENGTH(content),content where id=123;
where content is the BLOB field.
Regards.
see: LENGTH(str)