I have some numeric data that is given to me in INTEGER form. I insert it along with other data into SQLite but when I write it out The INTEGER Numbers need to be 8 digit Hex Numbers with leading zeros.
Ex.
Input
400
800
25
76
Output
00000190
00000320
00000019
0000004C
Originally I was converting them as I read them in and storing them as TEXT like this.
stringstream temp;
temp << right << setw(8) << setfill('0') << hex << uppercase << VALUE;
But life is never easy and now I have to create a second output in INTEGER form not HEX. Is there a way to convert INTEGER numbers to HEX or HEX numbers to INTEGERS in SQLite?
I'd like to avoid using C++ to change data after it is in SQLite because I've written a few convent export functions that take a queries result and print them to a file. If I need to touch the data during the queries return I couldn't use them.
I've looked at the HEX() function in SQLite but that didn't have desired results. Could I make a function or would that be very inefficient? I'm doing this over a really big data set so anything expensive should be avoided.
Note:
I'm using SQLites C/C++ interface with Visual Studios 2010.
You can use sqlite3 printf() function:
CREATE TABLE T (V integer);
insert into T(V) values(400), (800), (25), (76);
select printf('%08X', V) from T;
You can use sqlite3_create_function. See an example
Well not my favorite answer I've decided to add a column in the table that is the INTEGER value.
If someone finds a better way to do this I'm all ears.
EDIT:
After Implementing this answer and looking at it's effect on the program this appears to be a really good way to get the data without adding much to the run time load. This answer does add to the size of the Database a little but it doesn't require any extra processing to get the values from SQLite because this is just grabbing a different column in the query.
Also because I had these values to start with, this answer had a HUGE cost savings by adding them to the table verses processing later to get values I through away earlier in the program.
Related
I am simulating a DELETE command from SQL in c++. I've tried all the possible data types to temporary store the data from a certain table( a .bin file in my case), but none of them worked.
If I store the entire file at once in a char buffer[5000], the text is safe, but the integers somehow are stored on the address, and in the buffer they look like " }+;-." and so on.
If I select them one by one and concatenate each value to a similar buffer with strncat, and with itoa() if needed, I end up having them perfect but I can't navigate in it using specific data-type size.
I would apreciate very much a suggestion into this. I've literally searched all the internet by now. Thank you!
I want to store IP Address using c++ in sqlite3 DB in hex format. I am using TEXT as the format for storing hex values. I want to perform a check on the value being inserted in the DB. The value range I want to check for is 0x00000000 - 0xFFFFFFFF. For INTEGER type you can check by - CHECK (num BETWEEN 0 AND 100). Is there any way to add check constraint for Hex Values?
If there is an smarter way to store IP address in SQLITE please share with me.
Thanks
I think you have two main choices: (a) store as hex (i.e. text) and check "hex conformity", or (b) store as integer and print it as hex when reading data.
There may be several reasons for preferring the one over the other, e.g. if the application actually provides and receives integer values. Anyway, some examples for option (a) and option (b).
Option (a) - store as text and check hex conformity
the probably simplest way is to use a check based on GLOB as remarked by CL:
value TEXT CONSTRAINT "valueIsHex" CHECK(value GLOB "0x[a-fA-F0-9][a-fA-F0-9][a-fA-F0-9][a-fA-F0-9][a-fA-F0-9][a-fA-F0-9][a-fA-F0-9][a-fA-F0-9]")
If logic goes beyond that supported by GLOB, you could install a user defined function, either a general regexp() function or a custom function for your specific needs. This could be the case, if you want to store a complete IP-address in one field and still want to do a conformity check. For help on general regexp() function, confer this SO answer. For help on user defined functions confer, for example, this StackOverflow answer.
Option (b) - store as int and print as hex
If your application is actually working with integers, then change your db schema to integer and add a check like:
value INTEGER CONSTRAINT "valueIsValid" CHECK(value <= 0xFFFFFFFF)
For convenience, you can still print the value as hex using a query (or defining a corresponding view) like the following:
select printf('%08X', value) from theTable
I have searched around a bit and found some people asking a similar question, but I have not found an answer I can make work.
I have tab delimited .txt files which I need to read in to a SAS database. The files contain a serial number which is 18 numbers long so SAS imports this as "5.2231309E17".
Ideally SAS would import all the fields as if they were text, not numbers.
To add a complexity to this, the import files have 2 different formats, these are only visible once the file is open, I cannot tell which format the file is from the name. Also there are no column names in the file. So I don't know which column is which until I have read in the file.
Currently my starting point is:
data Readin;
infile foo dsd dlm='09'x truncover;
input item1-item25;
run;
foo is the file something like 'c:\myfile.txt'
Any help is appreciated.
There are two separate issues here. One is that the "9.234E17" is displaying in scientific notation, and two that you are reading in numbers that can't be stored exactly as numbers anyway.
First, this is how the BEST12. format works, which is the default numeric format for things like this. It's not truncating it in a meaningful way; if you simply change the format, to BEST32. for example, it will display the entire number, within the limits of precision, and it will always act as if it were the full number, again within the limits of precision; if I took 12345678, formatted BEST6., it would display as 1.23e7, but if I said if x=12345678 then do; put x; end;, it would put x, as it would be exactly equal to that value.
However, that last part is important, and the second part of your problem. You can't store 18 digit number precisely; 15 digits is the largest you can store precisely in Windows and similar Intel type environments, slightly different results on mainframes. So you definitely need these to be stored as character, unless you don't care about the last few digits (sounds like you do).
If you have a (anything)-delimited file, your best bet is to simply write a data step to read them in, at which point you can assign them as character yourself. Don't use proc import for most text files, unless they're really easy impossible to screw up sorts of things. What you can do is look at your log after PROC IMPORT ran, and copy that log into a program; then make adjustments to turn the serial number into a character field (and anything else you want to fix).
I had a similar problem, I was trying to import a file that had a 20 digit long field, one workaround i found for this was, opening the file in Excel and changing the attribute of the column from general to number, then when i imported the file, it was imported as a number and not in scientific notation
Background:
I am a new c++ programmer and am trying to build a program that returns a string stating the color of a given hex colorcode. The overall function is to request the hexcode of the pixel the mousepointer is on, and return a string that describes the color (like "Dark Red" for #8B0000). (I am colorblind, it would be a great help)
As a first try, I created a .txt file that contains all possible colorcodes on newlines. Needless to say, the document has 16777216 lines and is 134.2MB big. I have searched the internet and I've found that the only way to read a .txt file in C++ is line by line, start to end. That would result in 16777216 calls to "getline()" for the string "Black". This approach got my "hopeless" stamp on it for now.
Idea:
I would like to create a vector that contains 16777216 instances of (String colour) and use a hex-to-int conversion to use as an index to locate the correct String. This vector would also get quite big and pretty unhandy to build or use.
Problem:
I need to find the best way (if possible) to save/preserve a large object along with my c++ classes, so that I could just import the object and use it right away.
Thanks in advance.
A) Your file contains more than 16777216 lines, which means it contains more than the number of words in English, and probably Russian, Greek, Chinese and Japanese all combined.
B) You need to put things into ranges and then do a binary scan for the correct range. In other words map the range for navy blue as an object with a low and a high value into the range.
C) Put all the ranges into a big list and sort the list.
D) Then do a binary scan for any particular color and it will intersect the correct range.
For example:
// Navy blue might be this range
Low = RGB(0,0,170)
High = RGB(0,0,200)
// Light Red might be this range
Low = RGB(240,0,0)
High = RGB(255,0,0)
I mean, why would you want to name each color if you can name the range instead?
The OP posted:
I have searched the internet and I've found that the only way to read a .txt file in C++ is line by line, start to end.
This is incorrect. I'm not sure what C++ file reading/writing classes you are using, but if the one you are using doesn't support random access, then find a different one.
If you reset back and use fopen, you can use fseek to go a specific place in the file.
If you format all the records in your file to be the same length, you can then easily calculate the offset into the file as recordnumber*recordlength (assuming the first record is number 0).
Thanks for the discussion David and Alex :)
Simple solution for one way lookups value to name
So let me suggest first quantize the color space based in the four MSBs of every color value.
val = ((hexval & 0xf00000) >> 12) | ((hexval & 0x00f000) >> 8) | ((hexval & 0x0000f0) >> 4)
Also create a std::vector<std::string> with 4096 entries into which you read the color names.
std::vector<std::string> names(4096);
//Read file and do for each line
names[val] = /*name for the value*/
//Lookup
const std::string& name = names[val];
For bidirectional lookup I would still look to boost::bimap it can be configured to look like a vector when finding the names from color values. And configured as a hash table when finding a color values that match a certain named color.
If you want to "preserve" an object of a class then I suggest pickling it! Here is a library that I think will work for c++
These routines come from chooser.h in that library and should be usefull
// C++
DumpValToFile (const Val& thing_to_serialize,
const string& output_filename,
Serialization_e how_to_dump_the_data);
LoadValFromFile (const string& input_filename,
Val& result,
Serialization_e how_data_was_dumped);
I believe the parameter Val& is where you pass the object needing to be pickeled
What these tools do is serialize an object so that it can be stored easily on a hardrive.
I have never used this tool personally, but I have used something similar to it in python and so I suggest experimenting with pickling things in Python first. Google "python pickle" for more information on that.
I think you'll only want to actually 'name' a very small number of the possible 2^24 8-bit RGB values, so std::map is your friend here:
std::map<int, string> colors;
colors[0x000000] = "Black";
...
colors[0xFFFFFF] = "White";
You could start using the HTML color names from here: http://www.w3schools.com/html/html_colornames.asp
You'll also want to write a 'findNearest' function, (unless of course you actually have 16 million different names for colors). Your findNearest would compute the distance in RGB-space between each named colour and the target colour.
I would read it all into a std::map at the start of the program. Then use that map for fast lookups. If reading the text-file takes to long, consider converting it to some binary representation. Parsing the text file for every look-up will be slow.
If you want bidirectional lookups, i.e. from value to name and name to value. Check boost::multi_index
http://www.boost.org/doc/libs/1_53_0/libs/multi_index/doc/index.html
or boost::bimap
http://www.boost.org/doc/libs/1_53_0/libs/bimap/doc/html/index.html
I would also consider using boost::serialization to store and retrieve the data for the map in between runs.
http://www.boost.org/doc/libs/1_53_0/libs/serialization/doc/index.html
I need to be able to insert numbers like 098532 into my table. This is the exact same question as this but I actually need to store it with the leading zeros as I will be using the data for 3rd party API calls.
The given answer was to format the output which doesn't really solve my problem.
Is the best solution to change the datatype of the column to a CHAR field?
I think your best bet is to store it as a string in a charfield and then cast it when you need something else.
i.e.
num = "051"
int(num)
if you need it as an int. Could you give a little more information about what the api expects and why you need the leading zeroes?