Related
I want to use TinyGPS++ on an Arduino to parse NMEA data and display information on an OLED display. But, instead of using software serial and the TX/RX pins, the NMEA data will be received by USB.
I followed the examples from TinyGPS++, but i encountered two problems:
1)
Only the first 64 characters are received by the Arduino, when i send one NMEA sentence over the serial monitor (Windows, Arduino 1.6.9). How can I overcome this restriction? I help myself by deleting a couple of decimal places, but this is not the preferred way to go.
2)
In the TinyGPS++ BasicExample a sample NMEA string is defined in the read-only memory:
// A sample NMEA stream.
const char *gpsStream =
"$GPRMC,045103.0,A,3014.0,N,09748.0,W,36.88,65.02,030913,,,A*7C\r\n"
"$GPGGA,045104.0,3014.0,N,09749.0,W,1,09,1.2,211.6,M,-22.5,M,,*62\r\n"
"$GPRMC,045200.0,A,3014.0,N,09748.0,W,36.88,65.02,030913,,,A*77\r\n"
"$GPGGA,045201.0,3014.0,N,09749.0,W,1,09,1.2,211.6,M,-22.5,M,,*6C\r\n"
"$GPRMC,045251.0,A,3014.0,N,09748.0,W,36.88,65.02,030913,,,A*7D\r\n"
"$GPGGA,045252.0,3014.0,N,09749.0,W,1,09,1.2,211.6,M,-22.5,M,,*6F\r\n";
and parsed by
while (*gpsStream) {
Serial.print(*gpsStream);
gps.encode(*gpsStream++);
}
I receive my NMEA (unfortunately only one line) this way:
if (Serial.available()) {
while (Serial.available() > 0) {
if(index < 80)
{
inChar = Serial.read();
inData[index] = inChar;
index++;
inData[index] = '\0';
}
}
}
and try to parse it by:
index = 0;
while (index < 80) {
gps.encode(inData[index]);
Serial.print(inData[index]);
index++;
}
But this does not work as desired. Checking if the location isValid() always returns not to be true.
Unfortunately, i have several possible sources for this undesired behavior.
The too short sentences (unlikely)
Incorrect way of reading the data over serial.
I only submit one line.
Something else.
I am not that experienced with neither NMEA, nor the serial data communication, and i have only little experience with Arduino/C. Can you point me into a direction how to solve for this (these) problems?
Basically, you do not need to accumulate NMEA characters. Just feed them to the GPS library as you receive them. You don't provide the entire loop, but it is very common to have a problem there, too.
After struggling with several GPS libraries and their examples, I eventually wrote NeoGPS. It is faster and smaller than all other libraries, it validates the checksum, and the examples are structured correctly. Unlike other libraries, NeoGPS does not store GPS values as floating-point values, so it is able to retain the full accuracy of your GPS device.
If you'd like to try it, be sure to follow the Installation instructions. The NMEA.ino example will emit one line of info (CSV format) for each batch of GPS sentences that you send, ending with the default RMC sentence. Be sure to modify it to use the Serial object instead of gps_port, or simply define it that way:
#define gps_port Serial
It will also show the number of characters that have been parsed, how many good sentences have been received, and how many sentences had checksum errors. That could help with debugging if you are not generating the checksum correctly. This site is useful, too.
Those CSV lines will be sent back over the USB port (to the PC), but you can easily change it to send specific fields to the OLED (see NMEAloc.ino).
Although it is possible to develop something on a PC and then port it to an embedded environment like the Arduino, you have to be careful about (1) linear program structure and (2) ignoring resource limits (program size, MCU speed and RAM). There are a number of quirks with the Arduino environment that usually make it frustrating to port a "sketch" to/from a PC. :P
In my parallel program, there was a big matrix. Each process computed and stored a part of it. Then the program wrote the matrix to a file by letting each process wrote its own part of the matrix in the correct order. The output file is in "unformatted" form. But when I tried to read the file in a serial code (I have the correct size of the big matrix allocated), I got an error which I don't understand.
My question is: in an MPI program, how do you get a binary file as the serial version output for a big matrix which is stored by different processes?
Here is my attempt:
if(ThisProcs == RootProcs) then
open(unit = file_restart%unit, file = file_restart%file, form = 'unformatted')
write(file_restart%unit)psi
close(file_restart%unit)
endif
#ifdef USEMPI
call mpi_barrier(mpi_comm_world,MPIerr)
#endif
do i = 1, NProcs - 1
if(ThisProcs == i) then
open(unit = file_restart%unit, file = file_restart%file, form = 'unformatted', status = 'old', position = 'append')
write(file_restart%unit)psi
close(file_restart%unit)
endif
#ifdef USEMPI
call mpi_barrier(mpi_comm_world,MPIerr)
#endif
enddo
Psi is the big matrix, it is allocated as:
Psi(N_lattice_points, NPsiStart:NPsiEnd)
But when I tried to load the file in a serial code:
open(2,file=File1,form="unformatted")
read(2)psi
forrtl: severe (67): input statement requires too much data, unit 2 (I am using MSVS 2012+intel fortran 2013)
How can I fix the parallel part to make the binary file readable for the serial code? Of course one can combine them into one big matrix in the MPI program, but is there an easier way?
Edit 1
The two answers are really nice. I'll use access = "stream" to solve my problem. And I just figured I can use inquire to check whether the file is "sequential" or "stream".
This isn't a problem specific to MPI, but would also happen in a serial program which took the same approach of writing out chunks piecemeal.
Ignore the opening and closing for each process and look at the overall connection and transfer statements. Your connection is an unformatted file using sequential access. It's unformatted because you explicitly asked for that, and sequential because you didn't ask for anything else.
Sequential file access is based on records. Each of your write statements transfers out a record consisting of a chunk of the matrix. Conversely, your input statement attempts to read from a single record.
Your problem is that while you try to read the entire matrix from the first record of the file that record doesn't contain the whole matrix. It doesn't contain anything like the correct amount of data. End result: "input statement requires too much data".
So, you need to either read in the data based on the same record structure, or move away from record files.
The latter is simple, use stream access
open(unit = file_restart%unit, file = file_restart%file, &
form = 'unformatted', access='stream')
Alternatively, read with a similar loop structure:
do i=1, NPROCS
! read statement with a slice
end do
This of course requires understanding the correct slicing.
Alternatively, one can consider using MPI-IO for output, which is very similar to using stream output. Read this back in with stream access. You can find about this concept elsewhere on SO.
Fortran unformatted sequential writes in record files are not quite completely raw data. Each write will have data before and after the record in a processor dependent form. The size of your reads cannot exceed the record size of your writes. This means if psi is written in two writes, you will need to read it back in two reads, you cannot read it in at once.
Perhaps the most straightforward option is to instead use stream access instead of sequential. A stream file is indexed by bytes (generally) and does not contain record start and end information. Using this access method you can split the write but read all at once. Stream access is a feature of Fortran 2003.
If you stick with sequential access, you'll need to know how many MPI ranks wrote the file and loop over properly sized records to read the data as it was written. You could make the user specify the number of ranks or store that as the first record in the file and read that first to determine how to read the rest of the data.
If you are writing MPI, why not MPI-IO? Each process will call MPI_File_set_view to set a subarray view of the file, then each process can collectively write the data with MPI_FILE_WRITE_ALL . This approach is likely to scale really well on big machines (though your approach will be fine up to oh, maybe 100 processors.)
I sincerely apologize if this has been asked before, however I was unable to find a suitable answer that appeared similar to my current situation. I am developing an app with MoSync using the MAUI because of the same appearance across all platforms. I am running into issues with understanding MAHandles, as well as how to go about sending the SQLite information to a web address. The SQLite commands will then be converted to MySQL commands using a RedBean PHP script, and then sent to the permanent database.
My biggest concerns are 2 items:
1.Declaring connections that are usable through MAHandles (I have already gotten the SQLite commands working without using MAHandles, however declaring the database address in the resources.lstx still evades me)
2.Declaring MAHandles in general.
Also, I understand that strings are much more effective, however I disregard that fact due to the age of MAUI and it's capabilities appear much smoother when using char arrays.
I can provide additional clarification if needed so that I can help speed this process up.
Thank you ahead of time, and hopefully this will help others trying their hands at MoSync's immaculate product.
I have no experience with SQLite whatsoever, but I'm assuming handling SQLite commands is the job of your server-side application. To be clear, you are sending SQLite commands from your mobile app to a server-side app via a URL, correct? If you need help on this you should search "CGI". CGI is essentially a way to execute a server-side application with arguments via an http:// request.
This means your app should have a manager that constructs a URL with the right SQLite commands based on the input events sent to your mobile app (buttons, text fields, etc).
As far as Mosync goes, MAHandles can be used for many things including downloading.
Take a look at the MAUtil::DownloadListener class on Mosync's doxygen pages.
You will see that there are full descriptions of 5 pure virtual functions that you will need to implement.
The bulk of your code will probably be in finishedDownloading( Downloader* dl, MAHandle data ). It is here that the MAHandle "data", will point to the beginning of the data segment that you downloaded.
I read my data into a char* since I am downloading text.
Here's a snippet:
void MainScreen::finishedDownloading( Downloader* dl, MAHandle data )
{
char* mData = new char[ maGetDataSize( data ) + 1 ];
memset( mData, 0, maGetDataSize( data ) + 1 );
maReadData( data, mData, 0, maGetDataSize( data ) );
// Destroy the store
maDestroyObject( data );
// Do something with mData;
}
Here's one example of setting the font of NativeUI::Widget text using an MAHandle:
MAHandle font = maFontLoadDefault( FONT_TYPE_SERIF |
FONT_TYPE_MONOSPACE |
FONT_STYLE_NORMAL, 0, Dimensions::DIM_LIST_ELEM_FONT_SIZE );
ListViewItem* items = new ListViewItem();
items -> setFont( font );
I'm making simple dll packet sniffer using C++, that will hook to the apps, and write the received packet into INI file. Unfortunately after 20-30 minutes it crashed the main apps.
When the packet is received, receivedPacket() will be called. After 20+ minutes, WriteCount value is around 150,000-200,000.. and starting to get C++ runtime error/crash, GetLastError() code that I get is 0x8, which is ERROR_NOT_ENOUGH_MEMORY, and the WritePrivateProfileStringA() returns 0
void writeToINI(LPCSTR iSec,LPCSTR iKey,int iVal){
sprintf(inival, _T("%d"), iVal);
WritePrivateProfileStringA(iSec,iKey,inival,iniloc);
//sprintf(strc, _T("%d \n"), WriteCount);
//WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), strc, strlen(strc), 0, 0);
WriteCount++;
}
void receivedPacket(char *packet,WORD size){
switch ( packet[2] )
{
case 0x30:
// Size : 0x5F
ID = *(signed char*)&packet[0x10];
X = *(signed short*)&packet[0x20];
Y = *(signed short*)&packet[0x22];
Z = *(signed short*)&packet[0x24];
sprintf(inisec, _T("PACKET_%d"), (ID+1));
writeToINI(inisec,"id",ID);
writeToINI(inisec,"x",X);
writeToINI(inisec,"y",Y);
writeToINI(inisec,"z",Z);
}
[.....OTHER CASES.....]
}
Thanks :)
WritePrivateProfileString() and GetPrivateProfileString() are very slow (due to parsing INI file each call), instead you can:
use one of existing parsing libraries, but i am not sure about memory efficiency nor supporting sequential write.
write your own sequential INI writter:
read file (or only part, by part, if it is too big)
find section and key (if not found, create new section at end of file, or find insertion position, if you want sorted sections), save file position of key and next key
change value
save (beginning of original file to position of key + actual changed key + position of next key in original file to end of file) (if new section is created at end, you can simply append new section to original file) (if packets rewrite same ID often, you can add padding whitespace after each key, large to hold any value of desired type (example: change X=1---\n to X=100-\n (change - to whitespace), so you have constant size of key, you can update only part of file) )
database, for example MySQL
write binary file (fastest solution) and make program to read values, or to convert to text
Little note: I use GetPrivateProfileString() few years ago to read settings file (about 1KB of size), reading form HDD: 50ms, reading from USB flash disk: 1000ms!, after changing (1. read file to memory 2. run my own parser) it run in 1ms both on HDD and USB.
Thanks for the reply guys, but looks like the problem wasn't come from WritePrivateProfileStringA().
I just need to add extra size in malloc() for the Hook.
:)
I need to write a BLOB to a varbinary column in a SQL Server database. Sounds easy except that I have to do it in C++. I've been using ADO for the database operations (First question: is this the best technology to use?) So i've got the _Stream object, and a record set object created and the rest of the operation falls apart from there. If someone could provide a sample of how exactly to perform this seemingly simple operation that would be great!. My binary data is stored in a unsigned char array. Here is the codenstein that i've stitched together from what little I found on the internet:
_RecordsetPtr updSet;
updSet.CreateInstance(__uuidof(Recordset));
updSet->Open("SELECT TOP 1 * FROM [BShldPackets] Order by ChunkId desc",
_conPtr.GetInterfacePtr(), adOpenDynamic, adLockOptimistic, adCmdText);
_StreamPtr pStream ; //declare one first
pStream.CreateInstance(__uuidof(Stream)); //create it after
_variant_t varRecordset(updSet);
//pStream->Open(varRecordset, adModeReadWrite, adOpenStreamFromRecord, _bstr_t("n"), _bstr_t("n"));
_variant_t varOptional(DISP_E_PARAMNOTFOUND,VT_ERROR);
pStream->Open(
varOptional,
adModeUnknown,
adOpenStreamUnspecified,
_bstr_t(""),
_bstr_t(""));
_variant_t bytes(_compressStreamBuffer);
pStream->Write(_compressStreamBuffer);
updSet.GetInterfacePtr()->Fields->GetItem("Chunk")->Value = pStream->Read(1000);
updSet.GetInterfacePtr()->Update();
pStream->Close();
As far as ADO being the best technology in this case ... I'm not really sure. I personally think using ADO from C++ is a painful process. But it is pretty generic if you need that. I don't have a working example of using streams to write data at that level (although, somewhat ironically, I have code that I wrote using streams at the OLE DB level. However, that increases the pain level many times).
If, though, your data is always going to be loaded entirely in memory, I think using AppendChunk would be a simpler route:
ret = updSet.GetInterfacePtr()->Fields->
Item["Chunk"]->AppendChunk( L"some data" );