I'm just trying to get the contents of a page with their headers...but it seems that my buffer of size 1024 is either too large or too small for the last packet of information coming through...I don't want to get too much or too little, if that makes sense. Here's my code. It's printing out the page just fine with all the information, but I want to ensure that it's correct.
//Build HTTP Get Request
std::stringstream ss;
ss << "GET " << url << " HTTP/1.0\r\nHost: " << strHostName << "\r\n\r\n";
std::string req = ss.str();
// Send Request
send(hSocket, req.c_str(), strlen(req.c_str()), 0);
// Read from socket into buffer.
do
{
nReadAmount = read(hSocket, pBuffer, sizeof pBuffer);
printf("%s", pBuffer);
}
while(nReadAmount != 0);
nReadAmount = read(hSocket, pBuffer, sizeof pBuffer);
printf("%s", pBuffer);
This is broken. You can only use the %s format specifier for a C-style (zero-terminated) string. How is printf supposed to know how many bytes to print? That information is in nReadAmount, but you don't use it.
Also, you call printf even if read fails.
The simplest fix:
do
{
nReadAmount = read(hSocket, pBuffer, (sizeof pBuffer) - 1);
if (nReadAmount <= 0)
break;
pBuffer[nReadAmount] = 0;
printf("%s", pBuffer);
} while(1);
The correct way to read an HTTP reply is to read until you have received a full LF-delimited line (some servers use bare LF even though the official spec says to use CRLF), which contains the response code and version, then keep reading LF-delimited lines, which are the headers, until you encounter a 0-length line, indicating the end of the headers, then you have to analyze the headers to figure out how the remaining data is encoded so you know the proper way to read it and know how it is terminated. There are several different possibilities, refer to RFC 2616 Section 4.4 for the actual rules.
In other words, your code needs to use this kind of structure instead (pseudo code):
// Send Request
send(hSocket, req.c_str(), req.length(), 0);
// Read Response
std::string line = ReadALineFromSocket(hSocket);
int rescode = ExtractResponseCode(line);
std::vector<std::string> headers;
do
{
line = ReadALineFromSocket(hSocket);
if (line.length() == 0) break;
headers.push_back(line);
}
while (true);
if (
((rescode / 100) != 1) &&
(rescode != 204) &&
(rescode != 304) &&
(request is not "HEAD")
)
{
if ((headers has "Transfer-Encoding") && (Transfer-Encoding != "identity"))
{
// read chunks until a 0-length chunk is encountered.
// refer to RFC 2616 Section 3.6 for the format of the chunks...
}
else if (headers has "Content-Length")
{
// read how many bytes the Content-Length header says...
}
else if ((headers has "Content-Type") && (Content-Type == "multipart/byteranges"))
{
// read until the terminating MIME boundary specified by Content-Type is encountered...
}
else
{
// read until the socket is disconnected...
}
}
First time poster long time reader.
I've been playing round with reading in data from a bluetooth GPS unit.
I can connect to it using hyperterm and see the data
The following log is from the hyperterm
$GPRMC,195307.109,A,5208.2241,N,00027.7689,W,000.0,345.8,310712,,,A*7E
$GPVTG,345.8,T,,M,000.0,N,000.0,K,A*07
$GPGGA,195308.109,5208.2242,N,00027.7688,W,1,04,2.1,58.9,M,47.3,M,,0000*7E
$GPGSA,A,3,19,03,11,22,,,,,,,,,5.5,2.1,5.0*3F
$GPRMC,195308.109,A,5208.2242,N,00027.7688,W,000.0,345.8,310712,,,A*73
$GPVTG,345.8,T,,M,000.0,N,000.0,K,A*07
$GPGGA,195309.109,5208.2243,N,00027.7688,W,1,04,2.1,58.9,M,47.3,M,,0000*7E
END LOG
The following log is from my C++ program
$GPGSV,3,3,12,14,20,105,16,28,18,323,,08,07,288,,16,01,178,*7A
$GPRMC,195,3,2ÿþÿÿÿL.š945.109,A,5208.2386,N,00027.7592,W,000.0,169.5,8,323,,08,07,288,,16,01,178,*7A
$GPRMC,195,3,2ÿþÿÿÿL.š310712,,,A*70
$GPVTG,169.5,T,,M,000.0,N,000.0,K,A*06
8,07,288,,16,01,178,*7A
$GPRMC,195,3,2ÿþÿÿÿL.š310712,,,A*70
$GPVTG,169.5,T,,M,000.0,N,000.0,K,A*06
8,07,288,,16,01,178,*7A
$GPRMC,195,3,2ÿþÿÿÿL.š$GPGGA,195946.109,5208.2386,N,00027.7592,W,1.0,K,A*06
8,07,288,,16,01,178,*7A
END LOG
THE PROBLEM
I've left the line feeds as they come, the C++ output has extra line feeds, not sure why?
The C++ log also has some funky chars...?
The Code
for (int n=0;n<100;n++) {
char INBUFFER[100];
cv::waitKey(1000);
bStatus = ReadFile(comport, // Handle
&INBUFFER, // Incoming data
100, // Number of bytes to read
&bytes_read, // Number of bytes read
NULL);
cout << "bStatus " << bStatus << endl;
if (bStatus != 0)
{
// error processing code goes here
}
LogFile << INBUFFER;
}
I'm using settings...
comSettings.BaudRate = 2400;
comSettings.StopBits = ONESTOPBIT;
comSettings.ByteSize = 8;
comSettings.Parity = NOPARITY;
comSettings.fParity = FALSE;
...which as far as I can tell are the same as the settings used by hyperterm.
Any hints on what I'm doing wrong?
cheers!
UPDATE
So after updating to use bytes_read and account for the extra LF at the end of NMEA data I now have...
if (bytes_read!=0) {
for (int i=0; i < bytes_read; i++) {
LogFile << INBUFFER[i];
}
}
Which appears to have fixed things!
$GPGGA,215057.026,5208.2189,N,00027.7349,W,1,04,6.8,244.6,M,47.3,M,,0000*41
$GPGSA,A,3,32,11,01,19,,,,,,,,,9.7,6.8,7.0*3D
$GPRMC,215057.026,A,5208.2189,N,00027.7349,W,002.0,208.7,310712,,,A*74
$GPVTG,208.7,T,,M,002.0,N,003.8,K,A*09
$GPGGA,215058.026,5208.2166,N,00027.7333,W,1,04,6.8,243.1,M,47.3,M,,0000*42
Thanks folks, your help was much appreciated.
You have a bytes_read var, but you don't do anything with it? Seems to me that you're dumping the entire INBUFFER to the file, no matter how many/few bytes are actually loaded into it?
I'm working with a library which opens the same file many times. It checks the header of the file to make sure that it is the correct format. The first 1212 times it opens the file, it behaves correctly. The 1213th time, the bytes read out from the file are different. Can anyone suggest why this might be happening?
Unfortunately I can't make a small reproducible example - and it takes 20 minutes to run through to this point. So I'm wondering if there are any subtleties of fopen which I might have missed, or something else which might have a bearing on this execution.
The code is below. Many instances of the class are created, and each on has initialise() called with the same filename. The first 1212 times, the output is:
Expecting: '?'
?lon-1800????%#LYB1800????%#LYB100????%#LYB
lat-900??p-2?%#HYB900??p-2?%#HYB10??p-2?%#HYB
? soilcode0 ?? ?-2?&#AYB12 ?? ?-2?&#AYB1 ?? ?-2?&#AYBmtemp-600??x.2?YB600??x.2?YB10??x.2?YB
?mprec0???H2?&#.YB99999???H2?&#.YB1999???H2?&#.YB?msun0???A2?&#%YB1000???A2?&#%YB100???A2?&#%YB
?
Got: '?'
?lon-1800????%#LYB1800????%#LYB100????%#LYB
lat-900??p-2?%#HYB900??p-2?%#HYB10??p-2?%#HYB
? soilcode0 ?? ?-2?&#AYB12 ?? ?-2?&#AYB1 ?? ?-2?&#AYBmtemp-600??x.2?YB600??x.2?YB10??x.2?YB
?mprec0???H2?&#.YB99999???H2?&#.YB1999???H2?&#.YB?msun0???A2?&#%YB1000???A2?&#%YB100???A2?&#%YB
?
The last time I get:
Expecting: '?'
?lon-1800????%#LYB1800????%#LYB100????%#LYB
lat-900??p-2?%#HYB900??p-2?%#HYB10??p-2?%#HYB
? soilcode0 ?? ?-2?&#AYB12 ?? ?-2?&#AYB1 ?? ?-2?&#AYBmtemp-600??x.2?YB600??x.2?YB10??x.2?YB
?mprec0???H2?&#.YB99999???H2?&#.YB1999???H2?&#.YB?msun0???A2?&#%YB1000???A2?&#%YB100???A2?&#%YB
?
Got: ' lon lat year
The function is as follows:
class Archive {
private:
FILE* pfile;
<snip>
bool initialise(char* filename) {
int i;
unsigned char* pheader;
if (pfile) fclose(pfile);
pfile=fopen(filename,"rb");
if (!pfile || pfile == NULL ) {
printf("Could not open %s for input\n",filename);
return false;
}
pheader=new unsigned char[CRU_1901_2002_HEADERSIZE-4];
if (!pheader) {
printf("Out of memory\n");
fclose(pfile);
pfile=NULL;
return false;
}
::rewind(pfile);
fread(pheader,CRU_1901_2002_HEADERSIZE-4,1,pfile);
printf( "Expecting: '%s'\n", CRU_1901_2002_HEADER);
for( int j = 0; j < CRU_1901_2002_HEADERSIZE-4;j++ )
printf( "%c", CRU_1901_2002_HEADER[j]);
printf( "\nGot: '%s'\n", pheader);
for( int j = 0; j < CRU_1901_2002_HEADERSIZE-4;j++ )
printf( "%c", pheader[j]);
printf( "\n");
for (i=0;i<CRU_1901_2002_HEADERSIZE-4;i++) {
if (pheader[i]!=CRU_1901_2002_HEADER[i]) {
fclose(pfile);
pfile=NULL;
delete pheader;
return false;
}
}
delete pheader;
::rewind(pfile);
fseek(pfile,CRU_1901_2002_HEADERSIZE+CRU_1901_2002_DATA_LENGTH*CRU_1901_2002_NRECORD,SEEK_CUR);
recno=0;
iseof=false;
return true;
}
public:
Archive() {
pfile=NULL;
}
Archive() {
if (pfile) fclose(pfile);
}
Are you sure that there is data in the 1213th position? or, are these data are correct?
I suggest you mount a file with more than 1213th records and do a test to confirm if there is a read error or not in this position.
It turns out this is down to too many files being open. Changing the program elsewhere to open less files fixes it.
Checking fread returns 1, except for the last one, where it returns 0.
However, I don't understand why fopen gives back a non-null file pointer when it can't open the file. In test code, it returns NULL, which is then caught as expected.