How to properly delimit multiple images before sending them over a socket - c++

Question

let's say I need to send, for instance, five images from a client to a server over a socket and that I want to do it at once (not sending one and waiting for an ACK).
Questions:
I'd like to know if there are some best practices or guidelines for delimiting the end of each one.
What would be the safest approach for detecting the delimiters and processing each image once in the server? (In C/C++ if possible)
Thanks in advance!

Answer 1

Since images are binary data, it would be difficult to come up with delimiter that cannot be contained in the image. (And ultimately confusing the receiving side)
I would advice you to create a header that would be placed at the beginning of the transmission, or at the beginning of each image.
An example:
struct Header
{
uint32_t ImageLength;
// char ImageName[128];
} __attribute__(packed);
The sender should prepend this before each image and fill in the length correctly. The receiver would then know when the image ends and would expect another Header structure at that position.
The attribute(packed) is a safety, that makes sure the header will have the same alignment even if you compile server and client with different GCC versions. It's recomended in cases where structures are interpreted by different processes.
Data Stream:
Header
Image Data
Header
Image Data
Header
Image Data
...

Answer 2

You can use these function to send files (from client in java) to a server (in C). The idea is to send 4 bytes which indicates the file's size followed by the file content, when all files have been sent, send 4 bytes (all set to 0 zero) to indicate the end of the transfer.
// Compile with Microsoft Visual Studio 2008
// path, if not empty, must be ended with a path separator '/'
// for example: "C:/MyImages/"
int receiveFiles(SOCKET sck, const char *pathDir)
{
int fd;
long fSize=0;
char buffer[8 * 1024];
char filename[MAX_PATH];
int count=0;
// keep on receiving until we get the appropiate signal
// or the socket has an error
while (true)
{
if (recv(sck, buffer, 4, 0) != 4)
{
// socket is closed or has an error
// return what we've received so far
return count;
}
fSize = (int) ((buffer[0] & 0xff) << 24) |
(int) ((buffer[1] & 0xff) << 16) |
(int) ((buffer[2] & 0xff) << 8) |
(int) (buffer[3] & 0xff);
if (fSize == 0)
{
// received final signal
return count;
}
sprintf(filename, "%sIMAGE_%d.img", pathDir, count+1);
fd = _creat(filename, _S_IREAD | _S_IWRITE);
int iReads;
int iRet;
int iLeft=fSize;
while (iLeft > 0)
{
if (iLeft > sizeof(buffer)) iReads = sizeof(buffer);
else iReads=iLeft;
if ((iRet=recv(sck, buffer, iReads, 0)) <= 0)
{
_close(fd);
// you may delete the file or leave it to inspect
// _unlink(filename);
return count; // socket is closed or has an error
}
iLeft-=iRet;
_write(fd, buffer, iRet);
}
count++;
_close(fd);
}
}
The client part
/**
* Send a file to a connected socket.
* <p>
* First it send the file size if 4 bytes then the file's content.
* </p>
* <p>
* Note: File size is limited to a 32bit signed integer, 2GB
* </p>
*
* #param os
* OutputStream of the connected socket
* #param fileName
* The complete file's path of the image to send
* #throws Exception
* #see {#link receiveFile} for an example on how to receive the file from the other side.
*
*/
public void sendFile(OutputStream os, String fileName) throws Exception
{
// File to send
File myFile = new File(fileName);
int fSize = (int) myFile.length();
if (fSize == 0) return; // No empty files
if (fSize < myFile.length())
{
System.out.println("File is too big'");
throw new IOException("File is too big.");
}
// Send the file's size
byte[] bSize = new byte[4];
bSize[0] = (byte) ((fSize & 0xff000000) >> 24);
bSize[1] = (byte) ((fSize & 0x00ff0000) >> 16);
bSize[2] = (byte) ((fSize & 0x0000ff00) >> 8);
bSize[3] = (byte) (fSize & 0x000000ff);
// 4 bytes containing the file size
os.write(bSize, 0, 4);
// In case of memory limitations set this to false
boolean noMemoryLimitation = true;
FileInputStream fis = new FileInputStream(myFile);
BufferedInputStream bis = new BufferedInputStream(fis);
try
{
if (noMemoryLimitation)
{
// Use to send the whole file in one chunk
byte[] outBuffer = new byte[fSize];
int bRead = bis.read(outBuffer, 0, outBuffer.length);
os.write(outBuffer, 0, bRead);
}
else
{
// Use to send in a small buffer, several chunks
int bRead = 0;
byte[] outBuffer = new byte[8 * 1024];
while ((bRead = bis.read(outBuffer, 0, outBuffer.length)) > 0)
{
os.write(outBuffer, 0, bRead);
}
}
os.flush();
}
finally
{
bis.close();
}
}
To send the files from the client:
try
{
// The file name must be a fully qualified path
sendFile(mySocket.getOutputStream(), "C:/MyImages/orange.png");
sendFile(mySocket.getOutputStream(), "C:/MyImages/lemmon.png");
sendFile(mySocket.getOutputStream(), "C:/MyImages/apple.png");
sendFile(mySocket.getOutputStream(), "C:/MyImages/papaya.png");
// send the end of the transmition
byte[] buff = new byte[4];
buff[0]=0x00;
buff[1]=0x00;
buff[2]=0x00;
buff[3]=0x00;
mySocket.getOutputStream().write(buff, 0, 4);
}
catch (Exception e)
{
e.printStackTrace();
}

Answer 3

If you cannot easily send a header containing the length, use some likely delimiter. If the images are not compressed and consist of bitmap-stype data, maybe 0xFF/0XFFFF/0xFFFFFFF as fully-saturated luminance values are usually rare?
Use an escape-sequence to eliminate any instances of the delimiter that turn up inside your data.
This does mean iterating all the data at both ends, but depending on your data flows, and what is being done anyway, it may be a useful solution :(