Because of d3dcompiler*.dll becomes undesireable, I'm trying to exclude dependencies to it from my app. And there is something wrong with my new Compiled Shader Object (.cso) file loading code.
ifstream fstream;
fstream.open (vsfile);
if(fstream.fail())
return false;
fstream.seekg( 0, ios::end );
size_t size = size_t(fstream.tellg());
char* data = new char[size];
fstream.seekg(0, ios::beg);
fstream.read( data, size);
fstream.close();
XTRACE2(pDevice->CreateVertexShader(&data, size, 0, &m_pVertexShader))
The problem: CreateVertexShader() returning E_INVALIDARG error.
Old code with D3DReadFileToBlob() works fine. Blob returns buffer of the same size that is my char* or std::vector<char> and its equal to .cso file size.
I know, there are new Windows 8 examples on MSDN, but they use some new Metro stuff. I want to do It with plain C++.
XTRACE2 is just DirectX error checking macro.
Thanks in advance!
Error was caused by reading binary in text mode. Works as intended if add binary flag to read function. Something like that:
std::ifstream fstream;
fstream.open (filename, std::ifstream::in | std::ifstream::binary);
if(fstream.fail())
return false;
fstream.seekg( 0, std::ios::end );
size_t size = size_t(fstream.tellg());
data.resize(size);
fstream.seekg(0, std::ios::beg);
fstream.read( &data[0], size);
fstream.close();
Shame on me =\
Related
I did a sample project to read a file into a buffer.
When I use the tellg() function it gives me a larger value than the
read function is actually read from the file. I think that there is a bug.
here is my code:
EDIT:
void read_file (const char* name, int *size , char*& buffer)
{
ifstream file;
file.open(name,ios::in|ios::binary);
*size = 0;
if (file.is_open())
{
// get length of file
file.seekg(0,std::ios_base::end);
int length = *size = file.tellg();
file.seekg(0,std::ios_base::beg);
// allocate buffer in size of file
buffer = new char[length];
// read
file.read(buffer,length);
cout << file.gcount() << endl;
}
file.close();
}
main:
void main()
{
int size = 0;
char* buffer = NULL;
read_file("File.txt",&size,buffer);
for (int i = 0; i < size; i++)
cout << buffer[i];
cout << endl;
}
tellg does not report the size of the file, nor the offset
from the beginning in bytes. It reports a token value which can
later be used to seek to the same place, and nothing more.
(It's not even guaranteed that you can convert the type to an
integral type.)
At least according to the language specification: in practice,
on Unix systems, the value returned will be the offset in bytes
from the beginning of the file, and under Windows, it will be
the offset from the beginning of the file for files opened in
binary mode. For Windows (and most non-Unix systems), in text
mode, there is no direct and immediate mapping between what
tellg returns and the number of bytes you must read to get to
that position. Under Windows, all you can really count on is
that the value will be no less than the number of bytes you have
to read (and in most real cases, won't be too much greater,
although it can be up to two times more).
If it is important to know exactly how many bytes you can read,
the only way of reliably doing so is by reading. You should be
able to do this with something like:
#include <limits>
file.ignore( std::numeric_limits<std::streamsize>::max() );
std::streamsize length = file.gcount();
file.clear(); // Since ignore will have set eof.
file.seekg( 0, std::ios_base::beg );
Finally, two other remarks concerning your code:
First, the line:
*buffer = new char[length];
shouldn't compile: you have declared buffer to be a char*,
so *buffer has type char, and is not a pointer. Given what
you seem to be doing, you probably want to declare buffer as
a char**. But a much better solution would be to declare it
as a std::vector<char>& or a std::string&. (That way, you
don't have to return the size as well, and you won't leak memory
if there is an exception.)
Second, the loop condition at the end is wrong. If you really
want to read one character at a time,
while ( file.get( buffer[i] ) ) {
++ i;
}
should do the trick. A better solution would probably be to
read blocks of data:
while ( file.read( buffer + i, N ) || file.gcount() != 0 ) {
i += file.gcount();
}
or even:
file.read( buffer, size );
size = file.gcount();
EDIT: I just noticed a third error: if you fail to open the
file, you don't tell the caller. At the very least, you should
set the size to 0 (but some sort of more precise error
handling is probably better).
In C++17 there are std::filesystem file_size methods and functions, so that can streamline the whole task.
std::filesystem::file_size - cppreference.com
std::filesystem::directory_entry::file_size - cppreference.com
With those functions/methods there's a chance not to open a file, but read cached data (especially with the std::filesystem::directory_entry::file_size method)
Those functions also require only directory read permissions and not file read permission (as tellg() does)
void read_file (int *size, char* name,char* buffer)
*buffer = new char[length];
These lines do look like a bug: you create an char array and save to buffer[0] char. Then you read a file to buffer, which is still uninitialized.
You need to pass buffer by pointer:
void read_file (int *size, char* name,char** buffer)
*buffer = new char[length];
Or by reference, which is the c++ way and is less error prone:
void read_file (int *size, char* name,char*& buffer)
buffer = new char[length];
...
fseek(fptr, 0L, SEEK_END);
filesz = ftell(fptr);
will do the file if file opened through fopen
using ifstream,
in.seekg(0,ifstream::end);
dilesz = in.tellg();
would do similar
** more specific question **
How to read a 3d file (the type of data is not known) into memory to feed the following method .
i am simply trying to import a file (a 3d model.stl file) into memory and then pass it to the following method .
const aiScene* ReadFileFromMemory(
const void* pBuffer,
size_t pLength,
unsigned int pFlags,
const char* pHint = "");
but i don't know how to provide the following 2 arguments for the method mentioned above :
const void* pBuffer
size_t pLength
Can someone provide me with a block of code for this ?
How can i use ifstream to read a file from memory (like "model.stl") , and then put it inside a buffer and pass it to the mentioned method
Tnx
ps .
ReadFileFromMemory requires a pointer to the data. The data can be a char[], std::string, std::vector<uint8_t> or similar. Anything that holds raw data in contiguous memory and can provide a pointer to it
How to achieve this ?
=====================
UPDATE ======================
Solution
string readFile(const string& fileName)
{
ifstream ifs(fileName.c_str(), ios::in | ios::binary | ios::ate);
ifstream::pos_type fileSize = ifs.tellg();
ifs.seekg(0, ios::beg);
vector<char> bytes(fileSize);
ifs.read(bytes.data(), fileSize);
return string(bytes.data(), fileSize);
}
and then take the buffer and it's length as the following :
std::string my_file = readFile("models/model_file.ext");
const char* pbuffer = my_file.data();
size_t plenght = my_file.length();
To load a file to memory, you can use this,
std::ifstream file(file_path, std::ios::binary | std::ios::ate);
if(!file.is_open()) { /* Bail out if the file couldn't be loaded. */ }
size_t fileSize = file.tellg();
file.seekg(0);
char* pBuffer = new char[fileSize];
file.read(pBuffer, fileSize);
file.close();
// After everything is done.
delete[] pBuffer;
And then you can pass the pBuffer and fileSize to pBuffer and length parameters respectively.
Assimp also provides an easy ReadFile function in the Importer class which does all that for you. You just need to pass the asset file's path and the format flags to it.
as #MikeCAT said, pbuffer is pointer to the memory, where your file is stored and pLength is the size of that buffer in bytes. Check out function specs here:
http://assimp.sourceforge.net/lib_html/class_assimp_1_1_importer.html#a9b3c5e8b1042702f449e84a95b3324f6
Context: I am trying to read the content of a PNG picture in C++ to send it later to my Android app. To do so, I open the file in binary mode, read it's content by chuncks of 512 bytes, then send the data to the app. I'm on Windows.
Issue: I use an ifstream instance and the readsome() function as shown below, and it returns me 512, which is what I expected since I asked to read 512 bytes. However, it seems that I am far from really having 512 bytes in my buffer, which confuses me. While I debug my programm step by step, the number of char in the buffer seems random, but is never 512 as expected.
Code:
int currentByteRead = 0;
std::ifstream fl(imgPath.toStdString().c_str(), ios_base::binary);
fl.seekg( 0, std::ios::end );
int length = fl.tellg();
char *imgBytes = new char[512];
fl.seekg(0, std::ios::beg);
// Send the img content by blocks of 512 bytes
while(currentByteRead + 512 < length) {
int nbRead = fl.readsome(imgBytes, 512); // nbRead is always set to 512 here
if(fl.fail()) {
qDebug() << "Error when reading file content";
}
sendMessage(...);
currentByteRead += 512;
imgBytes = new char[512];
}
// Send the remaining data
int nbRemainingBytes = length - currentByteRead;
fl.readsome(imgBytes, nbRemainingBytes);
sendMessage(...);
fl.close();
currentByteRead += nbRemainingBytes;
The length I get at the beginning is the correct one, and it seems there is no error. But it is as if not all the data was copied into the buffer during the readsome() call.
Questions: Did I misunderstood something about the readsome() function ? Is there something related to Windows causing this behaviour ? Is there a more appropriate way to proceed ?
I finally found a way to do what I wanted, and as suggested by David Herring I will put here my answer.
My thoughts about the issue: If I use a std::ifstream::pos_type variable instead of an int, the correct number of bytes is read and put in the buffer. This was not the case when using an int, as if the chars were only written in the buffer until a given (random ?) point. I am not sure to understand why this behavior occurred. My guess was that I had issues with '\n' characters, but the randomness of the final content of the buffer is still unclear for me.
Correction: This is the working code I finally reached nonetheless. Starting with this, I was able to do what I had in mind.
std::ifstream ifs(imgPath.toStdString().c_str(), std::ios::binary|std::ios::ate);
std::ifstream::pos_type pos = ifs.tellg();
int length = ifs.tellg();
std::vector<char> result(pos);
ifs.seekg(0, std::ios::beg);
ifs.read(result.data(), pos);
ifs.close();
I hope this will help others. Thank you David for your suggestions.
my small OpenGL application works fine in Debug mode, but if I build it in Release mode I often get this error:
Shader Creation Error:
- Vertex shader failed to compile with the following errors:
ERROR: 0:22: error(#132) Syntax error: "]" parse error
ERROR: error(#273) 1 compilation errors. No code generateder code here
The strange thing is, that the error occurs most of the time, but sometimes the program works fine. I think it has something to do with the filestream, but I cannot figure out what it is.
This is the corresponding part of my code:
std::ifstream file(fp);
if(!file) crit_error("Shader Loading", ("file "+fp+" doesn't exist").c_str());
file.seekg(0, file.end);
GLint len = GLint(file.tellg());
file.seekg(0, file.beg);
GLchar* buf = new GLchar[len];
file.read(buf, len);
file.close();
std::string type = fp.substr(fp.size()-4, 4);
if(type == ".vsh")
id = glCreateShader(GL_VERTEX_SHADER);
else if(type == ".fsh")
id = glCreateShader(GL_FRAGMENT_SHADER);
else if(type == ".csh")
id = glCreateShader(GL_COMPUTE_SHADER);
glShaderSource(id, 1, (const GLchar**)&buf, &len);
glCompileShader(id);
delete[] buf;
Your problem lies here:
file.seekg(0, file.end);
GLint len = GLint(file.tellg());
file.seekg(0, file.beg);
GLchar* buf = new GLchar[len];
file.read(buf, len);
file.close();
This code reads exactly the length of the file and nothing more. And unfortunately file sizes don't actually tell you about how much there's actually to read; if the file read exits short it will leave in any garbage that was in the memory buf points to before it was allocated to your program. This explains why it works in debug mode: In debug mode buffers are usually allocated a little bit larger to allow for out-of-bounds access detection and variables and buffers left uninitialized by the programmer are set to zero. While useful for some debugging this may turn regular bugs into Heisenbugs.
Furthermore ifstream::read may return less than the requested amount of bytes, for example if you run into an end of file situation and leave the rest of the buffer untouched. As it happens ifstream::get will return NUL if you're hitting the end of file, so it will fill up your buffer with terminating NUL bytes.
The proper way to read a file being passed into C string processing functions is this:
file.seekg(0, file.end);
GLint len = GLint(file.tellg());
file.seekg(0, file.beg);
GLchar* buf = new GLchar[len + 1];
buf[len] = 0;
file.read(buf, len);
streamsize rb = file.gcount();
if( rb < len ) {
/* file read short */
/* either way zero out the remainder of
* the buffer untouched by the read. */
memset(buf + rb, 0, len - rb);
/* should also log some warning message here. */
}
file.close();
OK. I simply tried a different reading method and it worked.
GLchar* buf = new GLchar[len];
for(int i = 0; i < len; ++i)
buf[i] = file.get();
file.close();
I think this is a bug of the Visual C++ compiler.
I'm pretty close to losing my head here ;)
I'm developing a service that uses gsoap. I would like to return a mime response.
I have everything working, but when reading binary files, all kind of files like jpeg, pdf, etc... contains the \0 char several times over the data (if opened with notepad can see a lot of NUL).
So any code for reading a raw file fails miserably once it finds the end-of-file char. I have tried to replace the \0 but the file becomes incorrect to display.
I have also tried several methods including the example that comes with gsoap.
So resuming,
fstream generic code doesn't work.
for (i = 0; i < MAX_FILE_SIZE; i++)
{ if ((c = fgetc(fd)) == EOF)
break;
image.__ptr[i] = c;
}
doesn't work also
QFile::ReadAll works but when converting QString to char* the array is trimmed in the first NUL.
So, which is the best aproach to read an entire binary file? Its crazy how sometimes C++ at the basic.
Thanks in advance.
I have tried this as retnick suggested below
UrlToPdf urlToPdf;
urlToPdf.getUrl(&input, &result);
QByteArray raw = urlToPdf.getPdf(QString(result.data.c_str()));
int size = raw.toBase64().size();
char* arraydata = new char[size];
strcpy(arraydata, raw.toBase64().data());
soap_set_mime(this, "MIME_boundary", NULL);
if(soap_set_mime_attachment(this, arraydata, size, SOAP_MIME_BASE64, "application/pdf", NULL, NULL, NULL))
{
soap_clr_mime(this);
soapMessage = this->error;
}
but no luck... the mime response is bigger than the actual file...
David G Ortega
to read binary files use fread()
Once you read it treat it as an array of bytes not as a string. No string functions allowed.
EDIT: The gSOAP documentation section 14.1 explains how to send MIME attachments. I only refer to the relevant function (please read it all).
int soap_set_mime_attachment(struct soap *soap, char *buf_ptr, size_t buf_size,
enum soap_mime_encoding encoding,
const char *type, const char *id,
const char *location, const char *description);
char *buf_ptr is your buffer.
size_t buf_size is the length of your buffer.
So just do your QFile::ReadAll.
this gives you back a QByteArray. The QByteArray has the method
QByteArray QByteArray::toBase64 () const
this will return a
QByteArray base64image = QByteArray::toBase64(rawImage);
so now just do
soap_set_mime(soap, "MIME_boundary", "<boundary.xml#just-testing.com>");
/* add a base64 encoded image (base64image points to base64 data) */
soap_set_mime_attachment(soap,
base64image.data(), base64image.size(),
SOAP_MIME_BASE64, "image/jpeg",
"<boundary.jpeg#just-testing.com>", NULL, NULL);
I have not tested this but should be close to finished.
QFile::ReadAll works but when converting QString to char* the array is trimmed in the first NUL.
Are you sure it's actually trimmed or you just can't print/view the array in the debugger [since C-style strings are 0 terminated]?
If the QString itself is not enough for your needs you may want to convert it to a std::vector or similar using the range constructor or range assign, you'll have lots less grief towards the how much data the container holds.
EDIT:
Here's some sample code for fstream reading from a binary file:
std::ifstream image( <image_file_name>, std::ios_base::in | std::ios_base::binary );
std::istream_iterator< char > image_begin( image ), image_end;
std::vector< char > vctImage( image_begin, image_end );
The std::ios_base::binary is the most important part of the thing (similar to fopen/fread ["rb"] & probably QFile has something similar)
Also posting some sample code usually helps in getting the right answer.
HIH
I have the solution for this... As renick suggested I tried his idea but it failed without undestanding it so much... From a logical point of view recnick was right... bat the truth is that any king of string manipulation using QT QByteArray, std or mem is going to stop when findind the first \0 char, Qt QString can do it without problems but when converting it to c string (char*) the data will be again trimmed with the first \0
I found that using QDataStream::readRawData reads the file into a char* given the size to read. So thats how I accomplished the deal...
QFile file("test.pdf");
file.open(QIODevice::ReadOnly);
int size = file.size();
char* buffer = new char[size];
QDataStream stream(&file);
stream.readRawData(buffer, size);
soap_set_mime(this, "MIME_boundary", NULL);
if(soap_set_mime_attachment(this, buffer, size, SOAP_MIME_BINARY, "application/pdf", NULL, NULL, NULL))
{
soap_clr_mime(this);
soapMessage = this->error;
}
Note that in the line
if(soap_set_mime_attachment(this, buffer, size, SOAP_MIME_BINARY, "application/pdf", NULL, NULL, NULL))
I'm still using the size var instead of doing sizeof(buffer) or any other aproach since this one is going to trimm again the data qhen finding the first \0...
Hope this helps...
David G Ortega