I have a json file that I want to read in as a string, the problem I'm having is the read() function reads the first line like so:
json file contents:
{
"structure": [
{
"thing1": "1441",
"thing2": 1234,
"thing3": "2200.685715",
"thing4": "1793.190430",
}
}
what gets read is just the leading { (the first record/line), so when I print out I get this at the beginning and a whole bunch of blank spaces. How do I read in all the records/lines as a single pre-allocated string?
my code:
open (unit=11, status='scratch', access='stream', form='formatted')
call jsonHandler%print(outputJson, 11)
rewind(11)
inquire(11, SIZE=file_size)
! allocate the output based on the size of the file
allocate(character(len=file_size)::output)
read(11, *) output
close(11)
notes on the code:
I am using Jacob Williams' json-fortran library and I am aware that there is a serialize() function, however this is a very slow algorithm, because of the appends, this is the solution I came up with to have a pre-allocated string, and therefore will be fast
I am writing to a temporary file so normally this file object would not show up on the file system, the goal of this is to use a file objects powerful functionality without writing to the file system
This is more of a generalization of how to read multi-record data into a string, I am simply using json as an example
Do not use form="formatted". You do not want the formatting in the file to be interpretted in any way when reading the file content.
Related
I am familiar with Ruby and am trying to write a program in Crystal.
I have a file called special_file.txt that I want to read in my Crystal program, how do I do that?
Crystal is inspired by Ruby syntax and so you can often read and perform File operations in a similar manner. For example, Crystal has a File classclass which is an instance of the IO class containing a read method.
To read a file's contents on your filesystem you can instantiate a File object and invoke the gets_to_end method coming from the IO super class:
file = File.new("path/to/file")
content = file.gets_to_end
file.close
The gets_to_end method reads an entire IO objects data to a String variable.
You can also use a block invocation to achieve a similar result:
# Implicit close with `open`
content = File.open("path/to/file") do |file|
file.gets_to_end
end
Finally, the most idiomatic way to read the contents of an entire file would be the one line:
# Shortcut:
content = File.read("path/to/file")
In the following code I am about to create a temp file and then save the entire content of a txt file in this temp file. This is just a example. I know that makes no sense to read a text file and then write it in the temp file. But I want to demonstrate my question.
Well when I use the read() metod that means the entire contens of the temp file is saved in the RAM memory, right? I can't control the content size of the temp file. So I thinking about if there is a better way to protect the RAM memory. I don't want to inundate the RAM memory.
# Use the TemporaryFile context manager for easy clean-up
with tempfile.TemporaryFile(delete=True) as tmp:
with open('filename.txt', 'r') as my_file:
for line in my_file:
tmp.write(line)
tmp.seek(0)
exec(tmp.read())
The for line in my_file calls the file objects .next which does not buffer the entire file in memory when reading:
In order to make a for loop the most efficient way of looping over the
lines of a file (a very common operation), the next() method uses a
hidden read-ahead buffer
From the docs:
For reading lines from a file, you can loop over the file object. This
is memory efficient, fast, and leads to simple code
For the tmp.read() function, from the docs:
When size is omitted or negative, the entire contents of the file will
be read and returned; it’s your problem if the file is twice as large
as your machine’s memory.
So unless you read by line as you do when you write, or read with a fixed amount incrementally i.e tmp.read(100), you will read the entire file into memory.
I'm fairly new to Python, so I haven't done much in the way of reading files.
My question is this: if I use
with open(sendFile, 'r') as fileContent:
response = fileContent.read()
will the whole file always be read in to response at once or is there any chance that I'd have to call read() multiple times? Or does read() just handle that case for you?
I believe the file will be closed after this call, so I just want to make sure that I'm getting the whole file and not having to go back, open it again, and read more
Unless you specify a size, the read method reads the whole contents of the file.
From https://docs.python.org/2/library/stdtypes.html#file.read :
If the size argument is negative or omitted, read all data until EOF is reached.
I was looking for an easy way to write something into the first line of an already existing textfile. I tried using ofstream like this:
ofstream textFileWriter("Data/...txt");
if (textFileWriter.is_open())
{
textFileWriter << "HEADER: stuffstuff";
}
But it would delete everything which used to be in that file, even though the ofstream wasn't constructed with std::ofstream::trunc. I cannot use std::ofstream::app, since it is important to write into the first line.
Copying the whole textfile into a vector which has the line already and then writing it back would be my last option, but something I would really like to avoid, since the textfiles are quite large.
You can't simply "append" to the beginning of a file.
The common solution is to open a new (temporary) file, write your new header, write the rest of the original file to the temporary file, and then "rename" (using the OS system calls) the temporary file as the original file.
Or as you say in your question, read the original file into an in-memory buffer (e.g. a vector) and do the modification in that buffer, and then write the buffer to the file.
I would like to embed a text file with some data into my program.
let's call it "data.txt".
This text file is usually loaded with a function which requires the text file's file name as input and is eventually opened using a fopen() call... some something to the lines of
FILE* name = fopen("data.txt");
I can't really change this function and I would like the routine to open this same file every time it runs. I've seen people ask about embedding the file as a header but it seems that I wouldn't be able to call fopen() on a file that I embed into the header.
So my question is: is there a way to embed a text file as a callable file/variable to fopen()?
I am using VS2008.
Yes and No. The easiest way is to transform the content of the text file into an initialized array.
char data_txt[] = {
'd','a','t','a',' ','g','o','e','s',' ','h','e','r','e', //....
};
This transformation is easily done with a small perl script or even a small C program. You then compile and link the resulting module into your program.
An old trick to make this easier to manage with a Makefile is to make the script transform its data into the body of the initializer and write it to a file without the surrounding variable declaration or even the curly braces. If data.txt is transformed to data.inc, then it is used like so:
char data_txt[] = {
#include "data.inc"
};
Update
On many platforms, it is possible to append arbitrary data to the executable file itself. The trick then is to find it at run time. On platforms where this is possible, there will be file header information for the executable that indicates the length of the executable image. That can be used to compute an offset to use with fseek() after you have opened the executable file for reading. That is harder to do in a portable way, since it may not even be possible to learn the actual file name of your executable image at run time in a portable way. (Hint, argv[0] is not required to point to the actual program.)
If you cannot avoid the call to fopen(), then you can still use this trick to keep a copy of the content of data.txt, and put it back in a file at run time. You could even be clever and only write the file if it is missing....
If you can drop the call to fopen() but still need a FILE * pointing at the data, then this is likely possible if you are willing to play fast and loose with your C runtime library's implementation of stdio. In the GNU version of libc, functions like sprintf() and sscanf() are actually implemented by creating a "real enough" FILE * that can be passed to a common implementation (vfprintf() and vfscanf(), IIRC). That faked FILE is marked as buffered, and points its buffer to the users's buffer. Some magic is used to make sure the rest of stdio doesn't do anything stupid.
For any kind of file, base on RBerteig anwser you could do something simple as this with python:
This program will generate a text.txt.c file that can be compiled and linked to your code, to embed any text or binary file directly to your exe and read it directly from a variable:
import struct; # Needed to convert string to byte
f = open("text.txt","rb") # Open the file in read binary mode
s = "unsigned char text_txt_data[] = {"
b = f.read(1) # Read one byte from the stream
db = struct.unpack("b",b)[0] # Transform it to byte
h = hex(db) # Generate hexadecimal string
s = s + h; # Add it to the final code
b = f.read(1) # Read one byte from the stream
while b != "":
s = s + "," # Add a coma to separate the array
db = struct.unpack("b",b)[0] # Transform it to byte
h = hex(db) # Generate hexadecimal string
s = s + h; # Add it to the final code
b = f.read(1) # Read one byte from the stream
s = s + "};" # Close the bracktes
f.close() # Close the file
# Write the resultan code to a file that can be compiled
fw = open("text.txt.c","w");
fw.write(s);
fw.close();
Will generate something like
unsigned char text_txt_data[] = {0x52,0x61,0x6e,0x64,0x6f,0x6d,0x20,0x6e,0x75...
You can latter use your data in another c file using the variable with a code like this:
extern unsigned char text_txt_data[];
Right now I cant think of two ways to converting it to readable text. Using memory streams or converting it to a c-string.