I have a code preprocessor which inserts a #line directive in the source code. The directive contains a filename, line number, and character position. I have a lexer rule for the #line directive that calls a function called newFile. The newFile function sets the lexer line number and character position. But, I don't see a way to set the source name. There are functions for getting the source name, but not setting it. I tried setting the input stream source name, but that didn't seem to work (I have an errorListener that gets the filename from recognizer->getInputStream()->getSourceName() but it always returns the initial filename).
My code is (C++ target):
preprocessor pp(_defines, _incpaths);
ANTLRInputStream input(pp.preprocess(filename));
myLexer lexer(&input);
CommonTokenStream tokens(&lexer);
myParser parser(&tokens);
antlr4::tree::ParseTree* tree = parser.start();
And, the newFile code is:
void myLexer::newFile (std::string newFilename, int newLine, int newPos)
{
static_cast<ANTLRInputStream*>(_input)->name = newFilename; // doesn't work
setLine(newLine);
setCharPositionInLine(newPos);
}
Thanks for any and all help.
There's no built-in functionality like that. Keep this information in a separate structure and manage changes there. The input stream name is just a convenient feature, which is not flexible enough for that kind of processing.
Thanks for the info. I tried a few different ways to store file/location information in a separate structure, but it quickly got overly complex.
I resolved the problem by taking advantage of Antlr's line tracking functionality. I stored the filename in a list, then encoded the list index of the filename into the line number.
int fileIndex = filename_list.size();
filename_list.append(filename);
int line = (fileIndex << 20) + newLine;
setLine(line);
setCharPositionInLine(newPos);
Then, in the errorListener, or AST builder, it's easy to access the filename:
void errorListener::syntaxError(Recognizer* recognizer......)
{
int fileIndex = line >> 20;
line &= 0xFFFFF;
Related
I'm doing the project that convert the python code to C++, for better performance. That python project name is Adcvanced EAST, for now, I got the input data for nms function, in .csv file like this:
"[ 5.9358170e-04 5.2773970e-01 5.0061589e-01 -1.3098677e+00
-2.7747922e+00 1.5079222e+00 -3.4586751e+00]","[ 3.8175487e-05 6.3440394e-01 7.0218205e-01 -1.5393494e+00
-5.1545496e+00 4.2795391e+00 -3.4941311e+00]","[ 4.6003381e-05 5.9677261e-01 6.6983813e-01 -1.6515008e+00
-5.1606908e+00 5.2009044e+00 -3.0518508e+00]","[ 5.5172237e-05 5.8421570e-01 5.9929764e-01 -1.8425952e+00
-5.2444854e+00 4.5013981e+00 -2.7876694e+00]","[ 5.2929961e-05 5.4777789e-01 6.4851379e-01 -1.3151239e+00
-5.1559062e+00 5.2229333e+00 -2.4008298e+00]","[ 8.0250458e-05 6.1284608e-01 6.1014801e-01 -1.8556541e+00
-5.0002270e+00 5.2796564e+00 -2.2154367e+00]","[ 8.1256607e-05 6.1321974e-01 5.9887391e-01 -2.2241254e+00
-4.7920742e+00 5.4237065e+00 -2.2534993e+00]
one unit is 7 numbers, but a '\n' after first four numbers,
I wanna read this csv file into my C++ project,
so that I can do the math work in C++, make it more fast.
using namespace std;
void read_csv(const string &filename)
{
//File pointer
fstream fin;
//open an existing file
fin.open(filename, ios::in);
vector<vector<vector<double>>> predict;
string line;
while (getline(fin, line))
{
std::istringstream sin(line);
vector<double> preds;
double pred;
while (getline(sin, pred, ']'))
{
preds.push_back(preds);
}
}
}
For now...my code emmmmmm not working ofc,
I'm totally have no idea with this...
please help me with read the csv data into my code.
thanks
Unfortunately parsing strings (and consequently files) is very tedious in C++.
I highly recommend using a library, ideally a header-only one, like this one.
If you insist on writing it yourself, maybe you can draw some inspiration from this StackOverflow question on how to parse general CSV files in C++.
You could look at getdelim(',', fin, line),
But the other issue will be those quotes, unless you /know/ the file is always formatted exactly this way, it becomes difficult.
One hack I have used in the past that is NOT PERFECT, if the first character is a quote, then the last character before the comma must also be a matching quote, and not escaped.
If it is not a quote then getdelim() some more, but the auto-alloc feature of getdelim means you must use another buffer. In C++ I end up with a vector of all the pieces of getdelim results that then need to be concatenated to make the final string:
std::vector<char*> gotLine;
gotLine.push_back(malloc(2));
*gotLine.back() = fgetch();
gotLine.back()[1] = 0;
bool gotquote = *gotLine.back() == '"'; // perhaps different classes of quote
if (*gotLine.back() != ',')
for(;;)
{
char* gotSub= nullptr;
gotSub=getdelim(',');
gotLine.push_back(gotSub);
if (!gotquote) break;
auto subLen = strlen(gotSub);
if (subLen>1 && *(gotSub-1)=='"') // again different classes of quote
if (sublen==2 || *(gotSub-2)!='\\') // needs to be a while loop
break;
}
Then just concatenate all these string segments back together.
Note that getdelim supports null bytes. If you expect null bytes in the content, and not represented by the character sequences \000 or \# you need to store the actual length returned by getdelim, and use memcpy to concatenate them.
Oh, and if you allow utf-8 extended quotes it gets very messy!
The case this doesn't cover is a string that ends \\" or \\\\". Ideally you need to while count the number of leading backslashes, and accept the quote if the count is even.
Note that this leave the issue of unescaping the quoted content, i.e. converting any \" into ", and \\ into \, etc. Also discarding the enclosing quotes.
In the end a library may be easier if you need to deal with completely arbitrary content. But if the content is "known" you can live without.
Is it possible to edit text in a file using cpp code. Already there is related question on it, but it doesn't solve my problem. Kindly help me out.
I have given a rough code line on this.
seek() through the file and try to replace the contents with new string from that point till the end of line.
I need the "hello" string be placed and must be the end of line.
like if we have new.txt as
ABCDEFGHIJKLMNOPQRST
If I want the file content to be changed as
ABCDEHELLO
I am getting the file content as
ABCDHELLOJKLMNOPQRST
fstream file("new.txt",fstream::in|fstream::out);
file.open();
while(getline(file,str))
{
if(value==strstr())
{
file.seekp(pos);
str.erase(pos,len);//len specifies the value till end of str
str.replace(pos,6,"hello");
char *d=new char[str.length()+1];
strcpy(d,str.c_str());
file.write(d,strlen(d));
delete [] d;
}
}
If I could copy the file contents to the string, manipulate it, then copy to the new file then it is possible.
Is it possible to change the contents in the same file. If so kindly help me out, I am struck in this. If the replacing string is longer than the one actually existing then this works, but if the replacing string is smaller than the one which is actually existing then I am unable to do.
if you case is only one line in the file you can easily separate the I/O process in two stages. Read the file and get the position of the text. then close the file and reopened as out then write the string you want. Note that this will work if you have one line in the file
check the following code
std::string value = "GFGHHFGHH";
std::string str;
std::fstream file("new.txt", std::ios::in);
std::size_t found;
while (file >> str)
{
found = str.find(value);
if (found != std::string::npos)
{
str.erase(value.length() );
str.replace(found, 6, "hello");
}
}
file.close();
file.open("new.txt", std::ios::out);
file << str;
file.close();
You can do it using system call for sed:
string s="sed -i s/hey/ho/g file0102.txt";
system(s.c_str());
I have a small program, that is meant to copy a small phrase from a file, but it appears that I am either misinformed as to how seekg() works, or there is a problem in my code preventing the function from working as expected.
The text file contains:
//Intro
previouslyNoted=false
The code is meant to copy the word "false" into a string
std::fstream stats("text.txt", std::ios::out | std::ios::in);
//String that will hold the contents of the file
std::string statsStr = "";
//Integer to hold the index of the phrase we want to extract
int index = 0;
//COPY CONTENTS OF FILE TO STRING
while (!stats.eof())
{
static std::string tempString;
stats >> tempString;
statsStr += tempString + " ";
}
//FIND AND COPY PHRASE
index = statsStr.find("previouslyNoted="); //index is equal to 8
//Place the get pointer where "false" is expected to be
stats.seekg(index + strlen("previouslyNoted=")); //get pointer is placed at 24th index
//Copy phrase
stats >> previouslyNotedStr;
//Output phrase
std::cout << previouslyNotedStr << std::endl;
But for whatever reason, the program outputs:
=false
What I expected to happen:
I believe that I placed the get pointer at the 24th index of the file, which is where the phrase "false" begins. Then the program would've inputted from that index onward until a space character would have been met, or the end of the file would have been met.
What actually happened:
For whatever reason, the get pointer started an index before expected. And I'm not sure as to why. An explanation as to what is going wrong/what I'm doing wrong would be much appreciated.
Also, I do understand that I could simply make previouslyNotedStr a substring of statsStr, starting from where I wish, and I've already tried that with success. I'm really just experimenting here.
The VisualC++ tag means you are on windows. On Windows the end of line takes two characters (\r\n). When you read the file in a string at a time, this end-of-line sequence is treated as a delimiter and you replace it with a single space character.
Therefore after you read the file you statsStr does not match the contents of the file. Every where there is a new line in the file you have replaced two characters with one. Hence when you use seekg to position yourself in the file based on numbers you got from the statsStr string, you end up in the wrong place.
Even if you get the new line handling correct, you will still encounter problems if the file contains two or more consecutive white space characters, because these will be collapsed into a single space character by your read loop.
You are reading the file word by word. There are better methods:
while (getline(stats, statsSTr)
{
// An entire line is read into statsStr.
std::string::size_type posn = statsStr.find("previouslyNoted=");
// ...
}
By reading entire text lines into a string, there is no need to reposition the file.
Also, there is a white-space issue when reading by word. This will affect where you think the text is in the file. For example, white space is skipped, and there is no telling how many spaces, newlines or tabs were skipped.
By the way, don't even think about replacing the text in the same file. Replacement of text only works if the replacement text has the same length as the original text in the file. Write to a new file instead.
Edit 1:
A better method is to declare your key strings as array. This helps with positioning pointers within a string:
static const char key_text[] = "previouslyNoted=";
while (getline(stats, statsStr))
{
std::string::size_type key_position = statsStr.find(key_text);
std::string::size_type value_position = key_position + sizeof(key_text) - 1; // for the nul terminator.
// value_position points to the character after the '='.
// ...
}
You may want to save programming type by making your data file conform to an existing format, such as INI or XML, and using appropriate libraries to parse them.
I am basically trying to reverse the contents of a text file. When I run this code, nothing happens. Code:
getArguments();
stringstream ss;
ss << argument;
string fileName;
ss >> fileName;
fstream fileToReverse(fileName);
if (fileToReverse.is_open()) {
send(sock, "[*] Contents is being written to string ... ", strlen("\n[*] Contents is being written to string ... "), 0);
string line;
string contentsOfFile;
while (getline(fileToReverse, line)) {
contentsOfFile.append(line);
line = "\0";
}
send(sock, "done\n[*] File is being reversed ... ", strlen("done\n[*] File is being reversed ... "), 0);
string reversedText(contentsOfFile.length(), ' ');
int i;
int j;
for(i=0,j=contentsOfFile.length()-1;i<contentsOfFile.length();i++,j--) {
reversedText[i] = contentsOfFile[j];
}
contentsOfFile = "\0";
fileToReverse << reversedText;
fileToReverse.close();
send(sock, "done\n", strlen("done\n"), 0);
}
fileName is created from user input, and I know that the file exists. It just doesn't do anything to the file. If anyone has any ideas that they would like to share that would be great.
UPDATE:
I now can write reversedText to the file but how can I delete all of the files contents?
In this particular case, when you have read all the input content, your file is in an "error state" (eof and fail bits set in the status).
You need to clear that with fileToReverse.clear();. Your file position will also be at the end of the file, so you need to use fileToReverse.seekp(0, ios_base::beg) to set the position to the beginning.
But I, just as g-makulik, prefer to have two files, one for input and one for output. Saves a large amount of messing about.
When you need to debug something like this - saying "all the functions are being run and all the variables are being created, and it compiled without any warnings" isn't really debugging.
Debugging - this doesn't work. Remove bits until you find what doesn't work. Like you said - all variables are what you expect them. So... try and see if, for example, the way you read and write from a file works. Just write a small program that opens a file like you open it, reads from it like you do and then writes... whatever back into it in the same way you do. See if that works.
In other words, try and find the smallest program that reproduces what you see.
I have a C++ project which is supposed to add <item> to the beginning of every line and </item > to the end of every line. This works fine with normal English text, but I have a Chinese text file I would like to do this to, but it does not work. I normally use .txt files, but for this I have to use .rtf to save the Chinese text. After I run my code, it becomes gibberish. Here's an example.
{\rtf1\adeflang1025\ansi\ansicpg1252\uc1\adeff31507\deff0\stshfdbch31506\stshfloch31506\stshfhich31506\stshfbi31507\deflang1033\deflangfe1033\themelang1033\themelangfe0\themelangcs0{\fonttbl{\f2\fbidi
\fmodern\fcharset0\fprq1{*\panose
02070309020205020404}Courier
New;}
Code:
int main()
{
ifstream in;
ofstream out;
string lineT, newlineT;
in.open("rawquote.rtf");
if(in.fail())
exit(1);
out.open("itemisedQuote.rtf");
do
{
getline(in,lineT,'\n');
newlineT += "<item>";
newlineT += lineT;
newlineT += "</item>";
if (lineT.length() >5)
{
out<<newlineT<<'\n';
}
newlineT = "";
lineT = "";
} while(!in.eof());
return 0;
}
That looks like RTF, which makes sense as you say this is an rtf file.
Basically, if you dump that file when you open, you'll see it looks like that...
Also, you should revisit your loop
std::string line;
while(getline(in, line, '\n'))
{
// do stuff here, the above check correctly that you have indeed read in a line!
out << "<item>" << line << "</item>" << endl;
}
You can't read the RTF code the same way as plain text as you'll just ignore format tags, etc. and might just break the code.
Try to save your chinese text as a text file using UTF-8 (without BOM) and your code should work. However this might fail if some other UTF-8 encoded character contains essentially a line break (not sure about this part right now), so you should try to do real UTF-8 conversion and read the file using wide chars instead of regular chars (as Chan suggested), which is a little bit tricky using C++.
It's kind of a miracle that this works for non-Chinese text. "\n" is not the line separator in RTF, "\par" is. The odds that more damage is done to the RTF header are certainly greater for Chinese.
C++ is not the best language to tackle this. It is a trivial 5 minute program in C# as long as the file doesn't get too large:
using System;
using System.Windows.Forms; // Add reference
class Program {
static void Main(string[] args) {
var rtb = new RichTextBox();
rtb.LoadFile(args[0], RichTextBoxStreamType.RichText);
var lines = rtb.Lines;
for (int ix = 0; ix < lines.Length; ++ix) {
lines[ix] = "<item>" + lines[ix] + "</item>";
}
rtb.Lines = lines;
rtb.SaveFile(args[0], RichTextBoxStreamType.RichText);
}
}
If C++ is a hard requirement then you'll have to find an RTF parser.
I think you should use 'wchar' for string instead of 'regular char'.
If I'm understanding the objective of this code, your solution is not going to work. A line break in an RTF document does not correspond to a line break in the visible text.
If you can't just use plain text (Chinese characters are not a problem with a valid encoding), take a look at the RTF spec. You'll discover that it is a nightmare. So you're best bet is probably a third-party library that can parse RTF and read it "line" by "line." I have never looked for such a library, so do not have any suggestions off the top of my head, but I'm sure they are out there.