Eliminate newlines in google app script using regex - regex

I'm trying to write part of an add-on for Google Docs that eliminates newlines within selected text using replaceText. The obvious text.replaceText("\n",""); gives the error Invalid argument: searchPattern. I get the same error with text.replaceText("\r","");. The following attempts do nothing: text.replaceText("/\n/","");, text.replaceText("/\r/","");. I don't know why Google App Script does not allow for the recognition of newlines in regex.
I am aware that there is an add-on that does this already, but I want to incorporate this function into my add-on.
This error occurs even with the basic
DocumentApp.getActiveDocument().getBody().textReplace("\n","");
My full function:
function removeLineBreaks() {
var selection = DocumentApp.getActiveDocument().getSelection();
if (selection) {
var elements = selection.getRangeElements();
for (var i = 0; i < elements.length; i++) {
var element = elements[i];
// Only deal with text elements
if (element.getElement().editAsText) {
var text = element.getElement().editAsText();
if (element.isPartial()) {
text.replaceText("\n","");
}
// Deal with fully selected text
else {
text.replaceText("\n","");
}
}
}
}
// No text selected
else {
DocumentApp.getUi().alert('No text selected. Please select some text and try again.');
}
}

It seems that in replaceText, to remove soft returns entered with Shift-ENTER, you can use \v:
.replaceText("\\v+", "")
If you want to remove all "other" control characters (C0, DEL and C1 control codes), you may use
.replaceText("\\p{Cc}+", "")
Note that the \v pattern is a construct supported by JavaScript regex engine, and is considered to match a vertical tab character (≡ \013) by the RE2 regex library used in most Google products.

The Google Apps Script function replaceText() still doesn't accept escape characters, but I was able to get around this by using getText(), then the generic JavaScript replace(), then setText():
var doc = DocumentApp.getActiveDocument();
var body = doc.getBody();
var bodyText = body.getText();
//DocumentApp.getUi().alert( "Does document contain \\t? " + /\t/.test( bodyText ) ); // \n true, \r false, \t true
bodyText = bodyText.replace( /\n/g, "" );
bodyText = bodyText.replace( /\t/g, "" );
body.setText( bodyText );
This worked within a Doc. Not sure if the same is possible within a Sheet (and, even if it were, you'd probably have to run this once cell at a time).

here is my pragmatic solution to eliminate newlines in Google Docs, or, more exact, to eliminate newlines from Gmail message.getPlainBody().
It looks that Google uses '\r\n\r\n' as a plain EOL and '\r\n' as a manuell Linefeed (Shift-Enter). The code should be self explainable.
It might help to get alone with the newline problem in Docs.
A solution possibly not very elegant, but works like a charm :-)
function GetEmails2Doc() {
var doc = DocumentApp.getActiveDocument();
var body = doc.getBody();
var pc = 0; // Paragraph Counter
var label = GmailApp.getUserLabelByName("_Send2Sheet");
var threads = label.getThreads();
var i = threads.length;
// LOOP Messages within a THREAT
for (i=threads.length-1; i>=0; i--) {
for (var j = 0; j < messages.length; j++) {
var message = messages[j];
/* Here I do some ...
body.insertParagraph(pc++, Utilities.formatDate(message.getDate(), "GMT",
"dd.MM.yyyy (HH:mm)")).setHeading(DocumentApp.ParagraphHeading.HEADING4)
str = message.getFrom() + ' to: ' + message.getTo();
if (message.getCc().length >0) str = str + ", Cc: " + message.getCc();
if (message.getBcc().length >0) str = str + ", Bcc: " + message.getBcc();
body.insertParagraph(pc++,str);
*/
// Body !!
var str = processBody(message.getPlainBody()).split("pEOL");
Logger.log(str.length + " EOLs");
for (var k=0; k<str.length; k++) body.insertParagraph(pc++,str[k]);
}
}
}
function processBody(tx) {
var s = tx.split(/\r\n\r\n/g);
// it looks like message.getPlainBody() [of mail] uses \r\n\r\n as EOL
// so, I first substitute the 'EOL's with the string pattern "pEOL"
// to be replaced with body.insertParagraph in the main function
tx = '';
for (k=0; k<s.length; k++) tx = tx + s[k] + "pEOL";
// then replace all remaining simple \r\n with a blank
s = tx.split(/\r\n/g);
tx = '';
for (k=0; k<s.length; k++) tx = tx + s[k] + " ";
return tx;
}

I have now found out through much trial and error -- and some much needed help from Wiktor Stribiżew (see other answer) -- that there is a solution to this, but it relies on the fact that Google Script does not recognise \n or \r in regex searches. The solution is as follows:
function removeLineBreaks() {
var selection = DocumentApp.getActiveDocument()
.getSelection();
if (selection) {
var elements = selection.getRangeElements();
for (var i = 0; i < elements.length; i++) {
var element = elements[i];
// Only deal with text elements
if (element.getElement()
.editAsText) {
var text = element.getElement()
.editAsText();
if (element.isPartial()) {
var start = element.getStartOffset();
var finish = element.getEndOffsetInclusive();
var oldText = text.getText()
.slice(start, finish);
if (oldText.match(/\r/)) {
var number = oldText.match(/\r/g)
.length;
for (var j = 0; j < number; j++) {
var location = oldText.search(/\r/);
text.deleteText(start + location, start + location);
text.insertText(start + location, ' ');
var oldText = oldText.replace(/\r/, ' ');
}
}
}
// Deal with fully selected text
else {
text.replaceText("\\v+", " ");
}
}
}
}
// No text selected
else {
DocumentApp.getUi()
.alert('No text selected. Please select some text and try again.');
}
}
Explanation
Google Docs allows searching for vertical tabs (\v), which match newlines.
Partial text is a whole other problem. The solution to dealing with partially selected text above finds the location of newlines by extracting a text string from the text element and searching in that string. It then uses these locations to delete the relevant characters. This is repeated until the number of newlines in the selected text has been reached.

This Stack Overflow answer removes, specifically, "\n". It may help, it helped me indeed.

Related

Typescript regex exclude whole string if followed by specific string

I'm been running into weird issues with regex and Typescript in which I'm trying to have my expression replace the value of test minus the first instance if followed by test. In other words, replace the first two lines that have test but for the third line below, replace only the second value of test.
[test]
[test].[db]
[test].[test]
Where it should look like:
[newvalue]
[newvalue].[db]
[test].[newvalue]
I've come up with lots of variations but this is the one that I thought was simple enough to solve it and regex101 can confirm this works:
\[(\w+)\](?!\.\[test\])
But when using Typescript (custom task in VSTS build), it actually replaces the values like this:
[newvalue]
[newvalue].[db]
[newvalue].[test]
Update: It looks like a regex like (test)(?!.test) breaks when changing the use cases removing the square brackets, which makes me think this might be somewhere in the code. Could the problem be with the index that the value is replaced at?
Some of the code in Typescript that is calling this:
var filePattern = tl.getInput("filePattern", true);
var tokenRegex = tl.getInput("tokenRegex", true);
for (var i = 0; i < files.length; i++) {
var file = files[i];
console.info(`Starting regex replacement in [${file}]`);
var contents = fs.readFileSync(file).toString();
var reg = new RegExp(tokenRegex, "g");
// loop through each match
var match: RegExpExecArray;
// keep a separate var for the contents so that the regex index doesn't get messed up
// by replacing items underneath it
var newContents = contents;
while((match = reg.exec(contents)) !== null) {
var vName = match[1];
// find the variable value in the environment
var vValue = tl.getVariable(vName);
if (typeof vValue === 'undefined') {
tl.warning(`Token [${vName}] does not have an environment value`);
} else {
newContents = newContents.replace(match[0], vValue);
console.info(`Replaced token [${vName }]`);
}
}
}
Full code is for the task I'm using this with: https://github.com/colindembovsky/cols-agent-tasks/blob/master/Tasks/ReplaceTokens/replaceTokens.ts
For me this regex is working like you are expecting:
\[(test)\](?!\.\[test\])
with a Typescript code like that
myString.replace(/\[(test)\](?!\.\[test\])/g, "[newvalue]");
Instead, the regex you are using should replace also the [db] part.
I've tried with this code:
class Greeter {
myString1: string;
myString2: string;
myString3: string;
greeting: string;
constructor(str1: string, str2: string, str3: string) {
this.myString1 = str1.replace(/\[(test)\](?!\.\[test\])/g, "[newvalue]");
this.myString2 = str2.replace(/\[(test)\](?!\.\[test\])/g, "[newvalue]");
this.myString3 = str3.replace(/\[(test)\](?!\.\[test\])/g, "[newvalue]");
this.greeting = this.myString1 + "\n" + this.myString2 + "\n" + this.myString3;
}
greet() {
return "Hello, these are your replacements:\n" + this.greeting;
}
}
let greeter = new Greeter("[test]", "[test].[db]", "[test].[test]");
let button = document.createElement('button');
button.textContent = "Say Hello";
button.onclick = function() {
alert(greeter.greet());
}
document.body.appendChild(button);
Online playground here.

Gmail App search criteria

I have the following search criteria working very well in Gmail:
user#domain from:/mail delivery/ || /postmaster/ ||/Undeliverable/
I am trying to write Goole Apps code to return the same results. Here is the code:
var thread=GmailApp.search("user#domain from:/mail delivery/ || /postmaster/ ||/Undeliverable/ ");
I am getting different results. I am new to both Regex and Google Apps.
Try Amit Agarwal's tutorial on Gmail Search with Google Apps Script which includes Using Regular Expressions to Find Anything in your Gmail Mailbox:
function Search() {
var sheet = SpreadsheetApp.getActiveSheet();
var row = 2;
// Clear existing search results
sheet.getRange(2, 1, sheet.getMaxRows() - 1, 4).clearContent();
// Which Gmail Label should be searched?
var label = sheet.getRange("F3").getValue();
// Get the Regular Expression Search Pattern
var pattern = sheet.getRange("F4").getValue();
// Retrieve all threads of the specified label
var threads = GmailApp.search("in:" + label);
for (var i = 0; i < threads.length; i++) {
var messages = threads[i].getMessages();
for (var m = 0; m < messages.length; m++) {
var msg = messages[m].getBody();
// Does the message content match the search pattern?
if (msg.search(pattern) !== -1) {
// Format and print the date of the matching message
sheet.getRange(row,1).setValue(
Utilities.formatDate(messages[m].getDate(),"GMT","yyyy-MM-dd"));
// Print the sender's name and email address
sheet.getRange(row,2).setValue(messages[m].getFrom());
// Print the message subject
sheet.getRange(row,3).setValue(messages[m].getSubject());
// Print the unique URL of the Gmail message
var id = "https://mail.google.com/mail/u/0/#all/"
+ messages[m].getId();
sheet.getRange(row,4).setFormula(
'=hyperlink("' + id + '", "View")');
// Move to the next row
row++;
}
}
}
}

Removing line breaks using Docs GAS

I want to remove all newlines with spaces using google apps script for Docs
var doc = DocumentApp.getActiveDocument();
var body = doc.getBody().editAsText();
body.replaceText("\\t", " "); //works properly for tabs
body.replaceText("\\n", " "); //doesnt work
https://developers.google.com/apps-script/reference/document/body#replacetextsearchpattern-replacement
Any suggestions.?
It seems that the body.replaceText does not allow replacements of what is between paragraphs (the paragraph breaks).
You need to somehow merge all paragraphs into 1 paragraph. You may do it roughly with the following code:
function mergePars() {
var doc = DocumentApp.getActiveDocument();
var body = doc.getBody();
var pars = body.getParagraphs();
for( var j = 0; j < pars.length; ++j ) {
try {
pars[j].merge();
}
catch (e) {
Logger.log(e); // It will log "Exception: Element must be preceded by an element of the same type."
}
}
}
You may get rid of the try-catch if you get the number of all children in the document (with .getNumChildren()) and then loop through the items checking their DocumentApp.ElementType, and if the previous node was of type DocumentApp.ElementType.PARAGRAPH, apply .merge().

Program hangs due to too many regex-based replacements

In my input file, I need to do lots of string-manipulations (find/replaces) using Regex depending on various conditions. Like, if the content's one block meets the condition, I need to go to previous block and do replacement in that block.
For this reason, I am splitting the content to many substrings, so that I can move back to previous block (here, previous substring); and do the REGEX replacement.
But the Program hangs in the middle if the File content is more(or may be, no. of substrings exceeds).
Here is the code snippet.
string content = string.Empty;
string target_content = string.Empty;
string[] active_doc_nos;
byte[] content_bytes;
FileInfo input_fileinfo = new FileInfo(input_file);
long file_length = input_fileinfo.Length;
using (FileStream fs_read = new FileStream(input_file, FileMode.Open, FileAccess.Read))
{
content_bytes = new byte[Convert.ToInt32(file_length)];
fs_read.Read(content_bytes, 0, Convert.ToInt32(file_length));
fs_read.Close();
}
content = ASCIIEncoding.ASCII.GetString(content_bytes);
if (Regex.IsMatch(content, "<\\?CLG.MDFO ([^>]*) LEVEL=\"STRUCTURE\""))
{
#region Logic-1: TWO PAIRS of MDFO-MDFC-s one pair following the other
content = Regex.Replace(content, "(<\\?CLG.MDFO)([^>]*)(LEVEL=\"STRUCTURE\")", "<MDFO_VALIDATOR>$1$2$3");
string[] MDFO_Lines = Regex.Split(content, "<MDFO_VALIDATOR>");
active_doc_nos = new string[MDFO_Lines.GetLength(0)];
active_doc_nos[0] = Regex.Match(MDFO_Lines[0], "ACTIVE DOC=\"([^>]*)\"\\s+").ToString();
for (int i = 1; i < MDFO_Lines.GetLength(0); i++)
{
active_doc_nos[i] = Regex.Match(MDFO_Lines[i], "ACTIVE DOC=\"([^>]*)\"\\s+").ToString();
if (Regex.IsMatch(MDFO_Lines[i - 1], "(<\\?CLG.MDFC)([^>]*)(\\?>)(<\\S*\\s*\\S*>)*$"))
{
MDFO_Lines[i - 1] = Regex.Replace(MDFO_Lines[i - 1], "(<\\?CLG.MDFC)([^>]*)(\\?>)(<\\S*\\s*\\S*>)*$", "<?no_smark?>$1$2$3$4");
if (Regex.IsMatch(MDFO_Lines[i - 1], "^<\\?CLG.MDFO ([^>]*) ACTION=\"DELETED\""))
{
MDFO_Lines[i - 1] = Regex.Replace(MDFO_Lines[i - 1], "^<\\?CLG.MDFO ([^>]*) ACTION=\"DELETED\"", "<?no_bmark?><?CLG.MDFO $1 ACTION=\"DELETED\"");
}
if (active_doc_nos[i] == active_doc_nos[i - 1])
{
MDFO_Lines[i] = Regex.Replace(MDFO_Lines[i], "^<\\?CLG.MDFO ([^>]*) " + active_doc_nos[i], "<?no_smark?><?CLG.MDFO $1 " + active_doc_nos[i]);
}
}
}
foreach (string str_piece in MDFO_Lines)
{
target_content += str_piece;
}
byte[] target_bytes = ASCIIEncoding.ASCII.GetBytes(target_content);
using (FileStream fs_write = new FileStream(input_file, FileMode.Create, FileAccess.Write))
{
fs_write.Write(target_bytes, 0, target_bytes.Length);
fs_write.Close();
}
Do I have any other option to achieve this task??
Hard to say without seeing your data, but I have a suspicion that this part of some of your regexes may be the culprit:
(<\\S*\\s*\\S*>)*
Because \S can also match < and >, because everything is optional, and because you've got nested quantifiers, it's possible that this part of the regex leads to catastrophic backtracking.
What happens if you replace these parts with (?>(<\\S*\\s*\\S*>))*?

splitting function attributes

Hi would it be possible to correctly split function attributes using regex ?
i want an expression that splits all attributes in a comma seperated list. But the attributes themselves could be an Array or Object or something that can also contain commas :
ex :
'string1','string2',sum(1,5,8), ["item1","item2","item3"], "string3" , {a:"text",b:"text2"}
this should be split up as :
'string1'
'string2'
sum(1,5,8)
["item1","item2","item3"]
"string3"
{a:"text",b:"text2"}
so the expression should split all commas , but not commas that are surrounded by (), {} or [].
i am trying this in as3 btw
here is some code that will split all the commas (which is ofcourse not what i want) :
var attr:String = "'string1','string2',sum(1,5,8), ['item1','item2','item3'], 'string3' , {a:'text',b:'text2'}";
var result:Array = attr.match(/([^,]+),/g);
trace(attr);
for(var a:int=0;a<result.length;a++){
trace(a,result[a]);
}
here is an expression that allows nested round brackets , but not the others...
/([^,]+\([^\)]+\)|[^,]+),*/g
I've created a little example how to tackle a problem like this, only tested on your input so it might contain horrible mistakes. It only takes into account the parentheses and not the (curly) braces, but those can be easily added.
Basic idea is that you iterate over the characters in the input and add them to the current token if they are not a separator char, and push the current token into the result array when encountering a separator. You have to add a stack that will keep track how 'deep' you are nested to determine of a comma is a separator or part of a token.
For any issue more complicated than this you'll probably be better of using a 'real' parser (and probably a parser-generator), but in this case I think you'll be ok using some custom code.
As you can see parsing code like this quickly becomes quite hard to understand/debug. In a real-case scenario I'd recommend adding more comments, but also a good batch of tests to explain your expected behavior.
package {
import flash.display.Sprite;
public class parser extends Sprite
{
public function parser()
{
var input:String = "'string1','string2',sum(1,5,8), [\"item1\",\"item2\",\"item3\"], \"string3\" , {a:\"text\",b:\"text2\"}"
var result:Array = parseInput(input);
for each (var item:String in result)
{
trace(item);
}
}
// this function only takes into account the '(' and ')' - adding the others is similar.
private function parseInput(input:String):Array
{
var result:Array = [];
trace("parsing: " + input);
var token:String = "";
var parenthesesStack:Array = [];
var currentChar:String;
for (var i:int = 0; i < input.length; i++)
{
currentChar = input.charAt(i)
switch (currentChar)
{
case "(":
parenthesesStack.push("(");
break;
case ")":
if (parenthesesStack.pop() != "(")
{
throw new Error("Parse error at index " + i);
}
break;
case ",":
if (parenthesesStack.length == 0)
{
result.push(token);
token = "";
}
break;
}
// add character to the token if it is not a separating comma
if (currentChar != "," || parenthesesStack.length != 0)
{
token = token + currentChar;
}
}
// add the last token
if (token != "")
{
result.push(token);
}
return result;
}
}
}