I'm trying to create a loose word wrapping system via a regex in Perl. What I would like is about every 70 characters or so to check for the next whitespace occurrence and replace that space with a newline, and then do this for the whole string. The string I'm operating on may already have newlines in it already, but the amount of text between newlines tends to be very lengthy.
I'd like to avoid looping one character at a time or using substr if I can, and I would prefer to edit this string in place as opposed to creating new string objects. These are just preferences, though, and if I can't achieve what I'm looking for without breaking these preferences then that's fine.
Thoughts?
Look at modules like Text::Wrap or Text::Autoformat.
Depending on your needs, even the GNU core utility fold(1) may be an option.
s/(.{70}[^\s]*)\s+/$1\n/
Consume the first 70 characters, then stop at the next whitespace, capturing everything in the process. Then, emit the captured string, omitting the whitespace at the end, adding a newline.
This doesn't guarantee your lines will cut off strictly at 80 characters or something. There's no guarantee the last word it consumes won't be a billion characters long.
Welbog's answer wraps at the first space after 70 characters. This has the flaw that long words beginning close to the end of the line make an overlong line. I would suggest instead wrapping at the last space within the first, say, 81 characters, or wrapping at the first space if you have a >80 character "word", so that only truly unbreakable lines are overlong:
s/(.{1,79}\S|\S+)\s+/$1\n/g;
In modern perl:
s/(?:.{1,79}\S|\S+)\K\s+/\n/g;
You can get much, much more control and reliability by using Text::Format
use Text::Format;
print Text::Format->new({columns => 70})->format($text);
This is the one I've always used.
Unlike the accepted solution, it will wrap BEFORE the wrap-length (in this case, 70 characters), unless there's a really long "word" without spaces (such as a URL), in which case it will just place that word on its own line, rather than break it.
s/(?=.{70,})(.{0,70}\n?)( )/\1\2\n /g
This second form handles all line endings: Mac \r, Unix \n, Windows \r\n, and Teletype \n\r, but which one it uses as a replacement still depends on what you put in the replacement clause: I've used \n.
s/(?=.{70,})(.{0,70}(?:\r\n?|\n\r?)?)( )/\1\2\n /g
Both versions also indent all wrapped lines after the first by one space: remove the space before the last /g if you don't want that, but I usually find it nicer.
Related
I have a C++ source file containing many functions.
I want to find the beginning of every function quickly.
How can I form an expression for )newline{newline?
The newline symbol can be either one of the following:
\n
\r
\n\r
\r\n
Presumably, the same symbol is used all across the file, so instead of a single expression for all options combined, I need a single expression for each option.
I assume that a regular-expression can be used, but I'm not sure how.
Thanks
Barak, before we look at individual options, for all options, this will do it:
\)[\r\n]+{[\r\n]+
The [\r\n] is a character class that allows either of \r or \n. It is quantified with a + which means we are looking for one or more of these characters.
You said you want individual options, so this can be turned to:
\)\r\n{\r\n
\)\r{\r
\)\n{\n
\)\n\r{\n\r (this sequence of newlines is quite surprising)
If you simply want to use the regex search in VS to find the beginning of each function then this should work for you:
\)\r?\n\s*{\r?\n
Although that assumes the { is always on the next line with no white space before the line break.
This would be less strict where white space is concerned, but still expect the { to be on the next line and to be followed by a line break:
\)\s*\r?\n\s*{\s*\r?\n
And this would basically just look for the 2 brackets even if they're on the same line:
\)\s*\r?\n?\s*{
And if you expect there could be several line breaks between the 2 brackets:
\)\s*(\r?\n\s*)*{
Last example should find anything that could resemble the beginning of a method. But not sure how strict you want your search to be.
I have this problem:
Input text:
this is my text text text and more text
this is my text myspace this is my text
this space is my text space this is my
this is my text this is my text
this space is my text space space myspace
Let say I want to search for "space"
I would like to have this as output:
this is my text text text and more text
space
space space
this is my text this is my text
space space space space
Matches on the same line have to be separated with a space.
Line without matches must remain as it is.
Same for all other search items.
I'm trying to realize this, this afternoon but without success.
Can anyone help me?
Solution:
:g/space/s/\(.*space\).*$/\1/|s/.\{-}space/ space/g|s/^ //
Explanation:
This is tricky, but it can be done. It can't be done with a single regular expression, though.
The first thing we do is get rid of anything after the last match (we actually exploit the fact that regular expressions are greedy by default here):
s/\(.*space\).*$/\1/
Then we remove anything between all the internal matches (notice we use the lazy version of * here, \{-}):
s/.\{-}space/ space/g
The previous step will leave an initial space in the result, so we get rid of that:
s/^ //
Fortunately, in vim, we can chain replacements together with the | character. So, putting it all together:
:g/space/s/\(.*space\).*$/\1/|s/.\{-}space/ space/g|s/^ //
is this tricky line ok for you?
:g/space/s/space/^G/g|s/[^^G]//g|s/^G/space /g
the ^G above you need press Ctrl-V Ctrl-G
the output of above command is same as your example except for the ending whitespace after pattern (space in this case). but it is easy to be fixed, e.g. chain another s/ $// after the :g line.
Kent's solution uses a nice trick that makes it work only for fixed strings, but it's clean and short. Ethan Brown's answer is more general, but also adds complexity with its three steps. I think the best solution can be developed based on the accepted answer in this very similar question.
Contrary to what Ethan Brown assumes, this can indeed be done with a single regular expression substitution. Here it is, in all its ugliness:
:g/space/s/\%(^\|\%(space \)*space\%( \%(.*space\)\#=\)\?\)\zs\%(\%(space \)*space\%( \%(.*space\)\#=\)\?\)\#!.\{-1,}\ze\%(\%(space \)*space\%( \%(.*space\)\#=\)\?\|$\)//g
It becomes somewhat more readable when you use the :DeleteExcept command from my PatternsOnText plugin:
:g/space/DeleteExcept/\%(space \)*space\%( \%(.*space\)\#=\)\?/
Explanation
This deletes everything except
potentially multiple sequential occurrences \%(space \)*
of the word space
including the trailing whitespace when it's not the last match in the line, i.e. there's a following match \%(.*space\)\#= so that the whitespace is not swallowed
or excluding (i.e. deleting) it \? after the last match in the line.
More practical alternative
Though it's a nice challenge to come up with the above solution, in practice, I would also favor a two-step approach, just because it's way simpler:
:g/space/DeleteExcept/space\%( \|$\)/
This leaves behind trailing whitespace that can be pruned with
:%s/ $//
I often work with text files which have a variable amount of whitespaces as word separators (text processors like Word do this, to distribute fairly the whitespace amount due to different sizes of letters in certain fonts and they put this annoying variable amount of spaces even when saving as plain text).
I would like to automate the process of replacing these sequences of whitespaces that have variable length with single spaces. I suspect a regex could do it, but there are also whitespaces at the beginning of paragraphs (usually four of them, but not always), which I would want to let unchanged, so basically my regex should also not touch the leading whitespaces and this adds to the complexity.
I'm using vim, so a regex in the vim regex dialect would be very useful to me, if this is doable.
My current progress looks like this:
:%s/ \+/ /g
but it doesn't work correctly.
I'm also considering to write a vim script that could parse text lines one by one, process each line char by char and skip the whitespaces after the first one, but I have a feeling this would be overkill.
this will replace 2 or more spaces
s/ \{2,}/ /g
or you could add an extra space before the \+ to your version
s/ \+/ /g
This will do the trick:
%s![^ ]\zs \+! !g
Many substitutions can be done in Vim easier than with other regex dialects by using the \zs and \ze meta-sequences. What they do is to exclude part of the match from the final result, either the part before the sequence (\zs, “s” for “start here”) or the part after (\ze, “e” for “end here”). In this case, the pattern must match one non-space character first ([^ ]) but the following \zs says that the final match result (which is what will be replaced) starts after that character.
Since there is no way to have a non-space character in front of line-leading whitespace, it will be not be matched by the pattern, so the substitution will not replace it. Simple.
In the interests of pragmatism, I tend to just do it as a three-stage process:
:g/^ /s//XYZZYPARA/g
:g/ \+/s// /g
:g/^XYZZYPARA/s// /g
I don't doubt that there may be a better way (perhaps using macros or even a pure regex way) but I usually find this works when I'm in a hurry. Of course, if you have lines starting with XYZZYPARA, you may want to adjust the string :-)
It's good enough to turn:
This is a new paragraph
spanning two lines.
And so is this but on one line.
into:
This is a new paragraph
spanning two lines.
And so is this but on one line.
Aside: If you're wondering why I use :g instead of :s, that's just habit mostly. :g can do everything :s can and so much more. It's actually a way to execute an arbitrary command on selected lines. The command to execute happens to be s in this case so there's no real difference but, if you want to become a vi power user, you should look into :g at some point.
There are lots of good answers here (especially Aristotle's: \zs and \ze are well worth learning). Just for completeness, you can also do this with a negative look-behind assertion:
:%s/\(^ *\)\#<! \{2,}/ /g
This says "find 2 or more spaces (' \{2,}') that are NOT preceded by 'the start of the line followed by zero or more spaces'". If you prefer to reduce the number of backslashes, you can also do this:
:%s/\v(^ *)#<! {2,}/ /g
but it only saves you two characters! You could also use ' +' instead of ' {2,}' if you don't mind it doing a load of redundant changes (i.e. changing a single space to a single space).
You could also use the negative look-behind to just check for a single non-space character:
:%s/\S\#<!\s\+/ /g
which is much the same as (a slightly modified version of Aristotle's to treat spaces and tabs as the same in order to save a bit of typing):
:%s/\S\zs \+/ /g
See:
:help \zs
:help \ze
:help \#<!
:help zero-width
:help \v
and (read it all!):
:help pattern.txt
Answered; but though i'd toss my work flow in anyway.
%s/ / /g
#:#:#:#:#:#:#:#:#:#:#:#:(repeat till clean)
Fast and simple to remember. There are a far more elegant solutions above; but just my .02.
Does this work?
%s/\([^ ]\) */\1 /g
I like this version - it is similar to the look ahead version of Aristotle Pagaltzis, but I find it easier to understand. (Probably just my unfamiliarity with \zs)
s/\([^ ]\) \+/\1 /g
or for all whitespace
s/\(\S\)\s\+/\1 /g
I read it as "replace all occurences of something other than a space followed by multiple spaces with the something and a single space".
I happened across this page full of super useful and rather cryptic vim tips at http://rayninfo.co.uk/vimtips.html. I've tried a few of these and I understand what is happening enough to be able to parse it correctly in my head so that I can possibly recreate it later. One I'm having a hard time getting my head wrapped around though are the following two commands to remove all spaces from the end of every line
:%s= *$== : delete end of line blanks
:%s= \+$== : Same thing
I'm interpreting %s as string replacement on every line in the file, but after that I am getting lost in what looks like some gnarly variation of :s and regex. I'm used to seeing and using :s/regex/replacement. But the above is super confusing.
What do those above commands mean in english, step by step?
The regex delimiters don't have to be slashes, they can be other characters as well. This is handy if your search or replacement strings contain slashes. In this case I don't know why they use equal signs instead of slashes, but you can pretend that the equals are slashes:
:%s/ *$//
:%s/ \+$//
Does that make sense? The first one searches for a space followed by zero or more spaces, and the second one searches for one or more spaces. Each one is anchored at the end of the line with $. And then the replacement string is empty, so the spaces are deleted.
I understand your confusion, actually. If you look at :help :s you have to scroll down a few pages before you find this note:
*E146*
Instead of the '/' which surrounds the pattern and replacement string, you
can use any other character, but not an alphanumeric character, '\', '"' or
'|'. This is useful if you want to include a '/' in the search pattern or
replacement string. Example:
:s+/+//+
I do not know vim syntax, but it looks to me like these are sed-style substitution operators. In sed, the / (in s/REGEX/REPLACEMENT/) can be uniformly replaced with any other single character. Here it appears to be =. So if you mentally replace = with /, you'll get
:%s/ *$//
:%s/ \+$//
which should make more sense to you.
I have a big paragraph which I need to split into lines such that each line must not have more than 100 characters and no words must be broken. How would I go about doing this? I guess with regular expressions is the best way but I'm not sure how.
Use Text::Wrap.
Text::Wrap::wrap() is a very simple paragraph formatter. It formats a single paragraph at a time by breaking lines at word boundaries. Indentation is controlled for the first line ($initial_tab) and all subsequent lines ($subsequent_tab) independently.
While you should use a library function if you have one, as KennyTM suggested, a simple regex to solve this can be:
.{1,100}\b
This will take 100 characters or less, and will not break words. It would break other characters though, for example the period at the end of a sentence may be parted from the last word (last word<\n>. new line).
If that's an issue, you can also try:
.{1,99}(\s|.$)
That assures the last character in every match is a white space.
All of these assume you count spaces as characters, and probably don't have newlines in your text (a single paragraph), and don't have word of over 100 characters.