Ignoring headers in log files being indexed in Splunk - regex

Being relatively new to Splunk (ver 6) and even newer to Reg-ex, I have log files that I and trying to index that have a header than I need to ignore. There are 6 header lines. The first 4 all begin with * and the last two are blank lines. I'm assuming they are just carriage returns. I'm looking for help with the regular expression that will ignore these lines in the transforms.conf file when adding the data. Below is an example from the log file I want to add:
*******************************
*** This is a Header ***
*** 07:32:06 Tue Jan 07 ***
*******************************
Jan-07 07:32:06 SERVERNAME:somedatainfo
Jan-07 07:32:06 SERVERNAME:moredatainfo

On the forwarder or indexer you can set the sourcetype for the file that is being monitored.
# in inputs.conf
[monitor:///path/to/your/file]
sourcetype=new_sourcetype
On the indexer, you can get rid of the first lines like this:
# in props.conf
[new_sourcetype]
TRANSFORMS-test=ignore_header
#in transforms.conf
[ignore_header]
REGEX=^\*\*\*+
DEST_KEY=queue
FORMAT=nullQueue

Related

'revtes4-1' template of aip journals and the error : File `aps10pt4-1.rtx' not found. ...intsize\expandafter{\#pointsize#default}

I am writing a manuscript in aip journals (revtex4-1 that can be found here). Unfortunately, when I run it with Texstoduio or TexMaker I get this eror:
File `aps10pt4-1.rtx' not found. ...intsize\expandafter{#pointsize#default}.
I copy aps10pt4-1.rtx from Github, but nothing fixed. This is the my log file:
entering extended mode
restricted \write18 enabled.
%&-line parsing enabled.
**./aipsamp.tex
(aipsamp.tex
LaTeX2e <2022-11-01> patch level 1
L3 programming layer <2022-12-17> (revtex4-1.cls
Document Class: revtex4-1 2020/09/30 4.1s (http://publish.aps.org/revtex4/ for
documentation)
Copyright (c) 2009 The American Physical Society.
mailto:revtex#aps.org
Licensed under the LPPL:
http://www.ctan.org/tex-archive/macros/latex/base/lppl.txt
Arthur Ogawa <arthur_ogawa at sbcglobal dot net>
Based on work by David Carlisle <david at dcarlisle.demon.co.uk>.
ltxutil[2020/09/30 4.1s utilities package (portions licensed from W. E. Baxter
web at superscript.com)]
LaTeX Info: Redefining \#centercr on input line 346.
Class revtex4-1 Info: Repairing hyperref-unfriendly LaTeX definition of \mpfoot
note on input line 471.
Class revtex4-1 Info: Repairing broken LateX \#sect on input line 793.
\intertabularlinepenalty=\count185
\#tbpen=\count186
\#arstrutbox#hline=\box51
Class revtex4-1 Info: Repairing broken LaTeX \#xbitor on input line 1824.
Class revtex4-1 Info: Making \typeout \long on input line 2089.
ltxfront[2020/09/30 4.1s frontmatter package (AO,DPC)]
\c#affil=\count187
\c#collab=\count188
\absbox=\box52
ltxgrid[2020/09/30 4.1s page grid package (portions licensed from W. E. Baxter
web at superscript.com)]
\c#linecount=\count189
\output#latex=\toks16
\box#size#1=\box53
\box#size#2=\box54
\#protection#box=\box55
\#topmark#saved=\toks17
\footins#saved=\box56
\footins#recovered=\box57
\column#recovered=\box58
\pagesofar=\box59
\footsofar=\box60
\pagegrid#col=\count190
Class revtex4-1 Info: Overriding \#vspace, \#vspacer, \#no#pgbk, \#newline, and
\\ on input line 5655.
\c#part=\count191
\c#section=\count192
\c#subsection=\count193
\c#subsubsection=\count194
\c#paragraph=\count195
\c#subparagraph=\count196
\abovecaptionskip=\skip48
\belowcaptionskip=\skip49
\c#figure=\count197
\fbox#1=\box61
\c#table=\count198
\fbox#2=\box62
\c#video=\count199
\fbox#4=\box63
\#bibdataout=\write3
\widetext#top=\box64
\widetext#bot=\box65
Class revtex4-1 Info: Selecting society aip on input line 7616.
(aip4-1.rtx
File: aip4-1.rtx 2010/07/25 4.1r AIP substyle for REVTeX
Class revtex4-1 Info: RevTeX society AIP selected on input line 75.
)
Class revtex4-1 Warning: No journal specified, using default cha.
Using journal substyle cha.
Class revtex4-1 Info: Reading file aps10pt4-1.rtx on input line 7621.
! LaTeX Error: File `aps10pt4-1.rtx' not found.
Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: rtx)
Enter file name:
! Emergency stop.
<read *>
l.7621 ...intsize\expandafter{\#pointsize#default}
%^^M
*** (cannot \read from terminal in nonstop modes)
Here is how much of TeX's memory you used:
1866 strings out of 476406
24196 string characters out of 2789095
1848787 words of memory out of 3000000
22315 multiletter control sequences out of 15000+600000
512287 words of font info for 32 fonts, out of 8000000 for 9000
1141 hyphenation exceptions out of 8191
33i,0n,45p,549b,22s stack positions out of 10000i,1000n,20000p,200000b,80000s
! ==> Fatal error occurred, no output PDF file produced!
I downloaded the template from overleaf (official source). I really appreciate any help

How to parse Apache Catalina.log

I am trying to find a way to parse a Catalina.log and i am really struggling.
This a piece of the code:
May 12, 2017 2:14:38 PM org.apache.coyote.AbstractProtocol init
SEVERE: Failed to initialize end point associated with ProtocolHandler ["http-apr-10.1.31.104-443"]
java.lang.Exception: Connector attribute SSLCertificateFile must be defined when using SSL with APR
at org.apache.tomcat.util.net.AprEndpoint.bind(AprEndpoint.java:490)
at org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:649)
at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:434)
at org.apache.catalina.connector.Connector.initInternal(Connector.java:978)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.core.StandardService.initInternal(StandardService.java:559)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:821)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.startup.Catalina.load(Catalina.java:638)
at org.apache.catalina.startup.Catalina.load(Catalina.java:663)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:253)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:427)
I wanna get
Date = May 12, 2017 2:14:38 PM
class = org.apache.coyote.AbstractProtocol init
Error level = SEVERE
Error Msg = Failed to initialize end point associated with ProtocolHandler ["http-apr-10.1.321.224-443"]
Error Msg Body = java.lang.Exception: Connector attribute SSLCertificateFile must be defined when using SSL with APR
at org.apache.tomcat.util.net.AprEndpoint.bind(AprEndpoint.java:490)....
i don even know where to start :)
any ideas are very welcomed
I have prepared for you the following regex:
((Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\s+\d{1,2},\s+\d{4}\s+\d{1,2}:\d{1,2}:\d{1,2}\s(AM|PM))\s(.+)(\r)?\n(FATAL|SEVERE|ERROR|WARN(ING)?|INFO|CONFIG|INFO|DEBUG):\s(.+)(\r)?\n(.+)(\r)?\n(?=\s+at.+java:\d+\))
You can use the following back reference to capture your groups
DATE -> $1
CLASS -> $4
ERROR_LEVEL -> $6
ERROR_MSG -> $8
ERROR_BODY -> $10
The regex will only fetch strings that met the following conditions:
starts by a date in the format specified in your post
after the date, the first line is composed of the class name
the 2nd line is composed of the error level and the error msg
the 3rd line is your error msg body
followed by a java strack trace of n lines starting by \s at and ending by java:\d+)
The regex works in the following way:
((Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\s+\d{1,2},\s+\d{4}\s+\d{1,2}:\d{1,2}:\d{1,2}\s(AM|PM))
This part will fetch the date in the format of your post:
3 char month followed by space(s) then 1 or 2 digits, ',' then year in 4 digit
then space(s), then time(column char, followed by space(s) then AM or PM
\s(.+)(\r)?\n
this part of the regex will allow you to get the rest of your first line corresponding to your class
(FATAL|SEVERE|ERROR|WARN(ING)?|INFO|CONFIG|INFO|DEBUG):\s(.+)(\r)?\n(.+)(\r)?\n
This part will allow you to get the error level (in this exhaustive list) followed by column and the following 2 lines corresponding to your error msg/body
(?=\s+at.+java:\d+\))
This last part is a condition the enforce that your error is followed by a java stack trace.
You might need to adapt a bit some parts of the regex (like the number of lines of the error body, error message) or the stack trace conditions but I think this is a great starting point for your case.
CHEERS!!!

mutt reply function returns empty header fields

I have been using mutt for quite some time but I have reconfigured it for use with offlineimap. The problem is now that when I reply to an email the header fields "To, From, CC Subject ..." are left empty and it is impossible to send the reply. I have to paste the fields in by hand which is kind of annoying (what do we have computers for?).
Any hints, for I have not got a clue how to debug this problem.
Thanks
my muttrc
======================================================
# Boring details
set realname = "Ivo Keel"
set from = "<myid>#gmail.com"
set use_from = yes
# fills headers on reply
set fast_reply = yes
set envelope_from ="yes"
# Check for new mail in the folders
set mail_check=60 # mutt checks for new mails on every keystroke
# but not more often then once in 5 seconds
set beep_new = yes
# Use a signature
set signature="~/.signature"
# mailcap
set mailcap_path="~/.config/mutt/mailcap"
# Use msmtp rather than sendmail. Check that
# the path is correct for your system:
set sendmail="/usr/bin/msmtp"
# If not set in ~/.bashrc:
set spoolfile = "~/.maildir/"
set print_command="/usr/bin/muttprint %s"
set print_split set print="yes"
set sleep_time = 0
set reply_to = yes
#======================================================#
# Folders
set mbox_type=Maildir
#set mbox="~/.maildir/"
set folder="~/.maildir/" # Local mailboxes stored here
mailboxes =c32t/inbox =hibury/inbox =inbox
set record="+/hibury/sent/" # Where to store sent messages
set postponed="+/hibury/drafts/" # Where to store draft messages
#set move=no # Don't move mail from spool
set mask="!^\\.[^.]"
#======================================================#
# What to show and order of headers
ignore *
unignore Date: From: User-Agent: \
Cc: Reply-To: Subject: Mail-Followup-To:
hdr_order Date From Subject To CC
set sort=threads
set sort_browser=reverse-date
set sort_aux=reverse-last-date-received
#======================================================#
# which editor do you want to use?
# vim of course!
set editor="vim"
set edit_headers=yes # See the headers when editing
#======================================================#
# Aliases
set alias_file = ~/.config/mutt/alias # In their own file
source ~/.config/mutt/alias # Source them
set sort_alias=alias # Sort alphabetically
# Odds and ends
set markers # mark wrapped lines of text in the pager with a +
set smart_wrap # Don't wrap mid-word
set pager_context=5 # Retain 5 lines of previous page when scrolling.
set status_on_top # Status bar on top.
push <show-version> # Shows mutt version at startup
set duplicate_threads = yes
#email addresses with esc-1 and esc-2. Sets the status bar at the
# top too so you know which from address you're using
set mailcap_path = ~/.config/mutt/mailcap auto_view text/html
set query_command="abook --mutt-query '%s'"
# Pager View Options ---------------------------------
set pager_index_lines = 10 # number of index lines to show
set pager_context = 3 # number of context lines to show
set pager_stop # don't go to next message automatically
set menu_scroll # scroll in menus
set tilde # show tildes like in vim
unset markers # no ugly plus signs
set quote_regexp = "^( {0,4}[>|:#%]| {0,4}[a-z0-9]+[>|]+)+"
alternative_order text/plain text/enriched text/html
bind pager R group-reply
# View attachments properly.
bind attach <return> view-mailcap
#Skip some options
set include=yes
set forward_quote=yes
set copy=yes
set delete=yes
set confirmappend=no
#--------------------------------------------------------------------------
# Other configuration files
#--------------------------------------------------------------------------
#source ~/.config/mutt/account_hooks
#source ~/.config/mutt/folder_hooks
source ~/.config/mutt/colors
source ~/.config/mutt/sidebar
#source ~/.config/mutt/gpg
source ~/.config/mutt/macros
source ~/.config/mutt/lists
source ~/.config/mutt/alias
END of muttrc
Allowing mutt to automatically fill the From: header is done by adding:
set use_from = yes
to your .muttrc and allowing it to automatically fill the To: and Subject: header can be done by putting:
set fast_reply = yes
in your .muttrc. Those two options are very common, if you don't have them, you should consider looking at some of the .muttrc configurations available online to find other classical options (noconfirmappend, reverse_alias, sort=threads, etc.) which will greatly improve your mutt experience.
The CC: header will not be filled if you use the "reply" command instead of the "group-reply" one (the default keyboard shortcut is g).
I had this problem, but it turned out to be a problem with my workflow (the buggy system located between the keyboard and chair). Perhaps this was also your issue?
When I reply to a message in Mutt, Mutt drops me into a text editor session with pre-populated reply headers, like this:
1: From: Steve HHH <steve#hhhdomain.com>
2: To: Jonny Appleseed <jonny#hhhdomain.com>
3: Cc:
4: Bcc:
5: Subject: Re: Good morning
6: Reply-To:
7: In-Reply-To: <CABSQ23i=yOa3GUwvbUPmHE+_i1_gLw#mail.gmail.com>
8:
9: On Mon, Sep 29, 2014 at 11:52:04AM -0700, Jonny Appleseed wrote:
10: > LOL
My mistake was to start typing the reply on line one. Mutt needs those headers on lines 1-7 (and the newline on line 8?) to stay intact. If I enter my reply on line nine, the reply fields are correctly populated when I exit the editor and arrive at the Compose menu.

Find a XML tag inside log file using Regular Expression?

<ns2:createOrder>
<ns2:serviceResponseContext >
<interactionId></interactionId>
<requestId>5</requestId>
<requestReceivedTimestamp>Mon Jul 15 10:06:45 MDT
</requestReceivedTimestamp>
<responseGeneratedTimestamp>Mon Jul 15 10:07:29 MDT </responseGeneratedTimestamp>
<serverHostPort></serverHostPort>
<serviceDuration>44541</serviceDuration>
<statusCode>0</statusCode>
</ns2:serviceResponseContext>
<ns2:createOrderResponse>
<ns2:statusLog>
<ns2:createOrderStep>PROVISION_CODE</ns2:createOrderStep>
<ns2:statusCode>0</ns2:statusCode>
<ns2:message></ns2:message>
</ns2:statusLog>
<ns2:statusLog>
<ns2:createOrderStep>VISION_CODE</ns2:createOrderStep>
<ns2:statusCode>0</ns2:statusCode>
<ns2:message></ns2:message>
</ns2:statusLog>
I need to extract the first <statusCode>from the XML file only when the statusCode=0
the <statusCode> should be inside the <createOrder> using regular Expressions.
I just need the first <statusCode>0<statusCode> and ignore the others inside the <createOrderStep>
Thank you for your help !!
Try the following this works for both
<statusCode>0</statusCode> and <ns2:statusCode>0</ns2:statusCode>
/<(ns2:)?statusCode>0<\/(ns2:)?statusCode>/

In search and replacing using VIM, how do I keep a grouped text and reuse it in the replacement part?

In VIM (edit MacVim snapshot 64), I am looking for a way to convert these lines of text
can_check_steps = 1
can_edit_work_type = 1
can_edit_vendor = 1
can_edit_place = 1
to
'can_check_steps': can_check_steps,
'can_edit_work_type': can_edit_work_type,
'can_edit_vendor': can_edit_vendor,
'can_edit_place': can_edit_place,
I know how to add the leading single quote and convert the = into : with multi-line character replacement (something I learned from this question) and basic search and replace, respectively.
But I can't really figure out how to repeat that the variable name per line. I'm also starting to think there should be a way to do this all in one command using like grouped expressions (using ( and ) to "capture" in regular expression).
I know how to do it in Python with lines of code, but I don't really know how to do it in VIM using the s/search_for_this/replace_that/options format.
EDIT:
After reading the answers, why doesn't this work?
^\s*(\w+) = 1\s*$/'\1': \1,/g
For #Jonathan Leffler, :set all in MacVim produces
--- Options ---
aleph=224 bufhidden= cscopequickfix= noerrorbells fsync includeexpr= nomacmeta omnifunc= report=2 showmatch tabline= nottimeout nowildignorecase
antialias buflisted nocscoperelative esckeys nofullscreen incsearch magic operatorfunc= norevins showmode tabpagemax=10 ttimeoutlen=-1 wildmenu
noarabic buftype= nocscopetag eventignore= nogdefault indentexpr= makeef= nopaste norightleft showtabline=1 tabstop=4 ttybuiltin wildmode=full
arabicshape cdpath=,, cscopetagorder=0 expandtab guifont= noinfercase makeprg=make pastetoggle= ruler sidescroll=0 tagbsearch nottyfast wildoptions=
noallowrevins cedit=^F nocscopeverbose noexrc guifontwide= noinsertmode matchtime=5 patchexpr= rulerformat= sidescrolloff=0 taglength=0 ttymouse= window=83
noaltkeymap charconvert= nocursorbind fileencoding= guipty isprint=#,161-255 maxcombine=2 patchmode= scroll=41 smartcase tagrelative ttyscroll=999 winheight=1
ambiwidth=single nocindent nocursorcolumn fileformat=unix guitablabel=%M%t joinspaces maxfuncdepth=100 nopreserveindent noscrollbind smartindent tags=./tags,tags undodir=. nowinfixheight
autochdir cinoptions= cursorline filetype= guitabtooltip= key= maxmapdepth=1000 previewheight=12 scrolljump=1 smarttab tagstack noundofile nowinfixwidth
autoindent cmdheight=1 debug= nofkmap helpheight=20 keymap= maxmem=751782 nopreviewwindow scrolloff=0 softtabstop=0 term=builtin_gui undolevels=1000 winminheight=1
noautoread cmdwinheight=7 nodelcombine foldclose= helplang=en keymodel= maxmemtot=751782 printdevice= nosecure nospell notermbidi undoreload=10000 winminwidth=1
noautowrite colorcolumn= dictionary= foldcolumn=0 nohidden keywordprg=man -s menuitems=25 printencoding= selectmode= spellfile= noterse updatecount=200 winwidth=20
noautowriteall columns=269 nodiff foldenable history=1000 langmap= modeline printfont=courier shell=/bin/bash spelllang=en textauto updatetime=4000 wrap
background=dark nocompatible diffexpr= foldexpr=0 nohkmap langmenu=none modelines=5 printmbcharset= shellcmdflag=-c spellsuggest=best notextmode verbose=0 wrapmargin=0
nobackup concealcursor= diffopt=filler foldignore=# nohkmapp laststatus=2 modifiable printmbfont= shellquote= nosplitbelow textwidth=0 verbosefile= wrapscan
backupcopy=auto conceallevel=0 nodigraph foldlevel=0 hlsearch nolazyredraw nomodified printoptions= shelltemp nosplitright thesaurus= virtualedit= write
backupext=~ completefunc= display= foldlevelstart=-1 icon nolinebreak more prompt shellxquote= startofline notildeop novisualbell nowriteany
balloondelay=600 noconfirm eadirection=both foldmethod=manual iconstring= lines=84 mouse=a pumheight=0 noshiftround suffixesadd= timeout warn writebackup
noballooneval nocopyindent noedcompatible foldminlines=1 noignorecase linespace=0 nomousefocus quoteescape=\ shiftwidth=4 swapfile timeoutlen=1000 noweirdinvert writedelay=0
balloonexpr= cpoptions=aABceFs encoding=utf-8 foldnestmax=20 noimcmdline nolisp mousehide noreadonly noshortname swapsync=fsync title whichwrap=b,s
nobinary cryptmethod=zip endofline formatexpr= imdisable nolist mousetime=500 redrawtime=2000 showbreak= switchbuf= titlelen=85 wildchar=<Tab>
nobomb cscopepathcomp=0 equalalways formatoptions=tcq iminsert=2 listchars=eol:$ number norelativenumber showcmd synmaxcol=3000 titlestring= wildcharm=0
browsedir=last cscopeprg=cscope equalprg= formatprg= imsearch=2 loadplugins numberwidth=4 remap noshowfulltag syntax= transparency=0 wildignore=
backspace=indent,eol,start
backupdir=.,~/tmp,~/
backupskip=/tmp/*,/var/folders/70/f8c54mjn4vg_wztz9bd17lp40000gn/T/*
breakat= ^I!#*-+;:,./?
casemap=internal,keepascii
cinkeys=0{,0},0),:,0#,!^F,o,O,e
cinwords=if,else,while,do,for,switch
clipboard=autoselect
comments=s1:/*,mb:*,ex:*/,://,b:#,:%,:XCOMM,n:>,fb:-
commentstring=/*%s*/
complete=.,w,b,u,t,i
completeopt=menu,preview
define=^\s*#\s*define
directory=.,~/tmp,/var/tmp,/tmp
errorfile=errors.err
errorformat=%*[^"]"%f"%*\D%l: %m,"%f"%*\D%l: %m,%-G%f:%l: (Each undeclared identifier is reported only once,%-G%f:%l: for each function it appears in.),%-GIn file included from %f:%l:%c:,%-GIn file included from %f:%l:%c\,,%-GIn file included from %f:%l:%c,%-GIn file
included from %f:%l,%-G%*[ ]from %f:%l:%c,%-G%*[ ]from %f:%l:,%-G%*[ ]from %f:%l\,,%-G%*[ ]from %f:%l,%f:%l:%c:%m,%f(%l):%m,%f:%l:%m,"%f"\, line %l%*\D%c%*[^ ] %m,%D%*\a[%*\d]: Entering directory `%f',%X%*\a[%*\d]: Leaving directory `%f',%D%*\a: Entering directory `%f
',%X%*\a: Leaving directory `%f',%DMaking %*\a in %f,%f|%l| %m
fileencodings=ucs-bom,utf-8,default,latin1
fileformats=unix,dos
fillchars=vert:|,fold:-
foldmarker={{{,}}}
foldopen=block,hor,mark,percent,quickfix,search,tag,undo
foldtext=foldtext()
formatlistpat=^\s*\d\+[\]:.)}\t ]\s*
fuoptions=maxvert,maxhorz
grepformat=%f:%l:%m,%f:%l%m,%f %l%m
grepprg=grep -n $* /dev/null
guicursor=n-v-c:block-Cursor/lCursor,ve:ver35-Cursor,o:hor50-Cursor,i-ci:ver25-Cursor/lCursor,r-cr:hor20-Cursor/lCursor,sm:block-Cursor-blinkwait175-blinkoff150-blinkon175
guioptions=egmrLtT
helpfile=/Applications/MacVim.app/Contents/Resources/vim/runtime/doc/help.txt
highlight=8:SpecialKey,#:NonText,d:Directory,e:ErrorMsg,i:IncSearch,l:Search,m:MoreMsg,M:ModeMsg,n:LineNr,r:Question,s:StatusLine,S:StatusLineNC,c:VertSplit,t:Title,v:Visual,V:VisualNOS,w:WarningMsg,W:WildMenu,f:Folded,F:FoldColumn,A:DiffAdd,C:DiffChange,D:DiffDelete
,T:DiffText,>:SignColumn,-:Conceal,B:SpellBad,P:SpellCap,R:SpellRare,L:SpellLocal,+:Pmenu,=:PmenuSel,x:PmenuSbar,X:PmenuThumb,*:TabLine,#:TabLineSel,_:TabLineFill,!:CursorColumn,.:CursorLine,o:ColorColumn
include=^\s*#\s*include
indentkeys=0{,0},:,0#,!^F,o,O,e
isfname=#,48-57,/,.,-,_,+,,,#,$,%,~,=
isident=#,48-57,_,192-255
iskeyword=#,48-57,_,192-255
lispwords=defun,define,defmacro,set!,lambda,if,case,let,flet,let*,letrec,do,do*,define-syntax,let-syntax,letrec-syntax,destructuring-bind,defpackage,defparameter,defstruct,deftype,defvar,do-all-symbols,do-external-symbols,do-symbols,dolist,dotimes,ecase,etypecase,eva
l-when,labels,macrolet,multiple-value-bind,multiple-value-call,multiple-value-prog1,multiple-value-setq,prog1,progv,typecase,unless,unwind-protect,when,with-input-from-string,with-open-file,with-open-stream,with-output-to-string,with-package-iterator,define-condition,h
andler-bind,handler-case,restart-bind,restart-case,with-simple-restart,store-value,use-value,muffle-warning,abort,continue,with-slots,with-slots*,with-accessors,with-accessors*,defclass,defmethod,print-unreadable-object
matchpairs=(:),{:},[:]
maxmempattern=1000
mkspellmem=460000,2000,500
mousemodel=popup_setpos
mouseshape=i-r:beam,s:updown,sd:udsizing,vs:leftright,vd:lrsizing,m:no,ml:up-arrow,v:rightup-arrow
nrformats=octal,hex
paragraphs=IPLPPPQPP TPHPLIPpLpItpplpipbp
path=.,/usr/include,,
printexpr=system('open -a Preview '.v:fname_in) + v:shell_error
printheader=%<%f%h%m%=Page %N
rightleftcmd=search
runtimepath=~/.vim,/Applications/MacVim.app/Contents/Resources/vim/vimfiles,/Applications/MacVim.app/Contents/Resources/vim/runtime,/Applications/MacVim.app/Contents/Resources/vim/vimfiles/after,~/.vim/after
scrollopt=ver,jump
sections=SHNHH HUnhsh
selection=inclusive
sessionoptions=blank,buffers,curdir,folds,help,options,tabpages,winsize
shellpipe=2>&1| tee
shellredir=>%s 2>&1
shortmess=filnxtToO
spellcapcheck=[.?!]\_[\])'"^I ]\+
statusline=%F%m%r%h%w[%L][%{&ff}]%y[%p%%][%04l,%04v]
suffixes=.bak,~,.o,.h,.info,.swp,.obj
termencoding=utf-8
titleold=Thanks for flying Vim
toolbar=icons,tooltips
toolbariconsize=small
ttytype=builtin_gui
viewdir=~/.vim/view
viewoptions=folds,options,cursor
viminfo='100,<50,s10,h
That should do the work:
%s/\v(\w+).*/'\1': \1,
I think your query doesn't work because the + symbols needs the slash \ before it.
In my query I used the \v 'very magic' option that allows me to skip some slashes.
see :h /magic
:s/^\([^ ]*\) .*$/'\1': \1,/
The match part starts at the beginning of the line and captures a sequence of non-blanks, followed by an uncaptured blank and anything else. The replace part starts with a quote, what you remembered, a quote, colon, blank, what you remembered again, and a comma.
Divergent behaviours in vim
Weird, backslashing ( and ) doesn't even match anything in my VIM, which is MacVim snapshot 64 (although it still says it's 7.3.390 by Bram Moolenar). Using the \v in #Tassos's answer seems to work.
On Mac OS X 10.7.4, I'm using the vim from /usr/bin which identifies itself as:
VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Jun 24 2011 20:00:09)
Compiled by root#apple.com
Normal version without GUI. Features included (+) or not (-):
-arabic +autocmd -balloon_eval -browse +builtin_terms +byte_offset +cindent
-clientserver -clipboard +cmdline_compl +cmdline_hist +cmdline_info +comments
-conceal +cryptv +cscope +cursorbind +cursorshape +dialog_con +diff +digraphs
-dnd -ebcdic -emacs_tags +eval +ex_extra +extra_search -farsi +file_in_path
+find_in_path +float +folding -footer +fork() -gettext -hangul_input +iconv
+insert_expand +jumplist -keymap -langmap +libcall +linebreak +lispindent
+listcmds +localmap -lua +menu +mksession +modify_fname +mouse -mouseshape
-mouse_dec -mouse_gpm -mouse_jsbterm -mouse_netterm -mouse_sysmouse
+mouse_xterm +multi_byte +multi_lang -mzscheme +netbeans_intg -osfiletype
+path_extra -perl +persistent_undo +postscript +printer -profile -python
-python3 +quickfix +reltime -rightleft -ruby +scrollbind +signs +smartindent
-sniff +startuptime +statusline -sun_workshop +syntax +tag_binary
+tag_old_static -tag_any_white -tcl +terminfo +termresponse +textobjects +title
-toolbar +user_commands +vertsplit +virtualedit +visual +visualextra +viminfo
+vreplace +wildignore +wildmenu +windows +writebackup -X11 -xfontset -xim -xsmp
-xterm_clipboard -xterm_save
system vimrc file: "$VIM/vimrc"
user vimrc file: "$HOME/.vimrc"
user exrc file: "$HOME/.exrc"
fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -D_FORTIFY_SOURCE=0 -Iproto -DHAVE_CONFIG_H -arch i386 -arch x86_64 -g -Os -pipe
Linking: gcc -arch i386 -arch x86_64 -o vim -lncurses
I also use vim on Linux machines, and it behaves the same w.r.t regular expressions. One of those versions identifies itself, in part, as:
VIM - Vi IMproved 7.0 (2006 May 7, compiled Nov 5 2008 11:07:46)
Included patches: 1, 3-4, 7-9, 11, 13-17, 19-26, 29-31, 34-44, 47, 50-56, 58-64, 66-73, 75, 77-92, 94-107, 109, 202, 234-235, 237
Modified by <bugzilla#redhat.com>
Compiled by <bugzilla#redhat.com>
Huge version without GUI.
I'm not sure if there's anything worth showing from :set all. This is the output from the Mac version of Vim:
ambiwidth=single foldclose= omnifunc= syntax=
noautochdir foldcolumn=0 operatorfunc= tabline=
autoindent foldenable nopaste tabpagemax=10
autoprint foldexpr=0 pastetoggle= tabstop=4
noautoread foldignore=# patchexpr= tagbsearch
noautowrite foldlevel=0 patchmode= taglength=0
noautowriteall foldlevelstart=-1 nopreserveindent tagrelative
background=light foldmethod=manual previewheight=12 tags=./tags,tags
backspace=2 foldminlines=1 nopreviewwindow tagstack
nobackup foldnestmax=20 printdevice= term=xterm-color
backupcopy=auto formatexpr= printencoding= termencoding=
backupext=~ formatoptions=tcq printfont=courier noterse
nobeautify formatprg= printmbcharset= textauto
nobinary fsync printmbfont= notextmode
nobomb nogdefault printoptions= textwidth=0
bufhidden= helpheight=20 prompt thesaurus=
buflisted helplang=en pumheight=0 notildeop
buftype= nohidden quoteescape=\ timeout
cedit=^F history=20 noreadonly timeoutlen=1000
charconvert= nohlsearch redrawtime=2000 notitle
nocindent noicon norelativenumber titlelen=85
cinoptions= iconstring= remap titlestring=
cmdheight=1 noignorecase report=2 nottimeout
cmdwinheight=7 iminsert=0 noruler ttimeoutlen=-1
colorcolumn= imsearch=0 rulerformat= ttybuiltin
columns=80 includeexpr= scroll=32 ttyfast
nocompatible noincsearch noscrollbind ttymouse=xterm
completefunc= indentexpr= scrolljump=1 ttyscroll=999
noconfirm noinfercase scrolloff=0 undodir=.
nocopyindent noinsertmode nosecure noundofile
cpoptions=aABceFs isprint=#,161-255 selectmode= undolevels=1000
cryptmethod=zip joinspaces shell=/bin/sh undoreload=10000
cscopepathcomp=0 key= shellcmdflag=-c updatecount=200
cscopeprg=cscope keymodel= shellquote= updatetime=4000
cscopequickfix= keywordprg=man -s shelltemp verbose=0
nocscopetag langmenu= shellxquote= verbosefile=
cscopetagorder=0 laststatus=1 noshiftround virtualedit=
nocscopeverbose nolazyredraw shiftwidth=4 novisualbell
nocursorbind nolinebreak noshortname warn
nocursorcolumn lines=66 showbreak= noweirdinvert
nocursorline nolisp noshowcmd whichwrap=b,s
debug= nolist noshowfulltag wildchar=<Tab>
nodelcombine listchars=eol:$ showmatch wildcharm=0
dictionary= loadplugins showmode wildignore=
nodiff magic showtabline=1 nowildmenu
diffexpr= makeef= sidescroll=0 wildmode=full
diffopt=filler makeprg=make sidescrolloff=0 wildoptions=
nodigraph matchtime=5 nosmartcase window=0
directory=/tmp maxcombine=2 nosmartindent winheight=1
display= maxfuncdepth=100 nosmarttab nowinfixheight
eadirection=both maxmapdepth=1000 softtabstop=0 nowinfixwidth
noedcompatible maxmem=450298 nospell winminheight=1
encoding=utf-8 maxmemtot=450298 spellfile= winminwidth=1
endofline menuitems=25 spelllang=en winwidth=20
equalalways modeline spellsuggest=best wrap
equalprg= modelines=0 nosplitbelow wrapmargin=0
noerrorbells modifiable nosplitright wrapscan
esckeys nomodified startofline write
eventignore= more statusline= nowriteany
expandtab mouse= suffixesadd= writebackup
noexrc mousemodel=extend swapfile writedelay=0
fileencoding= mousetime=500 swapsync=fsync
fileformat=unix number switchbuf=
filetype= numberwidth=4 synmaxcol=3000
The behaviour you're describing sounds as if there are PCRE (Perl-Compatible Regular Expressions) in use. Maybe the -perl in the configuration is relevant there. I don't see anything in the settings shown that could alter the regex patterns. Maybe you can run your MacVim and show the output of macvim --version and then the output from : set all. Oh, drat, you don't have enough privileges to edit them into this message. I found an 80-column window sensible for the :set all information and pasting to SO; I normally use 120 column windows.
Please find my email in my profile and send me the data. I am curious to know what's different about the MacVim settings (if only so I can set them back to 'normal' if I ever get around to using it).
From vimregex.com:
You can group parts of the pattern expression enclosing them with "("
and ")" and refer to them inside the replacement pattern by their
special number \1, \2 ... \9. Typical example is swapping first two
words of the line:
s:(\w+)(\s+)(\w+):\3\2\1:
where \1 holds the first word, \2 -
any number of spaces or tabs in between and \3 - the second word. How
to decide what number holds what pair of () ? - count opening "("
from the left.