I want to get all the changed file in a changeset but without having the letter that specify the change type.
I know this command
hg status --change changeset
However, this return result like,
A /path/to/file
M /path/to/file
I want the result to be
/path/to/file
without "A" or "M". I tried to do it with regular expression and it used to work but I don't know it didn't work in one scenario.
The regular expression,
/(?<=[MA]\s).+/
This the scenario that did not work
arwa/Documents/Projects/firefox/M db << notice M here.
Any thought?
You should add ^ (start of line) to your regex:
/(?<=^[MA]\s).+/
If you don't do that, (?<=[MA]\s) will lookabehind the M part and .+ will catch db.
As of Mercurial 3.5, you can use templates (still an experimental feature, you can see it with hg help status -v). Namely:
hg status --change <rev> --template '{path}\n'
Why not use log with template?
hg log -r CSET -T "{files % '{file}\n'}"
will produce clean output
lang/UTF-8/serendipity_lang_ru.inc.php
plugins/serendipity_event_bbcode/UTF-8/lang_ru.inc.php
plugins/serendipity_event_emoticate/UTF-8/lang_ru.inc.php
plugins/serendipity_event_karma/UTF-8/lang_ru.inc.php
plugins/serendipity_event_s9ymarkup/UTF-8/lang_ru.inc.php
plugins/serendipity_plugin_shoutbox/UTF-8/lang_ru.inc.php
contrary to templated status
'lang\UTF-8\serendipity_lang_ru.inc.php
''plugins\serendipity_event_bbcode\UTF-8\lang_ru.inc.php
''plugins\serendipity_event_emoticate\UTF-8\lang_ru.inc.php
''plugins\serendipity_event_karma\UTF-8\lang_ru.inc.php
''plugins\serendipity_event_s9ymarkup\UTF-8\lang_ru.inc.php
''plugins\serendipity_plugin_shoutbox\UTF-8\lang_ru.inc.php
'
Related
I have a very large log file (more than 2GB) and would like to remove all debug logs if they contain 'EntityFramework'.
info: Microsoft.EntityFrameworkCore.Migrations[20405]
No migrations were applied. The database is already up to date.
dbug: Microsoft.EntityFrameworkCore.Infrastructure[10407]
'IDCDbContext' disposed.
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.
info: Hangfire.PostgreSql.PostgreSqlStorage[0]
Start installing Hangfire SQL objects...
Here I would like to remove the log below and keep the others
dbug: Microsoft.EntityFrameworkCore.Infrastructure[10407]
'IDCDbContext' disposed.
What I've tried so far:
sed -i '/^dbug/{:b,N;/^[[:lower:]]/!bb};/.*EntityFramework.*/d' logs
However it results in sed: can't find label for jump to b'
Any idea?
This job is better suited for awk:
awk '!p || !/^[[:blank:]]/ {p = /^dbug:/} !p' file
info: Microsoft.EntityFrameworkCore.Migrations[20405]
No migrations were applied. The database is already up to date.
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.
info: Hangfire.PostgreSql.PostgreSqlStorage[0]
Start installing Hangfire SQL objects...
We keep a flag p to control whether to print or not. p is set to 1 when a line starts with /dbug:/ and remains set for lines following dbug: that start with whitespaces.
The current error is due to the comma after the b label, there must be a semi-colon. Also, you should include /.*EntityFramework.*/d (or better, /EntityFramework/d) into the command block so that it is executed only inside it:
sed -i '/^dbug/{:b;N;/^[[:lower:]]/!bb;/EntityFramework/d}' logs
See the online demo.
1st solution: With your shown samples, please try following awk program, written and tested in GNU awk. Simple explanation would be, using awk's match function to match regex \ndbug:[^\n]*\n[^\n]* and printing only those lines which are required by OP's shown output(not matched lines only).
awk -v RS= 'match($0,/\ndbug:[^\n]*\n[^\n]*/){print substr($0,1,RSTART-1) substr($0,RSTART+RLENGTH)}' Input_file
2nd solution: Using record separator capability of awk and print the appropriate values needed by OP.
awk -v RS='\ndbug:[^\n]*\n[^\n]*\n' '{gsub(/\n+$/,"")}1' Input_file
I am looking to replace full stops in a filename however I need to remove some and replace others.
The file names are structured like so:
A.A M12345678 SOMEWORD 20.08.2019.pdf
A.A M12345678 SOMEWORD1 SOMEWORD2 20.08.2019.pdf
I want the format to be the following:
AA M12345678 SOMEWORD 20-08-2019.pdf
AA M12345678 SOMEWORD1 SOMEWORD2 20-08-2019.pdf
So the first full stop should be removed but the full stop encountered in the date should be a hyphen (-).
I have been using command prompt but I am running into some issues as I am fairly new to regular expressions.
I have tried approaching the problem one step at a time namely by just focusing on replacing the date format.
I've tested my regex using https://regexr.com/ and it matches correctly.
[0-9]\K[.]
My understanding of the code above should match the full stops in the date.
However when I run the following command:
ren *[0-9]\K[.].pdf -
It fails to find the file.
Expected Result
AA M12345678 SOMEWORD 20-08-2019.pdf
Actual Result
The expression I use just returns this error when I use the REN command.
"The filename, directory name, or volume label syntax is incorrect."
Pretty sure you could do this in the command-line with sed as follows. Run the following command:
sed -i "s/(\d\d)\.(\d\d)\.(\d\d\d\d)/\1-\2-\3 ; s/^(.)\.(.*)/\1\2/" <YOUR_FILE>
The -i flag is for modification in place. Here, I'm running two separate search-and-replace functions on each line; the first is to match to a date, and then format it accordingly, and the second is to match your A.A metadata and format it accordigly.
More about capture groups here.
I have a few Regex expressions that I use with xVim for Xcode. Rather than repeatedly typing them out in the command bar with \<Regex>, I'd like to be able to invoke them with a custom command, like :Regex1. So I've added command Regex1 “/-\s*\(“ to my xvimrc file and restarted Xcode. When I run :Regex1 however nothing happens.
Your command wouldn't even work in original Vim. I don't know xVim, but try something along these lines:
" With cursor moving to match.
command Foo /foo/
" Just updating the search pattern (but less likely to be portable to xVim).
command Foo let #/ = 'foo'
If none of that works; try defining a mapping instead. As this is just translating keys, it has the highest chance of being supported.
I would suggest using this PERL Regex plugin since it already does what you want.
https://github.com/othree/eregex.vim
Abbreviations ...
I understand you often use the same regex. You can use abreviations instead of a command to do a search.
ab re -\s*(
then type / + re + space and your long regex (here just "-\s*(" should expand).
... Not user defined command
User defined commands are not available in ed nor in vi nor in vim without the +eval compilation flag (:h user-commands and scroll one line up).
For a list of ex commands: http://www.csb.yale.edu/userguides/wordprocess/vi-summary.html
For a list of ed commands: http://pubs.opengroup.org/onlinepubs/7908799/xcu/ed.html
I would like to be able to perform regular expression-type searches on Mercurial changesets and display results using log.
I've come up with the following function, which seems to work, but has a number of possible bugs (e.g. $1 is in line of text containing the word changeset).
function hgs { hg log `hg log | grep changeset | grep "$1" \
| sed 's/changeset: *//g' | sed 's/:.*$//g' | \
awk '{print " -r " $0}'`; }
export -f hgs
Am I trying to recreate something here that already exists as a well-tested solution elsewhere?
It pretty much looks like a combination of using hg grep, making use revsets and templated output could possibly help you (check hg help revsets, hg help templates, hg help grep and possibly also hg help fileset).
E.g. to find all changes to config.lib or where the commit message contains 'pkgconfig' which were made after 2010:
hg log -r"(file('config.lib') or desc('pkgconfig')) and date('>2010')"
revsets are very powerful. You can also sort, limit to a certain number of changesets, combine different requirements...
Using the --template argument to hg log can be used to format the output in any pattern you desire.
I'm trying to remove sensitive data like passwords from my Git history. Instead of deleting whole files I just want to substitute the passwords with removedSensitiveInfo. This is what I came up with after browsing through numerous StackOverflow topics and other sites.
git filter-branch --tree-filter "find . -type f -exec sed -Ei '' -e 's/(aSecretPassword1|aSecretPassword2|aSecretPassword3)/removedSensitiveInfo/g' {} \;"
When I run this command it seems to be rewriting the history (it shows the commits it's rewriting and takes a few minutes). However, when I check to see if all sensitive data has indeed been removed it turns out it's still there.
For reference this is how I do the check
git grep aSecretPassword1 $(git rev-list --all)
Which shows me all the hundreds of commits that match the search query. Nothing has been substituted.
Any idea what's going on here?
I double checked the regular expression I'm using which seems to be correct. I'm not sure what else to check for or how to properly debug this as my Git knowledge quite rudimentary. For example I don't know how to test whether 1) my regular expression isn't matching anything, 2) sed isn't being run on all files, 3) the file changes are not being saved, or 4) something else.
Any help is very much appreciated.
P.S.
I'm aware of several StackOverflow threads about this topic. However, I couldn't find one that is about substituting words (rather than deleting files) in all (ASCII) files (rather than specifying a specific file or file type). Not sure whether that should make a difference, but all suggested solutions haven't worked for me.
git-filter-branch is a powerful but difficult to use tool - there are several obscure things you need to know to use it correctly for your task, and each one is a possible cause for the problems you're seeing. So rather than immediately trying to debug them, let's take a step back and look at the original problem:
Substitute given strings (ie passwords) within all text files (without specifying a specific file/file-type)
Ensure that the updated Git history does not contain the old password text
Do the above as simply as possible
There is a tailor-made solution to this problem:
Use The BFG... not git-filter-branch
The BFG Repo-Cleaner is a simpler alternative to git-filter-branch specifically designed for removing passwords and other unwanted data from Git repository history.
Ways in which the BFG helps you in this situation:
The BFG is 10-720x faster
It automatically runs on all tags and references, unlike git-filter-branch - which only does that if you add the extraordinary --tag-name-filter cat -- --all command-line option (Note that the example command you gave in the Question DOES NOT have this, a possible cause of your problems)
The BFG doesn't generate any refs/original/ refs - so no need for you to perform an extra step to remove them
You can express you passwords as simple literal strings, without having to worry about getting regex-escaping right. The BFG can handle regex too, if you really need it.
Using the BFG
Carefully follow the usage steps - the core bit is just this command:
$ java -jar bfg.jar --replace-text replacements.txt my-repo.git
The replacements.txt file should contain all the substitutions you want to do, in a format like this (one entry per line - note the comments shouldn't be included):
PASSWORD1 # Replace literal string 'PASSWORD1' with '***REMOVED***' (default)
PASSWORD2==>examplePass # replace with 'examplePass' instead
PASSWORD3==> # replace with the empty string
regex:password=\w+==>password= # Replace, using a regex
Your entire repository history will be scanned, and all text files (under 1MB in size) will have the substitutions performed: any matching string (that isn't in your latest commit) will be replaced.
Full disclosure: I'm the author of the BFG Repo-Cleaner.
Looks OK. Remember that filter-branch retains the original commits under refs/original/, e.g.:
$ git commit -m 'add secret password, oops!'
[master edaf467] add secret password, oops!
1 file changed, 4 insertions(+)
create mode 100644 secret
$ git filter-branch --tree-filter "find . -type f -exec sed -Ei '' -e 's/(aSecretPassword1|aSecretPassword2|aSecretPassword3)/removedSensitiveInfo/g' {} \;"
Rewrite edaf467960ade97ea03162ec89f11cae7c256e3d (2/2)
Ref 'refs/heads/master' was rewritten
Then:
$ git grep aSecretPassword `git rev-list --all`
edaf467960ade97ea03162ec89f11cae7c256e3d:secret:aSecretPassword2
but:
$ git lola
* e530e69 (HEAD, master) add secret password, oops!
| * edaf467 (refs/original/refs/heads/master) add secret password, oops!
|/
* 7624023 Initial
(git lola is my alias for git log --graph --oneline --decorate --all). Yes, it's in there, but under the refs/original name space. Clear that out:
$ rm -rf .git/refs/original
$ git reflog expire --expire=now --all
$ git gc
Counting objects: 6, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (6/6), done.
Total 6 (delta 0), reused 0 (delta 0)
and then:
$ git grep aSecretPassword `git rev-list --all`
$
(as always, run filter-branch on a copy of the repo Just In Case; and then removing original refs, expiring the reflog "now", and gc'ing, means stuff is Really Gone).