Remove\Change specific attributes in LDIF using vscode - regex

I am exporting an LDAP database into an LDIF, and I need to clean it up a bit. There are some attributes that I no longer want. If it is a one-line attribute it is simple:
# Entry 23: cn=dwhite,ou=ORG1,o=Root
dn: cn=dwhite,ou=ORG1,o=Root
objectclass: person
objectclass: inetorgperson
objectclass: webadmaccount
sn: white
cn: dwhite
description: don.white#company.com
givenname: Don
mail: 5551234567#txt.att.net
o: ORG1
uid: dwhite
userpassword: {SSHA}YrGx7L7L8xxxiMzuUzL+f0j+i73uFTYEPnu
webadmdata: OpenOTP.RejectCount={wcrypt}eAw+r6VX6l8lioB+1KX/6A==,OpenOTP.Las
tLogin={wcrypt}hQCwWl0KRdsiXyG7UPxPxo/WphEkyD6NPTKor8FCA5E=,OpenOTP.LoginCot
0f5vQVl1ippSdKFiGeACiJbFWHsfs=,DataMode=A8tMd3g=
# Entry 24: ou=ORG2,o=Root
dn: ou=ORG2,o=Root
objectclass: organizationalunit
ou: ORG2
# Entry 25: cn=jlee,ou=ORG2,o=Root
dn: cn=jlee,ou=ORG2,o=Root
objectclass: person
objectclass: inetorgperson
objectclass: webadmaccount
sn: Lee
cn: jlee
userpassword: {SSHA}+CreNAoXMO3XMIWxxaWsA2r3ozyeCBEShn
description: jay.lee#company.com
givenname: Jay
mail: 5551234567#txt.att.net
o: ORG2
uid: jlee
webadmdata: OpenOTP.RejectCount={wcrypt}OrtNikuuHCyBruxUTOq6eg==,OpenOTP.Las
tLogin={wcrypt}WUa965R8rFV9iHx7LrXrfzxEE3Gjd7TGiFz8AEJBmwU=,OpenOTP.LoginCo
unt={wcrypt}Qbamc7jYTL+14K1yaZS9kA==,OpenOTP.LastOTP={wcrypt}t+fDlGfP6kGIse
at8PHJdxJy15LSN4R+tJXr7B7L7+4=,DataMode=A8tMd3g=
In the sample above to remove the 'objectclass: webadmaccount' attribute, a Find and Replace with a regex like: ^objectclass.*webadmaccount$ will do the job. I know that seems a little overkill with the anchors, but elsewhere in the LDIF, you can find that sting embedded in a longer line, so the anchors.
Now, my problem comes when I want to get rid of the 'webadmdata:' attributes. You can see they are multiline, and not all the same length. This is where I am stuck, a regex like this:
^webadmdata:.*[\n\s]+.* will get the first two lines, but then it quits. If I make the .*[\n\s]+.* a capture group and repeat it with '+' at the end: (.*[\n\s]+.*)+ it will select the whole file (or crash vscode ). It seems like I should be able to write a regex that will select everything from ^webadmdata to A8tMd3g=\n\n but I cannot for the life of me get it to work. I hope there is a regex or VSCode guru out there that can help. Thanks in advance.

OK, I don't know why I thought this didn't work before, but it does. The following regex will do what I want:
^webadmdata:.(.|\n)+?A8tMd3G=$\n
Apparently VSCode defaults to lazy qualifier. See below:
screenshot

Related

Regex Derived Fields

I'm trying to do Regex for the below log line and put into the Derived fields in Grafana Loki data source.
2022-03-03 11:59:18,008 ERROR XXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXX XXXXXXXXXXXXXXXXXX
I cover like this:
Date - ([0-9]{4}-[0-9]{2}-[0-9]{2})
Time - ([0-9]{2}:[0-9]{2}:[0-9]{2}?,[0-9]{3})
LogLevel - ((TRACE|DEBUG|INFO|NOTICE|WARN|WARNING|ERROR|SEVERE|FATAL)`
And rest after the ERROR should be a message but i can not manage to do this.
I tried most of the negation regexes but did not get final result to leave the right side after the ERROR as Message variable.
I expect that Message will contain everything after the ERROR so XXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXX XXXXXXXXXXXXXXXXXX

regex help to parse data - how to get all the text in between two points

I've seen so many of these pages and received so much help over the years.
First time askign a question.
Basically, I use automation tools such as Integromat/Make/Zapier.
I have a csv file with a lot of data for a self-made booking system.
For the automation system to work, I will need to parse the data into "chunks" that I can map and add the proper functionality to.
The text looks as follows (but much longer of course):
BEGIN:VEVENT
DTSTAMP:20220301T023567Z
UID: xxx
DTSTART;TZID=Asia/Tokyo:20210630T110000
DTEND;TZID=Asia/Tokyo:20210630T160000
DESCRIPTION:\n--------\n同日入替 : false\n前回ゲスト数 : 1\n次
SUMMARY: xxxxx (xxx / xxx xxx
END:VEVENT
BEGIN:VEVENT
DTSTAMP:20221105T072143Z
UID: xxx
DTSTART;TZID=Asia/Tokyo:20210501T110000
DTEND;TZID=Asia/Tokyo:20210701T160000
DESCRIPTION:\n--------\n同日入替 : false\n前回ゲスト数 : 1\n次
SUMMARY: xxx (xxx / xxx xxx)
END:VEVENT
BEGIN:VEVENT
DTSTAMP:20220905T023143Z
UID: xxx
DTSTART;TZID=Asia/Tokyo:20220227T110000
DTEND;TZID=Asia/Tokyo:20220227T160000
DESCRIPTION:\n--------\n同日入替 : true\n前回ゲスト数 : 2\n次回
SUMMARY:★ xxx (xxx / xxx xxx)
END:VEVENT
I need the parts that need to be together as parts as:
BEGIN:VEVENT
DTSTAMP:20220301T023567Z
UID: xxx
DTSTART;TZID=Asia/Tokyo:20210630T110000
DTEND;TZID=Asia/Tokyo:20210630T160000
DESCRIPTION:\n--------\n同日入替 : false\n前回ゲスト数 : 1\n次
SUMMARY: xxxxx (xxx / xxx xxx
END:VEVENT
BEGIN:VEVENT
DTSTAMP:20221105T072143Z
UID: xxx
DTSTART;TZID=Asia/Tokyo:20210501T110000
DTEND;TZID=Asia/Tokyo:20210701T160000
DESCRIPTION:\n--------\n同日入替 : false\n前回ゲスト数 : 1\n次
SUMMARY: xxx (xxx / xxx xxx)
END:VEVENT
BEGIN:VEVENT
DTSTAMP:20220905T023143Z
UID: xxx
DTSTART;TZID=Asia/Tokyo:20220227T110000
DTEND;TZID=Asia/Tokyo:20220227T160000
DESCRIPTION:\n--------\n同日入替 : true\n前回ゲスト数 : 2\n次回
SUMMARY:★ xxx (xxx / xxx xxx)
END:VEVENT
Meaning that everything in between two specific points, starting with "BEGIN:VEVENT" and ending with "END:VEVENT" needs to be separated.
I have been trying to learn regex but as I can see many others are saying online, its not the most welcoming thing ever .. Im struggling hard.
You may try matching on the following pattern:
BEGIN:VEVENT.*?END:VEVENT
Demo
The above regex should be run in dot all mode (with the /s flag) so that .* can match across all lines in each event.

Extract a motif in various url strings with regex in ruby

I have different type of strings (in fact logs):
2022-08-03T16:20:41 - INFO - server.py - 649 - 192.168.1.24,192.168.1.29 - - [03/Aug/2022 16:20:41] "GET /get_customer_by_id/0024-A HTTP/1.0" 200 554 0.007798
2022-08-03T16:20:56 - INFO - utils.py - 10 - GET - http://192.168.1.24/get_customer_by_id/0025-A
2022-08-03T16:21:13 - INFO - utils.py - 10 - POST - http://192.168.1.24/order
I want to extract the customer id in each get_customer_by_id url. So for the previous example, i'm looking for 0024-A and 0025-A
I tried with a regex \/get_result\/(.+) but it gives me all the end of line when there is something after the customer id.
You can have a detail of implementation here: https://rubular.com/r/FgBxR1kUyQAYSl
How can i solve this ?
Thanks a lot for your help !
I suppose you'd be looking for something like /\/get_customer_by_id\/(\S+)/. This will grab all non-whitespace characters (stopping before the HTTP/1.0 on the first line). If you know it's always dddd-s, then you could also use something like /\/get_customer_by_id\/(\d+-\w)/. Either way, it will be in the first capture group (link to info on ruby capture groups).

How can I make multiple replacements on the same file using one saltstack state?

Here's my target file:
Sonatype Nexus
# ==============
# This is the most basic configuration of Nexus.
# Jetty section
application-port=8081
application-host=0.0.0.0
nexus-webapp=${bundleBasedir}/nexus
nexus-webapp-context-path=/nexus
# Nexus section
nexus-work=/opt/nexuswork
runtime=${bundleBasedir}/nexus/WEB-INF
I know there's an easy way to do this with regex or a simple sed script:
sed -i 's/${bundleBasedir}\/..\/my\/second\/path\/002\/\/nexus/\/myfirstdir001\/g'
However, I would, ideally, prefer the saltstack way.
I would like it to look something like this:
Sonatype Nexus
# ==============
# This is the most basic configuration of Nexus.
# Jetty section
application-port=8081
application-host=0.0.0.0
nexus-webapp=/my/second/path/002/nexus # changed
nexus-webapp-context-path=/nexus
# Nexus section
nexus-work=/opt/nexuswork
runtime=/myfirstdir001/nexus/WEB-INF # changed
I haven't yet made sense of the saltstack documentation on this.
Saltstack's documentation for salt.states.file.replace seems fairly straightforward:
http://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html#salt.states.file.replace
Here's what I tried:
/opt/nexus-2.8.0/conf/nexus.properties
file: # state
- replace
- pattern: '\$\{bundleBasedir\}' # without escapes: '${bundleBasedir}/nexus'
- repl: '/my/second/path/002/nexus'
# - name: /opt/nexus-2.8.0/conf/nexus.properties
# - count=0
# - append_if_not_found=False
# - prepend_if_not_found=False
# - not_found_content=None
# - backup='.bak'
# - show_changes=True
- pattern: '\$\{bundleBasedir\}\/WEB-INF' # without escapes: ${bundleBasedir}/WEB-INF
- repl: '/myfirstdir001/'
I could maybe try multiple state IDs, but that seems inelegant.
If there's anything else I'm fuffing up, please advise!
I'd shore love to find a solution to this.
Also, if there's any demand for people improving the salt documentation, I think my team could be convinced to pitch in some.
Here's the closest thing I've found to someone else asking this question:
http://comments.gmane.org/gmane.comp.sysutils.salt.user/15138
For such a small file I would probably go with a template as ahus1 suggested.
If the file was bigger and/or we didn't want to control other lines just ensure that those two are correct, I think multiple state IDs (as mentioned by OP) is a good way to go. Something like:
/opt/nexus-2.8.0/conf/nexus.properties-jetty:
file:
- replace
- name: /opt/nexus-2.8.0/conf/nexus.properties
- pattern: '\$\{bundleBasedir\}' # without escapes: '${bundleBasedir}/nexus'
- repl: '/my/second/path/002/nexus'
/opt/nexus-2.8.0/conf/nexus.properties-nexus:
file:
- replace:
- name: /opt/nexus-2.8.0/conf/nexus.properties
- pattern: '\$\{bundleBasedir\}\/WEB-INF' # without escapes: ${bundleBasedir}/WEB-INF
- repl: '/myfirstdir001/'
I have a similar setup in my configuration but I use salt.states.file.line to replace some lines with my values. In addition I used salt.states.file.managed with a template and replace: False to initialize the file if it's missing but once it exists, only the line states are doing changes.
The salt way to do this as I understand it: Place a template file for nexus.properties inside salt and use file.managed like shown in the docs http://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html
You will end up with something like:
/opt/nexus-2.8.0/conf/nexus.properties:
file.managed:
- source: salt://nexus/nexus.properties.jinja
- template: jinja
- defaults:
bundleBasedir: "..."
You'll then use Jinja templating in your file:
# Jetty section
application-port=8081
application-host=0.0.0.0
nexus-webapp={{ bundleBasedir }}/nexus
nexus-webapp-context-path=/nexus
See here for Jinja templating: http://docs.saltstack.com/en/latest/ref/renderers/all/salt.renderers.jinja.html
I hope it helps.

ApacheDS: Password policy check not getting enforced

I am working on enabling ApacheDS (version 2.0.0-M16) to enforce strong passwords - combination of alphanumeric and special characters. I have created my own instance for testing.
As a start, I switched the password quality check to STRICT (2) and changed the minimum length value to 7:
dn: ads-pwdId=default,ou=passwordPolicies,ads-interceptorId=authenticationInterceptor,ou=interceptors,ads-directoryServiceId=default,ou=config
ads-pwdMinLength: 7
ads-pwdinhistory: 5
ads-pwdid: default
ads-pwdCheckQuality: 2
ads-pwdlockout: TRUE
ads-pwdlockoutduration: 0
ads-pwdvalidator: org.apache.directory.server.core.api.authn.ppolicy.DefaultPasswordValidator
ads-pwdmaxfailure: 5
ads-pwdfailurecountinterval: 30
ads-pwdattribute: userPassword
entryParentId: 4f761b8b-c70c-43a2-bb6f-d26e8c297f84
ads-enabled: TRUE
objectclass: top
objectclass: ads-base
objectclass: ads-passwordPolicy
entryuuid: c83189f9-8ee4-4aec-9180-6e3f6d8297c6
ads-pwdgraceauthnlimit: 5
modifyTimestamp: 20140508182112.262Z
entryCSN: 20140508182112.263000Z#000000#001#000000
ads-pwdexpirewarning: 600
modifiersName: 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system
I then tried to import an ldif entry through the Apache Directory Studio with an obviously invalid password but the validation does not kick in. User is created successfully:
dn: cn=SherlockHolmes,ou=people,o=sevenSeas
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
objectclass: top
cn: Sherlock Holmes
description: Sherlock Holmes
givenname: Sherlock
sn: Holmes
uid: SHolmes
mail: SHolmes#gmail.com
userpassword: watson
What am I doing wrong?
Also, is there any way I can configure the complexity of the password in the form of a regex? I don't see this in the conf file.
Did you restart the server after updating the password policy?. Server must be restarted to enforce the updated policy.