Jenkins send email with part of log line - regex

I have SOAPUI project with 10 test cases in Jenkins. I set up Jenkins to send me email with information from console output (log). I have set up email notification content as HTML (text/html).
I have this log in console output in Jenkins:
07:25:05,957 INFO [SoapUITestCaseRunner] Running SoapUI testcase [Login with username and password]
07:25:05,957 INFO [SoapUITestCaseRunner] running step [Clear access token]
07:25:05,957 INFO [log] Environment URL: url.test.environment
07:25:05,958 INFO [SoapUITestCaseRunner] running step [Retrieve accessToken]
07:25:05,959 DEBUG [HttpClientSupport$SoapUIHttpClient] Stale connection check
07:25:05,960 DEBUG [HttpClientSupport$SoapUIHttpClient] Attempt 1 to execute request
07:25:05,960 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Sending request: POST /api/v2/path HTTP/1.1
07:25:06,010 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Receiving response: HTTP/1.1 200
07:25:06,011 DEBUG [HttpClientSupport$SoapUIHttpClient] Connection can be kept alive indefinitely
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [JsonPath Existence Match] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [Valid HTTP Status Codes] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [JsonPath Existence Match 1] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [JsonPath Existence Match 2] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] running step [Pass accessToken]
07:25:06,019 INFO [SoapUITestCaseRunner] Finished running SoapUI testcase [Login with username and password], time taken: 51ms, status: FINISHED
I have set up Jenkins to send email just with this line from log:
07:25:06,019 INFO [SoapUITestCaseRunner] Finished running SoapUI testcase [Login with username and password], time taken: 51ms, status: FINISHED
To that I use this regex to find just that line:
<pre>${BUILD_LOG_REGEX, regex="Finished running SoapUI testcase \\[Login with username and password\\]", showTruncatedLines=false}</pre>
But I want to have in email notification just parts of that line, something like this:
"Login with username and password: FINISHED"
or
"Login with username and password: FAILED"
Is there any way to send email with just part of that line?

Thanks to Aaron I found a solution.
Now I use this regex:
<b>Login with username and password: </b> <font color="green">${BUILD_LOG_REGEX, regex=".*Finished running SoapUI testcase \\[Login with username and password\\].*status: FINISHED", showTruncatedLines=false, substText="SUCCESS"}</font>
<font color="red">${BUILD_LOG_REGEX, regex=".*Finished running SoapUI testcase \\[Login with username and password\\].*status: FAILED", showTruncatedLines=false, substText="FAILED"}</font>
E-mail notification now looks like this:
Login with username and password: SUCCESS
or this:
Login with username and password: FAILED
"SUCCESS" text has green color and "FAILED" text has red color

It seems like you can use a substText argument to define a replacement pattern :
<pre>${BUILD_LOG_REGEX, regex="Finished running SoapUI testcase \\[Login with username and password\\].*status: ([A-Z]+)", showTruncatedLines=false, substText="Login with username and password : \\1"}</pre>
This will match only the lines such as the last one from your sample, select the success/failure status in the first capturing group which will be referred in the replacement pattern.

Related

Custom "statusMsg" not working as expected

After adding the following adaptive authentication script, if the username is not according to the format I should be getting the custom status message saying, Access Denied, invalid username format. But instead I get the default status message, Something went wrong during the authentication process. Please try signing in again.
function onLoginRequest(context) {
executeStep(1, {
onSuccess: function(context) {
var user = context.currentKnownSubject;
if(user!= null && user.username != null && !user.username.equals('')) {
Log.info("username: " + user.username);
} else {
sendError('',{'status':'AUTHENTICATION USERNAME ERROR', 'statusMsg': 'Access denied, invalid username format.'});
}
}
});
}
In addition, I get the following error in the wso2carbon.log file as well.
TID: [-1234] [authenticationendpoint] [2022-10-05 15:44:12,715] [37951f7d-8240-48d4-ad4f-1d4c8a6a3ec4] ERROR {org.wso2.carbon.identity.application.authentication.endpoint.util.AuthContextAPIClient} - Sending GET request to URL : https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/0b0efc37-819d-4b39-85b2-517126c3c9cb, failed. java.io.IOException: Server returned HTTP response code: 401 for URL: https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/0b0efc37-819d-4b39-85b2-517126c3c9cb
...
org.wso2.carbon.identity.application.authentication.endpoint.util.AuthContextAPIClient.getContextProperties(AuthContextAPIClient.java:70)
at org.apache.jsp.retry_jsp._jspService(retry_jsp.java:194)
...
org.wso2.carbon.ui.filters.cache.ContentTypeBasedCachePreventionFilter.doFilter(ContentTypeBasedCachePreventionFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
...
org.wso2.carbon.identity.application.authentication.endpoint.util.filter.AuthenticationEndpointFilter.doFilter(AuthenticationEndpointFilter.java:190)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
...
The <IS_HOME>/repository/conf/deployment.toml configurations for [server] are as follows.
[server]
hostname = "dev.wso2istemp.com"
node_ip = "127.0.0.1"
base_path = "https://$ref{server.hostname}:${carbon.management.port}"
What is the reason for the following issue in the wso2carbon.log and why the custom status message is not shown properly?
When the adaptive authentication script is running, the values are passed to the Identity Server(IS) encrypted. In above case, the encrypted data chunk is passed to the IS,
https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/24e56d99-9494-4989-a3e2-4008b73ebd9b
with the last segment of the URL is being the data chunk. When the server tries to get that data chunk with a GET request, java.io.IOException: Server returned HTTP response code: 401 for URL which is the code for unauthorized is thrown. Since that data chunk is not properly received, the default status message is shown instead of the custom status message. The steps to resolve this are given below.
First clarify whether the dev.wso2istemp.com which you were using is not mapped with the localhost in /etc/hosts file.
Go to <IS_HOME>/repository/conf/deployment.toml file and check for the following configuration [identity.auth_framework.endpoint] and check whether the mutual SSL is set to false via mutual_ssl_manager_enabled=false
If so, enable that by commenting the mutual_ssl_manager_enabled=false, since using mutual SSL is recommended for IS. If you go to the <IS_HOME>/repository/resources/conf/default.json file, you can notice that the default value for mutual_ssl_manager_enabled is true
In a multi-node situation, the above error can occur if the mutual SSL is not enabled, If that does not resolves the issue, then you have to check whether the internal_hostname has been set properly so that the internal API calls are being sent properly.
To do that, if you have not added the following configuration to <IS_HOME>/repository/conf/deployment.toml file, you can check whether it has been properly applied to the <IS_HOME>/repository/conf/identity/identity.xml by checking for <ServerHostName>localhost</ServerHostName>
[server]
internal_hostname="localhost"
If you are using a multi-node deployment, this localhost value should be added to the SAN for the certificate when the certificate is generated (-ext SAN=dns:localhost)
keytool -genkey -alias newcert -keyalg RSA -keysize 2048 -keystore newkeystore.jks -dname "CN=dev.wso2istemp.com, OU=Is,O=Wso2,L=SL,S=WS,C=LK" -storepass mypassword -keypass mypassword -ext SAN=dns:localhost
But if you are using a single node you can add the following configuration to the deployment.toml and check whether it resolves things. (In single node case the internal_hostname should be similar to hostname)
[server]
hostname = "dev.wso2istemp.com"
internal_hostname = "dev.wso2istemp.com"
If that is not working[https://github.com/wso2/product-is/issues/11878] then go to <IS_HOME>/repository/deployment/server/webapps/authenticationendpoint/WEB-INF/web.xml and uncomment the following commented snippet.
<!--context-param>
<param-name>AuthenticationRESTEndpointURL</param-name>
<param-value>https://localhost:9443/api/identity/auth/v1.1/</param-value>
</context-param-->
This might route the internal_hostname being reflected on the request since the internal API calls are blocked when hostname of the server being replaced instead of the internal_hostname for the internal API calls(https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/24e56d99-9494-4989-a3e2-4008b73ebd9b).

fluentd regexp to extract events from a log file

I'm new to fluentd.
I have a log that I want to push to AWS with fluentd but I can't figure out what the regexp should be.
All the log lines, except the multilines, start with a UUID.
Here's a sample log:
6b0815f2-8ff1-4181-a4e6-058148288281 2020-11-03 13:00:05.976366 [DEBUG] switch_core_state_machine.c:611 (some_other_data) State Change CS_REPORTING -> CS_DESTROY
And, I'm trying to get UUID, DateTime, and Message.
With this regex:
/^(?<UUID>[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}) (?<time>.*) (?<message>[^ ]*)/gm
I'm getting the last word CS_DESTROY.
I tried fluentular and still got:
text:
f6a6e1ae-e52e-4aba-a8a5-4e3cc7f40914 2020-11-03 14:32:34.975779 [CRIT] mod_dptools.c:1866 audio3: https://mydomain.s3-eu-west-1.amazonaws.com/media/576d06e5-04fc-11eb-a52c-020fd8c14d18/5f9ddf2d5df0f698094395.mpg
regexp:
^(?<UUID>[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}) (?<time>.*) (?<message>[^ ]*)$
and got:
time 2020/11/03 14:32:34 +0000
UUID f6a6e1ae-e52e-4aba-a8a5-4e3cc7f40914
message https://mydomain.s3-eu-west-1.amazonaws.com/media/576d06e5-04fc-11eb-a52c-020fd8c14d18/5f9ddf2d5df0f698094395.mpg
It's missing what's between the datetime and "https".
Try:
^(?<UUID>[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}) (?<time>[^\[]*) (?<message>\[.*)$
Live at rubular: https://rubular.com/r/JQQXs5VTkr2IxM
Here's the output for both logs:
Match 1
UUID 6b0815f2-8ff1-4181-a4e6-058148288281
time 2020-11-03 13:00:05.976366
message [DEBUG] switch_core_state_machine.c:611 (some_other_data) State Change CS_REPORTING -> CS_DESTROY
Match 2
UUID f6a6e1ae-e52e-4aba-a8a5-4e3cc7f40914
time 2020-11-03 14:32:34.975779
message [CRIT] mod_dptools.c:1866 audio3: https://mydomain.s3-eu-west-1.amazonaws.com/media/576d06e5-04fc-11eb-a52c-020fd8c14d18/5f9ddf2d5df0f698094395.mpg

OpenVPN: Authentication Failed?

When I use the plugin for authentication at server.conf, authentication wont work, but without it, non existent users can authenticate also.
I have added the following lines in the server conf and clinet
Commands in the server.conf file
================================
mode server
tls-server
plugin /usr/lib64/openvpn/plugin/lib/openvpn-auth-pam.so login
key-direction 0
================================
Commands in the client file
=================================
port 1194
proto udp
dev tun
nobind
key-direction 1
redirect-gateway def1
tls-version-min 1.2
auth SHA256
auth-user-pass
tls-client
remote-cert-tls server
resolv-retry infinite
persist-key
persist-tun
verb 3
===============================
Logs:
==============================================================
PLUGIN_CALL: POST /usr/lib64/openvpn/plugin/lib/openvpn-auth-pam.so/PLUGIN_AUTH_USER_PASS_VERIFY status=1
PLUGIN_CALL: plugin function PLUGIN_AUTH_USER_PASS_VERIFY failed with status 1: /usr/lib64/openvpn/plugin/lib/openvpn-auth-pam.so
TLS Auth Error: Auth Username/Password verification failed for peer
Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #7 / time = (1559124952) Wed May 29 10:15:52 2019 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
TLS Error: incoming packet authentication failed from [AF_INET6]::ffff:
openvpn[10420]: pam_unix(login:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost= user=*****```
==============================================================
I have used differen approached, although in production plugin /usr/lib64/openvpn/plugin/lib/openvpn-auth-pam.so login is recommended way, but I have taken one shell script and got authentication, but remember it is dangerous.
add following lines in your /etc/openvpn/server.conf file
--verify-cline-cert none
script-security 2
auth-user-pass-verify /etc/openvpn/example.sh via-file
Now create a file in /etc/openvpn/example.sh with following content
!/bin/bash
echo "started"
username=`head -1 $1`
password=`tail -1 $1`
if grep "$username:$password" $0.passwd > /dev/null 2>&1
then
exit 0
else
if grep "$username" $0.passwd > /dev/null 2>&1
then
echo "auth-user-pass-verify: Wrong password entered for user '$username'"
else
echo "auth-user-pass-verify: Unknown user '$username'"
fi
exit 1
fi
Now create username and password in /etc/openvpn/example.sh.passwd with following content
userone:securepassworduserone
usertwo:securepasswordusertwo
Now create a client file and import and connect using your password, but this where I am stack as I don't want to provide client file.

How to setup outgoing mail server through outlook office365 in odoo 9

I will try to setup outgoing mail server in odoo 9 .so i fill all the field and test connection and the connection also success , but at the time of send mail it will generate an error .
Field fill like that:-
Name : sendmail
Priority: 10
SMTP Server : smtp.office365.com
SMTP Port:25
Debugging: enable
Connection Security:TLS (STARTTLS)
Username:my yser name
Password:password
But, when we send any mail then it will generate the below error
16-12-06 10:04:28,440 426 INFO test openerp.addons.base.ir.ir_mail_server: Mail delivery failed via SMTP server 'smtp.office365.com'.
SMTPDataError: 550
5.7.60 SMTP; Client does not have permissions to send as this sender
2016-12-06 10:04:28,443 426 ERROR test openerp.addons.mail.models.mail_mail: failed sending mail (id: 136) due to Mail Delivery Failed
Mail delivery failed via SMTP server 'smtp.office365.com'.
SMTPDataError: 550
5.7.60 SMTP; Client does not have permissions to send as this sender
Traceback (most recent call last):
File "/usr/lib/python2.7/dist- packages/openerp/addons/mail/models/mail_mail.py", line 262, in send
res = IrMailServer.send_email(msg, mail_server_id=mail.mail_server_id.id)
File "/usr/lib/python2.7/dist-packages/openerp/api.py", line 248, in wrapper
return new_api(self, *args, **kwargs)
File "/usr/lib/python2.7/dist-packages/openerp/api.py", line 490, in new_api
result = method(self._model, cr, uid, *args, **old_kwargs)
File "/usr/lib/python2.7/dist-packages/openerp/addons/base/ir/ir_mail_server.py", line 483, in send_email
raise MailDeliveryException(_("Mail Delivery Failed"), msg)
MailDeliveryException: (u'Mail Delivery Failed', u"Mail delivery failed via SMTP server 'smtp.office365.com'.\nSMTPDataError: 550\n5.7.60 SMTP; Client does not have permissions to send as this sender")
So i tried too much for this , but i am not getting any solution , if you have any solutin , please share with me .
Use port 587.
The error message tells you that the sender is invalid - you can only send as the mailbox owner (primary SMTP address) or as one of the proxy addresses associated with the mailbox.
Removing all catchall parameters (mail.catchall.domain and mail.catchall.alias) under "Settings" -> "Technical" -> "Parameters" -> "System Parameters" and it work like charm .
WORKS LIKE CHARM :
Removing all catchall parameters (mail.catchall.domain and mail.catchall.alias) under "Settings" -> "Technical" -> "Parameters" -> "System Parameters" and it work like charm . TY Debasish
I'm on Odoo V12
It was not suffisient for my problem, I had to delete the alias domain but there another thing to check :
I initialy created my odoo installation with a GMAIL address, worked a bit but had to switch for a pro e-mail because all my invitations e-mail was blocked by Google Bot beacause it look liked suspicious. It did this only in Odoo v12 because there is more links in the mail.
So I configured my real smtp server in Odoo but get the error 550. Odoo kept in the COMPANY settings the primary gmail address and tried to send on my other smtp server with the gmail name. The other server didn't accepted it so sent me back error 550.
Once i putted my new e-mail address in the company description, and deleted alias domain it worked !!
PS : Don't try to edit ir_mail_server.py to put in bruteforce your e-mail ... Doesn't work ..

Fail2ban regex for exim

I have tried to write a regex for fail2ban for my exim mail server but I can't seem to get any matches. even on the rejected logfile.
Here is a line from my exim_mainlog:
2014-09-18 16:34:30 dovecot_login authenticator failed for xx-xx-78-xx.dedicated.abac.net (User) [xx.xx.78.xx]:64298: 535 Incorrect authentication data (set_id=sexy)
2014-09-18 16:50:17 dovecot_login authenticator failed for (User) [xx.xx.xx.231]:9859: 535 Incorrect authentication data (set_id=evans)
2014-09-18 16:52:30 dovecot_login authenticator failed for (User) [xx.xx.16.128]:60350: 535 Incorrect authentication data (set_id=orange)
2014-09-18 17:10:19 dovecot_login authenticator failed for XXXX.onlinehome-server.com (User) [xx.xx.96.171]:52799: 535 Incorrect authentication data (set_id=matrix)
the same entries are in my exim_rejectlog.
Here is the filter I tried on both the exim_mainlog and exim_reject log
failregex = \[<HOST>\]: 535 Incorrect authentication data
But I don't get any hits even from the ones that are in the rejectlog
Lines: 257 lines, 0 ignored, 0 matched, 257 missed
Im not very good at regex and would appreciate any help to keep these spammers away
Your server is configured to also log the incoming port, not just the ip address. Change your regex to accommodate the extra characters:
failregex = \[<HOST>\]:\d+: 535 Incorrect authentication data
Official released version with exim parsing (it now handles the port if it's present). https://github.com/fail2ban/fail2ban/blob/master/config/filter.d/exim.conf