I have tried to write a regex for fail2ban for my exim mail server but I can't seem to get any matches. even on the rejected logfile.
Here is a line from my exim_mainlog:
2014-09-18 16:34:30 dovecot_login authenticator failed for xx-xx-78-xx.dedicated.abac.net (User) [xx.xx.78.xx]:64298: 535 Incorrect authentication data (set_id=sexy)
2014-09-18 16:50:17 dovecot_login authenticator failed for (User) [xx.xx.xx.231]:9859: 535 Incorrect authentication data (set_id=evans)
2014-09-18 16:52:30 dovecot_login authenticator failed for (User) [xx.xx.16.128]:60350: 535 Incorrect authentication data (set_id=orange)
2014-09-18 17:10:19 dovecot_login authenticator failed for XXXX.onlinehome-server.com (User) [xx.xx.96.171]:52799: 535 Incorrect authentication data (set_id=matrix)
the same entries are in my exim_rejectlog.
Here is the filter I tried on both the exim_mainlog and exim_reject log
failregex = \[<HOST>\]: 535 Incorrect authentication data
But I don't get any hits even from the ones that are in the rejectlog
Lines: 257 lines, 0 ignored, 0 matched, 257 missed
Im not very good at regex and would appreciate any help to keep these spammers away
Your server is configured to also log the incoming port, not just the ip address. Change your regex to accommodate the extra characters:
failregex = \[<HOST>\]:\d+: 535 Incorrect authentication data
Official released version with exim parsing (it now handles the port if it's present). https://github.com/fail2ban/fail2ban/blob/master/config/filter.d/exim.conf
Related
After adding the following adaptive authentication script, if the username is not according to the format I should be getting the custom status message saying, Access Denied, invalid username format. But instead I get the default status message, Something went wrong during the authentication process. Please try signing in again.
function onLoginRequest(context) {
executeStep(1, {
onSuccess: function(context) {
var user = context.currentKnownSubject;
if(user!= null && user.username != null && !user.username.equals('')) {
Log.info("username: " + user.username);
} else {
sendError('',{'status':'AUTHENTICATION USERNAME ERROR', 'statusMsg': 'Access denied, invalid username format.'});
}
}
});
}
In addition, I get the following error in the wso2carbon.log file as well.
TID: [-1234] [authenticationendpoint] [2022-10-05 15:44:12,715] [37951f7d-8240-48d4-ad4f-1d4c8a6a3ec4] ERROR {org.wso2.carbon.identity.application.authentication.endpoint.util.AuthContextAPIClient} - Sending GET request to URL : https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/0b0efc37-819d-4b39-85b2-517126c3c9cb, failed. java.io.IOException: Server returned HTTP response code: 401 for URL: https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/0b0efc37-819d-4b39-85b2-517126c3c9cb
...
org.wso2.carbon.identity.application.authentication.endpoint.util.AuthContextAPIClient.getContextProperties(AuthContextAPIClient.java:70)
at org.apache.jsp.retry_jsp._jspService(retry_jsp.java:194)
...
org.wso2.carbon.ui.filters.cache.ContentTypeBasedCachePreventionFilter.doFilter(ContentTypeBasedCachePreventionFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
...
org.wso2.carbon.identity.application.authentication.endpoint.util.filter.AuthenticationEndpointFilter.doFilter(AuthenticationEndpointFilter.java:190)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
...
The <IS_HOME>/repository/conf/deployment.toml configurations for [server] are as follows.
[server]
hostname = "dev.wso2istemp.com"
node_ip = "127.0.0.1"
base_path = "https://$ref{server.hostname}:${carbon.management.port}"
What is the reason for the following issue in the wso2carbon.log and why the custom status message is not shown properly?
When the adaptive authentication script is running, the values are passed to the Identity Server(IS) encrypted. In above case, the encrypted data chunk is passed to the IS,
https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/24e56d99-9494-4989-a3e2-4008b73ebd9b
with the last segment of the URL is being the data chunk. When the server tries to get that data chunk with a GET request, java.io.IOException: Server returned HTTP response code: 401 for URL which is the code for unauthorized is thrown. Since that data chunk is not properly received, the default status message is shown instead of the custom status message. The steps to resolve this are given below.
First clarify whether the dev.wso2istemp.com which you were using is not mapped with the localhost in /etc/hosts file.
Go to <IS_HOME>/repository/conf/deployment.toml file and check for the following configuration [identity.auth_framework.endpoint] and check whether the mutual SSL is set to false via mutual_ssl_manager_enabled=false
If so, enable that by commenting the mutual_ssl_manager_enabled=false, since using mutual SSL is recommended for IS. If you go to the <IS_HOME>/repository/resources/conf/default.json file, you can notice that the default value for mutual_ssl_manager_enabled is true
In a multi-node situation, the above error can occur if the mutual SSL is not enabled, If that does not resolves the issue, then you have to check whether the internal_hostname has been set properly so that the internal API calls are being sent properly.
To do that, if you have not added the following configuration to <IS_HOME>/repository/conf/deployment.toml file, you can check whether it has been properly applied to the <IS_HOME>/repository/conf/identity/identity.xml by checking for <ServerHostName>localhost</ServerHostName>
[server]
internal_hostname="localhost"
If you are using a multi-node deployment, this localhost value should be added to the SAN for the certificate when the certificate is generated (-ext SAN=dns:localhost)
keytool -genkey -alias newcert -keyalg RSA -keysize 2048 -keystore newkeystore.jks -dname "CN=dev.wso2istemp.com, OU=Is,O=Wso2,L=SL,S=WS,C=LK" -storepass mypassword -keypass mypassword -ext SAN=dns:localhost
But if you are using a single node you can add the following configuration to the deployment.toml and check whether it resolves things. (In single node case the internal_hostname should be similar to hostname)
[server]
hostname = "dev.wso2istemp.com"
internal_hostname = "dev.wso2istemp.com"
If that is not working[https://github.com/wso2/product-is/issues/11878] then go to <IS_HOME>/repository/deployment/server/webapps/authenticationendpoint/WEB-INF/web.xml and uncomment the following commented snippet.
<!--context-param>
<param-name>AuthenticationRESTEndpointURL</param-name>
<param-value>https://localhost:9443/api/identity/auth/v1.1/</param-value>
</context-param-->
This might route the internal_hostname being reflected on the request since the internal API calls are blocked when hostname of the server being replaced instead of the internal_hostname for the internal API calls(https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/24e56d99-9494-4989-a3e2-4008b73ebd9b).
This is the an example log:
2022-01-07 11:05:01,185 [http-nio-8080-exec-526] WARN de.web.Main| = - Error while execute Request
javax.servlet.ServletException: com.cg.blart.web.HttpInvocationException: Invalid tenant session
at com.cg.blume.web.DispatchingServlet.doGet(DispatchingServlet.java:169)
at de.jinx.lee.web.AutoDBUpgradeDispatchingServlet.lambda$doGet$0(AutoDBUpgradeDispatchingServlet.java:200)
at de.jinx.lee.web.AutoDBUpgradeDispatchingServlet.addMDCKey(AutoDBUpgradeDispatchingServlet.java:297)
Caused by: com.cg.blume.web.HttpInvocationException: Invalid tenant session
at com.cg.blume.web.procedure.HttpSessionManager.get(HttpSessionManager.java:190)
at de.jinx.lee.web.session.leeHttpSessionManager.get(leeHttpSessionManager.java:76)
... 41 more
This ist the pattern so far:
^.*?Exception: +%{DATA:exception}$
The output i get is close but no cigar: "com.cg.blart.web.HttpInvocationException: Invalid tenant session". For the live of me i dont get it to match the whole line. Can you show me the way? The desired output is:
javax.servlet.ServletException: com.cg.blart.web.HttpInvocationException: Invalid tenant session
Here is a link for a test tool: https://grokdebug.herokuapp.com/
Solution was this pattern:
(?<DATA:mps_exception>.*?(Exception:).*)
It is explained here: https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#_custom_patterns
I have SOAPUI project with 10 test cases in Jenkins. I set up Jenkins to send me email with information from console output (log). I have set up email notification content as HTML (text/html).
I have this log in console output in Jenkins:
07:25:05,957 INFO [SoapUITestCaseRunner] Running SoapUI testcase [Login with username and password]
07:25:05,957 INFO [SoapUITestCaseRunner] running step [Clear access token]
07:25:05,957 INFO [log] Environment URL: url.test.environment
07:25:05,958 INFO [SoapUITestCaseRunner] running step [Retrieve accessToken]
07:25:05,959 DEBUG [HttpClientSupport$SoapUIHttpClient] Stale connection check
07:25:05,960 DEBUG [HttpClientSupport$SoapUIHttpClient] Attempt 1 to execute request
07:25:05,960 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Sending request: POST /api/v2/path HTTP/1.1
07:25:06,010 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Receiving response: HTTP/1.1 200
07:25:06,011 DEBUG [HttpClientSupport$SoapUIHttpClient] Connection can be kept alive indefinitely
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [JsonPath Existence Match] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [Valid HTTP Status Codes] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [JsonPath Existence Match 1] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [JsonPath Existence Match 2] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] running step [Pass accessToken]
07:25:06,019 INFO [SoapUITestCaseRunner] Finished running SoapUI testcase [Login with username and password], time taken: 51ms, status: FINISHED
I have set up Jenkins to send email just with this line from log:
07:25:06,019 INFO [SoapUITestCaseRunner] Finished running SoapUI testcase [Login with username and password], time taken: 51ms, status: FINISHED
To that I use this regex to find just that line:
<pre>${BUILD_LOG_REGEX, regex="Finished running SoapUI testcase \\[Login with username and password\\]", showTruncatedLines=false}</pre>
But I want to have in email notification just parts of that line, something like this:
"Login with username and password: FINISHED"
or
"Login with username and password: FAILED"
Is there any way to send email with just part of that line?
Thanks to Aaron I found a solution.
Now I use this regex:
<b>Login with username and password: </b> <font color="green">${BUILD_LOG_REGEX, regex=".*Finished running SoapUI testcase \\[Login with username and password\\].*status: FINISHED", showTruncatedLines=false, substText="SUCCESS"}</font>
<font color="red">${BUILD_LOG_REGEX, regex=".*Finished running SoapUI testcase \\[Login with username and password\\].*status: FAILED", showTruncatedLines=false, substText="FAILED"}</font>
E-mail notification now looks like this:
Login with username and password: SUCCESS
or this:
Login with username and password: FAILED
"SUCCESS" text has green color and "FAILED" text has red color
It seems like you can use a substText argument to define a replacement pattern :
<pre>${BUILD_LOG_REGEX, regex="Finished running SoapUI testcase \\[Login with username and password\\].*status: ([A-Z]+)", showTruncatedLines=false, substText="Login with username and password : \\1"}</pre>
This will match only the lines such as the last one from your sample, select the success/failure status in the first capturing group which will be referred in the replacement pattern.
I have fail2ban 0.9.1 with Asterisk 11 on Fedora 21 using IPTables.
The IP addresses that attack my server are not getting written to IP Tables automatically (see below about them working when manually running banip). Do you see any errors that would be causing this?
I get messages in my /var/log/asterisk/messages log about miscreants trying erroneous extensions.
My Regex works because when I run
fail2ban-regex /var/log/asterisk/messages /etc/fail2ban/filter.d/asterisk.conf
I get
Lines: 2985 lines, 0 ignored, 597 matched, 2388 missed [processed in 0.66 sec]
This means that 597 lines matched the regular expression. Right? Is there a way to show what lines were matched? and what the variables were?
I can also do:
fail2ban-client set asterisk banip 107.150.44.222
and IPTables is properly updated and the IP is banned. (Yes, I know I used a real IP address -- and as far as I am concerned everyone is welcome to ban the ba$%*$#rd)
jail.local
[asterisk]
enabled=yes
filter=asterisk
protocol=all
logpath = /var/log/asterisk/messages
banaction=iptables-multiport
port = 5060,5061
action = %(banaction)s[name=%(__name__)s-tcp, port="%(port)s", protocol="tcp", chain="%(chain)s", actname=%(banaction)s-tcp]
%(banaction)s[name=%(__name__)s-udp, port="%(port)s", protocol="udp", chain="%(chain)s", actname=%(banaction)s-udp]
%(mta)s-whois[name=%(__name__)s, dest="%(destemail)s"]
maxretry = 3
bantime=432000
findtime =86400
I removed the reference to Asterisk in jail.conf to avoid conflicts
filter.d/asterisk.conf
[INCLUDES]
# Read common prefixes. If any customizations available -- read them from
# common.local
before = common.conf
[Definition]
# Option: failregex
# Notes.: regex to match the password failures messages in the logfile.
# Values: TEXT
#
log_prefix= \[\]\s*(?:NOTICE|SECURITY)%(__pid_re)s:?(?:\[\S+\d*\])? \S+:\d*
failregex = ^%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - Wrong password$
^%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - No matching peer found$
^%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - Username/auth name mismatch$
^%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - Device does not match ACL$
^%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - Peer is not supposed to register$
^%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - ACL error \(permit/deny\)$
^%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - Not a local domain$
^%(log_prefix)s Call from '[^']*' \(<HOST>:\d+\) to extension '\d+' rejected because extension not found in context 'default'\.$
^%(log_prefix)s Host <HOST> failed to authenticate as '[^']*'$
^%(log_prefix)s No registration for peer '[^']*' \(from <HOST>\)$
^%(log_prefix)s Host <HOST> failed MD5 authentication for '[^']*' \([^)]+\)$
^%(log_prefix)s Failed to authenticate (user|device) [^#]+#<HOST>\S*$
^%(log_prefix)s (?:handle_request_subscribe: )?Sending fake auth rejection for (device|user) \d*<sip:[^#]+#<HOST>>;tag=\w+\S*$
^%(log_prefix)s SecurityEvent="(FailedACL|InvalidAccountID|ChallengeResponseFailed|InvalidPassword)",EventTV="[\d-]+",Severity="[\w]+",Service="[\w]+",EventVersion="\d+",AccountID="\d+",SessionID="0x[\da-f]+",LocalAddress="IPV[46]/(UD|TC)P/[\da-fA-F:.]+/\d+",RemoteAddress="IPV[46]/(UD|TC)P/(<HOST>)/[0-9]{4}"(,Challenge="\w+",ReceivedChallenge="\w+")?(,ReceivedHash="[\da-f]+")?$
# Option: ignoreregex
# Notes.: regex to ignore. If this regex matches, the line is ignored.
# Values: TEXT
#
ignoreregex =
Your asterisk.conf and jail.local entry look fine, though I typically add the jail name after the banaction. For example: banaction=iptables-multiport[name=asterisk]
Restart the fail2ban service and check your fail2ban log for any errors. A common one that didn't get fixed until v0.9.2 is:
Error in FilterPyinotify callback: 'module' object has no attribute '_strptime_time'
To fix it, update fail2ban to v0.9.2 or edit the file: /usr/share/fail2ban/common/__init__.py
and add the following text to the end of the file:
from time import strptime
# strptime thread safety hack-around - http://bugs.python.org/issue7980
strptime("2012", "%Y")
Is there a way to show what lines were matched? and what the variables were?
You'll want to use the -v option with fail2ban-regex. It won't give you matched variables, but will list each IP Address associated with the matched line. You can then examine details for that IP in your asterisk logs.
fail2ban-regex -v /var/log/asterisk/messages /etc/fail2ban/filter.d/asterisk.conf
I've configurated ProFTPD to log all authentifications on a Plesk powered server. This setting is not set by default since Plesk 10 (whyever...). I want to configure fail2ban, to detect unsuccessful login-attempts for Brute Force prevention.
/etc/proftpd.include:
ExtendedLog /var/log/proftpd/auth.log AUTH auth
LogFormat auth "%v [%h] %s"
Example of unsuccessfull login (530):
/var/log/proftpd/auth.log:
ProFTPD [12.89.47.3] 331
ProFTPD [12.89.47.3] 530
What's the correct regexp for fail2ban ? My configuration seems not to match the pattern:
/etc/fail2ban/filter.d/proftpd.conf:
failregex = ProFTPD(.)+\[<HOST>\] 530$
<HOST> seems to be a fail2ban variable, and $ ends up a rule (if you want to set several rules inside the failregex = variable).
For me the following is working on Plesk 10.4.
proftpd.include:
ExtendedLog /var/log/proftpd/auth.log AUTH auth
LogFormat auth "%v %t \"%r\" [%h] %s"
/etc/fail2ban/filter.d/proftpd.conf:
failregex = \[<HOST>\]\s+530$