WSO2IS logging client IP into file wso2carbon.log - wso2

I have an environment where WSO2IS is behind a balancer and would like the client IP also be logged into file wso2carbon.log beyond http_access.log file.

on application log we need to manually parse the value to log. you shall get the IP address from request. You may add this to Thread local and configure it as pattern layout[5].
String ipAddress = request.getHeader("X-FORWARDED-FOR");
if (ipAddress == null) {
ipAddress = request.getRemoteAddr();
}
You may use request header "X-Forwarded-For"[1].
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="${carbon.home}/repository/logs"
               prefix="http_access_management_console_" suffix=".log"
               pattern="%{X-Forwarded-For}i %h %l %u %t "%r" %s %b "%{Referer}i" "%{User-Agent}i"" />
[2]
[1].https://tomcat.apache.org/tomcat-7.0-doc/config/valve.html#Proxies_Support
[2].https://docs.wso2.com/display/ESB490/Access+Logs
[3].http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html#Access_Log_Valve
[4].http://www.techstacks.com/howto/configure-access-logging-in-tomcat.html
[5].https://logging.apache.org/log4j/2.x/manual/thread-context.html

If you can programmatically get the User's IP, then you can write a User Operation Event Listener [1] and print logs. A sample is in [2].
[1] http://tharindue.blogspot.com/2016/08/user-operation-event-listener-in-wso2.html
[2] http://tharindue.blogspot.com/2016/08/writing-user-operation-event-listener.html

Related

Custom "statusMsg" not working as expected

After adding the following adaptive authentication script, if the username is not according to the format I should be getting the custom status message saying, Access Denied, invalid username format. But instead I get the default status message, Something went wrong during the authentication process. Please try signing in again.
function onLoginRequest(context) {
executeStep(1, {
onSuccess: function(context) {
var user = context.currentKnownSubject;
if(user!= null && user.username != null && !user.username.equals('')) {
Log.info("username: " + user.username);
} else {
sendError('',{'status':'AUTHENTICATION USERNAME ERROR', 'statusMsg': 'Access denied, invalid username format.'});
}
}
});
}
In addition, I get the following error in the wso2carbon.log file as well.
TID: [-1234] [authenticationendpoint] [2022-10-05 15:44:12,715] [37951f7d-8240-48d4-ad4f-1d4c8a6a3ec4] ERROR {org.wso2.carbon.identity.application.authentication.endpoint.util.AuthContextAPIClient} - Sending GET request to URL : https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/0b0efc37-819d-4b39-85b2-517126c3c9cb, failed. java.io.IOException: Server returned HTTP response code: 401 for URL: https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/0b0efc37-819d-4b39-85b2-517126c3c9cb
...
org.wso2.carbon.identity.application.authentication.endpoint.util.AuthContextAPIClient.getContextProperties(AuthContextAPIClient.java:70)
at org.apache.jsp.retry_jsp._jspService(retry_jsp.java:194)
...
org.wso2.carbon.ui.filters.cache.ContentTypeBasedCachePreventionFilter.doFilter(ContentTypeBasedCachePreventionFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
...
org.wso2.carbon.identity.application.authentication.endpoint.util.filter.AuthenticationEndpointFilter.doFilter(AuthenticationEndpointFilter.java:190)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
...
The <IS_HOME>/repository/conf/deployment.toml configurations for [server] are as follows.
[server]
hostname = "dev.wso2istemp.com"
node_ip = "127.0.0.1"
base_path = "https://$ref{server.hostname}:${carbon.management.port}"
What is the reason for the following issue in the wso2carbon.log and why the custom status message is not shown properly?
When the adaptive authentication script is running, the values are passed to the Identity Server(IS) encrypted. In above case, the encrypted data chunk is passed to the IS,
https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/24e56d99-9494-4989-a3e2-4008b73ebd9b
with the last segment of the URL is being the data chunk. When the server tries to get that data chunk with a GET request, java.io.IOException: Server returned HTTP response code: 401 for URL which is the code for unauthorized is thrown. Since that data chunk is not properly received, the default status message is shown instead of the custom status message. The steps to resolve this are given below.
First clarify whether the dev.wso2istemp.com which you were using is not mapped with the localhost in /etc/hosts file.
Go to <IS_HOME>/repository/conf/deployment.toml file and check for the following configuration [identity.auth_framework.endpoint] and check whether the mutual SSL is set to false via mutual_ssl_manager_enabled=false
If so, enable that by commenting the mutual_ssl_manager_enabled=false, since using mutual SSL is recommended for IS. If you go to the <IS_HOME>/repository/resources/conf/default.json file, you can notice that the default value for mutual_ssl_manager_enabled is true
In a multi-node situation, the above error can occur if the mutual SSL is not enabled, If that does not resolves the issue, then you have to check whether the internal_hostname has been set properly so that the internal API calls are being sent properly.
To do that, if you have not added the following configuration to <IS_HOME>/repository/conf/deployment.toml file, you can check whether it has been properly applied to the <IS_HOME>/repository/conf/identity/identity.xml by checking for <ServerHostName>localhost</ServerHostName>
[server]
internal_hostname="localhost"
If you are using a multi-node deployment, this localhost value should be added to the SAN for the certificate when the certificate is generated (-ext SAN=dns:localhost)
keytool -genkey -alias newcert -keyalg RSA -keysize 2048 -keystore newkeystore.jks -dname "CN=dev.wso2istemp.com, OU=Is,O=Wso2,L=SL,S=WS,C=LK" -storepass mypassword -keypass mypassword -ext SAN=dns:localhost
But if you are using a single node you can add the following configuration to the deployment.toml and check whether it resolves things. (In single node case the internal_hostname should be similar to hostname)
[server]
hostname = "dev.wso2istemp.com"
internal_hostname = "dev.wso2istemp.com"
If that is not working[https://github.com/wso2/product-is/issues/11878] then go to <IS_HOME>/repository/deployment/server/webapps/authenticationendpoint/WEB-INF/web.xml and uncomment the following commented snippet.
<!--context-param>
<param-name>AuthenticationRESTEndpointURL</param-name>
<param-value>https://localhost:9443/api/identity/auth/v1.1/</param-value>
</context-param-->
This might route the internal_hostname being reflected on the request since the internal API calls are blocked when hostname of the server being replaced instead of the internal_hostname for the internal API calls(https://dev.wso2istemp.com/api/identity/auth/v1.1/data/AuthenticationError/24e56d99-9494-4989-a3e2-4008b73ebd9b).

Cannot send SMTP email from JBoss java app in Amazon EC2 instance: Could not convert socket to TLS

I'm running a java app in JBoss 6.4.0 in an Amazon Web Services red hat 8 EC2 instance.
When my app tries to send an email via javax.mail I'm getting an error "Could not convert socket to TLS".
I then coded up the AmazonSESSample.java sample program and tried it. I ran it in my EC2 instance outside JBoss and it ran successfully. (The AmazonSESSample program can be found here: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/examples-send-using-smtp.html)
Then I commented out the email code in my java app, and replaced it with the code in AmazonSESSample.java. When I run my java app with the AmazonSESSample code in JBoss I get the same error: "Could not convert socket to TLS". So the AmazonSESSample works fine outside JBoss, and gives an error when running inside JBoss.
Here is the AmazonSESSample code in my app. Can somebody help me fix the "Could not convert socket to TLS" error?:
public class AmazonSESSample {
private static final Logger logger = LogManager.getFormatterLogger("AmazonSESSample");
// Replace sender#example.com with your "From" address.
// This address must be verified.
static final String FROM = "email1#gmail.com";
static final String FROMNAME = "Steve";
// Replace recipient#example.com with a "To" address. If your account
// is still in the sandbox, this address must be verified.
static final String TO = "email2#gmail.com";
// Replace smtp_username with your Amazon SES SMTP user name.
static final String SMTP_USERNAME = "thisIsNotActualghijikl";
// Replace smtp_password with your Amazon SES SMTP password.
static final String SMTP_PASSWORD = "abcdefThisIsNotActual";
// Amazon SES SMTP host name. This example uses the US West (Oregon) region.
// See https://docs.aws.amazon.com/ses/latest/DeveloperGuide/regions.html#region-endpoints
// for more information.
static final String HOST = "email-smtp.us-east-2.amazonaws.com";
// The port you will connect to on the Amazon SES SMTP endpoint.
static final int PORT = 587;
static final String SUBJECT = "Amazon SES test (SMTP interface accessed using Java)";
static final String BODY = String.join(
System.getProperty("line.separator"),
"<h1>Amazon SES SMTP Email Test</h1>",
"<p>This email was sent with Amazon SES using the ",
"<a href='https://github.com/javaee/javamail'>Javamail Package</a>",
" for <a href='https://www.java.com'>Java</a>."
);
public int sendEmail(DisplayEmailMessage emailMessage) throws UnsupportedEncodingException, MessagingException {
// Create a Properties object to contain connection configuration information.
Properties props = System.getProperties();
props.put("mail.transport.protocol", "smtp");
props.put("mail.smtp.port", PORT);
props.put("mail.smtp.starttls.enable", "true");
props.put("mail.smtp.auth", "true");
// Create a Session object to represent a mail session with the specified properties.
Session session = Session.getDefaultInstance(props);
// Create a message with the specified information.
MimeMessage msg = new MimeMessage(session);
msg.setFrom(new InternetAddress(FROM, FROMNAME));
msg.setRecipient(Message.RecipientType.TO, new InternetAddress(TO));
msg.setSubject(SUBJECT);
msg.setContent(BODY, "text/html");
// Create a transport.
Transport transport = session.getTransport();
// Send the message.
try {
System.out.println("Sending...");
// Connect to Amazon SES using the SMTP username and password you specified above.
transport.connect(HOST, SMTP_USERNAME, SMTP_PASSWORD);
// Send the email.
transport.sendMessage(msg, msg.getAllRecipients());
System.out.println("Email sent!");
}
catch (Exception ex) {
System.out.println("The email was not sent.");
System.out.println("Error message: " + ex.getMessage());
}
finally {
// Close and terminate the connection.
transport.close();
}
return 0;
}
}
Here is the javamail debug output:
DEBUG: setDebug: JavaMail version 1.4.5.redhat-2
Sending email to 123#gmail.com
DEBUG: getProvider() returning javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Sun Microsystems, Inc]
Starting to connect at Sun Dec 26 13:14:23 UTC 2021 to email 123#gmail.com
DEBUG SMTP: useEhlo true, useAuth true
DEBUG SMTP: trying to connect to host "smtp.dreamhost.com", port 587, isSSL false
220 pdx1-sub0-mail-a290.dreamhost.com ESMTP
DEBUG SMTP: connected to host "smtp.dreamhost.com", port: 587
EHLO ip-172-31-29-30.us-east-2.compute.internal
250-pdx1-sub0-mail-a290.dreamhost.com
250-PIPELINING
250-SIZE 40960000
250-ETRN
250-STARTTLS
250-AUTH PLAIN LOGIN
250-AUTH=PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 CHUNKING
DEBUG SMTP: Found extension "PIPELINING", arg ""
DEBUG SMTP: Found extension "SIZE", arg "40960000"
DEBUG SMTP: Found extension "ETRN", arg ""
DEBUG SMTP: Found extension "STARTTLS", arg ""
DEBUG SMTP: Found extension "AUTH", arg "PLAIN LOGIN"
DEBUG SMTP: Found extension "AUTH=PLAIN", arg "LOGIN"
DEBUG SMTP: Found extension "ENHANCEDSTATUSCODES", arg ""
DEBUG SMTP: Found extension "8BITMIME", arg ""
DEBUG SMTP: Found extension "CHUNKING", arg ""
STARTTLS
220 2.0.0 Ready to start TLS
MessagingException
javax.mail.MessagingException: Could not convert socket to TLS
I fixed this by upgrading my JBoss to 7.4.0.

Request param is logged in access log with embedded jetty server of spring boot application

I have got an issue with my application, it logs request along with its query param which may contain sensitive data in access log. application is configured with logback.xml & embedded jetty.
jetty server is customized with below accessLogCustomer
public JettyServerCustomizer accessLogCustomizer() {
return server -> {
Slf4jRequestLog requestLog = new Slf4jRequestLog();
requestLog.setExtended(true);
requestLog.setLogLatency(true);
requestLog.setPreferProxiedForAddress(true);
requestLog.setLogTimeZone(userTimezone == null ? ZoneId.systemDefault().getId() : userTimezone);
requestLog.setLogDateFormat("Y-MM-dd HH:mm:ss, SSS Z");
RequestLogHandler requestLogHandler = new RequestLogHandler();
requestLogHandler.setRequestLog(requestLog);
requestLogHandler.setHandler(server.getHandler());
server.setHandler(requestLogHandler);
};
}
logback.xml
<appender name="access" class="ch.qos.logback.core.rolling.RollingFileAppender">
<File>${logs.dir}/abc-access.log</File>
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>%m %n</Pattern>
</layout>
<charset>UTF-8</charset>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<FileNamePattern>${logs.dir}/abc-access.%d.log.gz</FileNamePattern>
</rollingPolicy>
</appender>
<logger name="org.eclipse.jetty.server.RequestLog" additivity="false">
<appender-ref ref="access"/>
</logger>
request logged in access log
192.168.0.100 - - [2021-05-20 15:48:15,093 +0530] "POST /myAPI/v2/customer/message?myID=123&messageText=hello HTTP/1.0" 200 0 "-" "PostmanRuntime/7.26.8" 475
I am trying to avoid messageText from access log, but not getting any solution.
Use the CustomRequestLog and Slf4jRequestLogWriter instead.
You'll want the special format option %U which emits the URL path, without the query string (which is available as %q btw)
Your resulting configuration would look like this ...
Slf4jRequestLogWriter slfjRequestLogWriter = new Slf4jRequestLogWriter();
String format = "%{client}a - %u %t %m \"%U\" %s %O \"%{Referer}i\" \"%{User-Agent}i\"";
CustomRequestLog customRequestLog = new CustomRequestLog(slfjRequestLogWriter, format);
server.setRequestLog(customRequestLog);
Play with the format line, read the Javadoc on CustomRequestLog to know what you can do.
Some notes:
The example format is not strictly following the Extended NCSA format (as it's missing the HTTP version portion, and the HTTP method is outside of the quoted section, but that is usually not a problem for many users)
Slf4jRequestLogWriter is only concerned with taking the formatted log line and sending it to the slf4j-api, it does nothing else.
RequestLogHandler is deprecated and not a recommended usage anymore (as it does not log bad requests and context-less requests), use the Server.setRequestLog(RequestLog) instead.
Jetty will use the CustomRequestLog's Pattern to produce a String, this String is forwarded to the Slf4jRequestLogWriter as a slf4j logging event message, which is then logged per your existing slf4j + logback configuration.

How to setup outgoing mail server through outlook office365 in odoo 9

I will try to setup outgoing mail server in odoo 9 .so i fill all the field and test connection and the connection also success , but at the time of send mail it will generate an error .
Field fill like that:-
Name : sendmail
Priority: 10
SMTP Server : smtp.office365.com
SMTP Port:25
Debugging: enable
Connection Security:TLS (STARTTLS)
Username:my yser name
Password:password
But, when we send any mail then it will generate the below error
16-12-06 10:04:28,440 426 INFO test openerp.addons.base.ir.ir_mail_server: Mail delivery failed via SMTP server 'smtp.office365.com'.
SMTPDataError: 550
5.7.60 SMTP; Client does not have permissions to send as this sender
2016-12-06 10:04:28,443 426 ERROR test openerp.addons.mail.models.mail_mail: failed sending mail (id: 136) due to Mail Delivery Failed
Mail delivery failed via SMTP server 'smtp.office365.com'.
SMTPDataError: 550
5.7.60 SMTP; Client does not have permissions to send as this sender
Traceback (most recent call last):
File "/usr/lib/python2.7/dist- packages/openerp/addons/mail/models/mail_mail.py", line 262, in send
res = IrMailServer.send_email(msg, mail_server_id=mail.mail_server_id.id)
File "/usr/lib/python2.7/dist-packages/openerp/api.py", line 248, in wrapper
return new_api(self, *args, **kwargs)
File "/usr/lib/python2.7/dist-packages/openerp/api.py", line 490, in new_api
result = method(self._model, cr, uid, *args, **old_kwargs)
File "/usr/lib/python2.7/dist-packages/openerp/addons/base/ir/ir_mail_server.py", line 483, in send_email
raise MailDeliveryException(_("Mail Delivery Failed"), msg)
MailDeliveryException: (u'Mail Delivery Failed', u"Mail delivery failed via SMTP server 'smtp.office365.com'.\nSMTPDataError: 550\n5.7.60 SMTP; Client does not have permissions to send as this sender")
So i tried too much for this , but i am not getting any solution , if you have any solutin , please share with me .
Use port 587.
The error message tells you that the sender is invalid - you can only send as the mailbox owner (primary SMTP address) or as one of the proxy addresses associated with the mailbox.
Removing all catchall parameters (mail.catchall.domain and mail.catchall.alias) under "Settings" -> "Technical" -> "Parameters" -> "System Parameters" and it work like charm .
WORKS LIKE CHARM :
Removing all catchall parameters (mail.catchall.domain and mail.catchall.alias) under "Settings" -> "Technical" -> "Parameters" -> "System Parameters" and it work like charm . TY Debasish
I'm on Odoo V12
It was not suffisient for my problem, I had to delete the alias domain but there another thing to check :
I initialy created my odoo installation with a GMAIL address, worked a bit but had to switch for a pro e-mail because all my invitations e-mail was blocked by Google Bot beacause it look liked suspicious. It did this only in Odoo v12 because there is more links in the mail.
So I configured my real smtp server in Odoo but get the error 550. Odoo kept in the COMPANY settings the primary gmail address and tried to send on my other smtp server with the gmail name. The other server didn't accepted it so sent me back error 550.
Once i putted my new e-mail address in the company description, and deleted alias domain it worked !!
PS : Don't try to edit ir_mail_server.py to put in bruteforce your e-mail ... Doesn't work ..

Regexp: ProFTPD auth-logs

I've configurated ProFTPD to log all authentifications on a Plesk powered server. This setting is not set by default since Plesk 10 (whyever...). I want to configure fail2ban, to detect unsuccessful login-attempts for Brute Force prevention.
/etc/proftpd.include:
ExtendedLog /var/log/proftpd/auth.log AUTH auth
LogFormat auth "%v [%h] %s"
Example of unsuccessfull login (530):
/var/log/proftpd/auth.log:
ProFTPD [12.89.47.3] 331
ProFTPD [12.89.47.3] 530
What's the correct regexp for fail2ban ? My configuration seems not to match the pattern:
/etc/fail2ban/filter.d/proftpd.conf:
failregex = ProFTPD(.)+\[<HOST>\] 530$
<HOST> seems to be a fail2ban variable, and $ ends up a rule (if you want to set several rules inside the failregex = variable).
For me the following is working on Plesk 10.4.
proftpd.include:
ExtendedLog /var/log/proftpd/auth.log AUTH auth
LogFormat auth "%v %t \"%r\" [%h] %s"
/etc/fail2ban/filter.d/proftpd.conf:
failregex = \[<HOST>\]\s+530$