I'm facing some problems converting user-mgt.xml file from wso2is:5.8.0 to wso2is:5.10.0 version. I have some configurations in user-mgt.xml to primary user store using ldap and now when I use deployment.toml I can't start the server.
docker container log:
[2021-03-01 21:42:19,627] ERROR {org.wso2.config.mapper.TomlParser} - Invalid escape sequence '\.' (line 44, column 53)
[2021-03-01 21:42:19,627] ERROR {org.wso2.config.mapper.TomlParser} - Invalid escape sequence '\.' (line 45, column 59)
[2021-03-01 21:42:19,627] ERROR {org.wso2.config.mapper.TomlParser} - Invalid escape sequence '\S' (line 47, column 24)
[2021-03-01 21:42:19,628] ERROR {org.wso2.config.mapper.TomlParser} - Invalid escape sequence '\S' (line 48, column 30)
[2021-03-01 21:42:19,628] ERROR {org.wso2.config.mapper.TomlParser} - Invalid escape sequence '\-' (line 50, column 34)
[2021-03-01 21:42:19,628] ERROR {org.wso2.config.mapper.TomlParser} - Invalid escape sequence '\S' (line 51, column 30)
user-mgt.xml from old version 5.8.0
<UserStoreManager class="org.wso2.carbon.user.core.ldap.ReadWriteLDAPUserStoreManager">
<Property name="UsernameJavaRegEx">^[a-zA-Z0-9._-]+#[a-zA-Z0-9.-]+\.[a-zA-Z]{2,4}$</Property>
<Property name="UsernameJavaScriptRegEx">^[a-zA-Z0-9._-]+#[a-zA-Z0-9.-]+\.[a-zA-Z]{2,4}$</Property>
<Property name="UsernameJavaRegExViolationErrorMsg">Username pattern policy violated</Property>
<Property name="PasswordJavaRegEx">^[\S]{5,30}$</Property>
<Property name="PasswordJavaScriptRegEx">^[\S]{5,30}$</Property>
<Property name="PasswordJavaRegExViolationErrorMsg">Password length should be within 5 to 30 characters</Property>
<Property name="RolenameJavaRegEx">[a-zA-Z0-9._\-|//]{3,30}$</Property>
<Property name="RolenameJavaScriptRegEx">^[\S]{3,30}$</Property>
</UserStoreManager>
deployment.toml from newer version 5.10.0:
[user_store.properties]
...
UsernameJavaRegEx = "^[a-zA-Z0-9._-]+#[a-zA-Z0-9.-]+\.[a-zA-Z]{2,4}$"
UsernameJavaScriptRegEx = "^[a-zA-Z0-9._-]+#[a-zA-Z0-9.-]+\.[a-zA-Z]{2,4}$"
UsernameJavaRegExViolationErrorMsg = "Username pattern policy violated"
PasswordJavaRegEx = "^[\S]{5,30}$"
PasswordJavaScriptRegEx = "^[\S]{5,30}$"
PasswordJavaRegExViolationErrorMsg = "Password length should be within 5 to 30 characters"
RolenameJavaRegEx = "[a-zA-Z0-9._\-|//]{3,30}$"
RolenameJavaScriptRegEx = "^[\S]{3,30}$"
....
How can I fix this problem?
In order to use \ in the configs, you need to use the escape character(\). (eg: ^[\\S]{5,30}$)
Try out these.
[user_store]
username_java_regex = "^[a-zA-Z0-9._-]+#[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,4}$"
username_java_script_regex = "^[a-zA-Z0-9._-]+#[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,4}$"
username_java_regex_violation_error_msg = "Username pattern policy violated"
password_java_regex = "^[\\S]{5,30}$"
password_java_script_regex = "^[\\S]{5,30}$"
password_java_regrx_violation_error_msg = "Password length should be within 5 to 30 characters"
rolename_java_regex = "[a-zA-Z0-9._\\-|//]{3,30}$"
rolename_java_script_regex = "^[\\S]{3,30}$"
Related
I am trying to trim extra characters over 48 with the following addition, but it doesn't work. What could be wrong?
rsyslog.conf (without change)
$ActionQueueType LinkedList
$ActionQueueFileName srvrfwd
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on
$ModLoad imudp #loads the udp module
$UDPServerAddress XX.XX.YY.ZZ
$UDPServerRun 514
*.* #127.0.0.1:6514;RSYSLOG_SyslogProtocol23Format
rsyslog.conf (with addition)
$ActionQueueType LinkedList
$ActionQueueFileName srvrfwd
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on
$ModLoad imudp #loads the udp module
$UDPServerAddress XX.XX.YY.ZZ
$UDPServerRun 514
set $.APPNAME47 = substring($app-name, 0, 47);
template(name="trimmer" type="string"
string="<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %$.APPNAME47% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n")
*.* #127.0.0.1:6514;trimmer
this worked fine.
template(name="trimmer" type="string" string="<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME:1:46% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n")
*.* #127.0.0.1:6514;trimmer
This is my FluentD parser config:
<filter format.3.**>
#type parser
format /^\[(?<module>[^\]]+)\] (?<time>.+): (?<msg>.*)$/
time_format %Y-%m-%d %H:%M:%S
key_name log
keep_time_key true
reserve_data false
</filter>
And that's an example log line:
[Macaron] 2017-04-26 16:54:26: Started GET / for 172.20.0.0
In the FluentD error log I'm getting:
2017-04-27 12:01:58 +0000 [warn]: plugin/filter_parser.rb:69:rescue in block in filter_stream: invalid time format: value = 2017-04-27 12:01:58, error_class = ArgumentError, error = invalid strptime format - `%Y-%m-%d %H:%M:%S'
The important part is this:
invalid time format: value = 2017-04-27 12:01:58, error_class = ArgumentError, error = invalid strptime format - `%Y-%m-%d %H:%M:%S'
But I can't see how %Y-%m-%d %H:%M:%S doesn't match 2017-04-27 12:01:58, or why this format would be invalid.
according to this tool it should match
I figured it out, there were some special characters to set colors in the log. Apparently when copy-pasting them into fluentular.herokuapp.com they did not get copied and that's why it worked there:
{"log":"[Macaron] \u001b[1;32m2017-04-27 12:34:07: Completed /node 200 OK in 1.163351ms\u001b[0m\n","stream":"stdout","time":"2017-04-27T12:34:07.953993591Z"}
I am unable to start namenode. The hadoop version is hadoop-1.2.1. I have formated the name node ,cleared the tmp directory in the /app/hadoop/tmp. Below mentioned is the configuration in core-site.xml
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>>
hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
mapred-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>
This is the error getting displayed
priyank#priyank-Ideapad-Z570:~$ hadoop namenode
16/04/22 19:03:35 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = priyank-Ideapad-Z570/127.0.1.1
STARTUP_MSG: args = []
STARTUP_MSG: version = 1.2.1
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013
STARTUP_MSG: java = 1.7.0_95
************************************************************/
[Fatal Error] core-site.xml:11:17: Content is not allowed in trailing section.
16/04/22 19:03:35 FATAL conf.Configuration: error parsing conf file: org.xml.sax.SAXParseException; systemId: file:/home/priyank/Desktop/Softwares/Hadoop/hadoop-1.2.1/conf/core-site.xml; lineNumber: 11; columnNumber: 17; Content is not allowed in trailing section.
16/04/22 19:03:35 ERROR namenode.NameNode: java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:/home/priyank/Desktop/Softwares/Hadoop/hadoop-1.2.1/conf/core-site.xml; lineNumber: 11; columnNumber: 17; Content is not allowed in trailing section.
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1249)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1107)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1053)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:420)
at org.apache.hadoop.hdfs.server.namenode.NameNode.setStartupOption(NameNode.java:1374)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1463)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1488)
Caused by: org.xml.sax.SAXParseException; systemId: file:/home/priyank/Desktop/Softwares/Hadoop/hadoop-1.2.1/conf/core-site.xml; lineNumber: 11; columnNumber: 17; Content is not allowed in trailing section.
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:338)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:177)
Stopped all the Demons using stop-all.sh. Cleared all the files in /tmp directory. Started all the demons using start-all.sh .
I checked the logs before and after and here's what I found.
Bad logs:
2016-04-23 08:07:18,729 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered FSNamesystemStateMBean and NameNodeMXBean
2016-04-23 08:07:18,800 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names occuring more than 10 times
2016-04-23 08:07:18,816 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /tmp/hadoop-priyank/dfs/name does not exist.
2016-04-23 08:07:18,822 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-priyank/dfs/name is in an inconsistent state: storage d
irectory does not exist or is not accessible.
at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
Good logs:
2016-04-23 09:28:30,205 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered FSNamesystemStateMBean and NameNodeMXBean
2016-04-23 09:28:30,251 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names occuring more than 10 times
2016-04-23 09:28:30,332 INFO org.apache.hadoop.hdfs.server.common.Storage: Number of files = 1
2016-04-23 09:28:30,344 INFO org.apache.hadoop.hdfs.server.common.Storage: Number of files under construction = 0
2016-04-23 09:28:30,344 INFO org.apache.hadoop.hdfs.server.common.Storage: Image file of size 113 loaded in 0 seconds.
2016-04-23 09:28:30,345 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: EOF of /tmp/hadoop-priyank/dfs/name/current/edits, reached end of
Perhaps the name node was not getting formated with the script and once I formated the /tmp directory . It started!!
I am trying to get Django to email me 500 errors. I have got DEBUG mode turned off as required, and am sending to a local SMTP server. I'm using Python 2.7.2 and Django 1.5.1.
Django is sending using the incorrect and invalid recipients "n, e":
# python -m smtpd -n -c DebuggingServer localhost:25
---------- MESSAGE FOLLOWS ----------
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /
From: sender#example.com
To: n, e
Extract from settings.py:
ADMINS = (
('Me', 'me#example.com')
)
SEND_BROKEN_LINK_EMAILS = True
MANAGERS = ADMINS
SERVER_EMAIL = 'sender#example.com'
ADMINS = (
('Me', 'me#example.com'),
# ^ missing comma
)
Parenthesis do not create a tuple, the comma does (and empty parenthesis):
>> 1 == (1)
True
>> a = 1,
>> a == (1,)
True
Refer to Tuple Syntax for explanation.
What would the easiest way to parse the Data section from this STAF command be?
Cannot find a STAF parameter which I can pass to the command to automatically do this,
so looks like parsing/regular expression might be best option?
Note: I do not want to use any external libraries.
[root#source ~]# STAF target PROCESS START SHELL COMMAND "ls" WAIT RETURNSTDOUT
Response
--------
{
Return Code: 0
Key : <None>
Files : [
{
Return Code: 0
Data : myFile.txt
myFile2.txt
myFile3.txt
}
]
}
Instead I would like the output/result to be formated like ..
[root#source ~]# STAF target PROCESS START SHELL COMMAND "ls" WAIT RETURNSTDOUT
myFile.txt
myFile2.txt
myFile3.txt
Best way to this is Create a XML file and use python script to access the data part of STAFResult since STAF Return data in Marshalled form as "CONTENT" and python can be use to grab that.
I will try to explain it with simple example, Its an HTTP request to server.
<stafcmd>
<location>'%s'%machineName</location>
<service>'http'</service>
<request>'DOGET URL %s?phno=%s&shortCode=%s&query=%s' % (url, phno, shortCode, escapeQuery)</request>
</stafcmd>
<if expr="RC == 0">
<sequence>
<call function="'func_Script'"></call>
<if expr="rc == 0"> <!-- Pass At First Query -->
<sequence>
<message>'PASS#Fisrt HTTPRequest: Keyword = %s,\nRequired Response = %s,\ncontent=%s' %(query, response, content)</message>
<tcstatus result="'pass'">'Pass:' </tcstatus>
</sequence>
<else> <!-- Check For MORE -->
<call function="'Validate_QueryMore'"> </call>
</else>
</if>
</sequence>
<else>
<message>'ERROR: HTTPRequest QUERY : RC = %s Result= %s' %(rc,STAFResult)</message>
</else>
</if>
<function name="func_Script">
<script>
import re
content = STAFResult['content'].lower()
response = response.lower()
test = content.find(response)
if test != -1:
rc = 0
else:
rc = 1
</script>
</function>
Hope It will give you some Help.
You can pipe the output of your command through an sed script that will filter out only the filenames for you. Here's a first cut:
sed -ne '/^[a-z]/p;/Data/s/[^:]*: \(.*\)/\1/p'
The idea is: If a line starts with a lower-case letter, that's a file name (expression up to the first semicolon). If the string "Data" is on the line, take everything that comes after the first colon in that line (expression after the semicolon). Everything else is ignored.
You might want to be more specific than just expecting a lower-case letter at the beginning (this would filter out the "Response" line at the beginning, but if your filename might start with an upper-case letter, that won't work). Also, just looking for the string "Data" might be a bit too general -- that string might occur in the filename as well. But hopefully you get the idea. To use this, run your command like this:
STAF ... | sed -ne ...