sed replace range of lines with a single word - regex

I tried my best to get this and had little luck, I have following file and I want to replace
first “<Connector port="8080"” xml block with ‘xxxx’
Before :
<!-- A "Connector" represents an endpoint by which requests are received
and responses are returned. Documentation at :
Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
Java AJP Connector: /docs/config/ajp.html
APR (HTTP/AJP) Connector: /docs/apr.html
Define a non-SSL HTTP/1.1 Connector on port 8080
-->
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
<!-- A "Connector" using the shared thread pool-->
<!--
<Connector executor="tomcatThreadPool"
port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
-->
After :
<!-- A "Connector" represents an endpoint by which requests are received
and responses are returned. Documentation at :
Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
Java AJP Connector: /docs/config/ajp.html
APR (HTTP/AJP) Connector: /docs/apr.html
Define a non-SSL HTTP/1.1 Connector on port 8080
-->
xxxx
<!-- A "Connector" using the shared thread pool-->
<!--
<Connector executor="tomcatThreadPool"
port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
-->
I got the matching string to print using following sed command :
sed -n '/<Connector port="8080"/,/>/p' filename
However I’m unable to develop this to get above output.

This sed should do the trick on your sample input (but gets overzealous if you have more than one <Connector port="8080" section):
sed '/<Connector port="8080"/,/>/{ s/<Connector.*/xxxx/; t; d }'
But handling this robustly calls for an XML parser. Example:
#!/usr/bin/env ruby
require 'rexml/document'
include REXML
d = Document.new(File.read(ARGV[0]))
e = d.get_elements('//Connector[#port="8080"]').first
if !e.nil?
e.parent.insert_after(e, Text.new('xxxx'))
e.parent.delete(e)
end
File.open(ARGV[0],'w'){|f| f.print(d) }

sed '/<Connector port="8080"/,/>/ {
N
/>/ s/.*/xxxx/
}' YourFile
load the section of connector before changing it to xxxx, this allow you to eventually test something on it (but not better than pobrelkey if not)

Thanks all for the answers , they worked well .Appriciate it:
Also i got following from one of my firend which also a answer to this using awk
awk '/<Connector port/ {
print "abc"
getline
do {
getline
} while (index($0, ">") == 0)
getline
}
// { print $0}' < 1.txt

Related

Powershell regex for replacing text between two strings

I am trying to use a powershell script to change the password between two strings, I am running into two issues.
A complex password seems to break my regex, If I use something as simple as "TestPassword" the regex does what I expect. However using a more complex password "6QAfD5PmMhWzUxTq1FO1bGJQQXRXu6tizN29h6MRUSI=" it breaks and results in
SSLEnabled="true" keystoreFile="C:\cert.pfx" $16QAfD5PmMhWzUxTq1FO1bGJQQXRXu6tizN29h6MRUSI=" keystoreType="PKCS12"/>
instead of
SSLEnabled="true" keystoreFile="C:\cert.pfx" keystorePass="6QAfD5PmMhWzUxTq1FO1bGJQQXRXu6tizN29h6MRUSI=" keystoreType="PKCS12"/>
I want to be less specific for the second match grouo, for example at the moment I must specify '" keystoreType' but I would prefer to be less specific and only specify the ending quote. This way if I change the position of the keystoreType parameter in the future I don't have to worry about changing the regex to suit.
Bellow is my powershell as it stands:
#Set new password in server.xml
$pass='6QAfD5PmMhWzUxTq1FO1bGJQQXRXu6tizN29h6MRUSI='
$server_xml=".\server.xml"
(Get-Content $server_xml) -replace '(keystorePass=")(.*)(" keystoreType)',('$1{0}$3' -f "$pass") | Set-Content $server_xml
Bellow is an extract from my xml:
<Connector port="443" relaxedPathChars="[]|" relaxedQueryChars="[]|{}^\`"<>"
maxThreads="150" minSpareThreads="25" connectionTimeout="20000" enableLookups="false"
maxHttpHeaderSize="8192" protocol="HTTP/1.1" useBodyEncodingForURI="true" redirectPort="8443"
acceptCount="100" disableUploadTimeout="true" bindOnInit="false" secure="true" scheme="https"
proxyName="test.example.com" proxyPort="443"
SSLEnabled="true" keystoreFile="C:\cert.pfx" keystorePass="123abc" keystoreType="PKCS12"/>
Parse
As pointed out by #the four bird and #codextor in the comments; peeking and poking directly into a serialized string (e.g. XML) using string methods (like -Replace) is a bad idea. Instead you should use the related parser for searching and replacing which has an easier syntax, takes care of both your issues and other pitfalls (e.g. double quotes $pass='Test"123').
Security
There is even a protentional security risk by ignoring the related parsers as a user (which is assumed only allowed to supply a password) could inject a new property in your xml (connector) by supplying a password like:
$pass = 'MyPass" maxParameterCount="0'
Examples
$Xml = [Xml]'<Connector port="443" relaxedPathChars="[]|" relaxedQueryChars="[]|{}^\`"<>" maxThreads="150" minSpareThreads="25" connectionTimeout="20000" enableLookups="false" maxHttpHeaderSize="8192" protocol="HTTP/1.1" useBodyEncodingForURI="true" redirectPort="8443" acceptCount="100" disableUploadTimeout="true" bindOnInit="false" secure="true" scheme="https" proxyName="test.example.com" proxyPort="443" SSLEnabled="true" keystoreFile="C:\cert.pfx" keystorePass="123abc" keystoreType="PKCS12"/>'
$Xml.Connector.keystorePass = '6QAfD5PmMhWzUxTq1FO1bGJQQXRXu6tizN29h6MRUSI='
$Xml.Connector
port : 443
relaxedPathChars : []|
relaxedQueryChars : []|{}^\`"<>
maxThreads : 150
minSpareThreads : 25
connectionTimeout : 20000
enableLookups : false
maxHttpHeaderSize : 8192
protocol : HTTP/1.1
useBodyEncodingForURI : true
redirectPort : 8443
acceptCount : 100
disableUploadTimeout : true
bindOnInit : false
secure : true
scheme : https
proxyName : test.example.com
proxyPort : 443
SSLEnabled : true
keystoreFile : C:\cert.pfx
keystorePass : 6QAfD5PmMhWzUxTq1FO1bGJQQXRXu6tizN29h6MRUSI=
keystoreType : PKCS12
$Xml.OuterXml
<Connector port="443" relaxedPathChars="[]|" relaxedQueryChars="[]|{}^\`"<>" maxThreads="150" minSpareThreads="25" connectionTimeout="20000" enableLookups="false" maxHttpHeaderSize="8192" protocol="HTTP/1.1" useBodyEncodingForURI="true" redirectPort="8443" acceptCount="100" disableUploadTimeout="true" bindOnInit="false" secure="true" scheme="https" proxyName="test.example.com" proxyPort="443" SSLEnabled="true" keystoreFile="C:\cert.pfx" keystorePass="6QAfD5PmMhWzUxTq1FO1bGJQQXRXu6tizN29h6MRUSI=" keystoreType="PKCS12" />
Addendum
(based on the additional info in the comments)
If there are more connectors in your xml, as e.g.:
$Xml = [Xml]'
<Connectors>
<Connector
port="80"
keystorePass="Pass1" />
<Connector
port="443"
keystorePass="Pass2" />
</Connectors>'
You might address the connectors like:
$Xml.Connectors.Connector[0].keystorePass = 'Pass80'
$Xml.Connectors.Connector.Where{ $_.port -eq '443' }.SetAttribute('keystorePass', 'Pass443')
$Xml.OuterXml
<Connectors><Connector port="80" keystorePass="Pass80" /><Connector port="443" keystorePass="Pass443" /></Connectors>
This person had a similar issue that I was able to use the regex in my code:
Hide passwords in string
I ended up with the following:
#Set new password in server.xml
$pass='6QAfD5PmMhWzUxTq1FO1bGJQQXRXu6tizN29h6MRUSI='
$server_xml=".\server.xml"
(Get-Content $server_xml) -replace '(?:(?<=keystorePass=")\S+(?="))',("$pass") | Set-Content $server_xml

aws web service not available on the internet

In unix using putty I am able to get a result:
curl http://localhost:8080/products-ut-wo-db/rest/products
In security group I have inbound: 8080
In /home/ec2-user/java/tomcat9/webapps/manager/META-INF/context.xml I comment out the valve following:
I tried these url nothing come:
http://ec2-18-236-111-143.us-west-2.compute.amazonaws.com:8080/products-ut-wo-db/rest/products
http://ec2-18-236-111-143.us-west-2.compute.amazonaws.com:8080
Here is my netstat -nat:
Here is the /home/ec2-user/java/tomcat9/conf/server.xml:
Here is my /home/ec2-user/java/tomcat9/conf/server.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- Note: A "Server" is not itself a "Container", so you may not
define subcomponents such as "Valves" at this level.
Documentation at /docs/config/server.html
-->
<Server port="8005" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.startup.VersionLoggerListener" />
<!-- Security listener. Documentation at /docs/config/listeners.html
<Listener className="org.apache.catalina.security.SecurityListener" />
-->
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
<!-- Prevent memory leaks due to use of particular java/javax APIs-->
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />
<!-- Global JNDI resources
Documentation at /docs/jndi-resources-howto.html
-->
<GlobalNamingResources>
<!-- Editable user database that can also be used by
UserDatabaseRealm to authenticate users
-->
<Resource name="UserDatabase" auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<!-- A "Service" is a collection of one or more "Connectors" that share
a single "Container" Note: A "Service" is not itself a "Container",
so you may not define subcomponents such as "Valves" at this level.
Documentation at /docs/config/service.html
-->
<Service name="Catalina">
<!--The connectors can use a shared executor, you can define one or more named thread pools-->
<!--
<Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
maxThreads="150" minSpareThreads="4"/>
-->
<!-- A "Connector" represents an endpoint by which requests are received
and responses are returned. Documentation at :
Java HTTP Connector: /docs/config/http.html
Java AJP Connector: /docs/config/ajp.html
APR (HTTP/AJP) Connector: /docs/apr.html
Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
-->
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
<!-- A "Connector" using the shared thread pool-->
<!--
<Connector executor="tomcatThreadPool"
port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
-->
<!-- Define a SSL/TLS HTTP/1.1 Connector on port 8443
This connector uses the NIO implementation. The default
SSLImplementation will depend on the presence of the APR/native
library and the useOpenSSL attribute of the
AprLifecycleListener.
Either JSSE or OpenSSL style configuration may be used regardless of
the SSLImplementation selected. JSSE style configuration is used below.
-->
<!--
<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="150" SSLEnabled="true">
<SSLHostConfig>
<Certificate certificateKeystoreFile="conf/localhost-rsa.jks"
type="RSA" />
</SSLHostConfig>
</Connector>
-->
<!-- Define a SSL/TLS HTTP/1.1 Connector on port 8443 with HTTP/2
This connector uses the APR/native implementation which always uses
OpenSSL for TLS.
Either JSSE or OpenSSL style configuration may be used. OpenSSL style
configuration is used below.
-->
<!--
<Connector port="8443" protocol="org.apache.coyote.http11.Http11AprProtocol"
maxThreads="150" SSLEnabled="true" >
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
<SSLHostConfig>
<Certificate certificateKeyFile="conf/localhost-rsa-key.pem"
certificateFile="conf/localhost-rsa-cert.pem"
certificateChainFile="conf/localhost-rsa-chain.pem"
type="RSA" />
</SSLHostConfig>
</Connector>
-->
<!-- Define an AJP 1.3 Connector on port 8009 -->
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
<!-- An Engine represents the entry point (within Catalina) that processes
every request. The Engine implementation for Tomcat stand alone
analyzes the HTTP headers included with the request, and passes them
on to the appropriate Host (virtual host).
Documentation at /docs/config/engine.html -->
<!-- You should set jvmRoute to support load-balancing via AJP ie :
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
-->
<Engine name="Catalina" defaultHost="localhost">
<!--For clustering, please take a look at documentation at:
/docs/cluster-howto.html (simple how to)
/docs/config/cluster.html (reference documentation) -->
<!--
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
-->
<!-- Use the LockOutRealm to prevent attempts to guess user passwords
via a brute-force attack -->
<Realm className="org.apache.catalina.realm.LockOutRealm">
<!-- This Realm uses the UserDatabase configured in the global JNDI
resources under the key "UserDatabase". Any edits
that are performed against this UserDatabase are immediately
available for use by the Realm. -->
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
</Realm>
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true">
<!-- SingleSignOn valve, share authentication between web applications
Documentation at: /docs/config/valve.html -->
<!--
<Valve className="org.apache.catalina.authenticator.SingleSignOn" />
-->
<!-- Access log processes all example.
Documentation at: /docs/config/valve.html
Note: The pattern used is equivalent to using pattern="common" -->
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="localhost_access_log" suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
</Host>
</Engine>
</Service>
</Server>
Any help or hint will be greatly appreciated it.
It seems that tomcat is not binding to a IPv4 address (according to netstat output, it is only bound to IPv6 - check first column).
In order to configure tomcat to bind to IPv4, refer to this answer. It describes the proper way to configure your tomcat JVM to preferIPv4Stack and preferIPv4Addresses.

Remote Stream Server from IceCast

I an trying to reproduce a video streaming in an IceCast server so I would like to configure it in icecast.xml server. I have the following questions:
In the mount section how can I configure the video stream for reading the input.I would like to read a stream of the following type:
rtsp://*******************/Streaming/Channels/101?transportmode=unicast
In fact, I have just configured the following iceCast.xml file:
<icecast>
<limits>
<clients>1000</clients>
<sources>42</sources>
<threadpool>5</threadpool>
<queue-size>524288</queue-size>
<client-timeout>30</client-timeout>
<header-timeout>15</header-timeout>
<source-timeout>10</source-timeout>
<burst-on-connect>1</burst-on-connect>
<burst-size>65535</burst-size>
</limits>
<authentication>
<!-- Sources log in with username 'source' -->
<source-password>hackme</source-password>
<!-- Relays log in username 'relay' -->
<relay-password>hackme</relay-password>
<!-- Admin logs in with the username given below -->
<admin-user>admin</admin-user>
<admin-password>hackme</admin-password>
</authentication>
<hostname>localhost</hostname>
<listen-socket>
<port>8000</port>
</listen-socket>
<relay>
<server>rtsp://172.31.8.44/Streaming/Channels/101?transportmode=unicast</server>
<port>8000</port>
<mount>/test.ogg</mount>
<on-demand>0</on-demand>
<relay-shoutcast-metadata>0</relay-shoutcast-metadata>
</relay>
<mount type="normal">
<mount-name>/test.ogg</mount-name>
<username>othersource</username>
<password>hackmemore</password>
<max-listeners>1</max-listeners>
<dump-file>/test.mp4</dump-file>
<burst-size>65536</burst-size>
<fallback-mount>/etc/icecast2/test.mp4</fallback-mount>
<fallback-override>1</fallback-override>
<fallback-when-full>1</fallback-when-full>
<intro>/etc/icecast2/videoDAT.mp4</intro>
<hidden>1</hidden>
<no-yp>1</no-yp>
<stream-url>rtsp://172.31.8.44/Streaming/Channels/101?transportmode=unicast</stream-url>
<on-connect>sh /home/stream-start.sh</on-connect>
</mount>
<fileserve>1</fileserve>
<paths>
<basedir>/usr/share/icecast2</basedir>
<logdir>/var/log/icecast2</logdir>
<webroot>/usr/share/icecast2/web</webroot>
<adminroot>/usr/share/icecast2/admin</adminroot>
<alias source="/" dest="/status.xsl"/>
</paths>
<logging>
<accesslog>access.log</accesslog>
<errorlog>error.log</errorlog>
<loglevel>3</loglevel> <!-- 4 Debug, 3 Info, 2 Warn, 1 Error -->
<logsize>10000</logsize> <!-- Max size of a logfile -->
</logging>
<security>
<chroot>0</chroot>
</security>
</icecast>
How could I configure it for reading a mp4 file ?
Icecast can't relay RTSP streams. You can only relay compatible HTTP Progressive streams (which are the streams servers like Icecast and SHOUTcast serve.)
Icecast also will not decode your MP4.
You need to use a tool like FFmpeg to relay to Icecast. Something like...
ffmpeg -i <your RTSP URL> icecast://example.com/stream

Consuming a Webservice Using Mule 3.4

I am trying to consume a webservice produced on by following the tutorials in the mule Documentation. have been able to build the webservice successfully, but having issues consuming it. I have two Java Clasess "HelloWorld" and "HelloWorldImpl". This is my flow
<flow name="helloService" doc:name="helloService">
<http:inbound-endpoint address="http://localhost:63081/hello" exchange-pattern="request-response" doc:name="HTTP">
<cxf:jaxws-service serviceClass="com.test.HelloWorld"/>
</http:inbound-endpoint>
<component class="com.test.HelloWorldImpl" doc:name="Java"/>
<cxf:jaxws-client serviceClass="com.test.HelloWorld" operation="sayHi" doc:name="SOAP" />
<outbound-endpoint address="http://localhost:63081/services/greeter" doc:name="Generic"/>
</flow>
What am I doing wrong?
When I access the outbound endpoint I get
Cannot bind to address "http://activate.adobe.com:63081/services/greeter" No component registered on that endpoint
You have to make your endpoint accept all sub-paths and then handle wrong ones with message routing:
Example:
<flow name="jfeed_fill_data">
<http:inbound-endpoint address="http://localhost:1212" />
<choice>
<when evaluator="header" expression="INBOUND:http.request.path=/jcore/insert/feed/">
<component class="main.java.com.joshlabs.jcore.Feed"/>
</when>
<otherwise>
<message-properties-transformer>
<add-message-property key="http.status" value="404"/>
</message-properties-transformer>
<expression-transformer>
<return-argument evaluator="string" expression="{Exception: "Invalid URL"}"/>
</expression-transformer>
</otherwise>
</choice>
</flow>
First issue: How can there be two services running on the same port (63081) on your localhost.
http://localhost:63081/hello
http://localhost:63081/services/greeter
Also As mentioned in your post, the web-service you have created is Hello service with the endpoint
http://localhost:63081/hello
So you web sevice should be as follows.
<flow name="helloService" doc:name="helloService">
<http:inbound-endpoint address="http://localhost:63081/hello" exchange-pattern="request-response" doc:name="HTTP">
<cxf:jaxws-service serviceClass="com.test.HelloWorld"/>
</http:inbound-endpoint>
<component class="com.test.HelloWorldImpl" doc:name="Java"/>
</flow>
In order to consume you can write another flow which has got the cxf:jaxws-client
<flow name="helloclient" >
<some inbound endpoint. >
....
<cxf:jaxws-client serviceClass="com.test.HelloWorld" operation="sayHi" doc:name="SOAP" />
<outbound-endpoint address="http://localhost:63081/hello" doc:name="Generic"/>
.....
</flow>
Hope this helps.
Your inbound endpoint is http://localhost:63081/hello which is the address you should call to consume your webservice.
Also your outbound endpoint seems to point to a address where there is no webservice to consume. Unless you have a second flow in your mule config that you do not show.
You've defined a flow which has a listener on service end-point http://localhost:63081/hello. In this flow request comes in and then it is forwarded using jaxws-client to another service listening at http://localhost:63081/services/greeter.
Now the error message says Cannot bind to address which means it cannot call the end-point. Is there a service running anywhere at the end-point you're trying to send request to? If you want to send request locally as look like from your flow, then you need another flow listening at that end-point similar to one you have but with different http-endpoint

Modify tomcat server.xml config using sed

I am making an Ubuntu package that depends on Tomcat7 through HTTPS. To make it convenient for our customers, I would like the install script of the package enable HTTPS in Tomcat7. This is pretty easy to do manually; in the file /etc/tomcat7/server.xml, one needs to uncomment the following block:
<!--
<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="150" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" />
-->
How could I do this from a shellscript? Preferebly in a way that it still works for slight modifications of the exact pattern. I think the rule would be something along the lines of search for '<Connector port="8443"' and then remove <!-- and --> before and after the block.
Consider apply a patch on your server.xml.
Generating a patch file:
diff -ruN server.xml.old server.xml.new > mydiff.patch
Where server.xml.old is the original file, and server.xml.new is the file as you want.
The patch (mydiff.patch) will look like this:
--- server.xml.old 2011-10-29 04:03:25.000000000 -0300
+++ server.xml.new 2011-10-29 04:04:03.000000000 -0300
## -1,10 +1,10 ##
(...)
- <!--
<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="150" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" />
- --->
(...)
Then, just apply the patch:
patch server.xml mydiff.patch
You can run the patch command with the flag -N. Thus, it will skip files that seems already patched.
diff should most probably be the tool of your choice. But if the original config file is changed frequently, diff could not be able to apply your script in future versions.
sed also has the ability to read in more than one line. You may want to look at this example that also deals with modifying an xml document.
This might work:
sed -nr '/^<!--/,/^-->/!{p;b};/^<!--/{h;d};H;/^-->/{x;/<Connector port="8443"/{s/(^<!--\s*\n|\n\s*-->)//g};p}'
This ignores all non-comment lines. Saves the comment lines in hold space then deletes the start/end comment delimiters if the comment contains <Connector port="8443" and then prints the comment/non-comment.