My build.xml has 134 targets, most of which are hidden (hidden="true"). Is there a way to list all targets from the commandline? Target-definitions are sometimes split over multiple-lines, and I usually use double-quote-characters for properties. I'm running this on Debian, and kudos for sorting targets and/or also displaying descriptions. :-)
Examples:
<target name="example1" hidden="false" />
<target name="example3" hidden="true" />
<target
description="Ipsum lorem"
hidden="true"
name='example3'
>
<phingcall target="example1" />
</target>
We can't do this with Phing, but we can within Phing. There's probably a cleaner, better way than this, but this works - assuming all other properties are wrapped in double quotes (i.e. it just passes example#3, above)
<target name="list_all" hidden="false">
<property name="command" value="
cat ${phing.file.foo}
| perl -pe 's|^\s*||g'
| perl -0pe 's|\n([^<])| \1|gs'
| grep '<target'
| perl -pe "s|name='([^']*)'|name=\"\1\"|g"
| perl -pe 's|^<target(\s?)||'
| perl -pe 's|(.*)([ ]?)depends="([^"]*)"([ ]?)(.*)|\1 \2|g'
| perl -pe 's|(.*)([ ]?)hidden="([^"]*)"([ ]?)(.*)|\1 \2|g'
| perl -pe 's|.*description="([^"]*).*name="([^"]*).*|name="\2" description="\1"|g'
| perl -pe 's|name="([^"]*)"|\1|g'
| perl -pe 's|description="([^"]*)"|[\1]|g'
| sort
| uniq
" override="true" />
<exec command="${command}" passthru="true" />
</target>
What do those lines do?
1) output the contents of build.xml. Here, I'm interested in my 'global' build.xml which is named 'foo';
2) remove all leading whitespace from each line;
3) remove line-breaks within each opening tag;
4) filter for lines starting "<target";
5) change single quote-marks on name-property to double;
6, 7,8) remove leading "<target", and depends and hidden properties;
9) move 'description' after 'name' on each line;
10) remove 'name=' and its quote-marks;
11) replace 'description=' and its quote-marks with square-brackets; and
12, 13) sort & remove duplicates (if any)
Sample output:
example1
example2
example3 [Ipsum lorem]
You can't with phing itself. The code simply skips the display if the target is set to "hidden".
Related
I am trying to build a PowerShell script such that I give it an input file and regex, it replaces the matching content with the environment variable.
For example,
If the input file contains the following:
<?xml version="1.0" encoding="utf-8"?>
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/Services" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<Parameter Name="IntegrationManager_PartitionCount" Value="1" />
<Parameter Name="IntegrationManager_MinReplicaSetSize" Value="2" />
<Parameter Name="IntegrationManager_TargetReplicaSetSize" Value="#{INT_MGR_IC}" />
<Parameter Name="EventManager_InstanceCount" Value="#{EVT_MGR_IC}" />
<Parameter Name="Entities_InstanceCount" Value="#{ENT_IC}" />
<Parameter Name="Profile_InstanceCount" Value="#{PRF_IC}" />
<Parameter Name="Identity_InstanceCount" Value="#{IDNT_IC}" />
</Parameters>
</Application>
I would like to build a script that replaces #{INT_MGR_IC} with the value of the INT_MGR_IC environment variable and so on.
If you know of such script or can point me in the right direction, it would be a great help. Specifically, I am interested to know how to:
Extract and loop over keys from the file such as: #{INT_MGR_IC}, #{EVT_MGR_IC}, etc.
Once I have the key, how do I replace it with an associated environment variable. For example, #{INT_MGR_IC} with INT_MGR_IC env. variable.
Thanks a lot for looking into this :)
UPDATE 1
This is the RegEx I am planning to use: /#{(.+)}/g
Just load the file using the Get-Content cmdlet, iterate over each Parmeter, filter all parameter that Value starts with an #using Where-Object and change the value. Finally, use the Set-Content cmdlet to write it back:
$contentPath = 'Your_Path_Here'
$content = [xml] (Get-Content $contentPath)
$content.DocumentElement.Parameters.Parameter | Where Value -Match '^#' | ForEach-Object {
$_.Value = "REPLACE HERE"
}
$content | Set-Content $contentPath
In case you need to determine an environment variable, you could use [Environment]::GetEnvironmentVariable($_.Value).
Thanks a lot for everyone who helped. Especially, #jisaak :)
Here is the final script I built that solved the problem in the question. Hopefully its useful to someone!
$configPath = 'Cloud.xml'
$config = [xml] (Get-Content $configPath)
$config.DocumentElement.Parameters.Parameter | Where {$_.Value.StartsWith("#{")} | ForEach-Object {
$var = $_.Value.replace("#{","").replace("}","")
$val = (get-item env:$var).Value
Write-Host "Replacing $var with $val"
$_.Value = $val
}
$config.Save($configPath)
I have next xml file:
<AutoTest>
<Source>EBS FX</Source>
<CreateCFF>No</CreateCFF>
<FoXML descriptor="pb.fx.spotfwd.trade.feed" version="2.0">
<FxSpotFwdTradeFeed>
<FxSpotFwd feed_datetime="17-Dec-2014 10:20:09"
cpty_sds_id="EBS" match_id="L845586141217" original_trade_id_feed="L80107141217"
value_date="20141218" trade_id_external="001-002141880445/5862" match_sds_id="EBSFeedCpty"
counter_ccy="USD" trade_id_feed="107" trade_type="S" feed_source_id="80" quoting_term="M"
deal_ccy="GBP" rate="1.5" trade_date="20141217" modified_by="automation" cpty_side="B" counter_amt="1500000"
smart_match="0" booking_status_id="10" trade_status_id="22" deal_amt="1000000" trade_direction="B">
<Notes />
</FxSpotFwd>
</FxSpotFwdTradeFeed>
<TestCases />
</FoXML>
</AutoTest>
How to get value of trade_id_external attribute by using sed?
I tried with this expression: sed -n '/trade_id_external/s/.*=//p' ./file.xml
but no luck
You don't even need a pattern /trade_id_external/ before the s///
$ sed -n 's/.*trade_id_external="\([^"]*\).*/\1/p' file
001-002141880445/5862
In basic sed , \(...\) called capturing groups which was used to capture the characters you want to print at the final.
Through grep,
$ grep -oP 'trade_id_external="\K[^"]*' file
001-002141880445/5862
-P would turn on the Perl-regex mode in grep. So we could use any PCRE regexes in grep with -P param enabled. \K in the above regex would discard the previously matched characters, that is, it won't consider the characters which are matched by the pattern exists before the \K
I am using sed to parse an xml file from yahoo.finance. the file contains a bunch of uninteresting information and all global stock symbols which i want to extract. It's a 1 liner xml file with a big amount of stock symbols which are represented like that:
symbol="VALUE"
i am using sed like this:
sed "s/.* symbol=\"\(.*\)\".*/\1/" list_stocksymbols.xml >> ./tmpfile.txt
my output looks like that:
<?xml version="1.0" encoding="UTF-8"?>
WRG.AX
<!-- engine8.yql.bf1.yahoo.com -->
problem
as you can see only 1 symbol is extracted (WRG.AX).
question
how would i go about getting sed to write out all symbols?
i tried
sed "s/.* symbol=\"\(.*\)\".*/\1/g" list_stocksymbols.xml >> ./tmpfile.txt
global flag, but it didnt work :/
**xml file extract **
<?xml version="1.0" encoding="UTF-8"?>
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng" yahoo:count="215" yahoo:created="2014-08-22T09:05:59Z" yahoo:lang="en-US">
<results><industry id="112" name="Agricultural Chemicals">
<company name="Adarsh Plant Protect Ltd" symbol="ADARSHPL.BO"/>
<company name="Agrium Inc" symbol="AGU.DE"/><company name="Agrium Inc" symbol="AGU.TO"/>
<company name="Agrium Inc." symbol="AGU"/>
<company name="Aimco Pesticides Ltd" symbol="AIMCO.BO"/>
<company name="American Vanguard Corp." symbol="AVD"/>
... and so on. The file is in 1 line only and not formatted like above.
** perl regex try **
perl -nle'print $& if m{(?<=symbol=")[^"]+}' list_stocksymbols
did also just bring out the first occurence
grep -Eo 'symbol="[^"]+' yahoo.txt | cut -c 9-
This works for all the grep versions without Perl support (as in Mac OS X in your case).
Also using only sed you could:
sed 's/.*symbol=\"//;s/\".*//' yahoo.txt
i have a file which contains these values:-
<property name="india" column="delhi" />
<property name="austrelia" column="sydney" />
<property name="uae" column="dubai" />
Now i want to extract value inside the first " ".
So result should be :-
india
austrelia
uae
i am using shell and my regex is "(.*?)" . But it selects both " " value. I want only first one.
Can someone suggest me correct regex for this.
try this:
sed -r 's/^[^"]+"([^"]*)".*/\1/' file
test with your data:
kent$ echo '<property name="india" column="delhi" />
<property name="austrelia" column="sydney" />
<property name="uae" column="dubai" />'|sed -r 's/^[^"]+"([^"]*)".*/\1/'
india
austrelia
uae
$ awk -F\" '{print $2}' file
Btw, probably shell is not ideal tool for parsing XML.
I want to print
userId = 1234
userid = 12345
timestamp = 88888888
js = abc
from my data
messssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
<input name="userId" value="1234" type="hidden"> messsssssssssssssssssss
<input name="userid" value="12345" type="hidden"> messssssssssssssssssss
<input name="timestamp" value="88888888" type="hidden"> messssssssssssss
<input name="js" value="abc" type="hidden"> messssssssssssssssssssssssss
messssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
How can I do this with AWK(or whatever)? Assume that my data is stored in the "$info" variable (single line data).
Edit : single line data i mean all data represent like this
messss...<input name="userId" value="1234" type="hidden">messsss...<input ....>messssssss
So i can't use grep to extract interest section.
I'm not sure I understand your "single line data" comment but if this is in a file, you can just do something like:
cat file
| grep '^<input '
| sed 's/^<input name="//'
| sed 's/" value="/ = /'
| sed 's/".*$//'
Here's the cut'n'paste version:
cat file | grep '^<input ' | sed 's/^<input name="//' | sed 's/" value="/ = /' | sed 's/".*$//'
This turns:
messssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
<input name="userId" value="1234" type="hidden"> messsssssssssssssssssss
<input name="userid" value="12345" type="hidden"> messssssssssssssssssss
<input name="timestamp" value="88888888" type="hidden"> messssssssssssss
<input name="js" value="abc" type="hidden"> messssssssssssssssssssssssss
messssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
quite happily into:
userId = 1234
userid = 12345
timestamp = 88888888
js = abc
The grep simply extracts the lines you want while the sed commandsrespectively:
strip off up to the first quote.
replace the section between the name and value with an "=".
remove everything following the value closing quote (including that quote).
This part should probably be a comment on Pax's answer, but it got a bit long for that little box. I'm thinking 'single line data' means you don't have any newlines in your variable at all? Then this will work:
echo "$info" | sed -n -r '/<input/s/<input +name="([^"]+)" +value="([^"]+)"[^>]*>[^<]*/\1 = \2\n/gp'
Notes on interesting bits:
- -n means don't print by default - we'll say when to print with that p at the end.
-r means extended regex
/<input/ at the beginning makes sure we don't even bother to work on lines that don't contain the desired pattern
That \n at the end is there to ensure all records end up on separate lines - any original newlines will still be there, and the fastest way to get rid of them is to tack on a '| grep .' on the end - you could use some sed magic but you wouldn't be able to understand it thirty seconds after you typed it in.
I can think of ways to do this in awk, but this is really a job for sed (or perl!).
To process variables that contain more than one line, you need to put the variable name in double quotes:
echo "$info"|sed 's/^\(<input\( \)name\(=\)"\([^"]*\)" value="\([^"]*\)"\)\?.*/\4\2\3\2\5/'
using perl
cat file | perl -ne 'print($1 . "=" . $2 . "\n") if(/name="(.*?)".*value="(.*?)"/);'
IMO, parsing HTML should be done with a proper HTML/XML parser. For example, Ruby has an excellent package, Nokogiri, for parsing HTML/XML:
ruby -e '
require "rubygems"
require "nokogiri"
doc = Nokogiri::HTML.parse(ARGF.read)
doc.search("//input").each do |node|
atts = node.attributes
puts "%s = %s" % [atts["name"], atts["value"]]
end
' mess.html
produces the output you're after
AWK:
BEGIN {
# Use record separator "<", instead of "\n".
RS = "<"
first = 1
}
# Skip the first record, as that begins before the first tag
first {
first = 0
next
}
/^input[^>]*>/ { #/
# make sure we don't match outside of the tag
end = match($0,/>/)
# locate the name attribute
pos = match($0,/name="[^"]*"/)
if (pos == 0 || pos > end) { next }
name = substr($0,RSTART+6,RLENGTH-7)
# locate the value attribute
pos = match($0,/value="[^"]*"/)
if (pos == 0 || pos > end) { next }
value = substr($0,RSTART+7,RLENGTH-8)
# print out the result
print name " = " value
}
Tools like awk and sed can be used together with XMLStarlet and HTML Tidy to parse HTML.