I am using kubectl and trying to run kubectl get nodes and exclude any master nodes from the results.
My end goal is to get a list of the IP addresses of the worker nodes.
This command gives me all of the nodes IP addresses (including master nodes):
kubectl get nodes -o jsonpath="{range .items[*]}{.status.addresses[?(#.type=='InternalIP')].address}{'\n'}{end}"
But as I said, I need to exclude any master nodes. The path to the name is: .metadata.name. I need to exclude from the range any .metadata.names that contain the text -master-. (UPDATE: Alternativly, I could only include ones that contain the text -worker-.)
I an find equal, and not equal. But I can't seem to find a way to do a "contains" or "regex" ability.
How can exclude items that match a pattern?
For such usecase you can use label selector.
Since all master nodes have node-role.kubernetes.io/master: "" label. you can use it to exclude these nodes. Use:
kubectl get no -l node-role.kubernetes.io/master!=""
If your worker nodes have some specific label you can use it as an atlernative selector.
The thing you are trying to do: match patterns; is not possible with built in jsonpath since its limited functionality, but you can use jq for this usecase if you really don't like lables and selectors.
Related
I am trying to build a connection string that requires pulling 3 IP addresses from another config file. When I get those values, I need to replace the port on each. I plan to replace each port using simple Bash find and replace ${string/pattern/replacement} but my problem is I'm stuck on the best way to parse the pattern out of the IP.
Here is what I have so far:
myFile.config:
ip.1=ip-ip-1-address:1234:5678
ip.2=ip-ip-2-address:1234:5678
ip.3=ip-ip-3-address:1234:5678
Copying some other simple process, I found I can pull the value of each IP like this:
IP1=`grep "ip.1=" /path/to/conf/myFile.config | awk -F "=" '{print $2}'`
which gives me ip.1=ip-ip-1-address:1234:5678. However, I need to replace 1234:5678 with 6543 for example. I've been looking around and I found this awesome answer that detailed using Bash prefix substitution but that relies on knowing the parameter. for example, I would have to do it this way:
test=${ip1##ip-ip-1-address:}
which results in $test being 1234:5678. That's fine but maybe I don't know the IP address as the parameter, so I'm back to considering regex unless there's a way for me to use * as the parameter or something, but I have been unsuccessful so far. For regex, I have tried a bunch such as test=${ip1/(?<=:).*/}.
Note that ${ip1/(?<=:).*/} you tried is an example of string manipulation syntax that does not support regex, only specific patterns.
You seem to want
x='ip.1=ip-ip-1-address:1234:5678'
echo "${x%%:*}:6543" # => ip.1=ip-ip-1-address:6543
The ${x%%:*} takes the value of x and removes all chars from the end till the first : including it. :6543 is added to the result of this manipulation using "${x%%:*}:6543".
To extract that value, you may also use
awk '/^ip\.1=/{sub("^[^:]+:", "");print}' myFile.config
The awk command finds lines starting with ip.1= and then removes all text from the start till the first colon including the colon and only prints these values.
This is followup, but slightly different problem to another question that I posted earlier at:
How to extract everything between two patterns (using sed?)?
During testing, I have also found that in some cases, more than one of the records are not only in a single file, but also all the records are just together in a single line, with no newlines between them, and when I run the sed command on that one combined line, it seems like it will find a match, but the match, but it is not the first match (I don't actually know why it is matching the one that it is finding).
I know that the first "line" is also the most recent record, so I would like to perform a sed type match to extract the time from the first record, but only from the first record.
Here a snippet to illustrate what I tried to describe above:
UPDATE SUCCESS (key/name): 7c061313-9cbd-4bd1-8613-139cbd2bd1da/snctest38
UPDATE SUCCESS (key/name): 7ebab268-05dc-4110-bab2-6805dc2110f6/snctest4
UPDATE SUCCESS (key/name): 8ef15385-3d23-40e9-b153-853d2310e9fd/snctest5
UPDATE SUCCESS (key/name): 978bf8b5-b4aa-4cd4-8bf8-b5b4aaacd4b6/snctest6
UPDATE SUCCESS (key/name): 34f28da7-1311-4cb7-b28d-a713116cb77d/snctest7
UPDATE SUCCESS (key/name): 7be77991-4c73-4c35-a779-914c73ec359a/snctest8
</message><refDesc>PUSH Task 0a64cff0-b826-420b-a4cf-f0b826420bae 2_ldapadd</refDesc><refKey>0a64cff0-b826-420b-a4cf-f0b826420bae</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:59.649Z</end><start>2020-01-22T05:36:59.639Z</start><jobType>TASK</jobType><key>c4a0f6fa-109e-4a25-a0f6-fa109e5a2505</key><refDesc>PROPAGATION Task 3a2457ef-556b-4f39-a457-ef556b6f3921 d228de62-2918-46fe-960a-a193c3bfa0c3</refDesc><refKey>3a2457ef-556b-4f39-a457-ef556b6f3921</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:59.567Z</end><start>2020-01-22T05:36:59.558Z</start><jobType>TASK</jobType><key>e9defd1b-966f-4830-9efd-1b966f183008</key><refDesc>PROPAGATION Task a47fee5a-fea3-4422-bfee-5afea3e422c5 e7394424-a4b7-41fe-90a1-46e3b309e6f2</refDesc><refKey>a47fee5a-fea3-4422-bfee-5afea3e422c5</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:59.485Z</end><start>2020-01-22T05:36:59.476Z</start><jobType>TASK</jobType><key>52ab4c49-e28d-49d3-ab4c-49e28d39d38e</key><refDesc>PROPAGATION Task 437e3eb4-487f-471c-be3e-b4487f971cdf 11f95ec1-ac16-4344-8741-ae362ec4b357</refDesc><refKey>437e3eb4-487f-471c-be3e-b4487f971cdf</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:59.394Z</end><start>2020-01-22T05:36:59.385Z</start><jobType>TASK</jobType><key>17c947c9-eb8d-4222-8947-c9eb8d622288</key><refDesc>PROPAGATION Task 67956b5a-86a2-4f0c-956b-5a86a24f0c35 d7df1a03-a7f7-4a7b-8cab-d098852de2f3</refDesc><refKey>67956b5a-86a2-4f0c-956b-5a86a24f0c35</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:59.312Z</end><start>2020-01-22T05:36:59.302Z</start><jobType>TASK</jobType><key>42a7e05a-5ed2-438c-a7e0-5a5ed2e38c64</key><refDesc>PROPAGATION Task d3d6026d-f4fd-4c64-9602-6df4fd1c6455 321861ed-3ede-4981-a6cc-cc193662a652</refDesc><refKey>d3d6026d-f4fd-4c64-9602-6df4fd1c6455</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:59.226Z</end><start>2020-01-22T05:36:59.216Z</start><jobType>TASK</jobType><key>09e1767b-e243-47ea-a176-7be243a7eabd</key><refDesc>PROPAGATION Task 3dd00b2e-e0ab-43c8-900b-2ee0ab43c8de a3b19123-38de-4702-907a-11d7a14d7a64</refDesc><refKey>3dd00b2e-e0ab-43c8-900b-2ee0ab43c8de</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:59.145Z</end><start>2020-01-22T05:36:59.135Z</start><jobType>TASK</jobType><key>b13e32e2-e361-4175-be32-e2e361917501</key><refDesc>PROPAGATION Task de139a01-fb49-4ea5-939a-01fb49cea584 2a6b616b-5319-423c-9d4a-47157dfe2cbc</refDesc><refKey>de139a01-fb49-4ea5-939a-01fb49cea584</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:59.064Z</end><start>2020-01-22T05:36:59.054Z</start><jobType>TASK</jobType><key>cb24ebcd-6448-444b-a4eb-cd6448944bbc</key><refDesc>PROPAGATION Task aa947230-0cb2-40d2-9472-300cb270d230 0fade8e2-cfdf-4dc4-87ba-6d1352d32840</refDesc><refKey>aa947230-0cb2-40d2-9472-300cb270d230</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.983Z</end><start>2020-01-22T05:36:58.973Z</start><jobType>TASK</jobType><key>04897743-9db6-496f-8977-439db6596fab</key><refDesc>PROPAGATION Task bc3ae42c-701e-4258-bae4-2c701e3258cc 98f9e4de-0e86-447d-9ffa-c87c96a947ca</refDesc><refKey>bc3ae42c-701e-4258-bae4-2c701e3258cc</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.901Z</end><start>2020-01-22T05:36:58.891Z</start><jobType>TASK</jobType><key>999bab8f-9e64-4f53-9bab-8f9e64df5320</key><refDesc>PROPAGATION Task 15371c79-1ac0-4282-b71c-791ac0e28230 fbdf2574-5e7d-4e97-af94-e4d6169fc02b</refDesc><refKey>15371c79-1ac0-4282-b71c-791ac0e28230</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.819Z</end><start>2020-01-22T05:36:58.810Z</start><jobType>TASK</jobType><key>81e907f6-d358-42b9-a907-f6d358e2b997</key><refDesc>PROPAGATION Task 5401a5d8-0407-4c2f-81a5-d804071c2fd0 81679e82-7006-4a2b-8914-f76fea587cec</refDesc><refKey>5401a5d8-0407-4c2f-81a5-d804071c2fd0</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.738Z</end><start>2020-01-22T05:36:58.728Z</start><jobType>TASK</jobType><key>a6b42457-4b18-4dcc-b424-574b180dcc51</key><refDesc>PROPAGATION Task 90cc1ef6-5b5a-4f4d-8c1e-f65b5acf4d16 42ac4e4e-71e8-4cfb-8c2e-ceced4cdf029</refDesc><refKey>90cc1ef6-5b5a-4f4d-8c1e-f65b5acf4d16</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.655Z</end><start>2020-01-22T05:36:58.645Z</start><jobType>TASK</jobType><key>8aba0dc9-3df4-4398-ba0d-c93df453989c</key><refDesc>PROPAGATION Task a074cbf2-de6c-4c15-b4cb-f2de6c4c15cd 6e37c044-5aa2-45e6-9626-27e0b2f52038</refDesc><refKey>a074cbf2-de6c-4c15-b4cb-f2de6c4c15cd</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.571Z</end><start>2020-01-22T05:36:58.561Z</start><jobType>TASK</jobType><key>723f4400-49cd-4507-bf44-0049cdd507a1</key><refDesc>PROPAGATION Task cb87d842-2849-45ca-87d8-422849e5caa8 8bcd6de0-9e6f-4261-89b3-e3d56400f7af</refDesc><refKey>cb87d842-2849-45ca-87d8-422849e5caa8</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.485Z</end><start>2020-01-22T05:36:58.474Z</start><jobType>TASK</jobType><key>a0b82ca0-8e2e-4b05-b82c-a08e2e8b0548</key><refDesc>PROPAGATION Task 828178dd-c1c8-4668-8178-ddc1c80668e7 d91141d7-2fd4-4fef-adc4-79cb81f1560b</refDesc><refKey>828178dd-c1c8-4668-8178-ddc1c80668e7</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.401Z</end><start>2020-01-22T05:36:58.391Z</start><jobType>TASK</jobType><key>0c2f6210-f095-427c-af62-10f095927c09</key><refDesc>PROPAGATION Task 1b9cb7c6-7b51-4f4a-9cb7-c67b51bf4ae9 440fa8f9-ecd4-4053-9b07-fd22ed49b139</refDesc><refKey>1b9cb7c6-7b51-4f4a-9cb7-c67b51bf4ae9</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.319Z</end><start>2020-01-22T05:36:58.309Z</start><jobType>TASK</jobType><key>77b14d6c-5be6-40e0-b14d-6c5be6f0e00b</key><refDesc>PROPAGATION Task 289a014d-3646-456d-9a01-4d3646356dd0 59fd7993-2737-46d6-8c0e-a26f8f02e234</refDesc><refKey>289a014d-3646-456d-9a01-4d3646356dd0</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.237Z</end><start>2020-01-22T05:36:58.227Z</start><jobType>TASK</jobType><key>0f59fe8b-f057-4039-99fe-8bf057d0394e</key><refDesc>PROPAGATION Task 3f9f7751-787f-4dd2-9f77-51787fedd2bf 106abcb7-a5f0-4333-8e9e-0a81e84af904</refDesc><refKey>3f9f7751-787f-4dd2-9f77-51787fedd2bf</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.154Z</end><start>2020-01-22T05:36:58.143Z</start><jobType>TASK</jobType><key>a5c44fb7-2b39-4e22-844f-b72b39de22ff</key><refDesc>PROPAGATION Task d9f712ea-d3a7-44a0-b712-ead3a784a05e efb1c4ca-6598-4cce-9aef-c749bf18e10b</refDesc><refKey>d9f712ea-d3a7-44a0-b712-ead3a784a05e</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:58.066Z</end><start>2020-01-22T05:36:58.056Z</start><jobType>TASK</jobType><key>a9c2b65a-2cc6-4e70-82b6-5a2cc6fe7083</key><refDesc>PROPAGATION Task e504ad04-6afc-4880-84ad-046afcd8806c a025af4f-b7b8-4d90-917c-d3ff79d13780</refDesc><refKey>e504ad04-6afc-4880-84ad-046afcd8806c</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:57.985Z</end><start>2020-01-22T05:36:57.975Z</start><jobType>TASK</jobType><key>a433e0da-71e5-4957-b3e0-da71e5d95744</key><refDesc>PROPAGATION Task 30597ec2-8cea-4744-997e-c28cea47446a 101d544c-5355-436b-9e93-aebcbf3b5da4</refDesc><refKey>30597ec2-8cea-4744-997e-c28cea47446a</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:57.902Z</end><start>2020-01-22T05:36:57.891Z</start><jobType>TASK</jobType><key>a854c35a-291f-43c8-94c3-5a291fb3c891</key><refDesc>PROPAGATION Task 58ed721f-0b25-4a39-ad72-1f0b255a39c6 404aa838-b14a-40fd-8e6f-5a0522cfcce8</refDesc><refKey>58ed721f-0b25-4a39-ad72-1f0b255a39c6</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:57.817Z</end><start>2020-01-22T05:36:57.807Z</start><jobType>TASK</jobType><key>19a4823a-b57e-4a51-a482-3ab57eaa5172</key><refDesc>PROPAGATION Task 9d0c6898-185b-4eac-8c68-98185b9eacb8 2b6dc9a1-8cf9-4b8c-a597-b1fc4ecd9b72</refDesc><refKey>9d0c6898-185b-4eac-8c68-98185b9eacb8</refKey><status>SUCCESS</status></syncope21:exec><syncope21:exec xmlns:syncope21="http://syncope.apache.org/2.1"><end>2020-01-22T05:36:57.728Z</end><start>2020-01-22T05:36:57.717Z</start><jobType>TASK</jobType><key>f5b0ccb5-304f-40d2-b0cc-b5304f30d21e</key><refDesc>PROPAGATION Task 98b26cc8-0ff9-487b-b26c-c80ff9487b59 ac164590-3e66-42a5-b7d7-c001fa641a24</refDesc><refKey>98b26cc8-0ff9-487b-b26c-c80ff9487b59</refKey><status>SUCCESS</status></syncope21:exec></syncope21:execs>
As you can see there are apparently multiple "lines" glommed together into one large line.
So how can some of the sed commands that were suggested in that previous thread be modified to match and extract just the first match?
Thanks,
Jim
If I'm reading your question right (and this is not easy given its dependence on other questions), you just want the shorted match between two given regexes (which are themselves just strings): <\/messages> and <\/start>.
sed '/<\/message>.*<\/start>/ { s:^.*</message>:</message>:; s:</start>.*$:</start>: }'
This uses POSIX sed since you have not specified that you're using GNU (e.g. via a Linux distribution). First, we check to ensure the line has a match. If it does, remove everything before the final </message> and then remove everything after the first </start>.
This would be more elegant with GNU (or BSD) sed since you can use backreferences to retain those tags:
sed -E '/<\/message>.*<\/start>/ { s:^.*(</message>):\1:; s:(</start>).*$:\1: }'
Personally, I'd use GNU grep (assuming it's compiled with libpcre to support -P):
grep -Po '</message>.*?</start>'
This will only display the matching text of the shortest match between those two given strings.
I have a list with URLs and IPs for Office365 in XML format. Now I'd like to either write a script or use a text editor's search and replace function (regex) to automatically change some of these URLs.
Example:
These URLs
<address>scus-odc.officeapps.live.com</address>
<address>scus-roaming.officeapps.live.com</address>
<address>sea-odc.officeapps.live.com</address>
Should be changed to
<address>*.officeapps.live.com</address>
<address>*.officeapps.live.com</address>
<address>*.officeapps.live.com</address>
I would appreciate any input on this issue. Thanks in advance.
Here is what I have tried so far:
1)Search for ..(?=[^.].[^.]*$) and replace with an empty string.This does a good job but unfortunately it removes the preceeding as well...
2)As pointed out by Tim, the list consists of FQDNs with different domains.The list is available from https://go.microsoft.com/fwlink/?LinkId=533185 (This list includes all FQDNs - The IPs will get deleted)
3) Solved with the help of Sergio's input. The solution was to
search for (>)[^.\n\s]+ and substitute with \1\*
I will have to write another script to delete the multiple domains but that was not part of the question so I consider this issue closed. Thank you for your input.
You can use the regex:
(>)[^.\n\s]+
and substitute with \1\*
I am working with a cloud environment and there is a command that will display all available information about VMs running. here is an example of some of the lines that pertain to one VM.
RESERVATION r-6D0F464B 170506678332 GroupD
INSTANCE i-E9B444A9 emi-376642D8 999.99.999.999 88.888.88.888 running lock_key 0 c1.xlarge 2013-06-17T18:40:56.270Z cluster01 eki-E7E242A3 monitoring-disabled 999.99.999.999 88.888.88.888 ebs
I need to be able to pull the i-********, emi-********, both IP address, its status, the lock_key, the c1.xlarge, and the monitoring-disabled/enabled.
I have been able to pull the whole line with some super simple regex but all of this is well beyond me. If there is another easier method of grabbing this data any suggestions are welcome.
Let's go by parts. Best way I can think of is redirecting the output to a file, in unix-like environments you do it like:
cat your-command > filename.txt
Second, you need to read the file line by line, I would use a python script or a perl script if you know any of those, or whatever language fits you.
Third, you can get values two different ways:
Read columns by position, you can get colums with a regex like: [^\s]+
Write regular expressions for every specific column, so for IP you could have something like this: ([0-9]{1,3}\.){4} for monitoring monitoring-([^\s]+) and so on.
As long as the fields will always be in the same order, all you need to is split on whitespace.
Pseudocode (well, it's ruby, but hopefully you get the idea):
vms = {}
File.open('vm-info').readlines.each do |line|
fields = line.split('\s+')
field_map = {}
vm_name = fields[<index_of_vm_name>]
field_map['emi'] = fields[<index_of_emi>]
field_map['ip_address'] = fields[<index_of_ip_address]
.
.
.
vms[vm_name] = field_map
end
After this, vms will be initialized to contain information about each vm. You can simply print them all out at this point, or continue running data manipulation on them.
I have a huge log file that has a structure like this:
ip=X.X.X.X
userAgent=Firefox
-----
Referer=hxxp://www.bla.org
I want to create a custom output like this:
ip:userAgent
for ex:
X.X.X.X:Firefox
and the pattern will ignore lines which don't start with ip= and userAgent=. (these two must form a pair as i mentioned above.)
I am a newbie administrator and our client needs a sorted file immediately.
Any help will be wonderful.
Thanks.
^ip=(\d+(?:\.\d+){3})[\r\n]+userAgent=(.+)$
Apply in global + multiline mode.
Group 1 will contain the IP, group 2 will contain the user agent string.
Edit: The above expression can be simplified a bit, we can remove the IP address format checking - assuming that there will be nothing but real IP addresses in the log file:
^ip=(\d+\.?)+[\r\n]+userAgent=(.+)$
You can use:
^ip=((?:[0-9]{1,3}\.){3}[0-9]{1,3})$
And
^userAgent=(.*)$
Get the group 1 for both and you will have the desired data.
give it a try (this is in no way robust if there are lines where your log file differs from the example snippet above):
sed -n -e '/^ip=/ {s///
N
s/\nuserAgent=/:/
p
}' HugeFile > customoutput