SPLUNK subsearch 2 CSV Files join together - computer-vision

I have 2 Files with order data saved in two different sourcetypes in splunk.
One file contains an orderid, plnum(praefix + orderid (one ordernumer contains 3 plnum)), model (type of the order). The second file contains the same plnum's and Materialnumbers to those plnum's.
I want to search for the top Materials used for one or more Models.
So I searched for how to setup a subsearch:
sourcetype=file1 [search sourcetype=file2 MODEL="someting"| fields MODEL] |stats values(MATNR) by MODEL
I dont know why the subsearch dont work.

Run the subsearch by itself to verify it works and produces the expected results. I suspect it is working and is returning a list of PLNUMs in the form foo bar baz.... Splunk puts an implicit AND between search terms so your main search is looking for events containing all PLNUMs, which is unlikely.
Try using format in your subsearch. It returns the results in foo OR bar OR baz... format, which should work better in the main search.
sourcetype=file1 [search sourcetype=file2 MODEL="someting"| fields PLNUM | format] |stats values(MATNR) by PLNUM

Related

Remove columns by name based on pattern

How can I remove a large number of columns by name based on a pattern?
A data set exported from Jira has a ton of extra columns that I've no interest in. 400 Log entries, 50 Comments, dozens of links or attachments. Problem is that they get random numbers assigned which means that removing them with hardcoded column names will not work. That would look like this and break as the numbers change:
= Table.RemoveColumns(#"Previous Step",{"Watchers", "Watchers_10", "Watchers_11", "Watchers_12", "Watchers_13", "Watchers_14", "Watchers_15", "Watchers_16", "Watchers_17", "Watchers_18", "Watchers_19", "Watchers_20", "Watchers_21", "Watchers_22", "Watchers_23", "Watchers_24", "Watchers_25", "Watchers_26", "Watchers_27", "Watchers_28", "Log Work", "Log Work_29", "Log Work_30", "Log Work_31", "Log Work_32", ...
How can I remove a large number of columns by using a pattern in the name? i.e. remove all "Log Work" columns.
The best way I've found is to use List.FindText on Table.ColumnNames to get a list of column names dynamically based on target string:
= Table.RemoveColumns(#"Previous Step", List.FindText(Table.ColumnNames(#"Previous Step"), "Log Work")
This works by first grabbing the full list of Column Names and keeping only the ones that match the search string. That's then sent to RemoveColumns as normal.
Limitation appears to be that FindText doesn't offer complex pattern matching.
Of course, when you want to remove a lot of different patterns, having individual steps isn't very interesting. A way to combine this is to use List.Combine to join the resulting column names together.
That becomes:
= Table.RemoveColumns(L, List.Combine({ List.FindText(Table.ColumnNames(L), "Watchers_"), List.FindText(Table.ColumnNames(L), "Log Work"), List.FindText(Table.ColumnNames(L), "Comment"), List.FindText(Table.ColumnNames(L), "issue link"), List.FindText(Table.ColumnNames(L), "Attachment")} ))
SO what's actually written there is:
Table.RemoveColumns(PreviousStep, List.Combine({ foundList1, foundlist2, ... }))
Note the { } that signifies a list! You need to use this as List.Combine only accepts a single argument which is itself already a List of lists. And the Combine call is required here.
Also note the L here instead of #"Previous Step". That's used to make the entire thing more readable. Achieved by inserting a step named "L" that just has = #"Promoted Headers".
This allows relatively maintainable removal of multiple columns by name, but it's far from perfect.

ArangoDB query find name consisting of two or more parts

i am quite new to arangodb and AQL. What i am trying to do is to find names and surnames with query that have two or more parts that may or may not include spaces (" ") to divide those said parts.
As i went through documents, I figured that those fileds should be indexed as full-text indexes and query would be something like
FOR u IN FULLTEXT(collection, field, "search") RETURN u
Now my question is how to utilize the above query for a Persian/Arabic names such as
سید امیر حسین
Keep in mind that this name comes from an input of a user and it could be in the following formats such as
سید امیر حسین/سید امیرحسین/سیدامیر حسین/سیدامیرحسین
Note the various types of names with/without spaces. I tried the follwoing query F
FOR u IN FULLTEXT(collection, field, "سید,|امیر,|حسین,|سیدامیرحسین") RETURN u
But the problem is this will also result of fetching the name because i reckon the use of OR ( | ) operator.
سید محمد
which is not my intention. So what am i doing wrong or missing here ?
Many thanks in advance and excuse my noobiness
PS : arango 3.8.3

Remove all lines does not contain in other file

I have file A 'Emails' with so many email , and file B 'Domain' with so many domain
Example File A 'Emails ':
ctv#ymail.com
kfi#aol.in
hi#axus.cc
0#gmail.com
igp#yahoo.com
encor#mail2.com
cjang#mail.com
vn#gmail.com
87#gmail.com
ee#maoyt.com
Example file B 'Domain'
#gmail.com
#yahoo.com
My expected result :
0#gmail.com
igp#yahoo.com
vn#gmail.com
87#gmail.com
is there a way to do with 2 file in emeditor .Thanks much
I would propose using the Join CSV function. #Abimanyu's regex method may work if you have less than 10 or so domains. More than that, it might take a while to process the data.
To prepare the document for joining, right click on the CSV/Sort toolbar and edit the User-defined separated format to use # as the delimiter.
Now on both file A and file B, change the CSV mode to User-defined separated. On the CSV/Sort toolbar, there is a button called "Join CSV".
Join CSV options:
Make sure the correct documents are selected
Key Column is the email domain columns
In the list at the bottom, select the output columns, which should be column 1 and 2 from file A
Press the Join Now button, change CSV mode to Normal mode and you will get an output which looks like this:
0#gmail.com
igp#yahoo.com
vn#gmail.com
87#gmail.com
May be this will be help to you :
Pattern : .*#gmail.com|.*#yahoo.com
Match groups:
Match 1
1. 0#gmail.com
Match 2
1. igp#yahoo.com
Match 3
1. vn#gmail.com
Match 4
1. 87#gmail.com
https://rubular.com/r/M3MVSoRj6qnSbl

How to create new column that parses correct values from a row to a list

I am struggling on creating a formula with Power Bi that would split a single rows value into a list of values that i want.
So I have a column that is called ID and it has values such as:
"ID001122, ID223344" or "IRRELEVANT TEXT ID112233, MORE IRRELEVANT;ID223344 TEXT"
What is important is to save the ID and 6 numbers after it. The first example would turn into a list like this: {"ID001122","ID223344"}. The second example would look exactly the same but it would just parse all the irrelevant text from between.
I was looking for some type of an loop formula where you could use the text find function to find ID starting point and use middle function to extract 8 characters from the start but I had no progress in finding such. I tried making lists from comma separator but I noticed that not all rows had commas to separate IDs.
The end results would be that the original value is on one column next to the list of parsed values which then could be expanded to new rows.
ID Parsed ID
"Random ID123456, Text;ID23456" List {"ID123456","ID23456"}
Any of you have former experience?
Hey I found the answer by myself using a good article similar to my problem.
Here is my solution without any further text parsing which i can do later on.
each let
PosList = Text.PositionOf([ID],"ID",Occurrence.All),
List = List.Transform(PosList, (x) => Text.Middle([ID],x,8))
in List
For example this would result "(ID343137,ID352973) ID358388" into {ID343137,ID352973,ID358388}
Ended up being easier than I thought. Suppose the solution relied again on the lists!

How not to order a list of pk's in a query?

I have a list of pk's and I would like to get the result in the same order that my list is defined... But the order of the elements is begging changed. How any one help me?
print list_ids
[31189, 31191, 31327, 31406, 31352, 31395, 31309, 30071, 31434, 31435]
obj_opor=Opor.objects.in_bulk(list_ids).values()
for o in obj_oportunidades:
print o
31395 31435 31434 30071 31309 31406 31189 31191 31352 31327
This object should be used in template to show some results to the user... But how you can see, the order is different from the original list_ids
Would have been nice to have this feature in SQL - sorting by a known list of values.
Instead, what you could do is:
obj_oportunidades=Opor.objects.in_bulk(list_ids).values()
all_opor = []
for o in obj_oportunidades:
print o
all_opor.append(o)
for i in list_ids:
if i in all_opor:
print all_opor.index(i)
Downside is that you have to get all the result rows first and store them before getting them in the order you want. (all_opor could be a dictionary above, with the table records stored in the values and the PKeys as dict keys.)
Other way, create a temp table with (Sort_Order, Pkey) and add that to the query:
Sort_Order PKey
1 31189
2 31191
...
So when you sort on Sort_Order and Opor.objects, you'll get Pkeys it in the order you specify.
I found a solution in: http://davedash.com/2010/02/11/retrieving-elements-in-a-specific-order-in-django-and-mysql/ it's suited me perfectly.
ids = [a_list, of, ordered, ids]
addons = Addon.objects.filter(id__in=ids).extra(
select={'manual': 'FIELD(id,%s)' % ','.join(map(str,ids))},
order_by=['manual'])
This code do something similiar to MySQL "ORDER BY FIELD".
This guy: http://blog.mathieu-leplatre.info/django-create-a-queryset-from-a-list-preserving-order.html
Solved the problem for both MySQL and PostgreSQL!
If you are using PostgreSQL go to that page.