Combining filters in boost log - c++

I use boost log and want to define a composed filter. I use boost::log::init_from_streamto read the configuration from a stream. Filtering on single conditions works fine. I can do
Filter = "%Channel% = A"
to get only log entries from channel A. I can do
Filter = "%Severity% >= warn"
to get only log entries that have a severity which is warning or above.
Here comes the question: I want to do someting like
Filter = " (%Channel% = A AND %Severity% >= warn)
OR (%Channel% = B AND %Severity% >= info)"
I was not able to find any documentation regarding such a combination of filters. Is there a way to do this when using boost::log::init_from_stream?


I've found this documentation page which documents the grammar:
Filter and formatter parsers
filter:
condition { op condition }
op:
&
and
|
or
condition:
!condition
not condition
(filter)
%attribute_name%
%attribute_name% relation operand
relation:
>
<
=
!=
>=
<=
begins_with
ends_with
contains
matches
Using this, the example given in the question can be expressed as follows:
Filter = "(%Channel% = A & %Severity% >= warn) | (%Channel% = B & %Severity% >= info)"

Related

M & Power Query: How to use the $Skip ODATA expression within a loop?

Good afternoon all,
I'm trying to call all of the results within an API that has:
6640 total records
100 records per page
67 pages of results (total records / records per page)
This is an ever growing list so I've used variables to create the above values.
I can obviously use the $Skip ODATA expression to get any one of the 67 pages by adding the expression to the end of the URL like so (which would skip the first 100, therefore returning the 2nd page:
https://psa.pulseway.com/api/servicedesk/tickets/?$Skip=100
What I'm trying to do though is to create a custom function that will loop through each of the 67 calls, changing the $Skip value by an increment of 100 each time.
I thought I'd accomplished the goal with the below code:
let
Token = "Token",
BaseURL = "https://psa.pulseway.com/api/",
Path = "servicedesk/tickets/",
RecordsPerPage = 100,
CountTickets = Json.Document(Web.Contents(BaseURL,[Headers = [Authorization="Bearer " & Token],RelativePath = Path & "count"])),
TotalRecords = CountTickets[TotalRecords],
GetJson = (Url) =>
let Options = [Headers=[ #"Authorization" = "Bearer " & Token ]],
RawData = Web.Contents(Url, Options),
Json = Json.Document(RawData)
in Json,
GetPage = (Index) =>
let Skip = "$Skip=" & Text.From(Index * RecordsPerPage),
URL = BaseURL & Path & "?" & Skip,
Json = GetJson(URL)
in Json,
TotalPages = Number.RoundUp(TotalRecords / RecordsPerPage),
PageIndicies = {0.. TotalPages - 1},
Pages = List.Transform(PageIndicies, each GetPage(_))
in
Pages
I got all happy when it successfully made the 67 API calls and combined the results into a list for me to load in to a Power Query table, however what I'm actually seeing is the first 100 records repeated 67 times.
That tells me that my GetPage custom function which handles the $Skip value isn't changing and is stuck on the first one. To make sure the Skip index was generating them properly I duplicated the query and changed the code to load in the $Skip values and see what they are, expecting them all to be $Skip=0, what I see though is the correct $Skip values as below:
Image showing correct Skip values
It seems everything is working as it should be, only I'm only getting the first page 67 times.
I've made a couple of posts on other community site around this issue before but I realise the problem I was (poorly) describing was far too broad to get any meaningful assistance. I think now I've gotten to the point where I understand what my own code is doing and have really zoomed in to the problem - I just don't know how to fix it when I'm at the final hurdle...
Any help/advice would be massively appreciated. Thank you.
Edit: Updated following #RicardoDiaz answer.
let
// Define base parameters
Filter = "",
Path = "servicedesk/tickets/",
URL = "https://psa.pulseway.com/api/",
Token = "Token",
Limit = "100",
// Build the table based on record start and any filters
GetEntityRaw = (Filter as any, RecordStart as text, Path as text) =>
let
Options = [Headers=[ #"Authorization" = "Bearer " & Token ]],
URLbase = URL & Path & "?bearer=" & Token & "&start=" & RecordStart & "&limit=" & Text.From(Limit),
URLentity = if Filter <> null then URLbase & Filter else URLbase,
Source = Json.Document(Web.Contents(URLentity, Options)),
Result = Source[Result],
toTable = Table.FromList(Result, Splitter.SplitByNothing(), null, null, ExtraValues.Error)
in
toTable,
// Recursively call the build table function
GetEntity = (optional RecordStart as text) as table =>
let
result = GetEntityRaw(Filter, RecordStart, Path),
nextStart = Text.From(Number.From(RecordStart) + Limit),
nextTable = Table.Combine({result, #GetEntity(nextStart)}),
check = try nextTable otherwise result
in
check,
resultTable = GetEntity("0")
in
resultTable

As I couldn't test your code, it's kind of hard to provide you a concrete answer.
Said that, please review the generic code I use to connect to an api and see if you can find where yours is not working
EDIT: Changed api_record_limit type to number (removed the quotation marks)
let
// Define base parameters
api_url_filter = "",
api_entity = "servicedesk/tickets/",
api_url = "https://psa.pulseway.com/api/",
api_token = "Token",
api_record_limit = 500,
// Build the table based on record start and any filters
fx_api_get_entity_raw = (api_url_filter as any, api_record_start as text, api_entity as text) =>
let
api_url_base = api_url & api_entity & "?api_token=" & api_token & "&start=" & api_record_start & "&limit=" & Text.From(api_record_limit),
api_url_entity = if api_url_filter <> null then api_url_base & api_url_filter else api_url_base,
Source = Json.Document(Web.Contents(api_url_entity)),
data = Source[data],
toTable = Table.FromList(data, Splitter.SplitByNothing(), null, null, ExtraValues.Error)
in
toTable,
// Recursively call the build table function
fxGetEntity = (optional api_record_start as text) as table =>
let
result = fx_api_get_entity_raw(api_url_filter, api_record_start, api_entity),
nextStart = Text.From(Number.From(api_record_start) + api_record_limit),
nextTable = Table.Combine({result, #fxGetEntity(nextStart)}),
check = try nextTable otherwise result
in
check,
resultTable = fxGetEntity("0"),
expandColumn = Table.ExpandRecordColumn(
resultTable,
"Column1",
Record.FieldNames(resultTable{0}[Column1]),
List.Transform(Record.FieldNames(resultTable{0}[Column1]), each _)
)
in
expandColumn
QUESTION TO OP:
Regarding this line:
Result = Source[Result],
Does the json return a field called result instead of data?

Q model and or query giving incorrect result

I want perform below SQL query using Q models
SELECT * FROM motor_vehicle_collision.collision_data_collisiondetails
where (numberOfCyclistInjured > 0 or numberOfCyclistKilled > 0) and
(longitude != '' and latitude != '');
for that I have written below query
query = Q(numberOfCyclistInjured__gt=0)
query.add(Q(numberOfCyclistKilled__gt=0), Q.OR)
query.add(~Q(latitude=''), Q.AND)
query.add(~Q(longitude=''), Q.AND)
but still I am getting data having latitude/longitude empty, how should I rectify it?

You can work with:
query = (Q(numberOfCyclistInjured__gt=0) | Q(numberOfCyclistKilled__gt=0))
& ~Q(longitude='')
& ~Q(latitude=''))
This makes it also more explicit how you bind the conditions together.

Django, use complex Q object query string on a record

We have a Q object string similar to
filter_string1 = {'question__startswith':'Who'}
filter_string2 = {'question__startswith':'What'}
filter = Q(**filter_string1) | Q(**filter_string2) && Q(**filter_string1)
This can be used as :
Models.objects.filter(filter)
Can we use this filter to check if a certain record (a single record, instance of a model) satisfy above conditions.
Edited to clear question
if the filter looks something like this:
filter = Q(question__startswith = "Who") | Q(question__startswith = "What")
I want to get below result without actually writing it again. Using just filter and the object.
object = Model.objects.filter(id = 1)
if object.question.startswith("Who") or object.question.startswith("What")
return True
else:
return False
Note: object may not be saved to database yet.

Access the previous record to compare the value in DAX POWER BI

I need to access the previous record of the DTH_REFER_PEDID column to make the IF comparison (DTH_REFER_PEDID-1 <> "A").
That is, I'm reading the index X, I need to compare with the index X-1
Addition_Stats = VAR Atendido_OV = PR_HIST_MOVIM_PEDID[OVITEM_Hist]
VAR linha_anterior2 = CALCULATE(values(PR_HIST_MOVIM_PEDID[STA_ITEM_PEDCL]);filter(PR_HIST_MOVIM_PEDID;EARLIER(PR_HIST_MOVIM_PEDID[DTH_REFER_PEDID])))
Return
if(PR_HIST_MOVIM_PEDID[DTH_REFER_PEDID].[Month]<PR_HIST_MOVIM_PEDID[DAT_MAIOR_PLANE].[Month];"Atraso mês ant";
if(PR_HIST_MOVIM_PEDID[STA_ITEM_PEDCL] = "A" && PR_HIST_MOVIM_PEDID[DTH_REFER_PEDID].[Day]<=PR_HIST_MOVIM_PEDID[DAT_MAIOR_PLANE].[Day];"Atendido no Prazo";
if((PR_HIST_MOVIM_PEDID[STA_ITEM_PEDCL]="P"||PR_HIST_MOVIM_PEDID[STA_ITEM_PEDCL]="L") && PR_HIST_MOVIM_PEDID[DTH_REFER_PEDID].[Day]<= PR_HIST_MOVIM_PEDID[DAT_MAIOR_PLANE].[Day];"Planejado no prazo";
if(PR_HIST_MOVIM_PEDID[STA_ITEM_PEDCL]<>"A" && PR_HIST_MOVIM_PEDID[DTH_REFER_PEDID].[Day]>PR_HIST_MOVIM_PEDID[DAT_MAIOR_PLANE].[Day];"Em atraso";
if(PR_HIST_MOVIM_PEDID[STA_ITEM_PEDCL] = "A"
&& linha_anterior2 <>"A"
&& PR_HIST_MOVIM_PEDID[DTH_REFER_PEDID].[Day]>PR_HIST_MOVIM_PEDID[DAT_MAIOR_PLANE].[Day];"Atend fora Prazo"
;IF((PR_HIST_MOVIM_PEDID[OVITEM_Hist]=Atendido_OV)&&(PR_HIST_MOVIM_PEDID[DTH_REFER_PEDID]>FIRSTDATE(PR_HIST_MOVIM_PEDID[DTH_REFER_PEDID].[Date]));"A retido";"NA")
)
)
)
)
)
//)
The error displayed is: A circular dependency has been detected: PR_HIST_MOVIM_PEDID [Addition_Stats].
How do I compare DTH_REFER_PEDID-1 <> "A"?

An easy way to work with previous or next records is:
Make sure your data is in a table with a primary key (=ID)
Make a query with all the fields as in your table and add one colum with ID+1. (or ID-1)
Make another query with the table and the query mentioned above and make a join between ID and ID+1 (or ID-1). Place all the fields of the table and the 1st query and you end up with all the values in 1 record. This way you can work with the previous or next values.

Regex / subString to extract all matching patterns / groups

I get this as a response to an API hit.
1735 Queries
Taking 1.001303 to 31.856310 seconds to complete
SET timestamp=XXX;
SELECT * FROM ABC_EM WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
38 Queries
Taking 1.007646 to 5.284330 seconds to complete
SET timestamp=XXX;
show slave status;
6 Queries
Taking 1.021271 to 1.959838 seconds to complete
SET timestamp=XXX;
SHOW SLAVE STATUS;
2 Queries
Taking 4.825584, 18.947725 seconds to complete
use marketing;
SET timestamp=XXX;
SELECT * FROM ABC WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
I have extracted this out of the response html and have it as a string now.I need to retrieve values as concisely as possible such that I get a map of values of this format Map(Query -> T1 to T2 seconds) Basically what this is the status of all the slow queries running on MySQL slave server. I am building an alert system over it . So from this entire paragraph in the form of String I need to separate out the queries and save the corresponding time range with them.
1.001303 to 31.856310 is a time range . And against the time range the corresponding query is :
SET timestamp=XXX; SELECT * FROM ABC_EM WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
This information I was hoping to save in a Map in scala. A Map of the form (query:String->timeRange:String)
Another example:
("use marketing; SET timestamp=XXX; SELECT * FROM ABC WHERE last_modified >= 'XXX' AND last_modified xyz ;"->"4.825584 to 18.947725 seconds")
"""###(.)###(.)\n\n(.*)###""".r.findAllIn(reqSlowQueryData).matchData foreach {m => println("group0"+m.group(1)+"next group"+m.group(2)+m.group(3)}
I am using the above statement to extract the the repeating cells to do my manipulations on it later. But it doesnt seem to be working;
THANKS IN ADvance! I know there are several ways to do this but all the ones striking me are inefficient and tedious. I need Scala to do the same! Maybe I can extract recursively using the subString method ?

If you want use scala try this:
val regex = """(\d+).(\d+).*(\d+).(\d+) seconds""".r // extract range
val txt = """
|1735 Queries
|
|Taking 1.001303 to 31.856310 seconds to complete
|
|SET timestamp=XXX; SELECT * FROM ABC_EM WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
|
|38 Queries
|
|Taking 1.007646 to 5.284330 seconds to complete
|
|SET timestamp=XXX; show slave status;
|
|6 Queries
|
|Taking 1.021271 to 1.959838 seconds to complete
|
|SET timestamp=XXX; SHOW SLAVE STATUS;
|
|2 Queries
|
|Taking 4.825584, 18.947725 seconds to complete
|
|use marketing; SET timestamp=XXX; SELECT * FROM ABC WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
""".stripMargin
def logToMap(txt:String) = {
val (_,map) = txt.lines.foldLeft[(Option[String],Map[String,String])]((None,Map.empty)){
(acc,el) =>
val (taking,map) = acc // taking contains range
taking match {
case Some(range) if el.trim.nonEmpty => //Some contains range
(None,map + ( el -> range)) // add to map
case None =>
regex.findFirstIn(el) match { //extract range
case Some(range) => (Some(range),map)
case _ => (None,map)
}
case _ => (taking,map) // probably empty line
}
}
map
}

Modified ajozwik's answer to work for SQL commands over multiple lines :
val regex = """(\d+).(\d+).*(\d+).(\d+) seconds""".r // extract range
def logToMap(txt:String) = {
val (_,map) = txt.lines.foldLeft[(Option[String],Map[String,String])]((None,Map.empty)){
(accumulator,element) =>
val (taking,map) = accumulator
taking match {
case Some(range) if element.trim.nonEmpty=> {
if (element.contains("Queries"))
(None, map)
else
(Some(range),map+(range->(map.getOrElse(range,"")+element)))
}
case None =>
regex.findFirstIn(element) match {
case Some(range) => (Some(range),map)
case _ => (None,map)
}
case _ => (taking,map)
}
}
println(map)
map
}