I am new to Esper and i am working on Storm-Esper collaboration.Through my main class,i send queries to a bolt which contains esper while the esper-bolt sends the tuple which contain the results to a printer bolt.My problem is that,although the result of a query is correct as for the values,the attribute values are not in the correct order.For example,i have a query which selects attributes from a pilot's table: name,surname,airline and i should have the result in the same order.However i get:name,airline,surname.I have tried everything concerning group by and order by.I suppose it must be an Esper's fault when creating the event's map which contains the attributes-values.I paste the main class code and the esper bolt code where the map is processed.Any idea why is that happening is most welcome!
**mainclass**
.addStatements(("insert into pilotStream " +
"select * " +
"from Log.win:time(120 second) A "))
.addStatements(("insert into employeeStream " +
"select * " +
"from Emp.win:time(120 second) A "))
.addStatements(("insert into CombinedEvent "+
"select tick.pilotName as p_name , " +
"tick.pilotSurname as p_surname , " +
"tick.airline as p_airline " +
"from pilotStream.win:time(120 second) as tick, " +
"employeeStream.win:time(120 second) as rom "+
"where tick.airline = rom.employeeAirline "+
))
**espebolt**
Map<String, Object> emap = (Map<String, Object>) newEvent.getUnderlying();
String Event_name = newEvent.getEventType().getName();
//System.out.println(Event_name);
for (Map.Entry<String, Object> entry : emap.entrySet()) {
// String key = entry.getKey();
String val = String.valueOf(entry.getValue()) ;
//System.out.println(key+" :"+val);
//System.out.println(val);
values.add(val);
}
collector.emit(Event_name, toTuple(newEvent, values, false));
values.removeAll(values);
The result should be : source: Esper-Print:2, stream: CombinedEvent, id: {}, [John, Snow, Lufthansa]
Instead,i get:source: Esper-Print:2, stream: CombinedEvent, id: {}, [John, Lufthansa, Snow]
P.S.The toTuple functions simply gets the values of the attributes through the values list of strings and puts them into a tuple which is emitted to printerbolt.In the espebolt code there is some printing in comments which helped me see that the problem is in the map which esper creates internally.
By default Esper generates Map events. This can be changed into object-array events when setting a configuration or with annotations. Map events use "HashMap" and not "LinkedHashMap". The "HashMap" is not ordered when iterating the key-value pairs but takes much less memory. Object-array is ordered. For ordered access to Map events there is the "EventType" that you can get from a statement which returns you the property names in order.
Related
I am trying to only query data that has a number > 0 in Column K within this complex string.
=QUERY('Form Responses 1'!A1:K,"SELECT *" & if(COUNTBLANK('Helper Formulas'!A1:A3)=3,""," WHERE " & TEXTJOIN(" AND ",TRUE,'Helper Formulas'!A1:A3)),1)
I tried...
=QUERY('Form Responses 1'!A1:K,"SELECT * WHERE K > 0" & if(COUNTBLANK('Helper Formulas'!A1:A3)=3,""," WHERE " & TEXTJOIN(" AND ",TRUE,'Helper Formulas'!A1:A3)),1)
But keep getting an error.
Not sure what I need to do.
try:
=QUERY('Form Responses 1'!A1:K,
"where K > 0"&IF(COUNTBLANK('Helper Formulas'!A1:A3)=3,,
" and "&TEXTJOIN(" and ", 1, 'Helper Formulas'!A1:A3)), 1)
The second formula has two WHEREs. Replace the second one with AND, something like:
=QUERY('Form Responses 1'!A1:K,"SELECT * WHERE K > 0" & if(COUNTBLANK('Helper Formulas'!A1:A3)=3,""," AND " & TEXTJOIN(" AND ",TRUE,'Helper Formulas'!A1:A3)),1)
You may need to tweak this if you end up with too many "AND"s.
When troubleshooting this sort of thing it's often helpful to just display the string in the cell (i.e. just the bit inside the query() function). That would have shown up the error quite quickly.
In Power BI Desktop i have a table from an excel file and i want to split a row based on a division between the value of a specific column and a default number.
In more details lets assume tha we have a table like this :
if the default value we want to devide column Amount is 50,then the desirable result would be something like that :
Do you have any idea how can i implement that in Power query editor or with dax?
Thanks
Tested this in Power Query for Excel, but hopefully should work for you in Power BI too. If you create a function like:
divisionToList = (numberToDivide as number, numberToDivideBy as number) as list =>
let
divisionResult = numberToDivide / numberToDivideBy,
isResultValid = (divisionResult >= 0) and (Number.Mod(divisionResult, 1) = 0),
errorIfInvalid = Error.Record("Cannot create a list with " & Text.From(divisionResult) & " items", Number.ToText(numberToDivide) & " / " & Number.ToText(numberToDivideBy) & " = " & Text.From(divisionResult), null),
listOrError = if isResultValid then List.Repeat({divisionResult}, divisionResult) else error errorIfInvalid
in listOrError,
It should divide two numbers and return a list of length d in which each element is d (d is the result of the division). This list can then, in the context of a table, be expanded into new rows.
There is some basic error handling in the function for cases where the division yields a problematic number (since you can't have a list with, for example, 5.1 elements or -1 elements). You can change/remove this handling if necessary.
I think this code below takes me from your first image to your second image -- and hopefully will give you some idea on how to go about achieving this.
let
mockData = Table.FromColumns({{200, 400}, {"A", "B"}}, type table [Amount = number, Description = text]),
defaultValue = 50, // Not sure what logic is required for arriving at this figure, so have simply assigned it.
divisionToList = (numberToDivide as number, numberToDivideBy as number) as list =>
let
divisionResult = numberToDivide / numberToDivideBy,
isResultValid = (divisionResult >= 0) and (Number.Mod(divisionResult, 1) = 0),
errorIfInvalid = Error.Record("Cannot create a list with " & Text.From(divisionResult) & " items", Number.ToText(numberToDivide) & " / " & Number.ToText(numberToDivideBy) & " = " & Text.From(divisionResult), null),
listOrError = if isResultValid then List.Repeat({divisionResult}, divisionResult) else error errorIfInvalid
in listOrError,
invokeFunction = Table.TransformColumns(mockData, {{"Amount", each divisionToList(_, defaultValue), type list}}),
expanded = Table.ExpandListColumn(invokeFunction, "Amount")
in
expanded
I know the title is a mouth-full - sorry about that but trying to be specific here.
DB: MySql (technically Maria)
ColdFusion (technically Lucee: 5.x)
The array looks like the following:
NOTE: the outter most array only shows part of 2 and could continue through into the 30's.
I'm looking to perform a loop over the array to insert the elements marked as "string" in the image into the database using one query. Query has been trimmed for the sake of clarity and conciseness:
for (outer = 1; outer <= ArrayLen(myArray); outer++) {
local.currentrow = local.currentrow + 1;
for (inner = 1; inner <= ArrayLen(myArray[outer]); inner++) {
local.sql = "
INSERT INTO table (uuid, typeID, menuId, activityID, userID)
VALUES (
'#local.uuid#',
#myArray[outer][inner][1]#,
#myArray[outer][inner][2]#,
#myArray[outer][inner][3]#,
#arguments.formDataStruct.userID#
)";
queryExecute(local.sql);
}
}
I'm looking for something along this line but as written, it isn't working:
local.sql = "
INSERT INTO table (uuid, typeID, menuId, activityID, userID)
VALUES (
if (local.currentrow gt 1) {
,
}
for (outer = 1; outer <= ArrayLen(myArray); outer++) {
local.currentrow = local.currentrow + 1;
for (inner = 1; inner <= ArrayLen(myArray[outer]); inner++) {
'#local.uuid#',
'#myArray[outer][inner][1]#',
'#myArray[outer][inner][2]#',
'#myArray[outer][inner][3]#',
'#arguments.formDataStruct.userID#'
}
})
";
queryExecute(local.sql);
The error message I'm getting is
Element at position [1] doesn't exist in array
but if I perform a writedump[1][3][3] (e.g.), I'll get the value 24.
I would recommend against looping over an INSERT statement and rather just loop over VALUES to generate a single INSERT statement. A single INSERT will perform significantly faster, plus it will minimize the connections to your database.
Build out the list of values with something like:
for (var outer in arguments.inArray) {
for (var inner in outer) {
// Concat elements of inner array to a SQL VALUE string. If UUID is supposed to be a unique identity for the row, use Maria's uuid() instead of CF (or skip the UUID insert and let Maria do it).
// inArray elements and inUserID should be sanitized.
local.values &= "( uuid(), '" & inner[1] & "','" & inner[2] & "','" & inner[3] & "'," & local.userID & ")," ;
}
}
local.values = left(local.values,len(local.values)-1) ; // Get rid of the last comma.
local.sql = "INSERT INTO table (uuid, typeID, menuId, activityID, userID) VALUES " & local.values ;
After you've built up the SQL INSERT string, execute the query to INSERT the records. (You would probably build the above function differently to handle building the query string and parameters and then executing it all in one place.)
Don't forget to sanitize your array and other inputs. Does the array come from a source you control or is it user input?
https://trycf.com/gist/7ad6af1e84906b601834b0cc5ff5a392/lucee5?theme=monokai
http://dbfiddle.uk/?rdbms=mariadb_10.2&fiddle=d11f45f30723ba910c58a1e3f7a7c30b
In Siddhi query, I am importing two stream S1 and S2. If I receive in S1 stream I will insert in event table T1, and when I receive in S2 I will update in the T1 table based on the id, and also I will send the updated values from the table into Output stream O1.
As a part of the requirement, I need to get the content which table T1, which is inserted before 5 min(ie, if a record resides more than 5 min) and send to another output stream O2.
#name('S1')
from S1
select id, srcId, 'null' as msgId, 'INP' as status
insert into StatusTable;
#name('S2')
from S2#window.time(1min) as g join StatusTable[t.status == 'INP'] as t
on ( g.srcId == t.id)
select t.id as id, g.msgId as msgId, 'CMP' as status
update StatusTable on TradeStatusTable.id == id;
#name('Publish')
from S2 as g join StatusTable[t.status == 'CMP'] as t on ( g.srcId == t.id and t.status == 'CMP')
select t.id as id, t.msgId as msgId, t.status as status
insert into O1;
How to add a query in this existing query to fetch the records from TradeStatus table, which receides more than 5 minutes. Since the table cannot be used alone, I need to join it with a stream, how to do this scenario?
String WebAttackSuccess = "" +
"#info(name = 'found_host_charged1') "+
"from ATDEventStream[ rid == 10190001 ]#window.timeBatch(10 sec) as a1 "+
"join ATDEventStream[ rid == 10180004 ]#window.time(10 sec) as a2 on a2.src_ip == a1.src_ip and a2.dst_ip == a1.dst_ip " +
" select UUID() as uuid,1007 as cid,a1.sensor_id as sensor_id,a1.interface_id as interface_id,a1.other_id as other_id,count(a1.uuid) as event_num,min(a1.timestamp) as first_seen,max(a2.timestamp) as last_seen,'' as IOC,a1.dst_ip as victim,a1.src_ip as attacker,a1.uuid as NDE4,sample:sample(a2.uuid) as Sample_NDE4 " +
" insert into found_host_charged1;"+
""+
"#info(name = 'found_host_charged2') "+
"from every a1 = found_host_charged1 " +
"-> a2 = ATDEventStream[dns_answers != ''] "+
"within 5 min "+
"select UUID() as uuid,1008 as cid,a2.sensor_id as sensor_id,a2.interface_id as interface_id,a2.other_id as other_id,count(a2.uuid) as event_num,a1.first_seen as first_seen,max(a2.timestamp) as last_seen,a2.dns_answers as IOC,a2.dst_ip as victim,a2.src_ip as attacker,a1.uuid as NDE5,sample:sample(a2.uuid) as Sample_NDE5 " +
"insert into found_host_charged2; ";
This is part of my work,i use two stream,maybe you can get the data from StatusTable in your second stream.If not yet resolved,you can change StatusTable to S1.
I am seeing a problem with some Scala 2.7.7 code I'm working on, that should not happen if it the equivalent was written in Java. Loosely, the code goes creates a bunch of card players and assigns them to tables.
class Player(val playerNumber : Int)
class Table (val tableNumber : Int) {
var players : List[Player] = List()
def registerPlayer(player : Player) {
println("Registering player " + player.playerNumber + " on table " + tableNumber)
players = player :: players
}
}
object PlayerRegistrar {
def assignPlayersToTables(playSamplesToExecute : Int, playersPerTable:Int) = {
val numTables = playSamplesToExecute / playersPerTable
val tables = (1 to numTables).map(new Table(_))
assert(tables.size == numTables)
(0 until playSamplesToExecute).foreach {playSample =>
val tableNumber : Int = playSample % numTables
tables(tableNumber).registerPlayer(new Player(playSample))
}
tables
}
}
The PlayerRegistrar assigns a number of players between tables. First, it works out how many tables it will need to break up the players between and creates a List of them.
Then in the second part of the code, it works out which table a player should be assigned to, pulls that table from the list and registers a new player on that table.
The list of players on a table is a var, and is overwritten each time registerPlayer() is called. I have checked that this works correctly through a simple TestNG test:
#Test def testRegisterPlayer_multiplePlayers() {
val table = new Table(1)
(1 to 10).foreach { playerNumber =>
val player = new Player(playerNumber)
table.registerPlayer(player)
assert(table.players.contains(player))
assert(table.players.length == playerNumber)
}
}
I then test the table assignment:
#Test def testAssignPlayerToTables_1table() = {
val tables = PlayerRegistrar.assignPlayersToTables(10, 10)
assertEquals(tables.length, 1)
assertEquals(tables(0).players.length, 10)
}
The test fails with "expected:<10> but was:<0>". I've been scratching my head, but can't work out why registerPlayer() isn't mutating the table in the list. Any help would be appreciated.
The reason is that in the assignPlayersToTables method, you are creating a new Table object. You can confirm this by adding some debugging into the loop:
val tableNumber : Int = playSample % numTables
println(tables(tableNumber))
tables(tableNumber).registerPlayer(new Player(playSample))
Yielding something like:
Main$$anon$1$Table#5c73a7ab
Registering player 0 on table 1
Main$$anon$1$Table#21f8c6df
Registering player 1 on table 1
Main$$anon$1$Table#53c86be5
Registering player 2 on table 1
Note how the memory address of the table is different for each call.
The reason for this behaviour is that a Range is non-strict in Scala (until Scala 2.8, anyway). This means that the call to the range is not evaluated until it's needed. So you think you're getting back a list of Table objects, but actually you're getting back a range which is evaluated (instantiating a new Table object) each time you call it. Again, you can confirm this by adding some debugging:
val tables = (1 to numTables).map(new Table(_))
println(tables)
Which gives you:
RangeM(Main$$anon$1$Table#5492bbba)
To do what you want, add a toList to the end:
val tables = (1 to numTables).map(new Table(_)).toList
val tables = (1 to numTables).map(new Table(_))
This line seems to be causing all the trouble - mapping over 1 to n gives you a RandomAccessSeq.Projection, and to be honest, I don't know how exactly they work, but a bit less clever initialising technique does the job.
var tables: Array[Table] = new Array(numTables)
for (i <- 0 to numTables) tables(i) = new Table(i)
Using the first initialisation method I wasn't able to change the objects (just like you), but using a simple array everything seems to be working.