In Siddhi query, I am importing two stream S1 and S2. If I receive in S1 stream I will insert in event table T1, and when I receive in S2 I will update in the T1 table based on the id, and also I will send the updated values from the table into Output stream O1.
As a part of the requirement, I need to get the content which table T1, which is inserted before 5 min(ie, if a record resides more than 5 min) and send to another output stream O2.
#name('S1')
from S1
select id, srcId, 'null' as msgId, 'INP' as status
insert into StatusTable;
#name('S2')
from S2#window.time(1min) as g join StatusTable[t.status == 'INP'] as t
on ( g.srcId == t.id)
select t.id as id, g.msgId as msgId, 'CMP' as status
update StatusTable on TradeStatusTable.id == id;
#name('Publish')
from S2 as g join StatusTable[t.status == 'CMP'] as t on ( g.srcId == t.id and t.status == 'CMP')
select t.id as id, t.msgId as msgId, t.status as status
insert into O1;
How to add a query in this existing query to fetch the records from TradeStatus table, which receides more than 5 minutes. Since the table cannot be used alone, I need to join it with a stream, how to do this scenario?
String WebAttackSuccess = "" +
"#info(name = 'found_host_charged1') "+
"from ATDEventStream[ rid == 10190001 ]#window.timeBatch(10 sec) as a1 "+
"join ATDEventStream[ rid == 10180004 ]#window.time(10 sec) as a2 on a2.src_ip == a1.src_ip and a2.dst_ip == a1.dst_ip " +
" select UUID() as uuid,1007 as cid,a1.sensor_id as sensor_id,a1.interface_id as interface_id,a1.other_id as other_id,count(a1.uuid) as event_num,min(a1.timestamp) as first_seen,max(a2.timestamp) as last_seen,'' as IOC,a1.dst_ip as victim,a1.src_ip as attacker,a1.uuid as NDE4,sample:sample(a2.uuid) as Sample_NDE4 " +
" insert into found_host_charged1;"+
""+
"#info(name = 'found_host_charged2') "+
"from every a1 = found_host_charged1 " +
"-> a2 = ATDEventStream[dns_answers != ''] "+
"within 5 min "+
"select UUID() as uuid,1008 as cid,a2.sensor_id as sensor_id,a2.interface_id as interface_id,a2.other_id as other_id,count(a2.uuid) as event_num,a1.first_seen as first_seen,max(a2.timestamp) as last_seen,a2.dns_answers as IOC,a2.dst_ip as victim,a2.src_ip as attacker,a1.uuid as NDE5,sample:sample(a2.uuid) as Sample_NDE5 " +
"insert into found_host_charged2; ";
This is part of my work,i use two stream,maybe you can get the data from StatusTable in your second stream.If not yet resolved,you can change StatusTable to S1.
Related
I'm using Google Cloud SDK (command-line) via C# and I want to select the information for the Current Date(today).
The select is working but I'm not able to bring the latest date on the column DATE
Below is the query I'm using:
var table = client.GetTable("projectId", "datasetId", "table");
var sql = $"" +
$"SELECT " +
$"sku, " +
$"FROM {table} " +
$"WHERE DATE=CurrentDate('America/Sao_Paulo') " +
$"LIMIT 10";
Schema: SKU - String
DATE - Timestamp
Try to use CURRENT_DATE instead of CurrentDate
var table = client.GetTable("projectId", "datasetId", "table");
var sql = $"" +
$"SELECT " +
$"sku, " +
$"FROM {table} " +
$"WHERE DATE=CURRENT_DATE('America/Sao_Paulo') " +
$"LIMIT 10";
I am new to Esper and i am working on Storm-Esper collaboration.Through my main class,i send queries to a bolt which contains esper while the esper-bolt sends the tuple which contain the results to a printer bolt.My problem is that,although the result of a query is correct as for the values,the attribute values are not in the correct order.For example,i have a query which selects attributes from a pilot's table: name,surname,airline and i should have the result in the same order.However i get:name,airline,surname.I have tried everything concerning group by and order by.I suppose it must be an Esper's fault when creating the event's map which contains the attributes-values.I paste the main class code and the esper bolt code where the map is processed.Any idea why is that happening is most welcome!
**mainclass**
.addStatements(("insert into pilotStream " +
"select * " +
"from Log.win:time(120 second) A "))
.addStatements(("insert into employeeStream " +
"select * " +
"from Emp.win:time(120 second) A "))
.addStatements(("insert into CombinedEvent "+
"select tick.pilotName as p_name , " +
"tick.pilotSurname as p_surname , " +
"tick.airline as p_airline " +
"from pilotStream.win:time(120 second) as tick, " +
"employeeStream.win:time(120 second) as rom "+
"where tick.airline = rom.employeeAirline "+
))
**espebolt**
Map<String, Object> emap = (Map<String, Object>) newEvent.getUnderlying();
String Event_name = newEvent.getEventType().getName();
//System.out.println(Event_name);
for (Map.Entry<String, Object> entry : emap.entrySet()) {
// String key = entry.getKey();
String val = String.valueOf(entry.getValue()) ;
//System.out.println(key+" :"+val);
//System.out.println(val);
values.add(val);
}
collector.emit(Event_name, toTuple(newEvent, values, false));
values.removeAll(values);
The result should be : source: Esper-Print:2, stream: CombinedEvent, id: {}, [John, Snow, Lufthansa]
Instead,i get:source: Esper-Print:2, stream: CombinedEvent, id: {}, [John, Lufthansa, Snow]
P.S.The toTuple functions simply gets the values of the attributes through the values list of strings and puts them into a tuple which is emitted to printerbolt.In the espebolt code there is some printing in comments which helped me see that the problem is in the map which esper creates internally.
By default Esper generates Map events. This can be changed into object-array events when setting a configuration or with annotations. Map events use "HashMap" and not "LinkedHashMap". The "HashMap" is not ordered when iterating the key-value pairs but takes much less memory. Object-array is ordered. For ordered access to Map events there is the "EventType" that you can get from a statement which returns you the property names in order.
I need to convert a query on excel to a Insert on SQL, I wont to create a function that will work on any query, and Invoke it, so he need to be quite reusable.
let
SQL = (TableX as table) => let
// Get Columns Name abd use on the columns INSERT
SOURCE = TableX,
Names = Table.ColumnNames(SOURCE),
ListN = Table.FromList(Names),
TransposeN = Table.Transpose(ListN),
CombineN = Table.CombineColumns(TransposeN,Table.ColumnNames(TransposeN),Combiner.CombineTextByDelimiter(",", QuoteStyle.None),"M"),
ColunmsAsText = List.Single(CombineN[M]),
// ColumnsasText is all the Columns Name
// This part I cant do I need to format all columns acording to the type and Combine They
COLUNMS = (ListN as text) => let
TYPE = some.function.to.get.type(SOURCE[ListN]),
if TYPE = date then FIELDS = FIELDS & "'" & Date.ToText([ListN], "yyyy-MM-dd") & "'," else
if TYPE = text then FIELDS = FIELDS & "'" & [ListN] & "'," else
if TYPE = number then FIELDS = FIELDS & [ListN] & "," else FIELDS = FIELDS & "#error,"
in
FIELDS
INSERT = "INSERT INTO " & "TABLENAME" & " (" & ColunmsAsText & ") VALUES (" & FIELDS & ");",
NEWCOLUMNADD = Table.AddColumn(SOURCE, "SQL", each INSERT)
in
NEWCOLUMNADD
in
SQL
The problem is the FIELDS variable where, I need to compose a string of all columns, and the function "some.function.to.get.type", and the output 'll be something like this, for each row:
INSERT INTO (Column_name1, Column_name2, Column_name3) value (15, '2017-05-20', 'Text hear');
Tanks for the help.
You want Value.Type
Value.Type("abc") = type text
I am trying to update a row in a table in an SQLite database using C++, but I want to update it relative to its current value.
This is what I have tried so far:
int val=argv[2];
string bal = "UPDATE accounts SET balance = balance + " + argv[1] + "WHERE account_id = " + bal + argv[2];
if (sqlite3_open("bank.db", &db) == SQLITE_OK)
{
sqlite3_prepare( db, balance.c_str(), -1, &stmt, NULL );//preparing the statement
sqlite3_step( stmt );//executing the statement
}
So that the first parameter is the account_id, and the second parameter is the current balance.
However, this does not work. What can I do to have the database successfully update?
Thank you!
EDIT: Sorry for the confusion. The primary situation is having a table with many entries, each with a unique account id. For example, one has an id of 1 with a balance of 5.
If I run this program with the parameters "1 5", the balance should now be 10. If I run it again with "1 7", it should be 17.
You cannot use the + operator to concatenate C-style strings and string literals. A quick and dirty fix:
string bal = string("UPDATE accounts SET balance = balance + ") + argv[1] + string( " WHERE account_id = " ) + argv[2];
I have a table with 10,000 rows and I want to select the first 1000 rows and then select again and this time, the next set of rows, which is 1001-2001.
I am using the BETWEEN clause in order to select the range of values. I can also increment the values. Here is my code:
count = cursor.execute("select count(*) from casa4").fetchone()[0]
ctr = 1
ctr1 = 1000
str1 = ''
while ctr1 <= count:
sql = "SELECT AccountNo FROM ( \
SELECT AccountNo, ROW_NUMBER() OVER (ORDER BY Accountno) rownum \
FROM casa4 ) seq \
WHERE seq.rownum BETWEEN " + str(ctr) + " AND " + str(ctr1) + ""
ctr = ctr1 + 1
ctr1 = ctr1 + 1000
cursor.execute(sql)
sleep(2) #interval in printing of the rows.
for row in cursor:
str1 = str1 + '|'.join(map(str,row)) + '\n'
print "Records:" + str1 #var in storing the fetched rows from database.
print sql #prints the sql statement(str) and I can see that the var, ctr and ctr1 have incremented correctly. The way I want it.
What I want to achieve is using a messaging queue, RabbitMQ, I will send this rows to another database and I want to speed up the process. Selecting all and sending it to the queue returns an error.
The output of the code is that it returns 1-1000 rows correctly on the 1st but, on the 2nd loop, instead of 1001-2001 rows, it returns 1-2001 rows, 1-3001 and so on.. It always starts on 1.
I was able to recreate your issue with both pyodbc and pypyodbc. I also tried using
WITH seq (AccountNo, rownum) AS
(
SELECT AccountNo, ROW_NUMBER() OVER (ORDER BY Accountno) rownum
FROM casa4
)
SELECT AccountNo FROM seq
WHERE rownum BETWEEN 11 AND 20
When I run that in SSMS I just get rows 11 through 20, but when I run it from Python I get all the rows (starting from 1).
The following code does work using pyodbc. It uses a temporary table named #numbered, and might be helpful in your situation since your process looks like it would do all of its work using the same database connection:
import pyodbc
cnxn = pyodbc.connect("DSN=myDb_SQLEXPRESS")
crsr = cnxn.cursor()
sql = """\
CREATE TABLE #numbered (rownum INT PRIMARY KEY, AccountNo VARCHAR(10))
"""
crsr.execute(sql)
cnxn.commit()
sql = """\
INSERT INTO #numbered (rownum, AccountNo)
SELECT
ROW_NUMBER() OVER (ORDER BY Accountno) AS rownum,
AccountNo
FROM casa4
"""
crsr.execute(sql)
cnxn.commit()
sql = "SELECT AccountNo FROM #numbered WHERE rownum BETWEEN ? AND ? ORDER BY rownum"
batchsize = 1000
ctr = 1
while True:
crsr.execute(sql, [ctr, ctr + batchsize - 1])
rows = crsr.fetchall()
if len(rows) == 0:
break
print("-----")
for row in rows:
print(row)
ctr += batchsize
cnxn.close()