Based on what I read on documentation and source code, WSO2CEP works as a passive software, so it only does something if a stream event arrives on it. So if I want to develop some "health check" on the stream, I need to implement it by myself. My question is, does anyone have a first step on how to do it? I am a little lost on where to start. Do I use InputEventAdapterFactory or TimeWindowProcessor? My scenario is "simple", if a stream does not arrive on a x time, I need to create a alert.
Thanks,
Marcelo Sabino
This concept will work.
#Plan:name('ExecutionPlan')
#Import('rss.in:1.0.0')
define stream inStream (meta_title string, meta_description string, meta_url string);
define trigger ThirtySecTriggerStream at every 30 sec;
from inStream#window.length(1)
select meta_title, time:currentTimestamp() as meta_time
insert into table1;
from ThirtySecTriggerStream join table1
select meta_title, meta_time
insert into tempStream;
from ThirtySecTriggerStream[time:timestampInMilliseconds(meta_time)-time:timestampInMilliseconds(time:curretTimestamp())>30000]
select meta_title
insert into OutStream
Related
I have a LTTNg trace, which i am parsing using babeltrace API. So i was wondering if I could count all events in trace (or stream) without iterating over them. What functions from publilc API I can use to do that ?
The very nature of CTF makes it impossible to count the event records of a given packet in constant time. The packet's context could include an event record count field somehow, but it's not specified, so generic tools would not use it.
Thus the only way to count events is to iterate the event records, unfortunately. The easiest way is to count the number of lines that the text format of the babeltrace(1) tool prints:
babeltrace /path/to/ctf/trace/directory | wc --lines
This works as long as there's one line per printed event record, which is the case unless an event record contains a string field which has a newline (currently not escaped in the text output).
You may also wish to consider discarded event records. They are not printed to the standard output by babeltrace(1), but the tool prints a message including the count to the standard error when they are detected.
There's no way with the current babeltrace(1) tool to only print the event records which belong to the packets of a given data stream. If you need this, what I suggest is that you remove all the data stream files except the one for which you need an event record count, and run the command above again.
Also consider the Babeltrace Python bindings, for example (not tested):
import babeltrace
def count_ctf_event_records(path):
trace_collection = babeltrace.TraceCollection()
trace_collection.add_trace(path, 'ctf')
return sum(1 for event in trace_collection.events)
if __name__ == '__main__':
import sys
print(count_ctf_event_records(sys.argv[1]))
Saved as count.py, you can try this:
python3 count.py /path/to/ctf/trace/directory
Counting the event records of a specific data stream with the Python bindings is left as an exercise for the reader.
Having said this, I don't know if the Python bindings approach is faster than the babeltrace(1) one.
for example, that is data:
1,1470732420000,0
2,1470732421000,0
3,1470732422000,0
4,1470732423000,86
5,1470732424000,87
6,1470732425000,88
7,1470732426000,84
8,1470732427000,0
9,1470732428000,0
10,1470732429000,0
11,1470732430000,89
12,1470732431000,89
13,1470732432000,87
14,1470732433000,89
15,1470732434000,85
16,1470732435000,89
17,1470732436000,89
18,1470732437000,87
19,1470732438000,86
20,1470732439000,88
21,1470732440000,0
22,1470732441000,0
23,1470732442000,0
24,1470732443000,87
25,1470732444000,85
26,1470732445000,86
27,1470732446000,0
28,1470732447000,0
29,1470732448000,0
30,1470732449000,0
column one is id,column two is timestamp,column three is value,1 sec interval between the timestamp.
i want monitoring the value of event,if i found out value>=85(e.g. id=4), i will starting counting,if the next two consecutive value>=85(e.g. id=5/id=6),then i will put the third value of event to OutputStream.(e.g. id=6,value=88,timestamp=1470732425000)
at the same time i clear the counting and wait value lower than 85(e.g. id=7,value=84), then i will monitoring again,when i found out value>=85(e.g. id=11,value=89) i will starting counting,if the next two consecutive value>=85(e.g. id=12/id=13),then i will put the third value of event to OutputStream.(e.g. id=13,value=87,timestamp=1470732432000)...
all this is i wanna do,before i post this ask, i've got an answer in this post,i've tried this code:
from every a1=InputStream[value>=85], a2=InputStream[value>=85]+, a3=InputStream[value<85]
select a2[1].id, a2[1].value
having (not (a2[1] is null))
insert into OutPutStream;
and it works,but i found out it will insert the value into OutputStream after the value<=85,and what i want is if i got three consecutive value>=85 then i insert into the value immediately.(i don't want to wait if the next value>=85 all the times)
in fact, i just wanna record value of third seconds in three consecutive seconds value(>=85) .
i'm using wso2das-3.1.0-SNAPSHOT.
Though DAS (Siddhi) supports sequence/pattern processing, for your requirement you might need to write a custom extension. I have written a sample window processor extension to cater your requirement (source code). Download and place siddhi-extension-condition-window-1.0.jar in <das_home>/repository/components/lib/ directory and restart the server. Refer to the test case to get an idea of the usage of the extension.
I just want to know that how to retrieve all the record from universe database table using universe basic subroutine.I am new in universe.
Perhaps something like this in the unibasic
OPEN "filename" to FIL ELSE STOP 201,"cannot open filename"
EXECUTE "SELECT filename"
LOOP WHILE READNEXT ID
READ REC FROM FIL,ID ELSE REC = ""
* you now have the entire row in REC
REPEAT
Can you provide more information on what you are trying to do?
Having a subroutine call return the entire contents of a UniVerse file could return a large amount of data. I would expect you would be better off only returning a subset of the items so the calling routine can process a bit at a time.
New content based on comment:
Ok, since you mentioned a type 19 file, I assume you want to read one file from the directory/folder the file points to.
In your subroutine, you can use OPEN on the type 19 file, and use the READ command to read the file. ( Note that you can also use READU, READL, MATREAD, MATREADU, or MATREADL to get the entire file in the directory/folder, depending on if/how you want to lock the item and if you want the data in a dynamic or dimensioned array. If you only need a specific attribute you can then use READV, READVL or READVU commands.
Or, since this is a type 19 file, you can use sequential reads. Open the file with OPENSEQ and read with the READSEQ or READBLK command.
There is an article and sample code on GitHub on how to execute U2 UniVerse Subroutine.
Execute Rocket MV U2 Subroutine Asynchronously using C# (async\await) and U2 Toolkit for .NET. Convert Subroutine Multi-Value Output to Json/Objects/DataTable
These sample code are based on C# (async\await), but you can use for synchronous programming as well with little code tweak.
For article:
Go to this link :
https://github.com/RocketSoftware/multivalue-lab/tree/master/U2/Demos/U2-Toolkit/AsyncAwait/Execute_Subroutine_Async
Read ‘Subroutine-Async.docx’ file.
Sample Code for this article on GitHub
Go to this link :
https://github.com/RocketSoftware/multivalue-lab/tree/master/U2/Demos/U2-Toolkit/AsyncAwait/Execute_Subroutine_Async
OPEN '',FILENAME TO F.FILE ELSE STOP
SELECT F.FILE
LOOP
READNEXT K.FILE ELSE EXIT
READ R.FILE FROM F.FILE, K.FILE ELSE NULL
PRINT R.FILE
REPEAT
PRINT "All over Red Rover"
Filename should be in quotes, i.e "MYFILE" or 'MYFILE'
The loop will repeat till all records have been read and will then exit.
REPEAT With FRAME:
prompt-for IN-SCAN3.scan.
if input IN-SCAN3.scan="" then Do:
Message "please input date.".
undo,retry.
end.
else DO:
FIND FIRST in-scan3 USING IN-SCAN3.scan NO-LOCK NO-WAIT NO-ERROR.
if avail In-scan3 then DO:
str="OK".
display str.
next-prompt IN-SCAN3.scan.
end.
else DO:
CREATE In-scan3.
ASSIGN IN-scan3.scan=INPUT in-scan3.scan.
str="NO". DISPLAY str.
next-prompt In-scan3.scan.
END.
end.
begin=begin + 1.
end.
Question desc:
There are 20 users using scanning at the same time,first find input data, if not found, then create one record in the database. The question is, at the same time operating will appear dead lock.
I try NO-LOCK NO-WAIT with record when find,operating will appear dead lock when create a record.
thanks any answer.
You have unfortunately fallen foul of ABL's basic "gotcha" -- record locking will default to SHARE-LOCK even if you specify otherwise, and this is probably not what you want.
The basic rule is that if your transaction scope is smaller than your record scope then the record will drop back to a SHARE-LOCK when you leave the transaction. But I recommend you read the relevant chapter in the ABL guide.
There are a number of ways to fix this. The RELEASE keyword is one. But I tend to favour the idea that you should use a separate buffer for actually locking the record. That way you make things very clear to other programmers.
For example:
def buffer b-in-scan3 for in-scan3.
repeat:
prompt-for in-scan3.scan.
/*** etc. ***/
else
do for b-in-scan3 transaction:
find first b-in-scan3 using in-scan3.scan
exclusive no-wait no-error.
if not avail b-in-scan3
and not locked b-in-scan3
then
do:
create b-in-scan3.
b-in-scan3.scan = input in-scan3.scan.
end.
end. /* of transaction */
end. /* of repeat */
This way if another programmer uses b-inscan3 outside of your transaction block, the program will fail to compile rather than start dropping back to SHARE-LOCK.
Remember that you can check LOCKED as well as AVAILABLE, but if a record is locked then it's not available.
I'm stuck on problem were I would like to ask for some help:
I have the task to print some files of different types using ShellExecuteEx with the "print" verb and need to guarantee print order of all files. Therefore I use FindFirstPrinterChangeNotification and FindNextPrinterChangeNotification to monitor the events PRINTER_CHANGE_ADD_JOB and PRINTER_CHANGE_DELETE_JOB using two different threads in the background which I start before calling ShellExecuteEx as I don't know anything about the application which will print the files etc. The only thing I know is that I'm the only one printing and which file I print. My solution seems to work well, my program successfully recognizes the event PRINTER_CHANGE_ADD_JOB for my file, I even verify that this event is issued for my file by checking what is give to me as additional info by specifying JOB_NOTIFY_FIELD_DOCUMENT.
The problem now is with the event PRINTER_CHANGE_DELETE_JOB, where I don't get any addition info about the print job, though my logic is exactly the same for both events: I've written one generic thread function which simply gets executed with the event it is used for. My thread is recognizing the PRINTER_CHANGE_DELETE_JOB event, but on each call to FindNextPrinterChangeNotification whenever this event occured I don't get any addition data in ppPrinterNotifyInfo. This works for the start event, though, I verified using my logs and the debugger. But with PRINTER_CHANGE_DELETE_JOB the only thing I get is NULL.
I already searched the web and there are some similar questions, but most of the time related to VB or simply unanswered. I'm using a C++ project and as my code works for the ADD_JOB-event I don't think I'm doing something completely wrong. But even MSDN doesn't mention this behavior and I would really like to make sure that the DELETE_JOB event is the one for my document, which I can't without any information about the print job. After I get the DELETE_JOB event my code doesn't even recognize other events, which is OK because the print job is done afterwards.
The following is what I think is the relevant notification code:
WORD jobNotifyFields[1] = {JOB_NOTIFY_FIELD_DOCUMENT};
PRINTER_NOTIFY_OPTIONS_TYPE pnot[1] = {JOB_NOTIFY_TYPE, 0, 0, 0, 1, jobNotifyFields};
PRINTER_NOTIFY_OPTIONS pno = {2, 0, 1, pnot};
HANDLE defaultPrinter = PrintWaiter::openDefaultPrinter();
HANDLE changeNotification = FindFirstPrinterChangeNotification( defaultPrinter,
threadArgs->event,
0, &pno);
[...]
DWORD waitResult = WAIT_FAILED;
while ((waitResult = WaitForSingleObject(changeNotification, threadArgs->wfsoTimeout)) == WAIT_OBJECT_0)
{
LOG4CXX_DEBUG(logger, L"Irgendein Druckereignis im Thread zum Warten auf Ereignis " << LogStringConv(threadArgs->event) << L" erkannt.");
[...]
PPRINTER_NOTIFY_INFO notifyInfo = NULL;
DWORD events = 0;
FindNextPrinterChangeNotification(changeNotification, &events, NULL, (LPVOID*) ¬ifyInfo);
if (!(events & threadArgs->event) || !notifyInfo || !notifyInfo->Count)
{
LOG4CXX_DEBUG(logger, L"unpassendes Ereignis " << LogStringConv(events) << L" ignoriert");
FreePrinterNotifyInfo(notifyInfo);
continue;
}
[...]
I would really appreciate if anyone could give some hints on why I don't get any data regarding the print job. Thanks!
https://forums.embarcadero.com/thread.jspa?threadID=86657&stqc=true
Here's what I think is going on:
I observe two events in two different threads for the start and end of each print job. With some debugging and logging I recognized that FindNextPrinterChangeNotification doesn't always return only the two distinct events I've notified for, but some 0-events in general. In those cases FindNextPrinterChangeNotification returns 0 as the events in pdwChange. If I print a simple text file using notepad.exe I only get one event for creation of the print job with value 256 for pdwChange and the data I need in notifyInfo to compare my printed file name against and comparing both succeeds. If I print a pdf file using current Acrobat Reader 11 I get two events, one has pdwChange as 256, but gives something like "local printdatafile" as the name of the print job started, which is obviously not the file I printed. The second event has a pdwChange of 0, but the name of the print job provided in notifyInfo is the file name I used to print. As I use FreePDF for testing pruproses, I think the first printer event is something internal to my special setup.
The notifications for the deletion of a print job create 0 events, too. This time those are sent before FindNextPrinterChangeNotification returns 1024 in pdwChange, and timely very close after the start of the print job. In this case the exactly one generated 0 event contains notifyInfo with a document name which equals the file name I started printing. After the 0 event there's exactly one additional event with pdwChange of 1024, but without any data for notifyInfo.
I think Windows is using some mechanism which provides additional notifications for the same event as 0 events after the initial event has been fired with it's real value the user notified with, e.g. 256 for PRINTER_CHANGE_ADD_JOB. On the other hand it seems that some 0 events are simply fired to provide data for an upcoming event which then gets the real value of e.g. 1024 for PRINTER_CHANGE_DELETE_JOB, but without anymore data because that has already been delivered to the event consumer with a very early 0 event. Something like "Look, there's more for the last events." and "Look, something is going to happen with the data I already provide now." Implementing such an approach my prints now seem to work as expected.
Of course what I wrote doesn't fit to what is documented for FindNextPrinterChangeNotification, but it makes a bit of sense to me. ;-)
You're not checking for overflows or errors.
The documentation for FindNextPrinterChangeNotification says this:
If the PRINTER_NOTIFY_INFO_DISCARDED bit is set in the Flags member of
the PRINTER_NOTIFY_INFO structure, an overflow or error occurred, and
notifications may have been lost. In this case, no additional
notifications will be sent until you make a second
FindNextPrinterChangeNotification call that specifies
PRINTER_NOTIFY_OPTIONS_REFRESH.
You need to check for that flag and do as described above, and you should also be checking the return code from FindNextPrinterChangeNotification.