Can ioHub be used with Eye Tribe glasses?
Using ioHub, is it possible to detect sacaddic, blinks, and fixations?
I have not used EyeTribe (and don't use ioHub), but we do use Psychopy for eye tracking. And for us we just connect to the eye tracker using tcp/ip communication from Python itself. And the EyeTribe website under the API section says:
If you favor any other programming language the open API relies on the standard TCP/IP protocol. If it can open a socket and parse strings then you’re covered.
And you can certainly do that in Python, so if you want to use these glasses I am sure you can, but it will likely require some work on your part. I had an undergrad who got a Mirametrix tracker talking to Python/Psychopy and he left some of his experience detailed here: https://brittlab.uwaterloo.ca/research-tips/
But make sure that the technical specifications (60 Hz and a 20 ms latency) will be sufficient for your needs. I find with eye trackers what you pay for is what you get. There is another nice open source option with a higher speed camera and very supportive developmen team at http://pupil-labs.com/pupil/
Good luck, Please add a comment later on your experience if you get these to work, because alot of us, like you, are looking for cheaper tracking options.
Related
I'm trying to make a mobile AR application. I want to track the user's handwriting in real time using smartphones while the user writes on paper. That is to track every stroke made by the user.
I know some SDKs and products like ManoMotion and Leap Motion provide relatively precise hand tracking and analysis, but since writing on paper doesn't involve many motions and gestures, I don't think they are suitable.
I have searched online and haven't found any resources for my particular use case. So I would like to ask if there are other resources I should take a look at, or I should rely on some lower level APIs like OpenCV.
mhc, kind of late to answer this. But regarding ManoMotion you need to apply for access to the SDK products through the official website at www.manomotion.com the approval process is manual and they require some information regarding the intention of use in a project so I highly suggest to provide with as much detail as possible.
I did my best to search all topics regarding to SCADA and developing your own C++ desktop application to communicate with PLCs, but could not find any recent, or in my opinion, relevant topics that fit what I needed. If I missed them, a link to them would be very much appreciated. If I also happened to post this in the wrong section, or you can think of a better section for me to post this in, I will take it there.
With that said, I thank you in advance for taking the time to read my questions, and appreciate any input you have to offer.
A little bit about what I'm doing
I'm currently in school for electromechanical engineering, and for my final year project I am developing a desktop application in C++ to monitor PLCs we have located within one of our labs.
Within this lab, I have a pre-existing ethernet network connecting all PLCs to single point, which I am tying into with a PC, and will be doing all my work from there.
I will be developing the application in Qt for an easy way to design the GUI, and giving me access to the QNetworkInterface as well as QTcpSocket.
With that said, I wouldn't go as far as saying I'm an experienced programmer, but I have been fooling around with a few languages (i.e.: python, c++, c, php) for quite a few years, and am still learning, considering the learning NEVER stops.
My questions
Is there any reference material I can read, that you can suggest, on the subject to more easily understand what sort of process I need to go through to receive information (i.e.: individual I/Os, status bits, tags, logs, etc...) from the PLCs directly, and not through an OPC server?
If an OPC server is required, I've never dealt with OPC links other than using Rockwell Automations RSLinx to grab tags and display their values within excel (I had created a prototype using that exact method to start, but would like to move away from excel, and if possible, the OPC server (RSLinx) as well). What would you suggest to someone who knows nothing about the subject of OPC servers, or to my knowledge, OPC in general?
Have any of you previously written your own application to do something similar, if not of the same nature to what I'm trying to accomplish?
What advice or suggestions would you give for someone who is attempting this type of project?
PS: As a start for this project, I would initially just want to get the reading of the I/Os (tags or addresses) to view what their current values are (closed or open for inputs, energized or not for outputs). But eventually I would also like to be able to write values to tags on the PLCs I'm monitoring based on the values I've received from them.
PSS: I would like to note again, that I am still a student, and am still learning about this subject in it's entirety. I would just like to ask for your patience, as I may not grasp things completely the first time!
If I've missed any information you feel is pertinent to be able to provide an answer, please let me know! I will do my best to come up with said information in a timely manner!
Thank you!
EDIT #1: Added in another question, and altered my first question slightly
EDIT #2: Fixed up question 2
IMHO a SCADA program should have as a minimum requirement to be able to connect to an OPC server. OPC is used for most commercial PLCs.
Strictly speaking there is no need to have an OPC server/client approach but it gives you flexibility and gives you an abstraction model. If you want to directly connect to PLCs using a protocol then that is of course possible as well. You then need to know more details about the protocols and their various versions.
Yes I worked for a few years in a team that developed a commercial SCADA application.
It is very easy to get lost in details in such a project so try to keep things as simple as possible. By using OPC you will save time instead of fiddling directly with the protocols. You could add the ability to add custom-drivers for other protocols - depending on your timeframe. Try to model up your project before you start coding to a birdsview of the model and avoid getting lost in the details.
I would stay well away from looking to write your own code to connect directly
to an AB PLC - there are products out there that you can use in your application:
http://www.rtaautomation.com/software/ethernetip/client/tagc/ControlWin.html
http://www.automatedsolutions.com/products/dotnet/ascomm/
You would be better to use OPC - you can write you own OPC client if you want and follow examples you find here:
http://www.opcconnect.com/source.php#freesource
According to this http://www.control.com/thread/1026173407 you should be able to get source code of Kepwares OPC Quick Client.
It would probably be easier to just use a library as in this example (RSLogix,C#):
http://www.mesta-automation.com/opc-client-with-c-an-how-to-video/
You might find this of use:
http://www.rockwellautomation.co.kr/applications/gs/ap/GSKR.nsf/files/rslinxsdk_ma_eng.pdf/$file/rslinxsdk_ma_eng.pdf
Some resources:
http://www.opcconnect.com/ ,
http://www.mesta-automation.com/
Answer to question #4 - realize that your lab technically could contain ANY manufacturer's PLCs in the future. If you ever took a Data Communications class, you realize that for N different PLC types, you would have to write N different communication drivers for your PLC client.
This is where standards are helpful. Without the use of a standard protocol, scaling your lab could become more time consuming and less manageable. This is why communications standards exist.
HOWEVER, not all PLCs necessarily support the standard(s) you may decide upon.
The best choice is OPC/UA. Many PLCs have server drivers readily available. That means that your client just needs to understand 1 protocol (OPC/UA), and then it can "easily" be connected to any PLC that has a driver for that standard.
After that, there is OPC. After that, Modbus (TCP and RTU flavors), a relatively simple industry standard that is supported by most PLCs. EtherNet/IP is also a possible choice, although not all PLCs support it in a "server" role (many do support it as a client, but that is not what you need).
have a look at pycomm in github or pylogix at github which are Python written drivers to link to clx plc.
I've received a dk 3200 kit recently. I know it's old but I would like to start using it to have more of a challenge than just arduino. It came with the board, an st flashlink FL-101B and some cables. I do not have the install disc, but I found the software for psd soft express online. It doesn't work with current windows 7 64bit. If I could use my arduino to program it, that would be great! Or maybe just do it through USB or the parallel ports? I've read that st provides a stm32 library used to help make code. If that works for my mcu, I'll use that.
Thanks
ST is a company that loves to re-organize their website and break links, but a data sheet for a part of what seems to be the same family as on your board is available from a toolchain vendor at:
http://www.keil.com/dd/docs/datashts/st/upsd321x_ds.pdf
On page 118 this seems to indicate that programs can be loaded using JTAG In System Configuration commands, which may be somewhat standardized though quick searching isn't producing a lot of leads. A reference is also made to ST's AN1153 which would be worth trying to locate a copy of, however it's not entirely clear if that will say anything about the programming, or if it only covers the additional two optional signals which have been added for hardware acceleration of the interface.
In theory, if you can find sufficient information about this device (looking at related devices may provide clues) then you should be able to craft a programming from an Arduino or anything else that can be adapted to twiddle bits at the appropriate voltage/threshold levels.
In practice, you may be better off trying to find an old Windows XP box, or even trying to run that software on a virtual/emulated environment and trap the I/O access either to proxy or to figure out what it is doing and reverse engineer your own programmer.
But unless you have a large installed base of boards using these chips, or particular need some unusual feature of them (I thought I saw something about a built-in CPLD?) it's really not going to be worth the effort.
I am really passionate about the machine learning,data mining and computer vision fields and I was thinking at taking things a little bit further.
I was thinking at buying a LEGO Mindstorms NXT 2.0 robot for trying to experiment machine learning/computer vision and robotics algorithms in order to try to understand better several existing concepts.
Would you encourage me into doing so? Do you recommend any other alternative for a practical approach in understanding these fields which is acceptably expensive like(nearly 200 - 250 pounds) ? Are there any mini robots which I can buy and experiment stuff with?
If your interests are machine learning, data mining and computer vision then I'd say a Lego mindstorms is not the best option for you. Not unless you are also interested in robotics/electronics.
Do do interesting machine learning you only need a computer and a problem to solve. Think ai-contest or mlcomp or similar.
Do do interesting data mining you need a computer, a lot of data and a question to answer. If you have an internet connection the amount of data you can get at is only limited by your bandwidth. Think netflix prize, try your hand at collecting and interpreting data from wherever. If you are learning, this is a nice place to start.
As for computer vision: All you need is a computer and images. Depending on the type of problem you find interesting you could do some processing of random webcam images, take all you holiday photo's and try to detect where all your travel companions are in them. If you have a webcam your options are endless.
Lego mindstorms allows you to combine machine learning and computer vision. I'm not sure where the datamining would come in, and you will spend (waste?) time on the robotics/electronics side of things, which you don't list as one of your passions.
Well, I would take a look at the irobot create... well within your budget, and very robust.
Depending on your age, you may not want to be seen with a "lego robot" if you are out of college :-)
Anyway, I buy the creates in batches for my lab. You can link to them with a hard cable(cheap) or put a blue tooth interface on it.
But a webcam on that puppy, hook it up to a multicore machine and you have an awesome working robot for the things you want to explore.
Also, the old roombas had a ttl level serial port (if that did not make sense to you , then skip it). I don't know about the new ones. So, it was possible to control any roomba vacuum from a laptop.
The Number One rule, and I cannot emphasize this enough: Have a reliable platform for experimentation. If you hand build something, just for basic functionality, you will spend all your time on minor issues and not get to the fun stuff.
Anyway. best of luck.
I'm on a project that among other video related tasks should eventually be capable of extracting the audio of a video and apply some kind of speech recognition to it and get a transcribed text of what's said on the video. Ideally it should output some kind of subtitle format so that the text is linked to a certain point on the video.
I was thinking of using the Microsoft Speech API (aka SAPI). But from what I could see it is rather difficult to use. The very few examples that I found for speech recognition (most are for Text-To-Speech which mush easier) didn't perform very well (they don't recognize a thing). For example this one: http://msdn.microsoft.com/en-us/library/ms717071%28v=vs.85%29.aspx
Some examples use something called grammar files that are supposed to define the words that the recognizer is waiting for but since I haven't trained the Windows Speech Recognition thoroughly I think that might be adulterating the results.
So my question is... what's the best tool for something like this? Could you provide both paid and free options? Well the best "free" (as it comes with Windows) option I believe it's SAPI, all the rest should be paid but if they are really good it might be worth it. Also if you have any good tutorials for using SAPI (or other API) on a context similar to this it would be great.
On the whole this is a big ask!
The issue with any speech recognition system is that it functions best after training. It needs context (what words to expect) and some kind of audio benchmark (what does each voice sound like). This might be possible in some cases, such as a TV series if you wanted to churn through hours of speech -separated for each character- to train it. There's a lot of work there though. For something like a film there's probably no hope of training a recogniser unless you can get hold of the actors.
Most film and TV production companies just hire media companies to transcribe the subtitles based on either direct transcription using a human operator, or converting the script. The fact that they still need humans in the loop for these huge operations suggests that automated systems just aren't up to it yet.
In video you have a plethora of things that make you life difficult, pretty much spanning huge swathes of current speech technology research:
-> Multiple speakers -> "Speaker Identification" (can you tell characters apart? Also, subtitles normally have different coloured text for different speakers)
-> Multiple simultaneous speakers -> The "cocktail party problem" - can you separate the two voice components and transcribe both?
-> Background noise -> Can you pick the speech out from any soundtrack/foley/exploding helicopters.
The speech algorithm will need to be extremely robust as different characters can have different gender/accents/emotion. From what I understand of the current state of recognition you might be able to get a single speaker after some training, but asking a single program to nail all of them might be tough!
--
There is no "subtitle" format that I'm aware of. I would suggest saving an image of the text using a font like Tiresias Screenfont that's specifically designed for legibility in these circumstances, and use a lookup table to cross-reference images against video timecode (remembering NTSC/PAL/Cinema use different timing formats).
--
There's a bunch of proprietary speech recognition systems out there. If you want the best you'll probably want to license a solution off one of the big boys like Nuance. If you want to keep things free the universities of RWTH and CMU have put some solutions together. I have no idea how good they are or how well they might be suited to the problem.
--
The only solution I can think of similar to what you're aiming at is the subtitling you can get on news channels here in the UK "Live Closed Captioning". Since it's live, I assume they use some kind of speech recognition system trained to the reader (although it might not be trained, I'm not sure). It's got better over the past few years, but on the whole it's still pretty poor. The biggest thing it seems to struggle with is speed. Dialogue is normally really fast, so live subtitling has the extra issue of getting everything done in time. Live closed captions quite frequently get left behind and have to miss a lot of content out to catch up.
Whether you have to deal with this depends on whether you'll be subtitling "live" video or if you can pre-process it. To deal with all the additional complications above I assume you'll need to pre-process it.
--
As much as I hate citing the big W there's a goldmine of useful links here!
Good luck :)
This falls into the category of dictation, which is a very large vocabulary task. Products like Dragon Naturally Speaking are amazingly good and that has a SAPI interface for developers. But it's not so simple of a problem.
Normally a dictation product is meant to be single speaker and the best products adapt automatically to that speaker, thereby improving the underlying acoustic model. They also have sophisticated language modeling which serves to constrain the problem at any given moment by limiting what is known as the perplexity of the vocabulary. That's a fancy way of saying the system is figuring out what you're talking about and therefore what types of words and phrases are likely or not likely to come next.
It would be interesting though to apply a really good dictation system to your recordings and see how well it does. My suggestion for a paid system would be to get Dragon Naturally Speaking from Nuance and get the developer API. I believe that provides a SAPI interface, which has the benefit of allowing you to swap in the Microsoft speech or any other ASR engine that supports SAPI. IBM would be another vendor to look at but I don't think you will do much better than Dragon.
But it won't work well! After all the work of integrating the ASR engine, what you will probably find is that you get a pretty high error rate (maybe half). That would be due to a few major challenges in this task:
1) multiple speakers, which will degrade the acoustic model and adaptation.
2) background music and sound effects.
3) mixed speech - people talking over each other.
4) lack of a good language model for the task.
For 1) if you had a way of separating each actor on a separate track that would be ideal. But there's no reliable way of separating speakers automatically in a way that would be good enough for a speech recognizer. If each speaker were at a distinctly different pitch, you could try pitch detection (some free software out there for that) and separate based on that, but this is a sophisticated and error prone task.) The best thing would be hand editing the speakers apart, but you might as well just manually transcribe the speech at that point! If you could get the actors on separate tracks, you would need to run the ASR using different user profiles.
For music (2) you'd either have to hope for the best or try to filter it out. Speech is more bandlimited than music so you could try a bandpass filter that attenuates everything except the voice band. You would want to experiment with the cutoffs but I would guess 100Hz to 2-3KHz would keep the speech intelligible.
For (3), there's no solution. The ASR engine should return confidence scores so at best I would say if you can tag low scores, you could then go back and manually transcribe those bits of speech.
(4) is a sophisticated task for a speech scientist. Your best bet would be to search for an existing language model made for the topic of the movie. Talk to Nuance or IBM, actually. Maybe they could point you in the right direction.
Hope this helps.