I am using libcurl, currently the easy api. I am making a request to a web server that responds with HTTP Chunked Encoding. I would like to know if there is a way to know when a chunk from the server has finished. I was looking for some type of callback. DEBUGDATA didn't seem to include it, and CHUNK_END_FUNCTION seemed to be unrelated.
If you use the CURLOPT_DEBUGFUNCTION option you will get the data "un-chunked" and then you can scan and parse the data yourself as you see fit and thus track the end of chunks or whatever you like. ()
As "n.m." already said, libcurl has no API that exposes chunks as it tries to make the transfer encoding completely transparent to users.
Alternatively, applications can tell libcurl to not decode chunked transfer encoding at all and instead pass it on as-is to the application with CURLOPT_HTTP_TRANSFER_DECODING.
No, libcurl doesn't have any API that would let you know that.
Related
I've bougth an IP camera. I then, discovered that it uses a standard called ONVIF. I searched about it and saw that it's a good way to put all the cameras in the world to be implemented in the same way.
Of course I've read the ONVIF specification PDFs. What I understand about how it works, is the following:
There is a WSDL file that describes how the camera (or another thing) can be accessed. Like, which settings can be modified, which methods are accepted, and so. When I read this WSDL file, I'll choose a method and send a type of message called SOAP, which will be a XML containing the response to some method on the WSDL file. SOAP is a XML that can go over HTTP (so it can go through firewalls), but also can be sent over other protocols.
So, as I understood, if I want to tell my camera to start streaming video, I just need to find the WSDL method that describes how to initiate a stream, and then send a SOAP message to tell it that I need to start a video stream over some protocol, at some port. Then, I can read and decode the stream.
By knowing this simple example I gave, I could do other things, like, control the camera and so. I don't know the public for which the PDFs at ONVIF are directed for, but I couldn't understand it enough so I could explore my camera. For example: I don't know how to receive this WSDL file. I've read somewhere that I should access http://ip_of_camera/onvif/device_service to get the file, and then I should send a SOAP message to it and watch the response.
I even tried to analyse the packets via wireshark, but they've seem nonsense to me. It was difficult because I couldn't even make my camera work with iSpy software. It only works perfectly (but with no audio or talk) in my android app, so I've set up a proxy at my raspberry pi and then analyzed the packets.
Could somebody give me a basic example of how to receive the WSDL file, and then how to call one method of it? I wanted to see how it works in a network level, but I can only find how to call libraries in C# that are ready to deal with it.
For example: I wanted to know what I need to send through TCP or over TCP/HTTP to the camera, and how to initate a stream. All of this in network level, no code needed at all.
I'm just trying to learn here, but this doesn't seem like a real "open" standard. It's very hard to understand if you never programmed anything like it. So I guess those PDFs are for people who work with it. I know how TCP and UDP works and how to send raw messages by them, so it's all a question of finding a good resource with examples, so I can understand what ONVIF is really about.
Couldn't the internet unite to make it more clear? Why there are no posts about it?
Thanks, amazing community <3
I'm writing a web app that is sending some straight files in response to some requests. I want to handle this in Java, not nginx, et al. In a standard servlet, the only option is to use the java.io.OutputStream of the HttpServletResponse.
File file = ...
response.setContentLength((int)file.length());
FileInputStream in = new FileInputStream(file);
IOUtils.copy(in, response.getOutputStream());
This copies byte buffers more than is necessary. I'd like to see if I can improve performance by using NIO buffers and channels. I know Jetty is using NIO under the hood because the "connector" in my server is of class org.eclipse.jetty.server.nio.SelectChannelConnector.
Is there a way to get at the underlying channels from the servlet? Or is there a way to define a Jetty-specific handler that uses java.nio instead of java.io?
Their docs show a Jetty "hello world" handler, but that is also using HttpServletResponse and java.io streams.
I would take a look at the DefaultServlet, it already does something like this and also serves as an example of how a servlet can use direct buffers to more efficiently serve out a static resource like that.
http://git.eclipse.org/c/jetty/org.eclipse.jetty.project.git/tree/jetty-servlet/src/main/java/org/eclipse/jetty/servlet/DefaultServlet.java
Line 758 is the start of the sendData method.
Typically we recommend folks just configure the DefaultServlet if they are concerned with performance in serving static content as it also lets you mess with caching headers and that ilk as well
good luck
This copies byte buffers more than is necessary.
Not if you use chunked streaming mode. When you do that there are no byte buffers at all, unless IOUtils.copy() does that, in which case don't use it, write the 4 lines of code yourself.
The reason for the ByteBuffers is that Java will attempt to set the Content-Length response header correctly, by accumulating your output in a BB and then getting its size before sending the headers and then the BB contents as the body. Setting the header yourself has I believe no actual effect.
I want my server app to be able to send data to be processed by a bunch of various clients, and then have the processed data returned to the server.
Ideally, I'd have some call like some_process = send_to_client_for_calculating(connection, data)
I just need to be able to send a bunch of data to a client, tell the client what to do (preferably in the same message, which can be done with an array [command, data]), and then return the data...
I'm breaking up pieces of a neural network (tis very large), and then assembling them all later.
If I need to be clearer, let me know how.
I'm shocked no one has thrown it out there... how about boost::asio.
Why don't you have a look at using Apache ActiveMQ? It's a Java JMS server, but it has C++ bindings, and does what you want with a minimum of writing networking code. You basically just subscribe to messages, and send responses back. The MQ server takes care of dispatch and message persistence for you.
You could try using beanstalkd, a fast working queue. I don't know if it fits your purposes. There is a client library written in C, which you should be able to use from C++.
I'd suggest looking at gSOAP, which implements SOAP in C++, including networking.
I use an applet to bundle some files into a zip and during that I create a description-object.
I already utilize JavaEE Webservices to send the description-object to the server but how do I transport my zip?
I'm fairly new to Java EE and want to know if there is a common way to do that already (since java ee offers a lot) or would I use something ordinary like ftp?
I imagine something like a takeFileWebService that I can use from my applet, which then calls some Method onFileReceived on the serverside to handle the file, dunno :/
Thank in advance,
philipp
I'm using kind of an uncommon, maybe even insecure way now. Since I need to send a describing Object anyway I just appended my file as an array of bytes.
I lookup my TakeStuffEJB via JNDI, and then I'll invoke a remote-method that wants a byte[] as parameter. I do CRC32 integrity checks on client and server side and it works. There is a limit to filesize though. Files above 20 MB whill throw a MarshalException("Not Completed") after a while.
Maybe this is insane because I don't think RMI was designed for filetransfers. But I'll give it a try since the files I'm sending a rarely larger than 1 MB.
When I say non-standard libraries, I am referring to things like Boost, libCurl and anything else that may be able to do this far easier than standard C++ can. The reason for this is that I am writing an application as a piece of coursework (the class is dedicated to C++) and I am required to use only standard libraries and functions.
I am looking to download a RSS file, using a URL that the user will supply (I'm building a rudimentary RSS client), and the biggest problem I'm facing is that I'm not sure how to get the file down. Once I get past that bit, parsing it for the xml tags and displaying the content will be relatively straightforward. I've been looking around and I've only found solutions that say to use non-standard libraries, usually libCurl. If someome could just give me a quick heads up about what I should be looking at for this, then I'd be grateful.
Also, if you think you're helping me cheat, you're not. The assignment is to build an application of our choice and we're being graded on our use of the various feature of the language (it must contain so many classes, use these variables types, etc).
Check out Beej's Guide to Network Programming for a quick but excellent introduction to sockets. If you cannot use any non-standard libraries, your only option is to manually connect on port 80 and make the request yourself.
Assuming even a beginner-level knowledge of C++, that should be all you need.
First off, it can't be done using only standard C++. There is no network interface in either standard C++ or standard C.
If you're required to take a "do-it-yourself" approach, then probably the intention is that you would use your platform's sockets API. In the case of linux, this is part of the POSIX standard, not C++, and is available from <sys/socket.h>.
The basic procedure is: parse the URL; look up the IP address of the domain; create a socket; connect the socket; write an HTTP request to the socket; read the HTTP response back from the socket; clean up.
Obviously, an HTTP library is far more convenient, especially since an HTTP download can get more complicated than what I describe above (for example, if the server responds with a redirect). Pretty much all linux distributions will provide libcurl, and/or the curl and wget programs.
Writing a program to make a socket connection is relatively trivial.
http://www.linuxhowtos.org/C_C++/socket.htm
Now that you have a socket open to an HTTP server you need to understand how to ask for a document and how to decode the reply:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html
Basically you need to send:
GET<SP><URL><SP>HTTP/1.1<CRLF>
Where:
SP: Single Space
CRLF: \r\n
URL: The Full URL of the page including the server name.
What you get back will be
http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html#sec6
HTTP/1.1<SP>200<SP>OK<CRLF>
(<Header><CRLF>)*
<CRLF>
<Document>
The above means:
The first line is the response line that should contain 200 OK.
If it does not then there is some kind of error and you should just give up.
This is followed by 0 or more header lines
Just ignore these lines
There will be 1 empty line to mark the end of the headers.
Then the document will be on the stream.
If you really want to do it without using libcurl you can always open a tcp socket and then send:
GET /myurl
(http 1.0 or preferably use http 1.1)
Basically you're writing a very simple http protocol client implementation.
You can download source code for wget standard utility
Since you are not allowed to use non-standard libraries, you could write your own primitive wrapper class for the linux "curl" command (I'm assuming you are using linux). Curl is a very powerful command, and it can probably do what you need it to.