I am trying to come up with a way to expose the specific build number of a web service (REST and SOAP interfaces) using HTTP in response to messages.
Just to be clear I am not trying to control which version of an application is being used by a client as this is handled by the traffic manager and service registry (also much discussed already), I only want to be able to debug later what version was used or is being used, e.g., to check the traffic manager is serving up the correct version.
I have spent a long time looking at the HTTP spec and can't decide between using Pragma
Pragma: application-version=1.0.0-SNAPSHOT
using an additional Server header
Server: Apache-Coyote/1.1
Server: MyService-1.0.0-SNAPSHOT
using a Via
Via: MyService-1.0.0-SNAPSHOT
or a customer header
X-ApplicationVersion: 1.0.0-SNAPSHOT
To my mind using a Via or a Server is semantically incorrect but using an X- header bears the risk of being dropped by proxies but Pragma is unconstrained and thus more difficult to parse if wanted to use it programmatically.
I also have a feeling that multiple Server headers won't be respected but there is no way to specify multiple values with the container I'm using (JBoss).
Is there any precedent for this?
Anyone have any tips?
Should I just shut up and use one?
To my mind using a Via or a Server is semantically incorrect but using
an X- header bears the risk of being dropped by proxies
I would not worry too much about the X headers being dropped. Take this call to the ebay finding service for example
http://svcs.ebay.com/services/search/FindingService/v1?OPERATION-NAME=findItemsByKeywords&SERVICE-VERSION=1.0.0&RESPONSE-DATA-FORMAT=XML&REST-PAYLOAD&keywords=harry%20potter
The response includes this header.
X-EBAY-SOA-SERVICE-VERSION: 1.12.0
I've seen the X headers used quite a lot for this purpose.
Related
There is good documentation (OpenTelemetry project, W3C) on generating, passing, and parsing distributed tracing headers (e.g., traceparent) on the server side, however I have not found anything authoritative on exposing trace IDs to the client side. E.g., I would like to configure my clients to use the trace ID from an error response to add context to a Sentry log, or allow users to include this information in a support request.
Google Cloud seems to add X-Cloud-Trace-Context to responses on at least Cloud Run services... but it's not documented and might not be intentional but present only as a convenient bug.
Is there a standard or even a broadly-used convention for this, or is this entirely up to implementations to fashion on their own?
What kind of existing options there is to make the client's GET "myserver://api/download/12345.jpg" to download from some_cloudfront_server://files/12345.jpg without redirecting the client to that CloudFront-path? Ie. I want the client see only myserver://api/download/12345.jpg all the time.
It should be some kind channeling solution, as downloading a full file first to Django-server and then sending it to the client is not applicable (takes so much time that the client will see timeout before a response to its query comes). Any existing libraries for this? If I have to create one myself, I welcome even just tips where to start from as Django's communication layer is not too familiar to me.
Problem here is that we are creating CloudFront signatures with wildcards to certain set of files, in format files/<object_id>*, thus allowing the client access only to certain object's all files in CloudFront-server directly. This works fine as long as file access traffic from clients is low. But if we start creating separate access signatures for hundred different files at same time, CloudFront starts throttling our requests. Solution I came up is to create and store to Django-server one generic allow-all signature for files/*, which is used only by Django-server and never given to any client, and then we let Django-server decide should it fetch files for the client or not. Thus, I can't give to the client the CloudFront-path with a allow-all signature, but I can channel CloudFront-data thru Django's endpoint to the client without showing signature to the client.
Environment I'm working with is Django v1.11, and Django REST Framework v3.4's ViewSets.
One colleague heard about my problem and mentioned that signatures can be made locally at server, thus no CloudFront connection needed every time the signature is created and we can keep signatures as file specific. He took this task for himself so I don't know details yet.
I have an iPhone App that gets the data by SOAP requests.
The SOAP calls are done by sudzc.com library.
I have to make SOAP Request to two servers.
Server A: is my own server, where I retrieve some informations, SOAP Response written by myself
Server B: a third party server that gives me some necessary informations
iOS 6
The app is working 100% correct.
iOS 7
Server A: working perfectly
Server B: SOAP Requests randomly fails. I am getting the following
error message sometimes:
< SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
< SOAP-ENV:Header/>
< SOAP-ENV:Body>
< SOAP-ENV:Fault>
< faultcode>SOAP-ENV:Server
< faultstring xml:lang="en">Could not access envelope: Unable to create envelope from given source: ; nested exception is com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Unable to create envelope from given source: : Unable to create envelope from given source: : org.xml.sax.SAXParseException: The markup in the document preceding the root element must be well-formed.: The markup in the document preceding the root element must be well-formed.
< /faultstring>
< detail/>
< /SOAP-ENV:Fault>
< /SOAP-ENV:Body>
< /SOAP-ENV:Envelope>
Anyone has an idea, why this only happens on iOS7 and how I could get rid of it?
UPDATE:
May it be related to the fact, that one server is running on https and the other runs on http?
Server A: http://www.xxx.xy
Server B: https://www.xxx.xy:443
I have written a support request to the iOS Team. here is what they replied... in Fact it has something to do with the htpps... Just in case someone runs into the same error, here might be the reason why:
I'm responding to your question as to why your SOAP requests are failing on iOS 7, but only when targeting one of your two servers. You wrote:
May it be related to the fact, that one server is running on https and
the other runs on http?
It turns out that your theory is correct. The problem here is that iOS 7 includes the recommended countermeasure for the BEAST attack.
http://en.wikipedia.org/wiki/Transport_Layer_Security#BEAST_attack
http://www.educatedguesswork.org/2011/11/rizzoduong_beast_countermeasur.html
https://www.imperialviolet.org/2012/01/15/beastfollowup.html
iOS applies this countermeasure when it talks to a TLS 1.0 or earlier server (TLS 1.1 and later include their own fix for this attack) that negotiates the use a block cypher (stream cyphers are not vulnerable to this attack).
Looking at a packet trace of the test app you sent me, I can see that it opens a TLS 1.2 connection to www.xxxx.xy:443. That server downgrades the connection to TLS 1.0 and then negotiates to use the AES-128 block cypher (SSL_RSA_WITH_AES_128_CBC_SHA). After that I can see the unmistakable signs of this countermeasure kicking in.
I confirmed my analysis by running your test app on the simulator (which accurately reproduces the problem) and then, using the debugger, setting the internal state of Secure Transport (the subsystem that implements TLS on iOS) to disable this countermeasure entirely. On doing that I found that the first tab in your app works fine.
A well-known side effect of this countermeasure is that it causes problem for poorly written HTTPS servers. That's because it breaks up the HTTP request (the one running over the TLS connection) into chunks, and the server must be coded to correctly receive these chunks and join them into a single HTTP request. Some servers don't do that properly, which results in a variety of interesting failures.
The best way to solve this problem is to fix the server. The server should be able to cope with receiving the HTTP message in chunks, detecting HTTP message boundaries in the manner prescribed by RFC 2616.
http://www.ietf.org/rfc/rfc2616.txt
If that fix is too hard to implement in the short term, you can work around the problem by simply upgrading your server to support TLS 1.2. This is a good idea anyway.
Another workaround, one that's a less good idea, is to tweak the server configuration to negotiate the use of a stream cypher.
If you don't control the server, I strongly encourage you to lobby the server operators for a server-side fix for this problem. iOS 7 is not the only client that's implementing this workaround; you'll find it in recent versions of both Chrome, Firefox and so on.
If a server-side fix is just not possible, your options on the client side are less than ideal:
o You could replace HTTPS with HTTP. Obviously this is not a good thing, and it also requires that the server support HTTP. Also, HTTP comes with its own array of un-fun problems (various middleboxes, especially those run by cellular carriers, like to monkey with HTTP messages).
o At the lowest level, you can disable this countermeasure for a given a Secure Transport context by way of the kSSLSessionOptionSendOneByteRecord flag (see ). This flag is not directly available to higher-level software, although it is possible to use it at the CFSocketStream layer (because that API gives you a way to get at the Secure Transport context it's using).
IMPORTANT: The high-level APIs, NSURLSession and NSURLConnection, don't give you access to the Secure Transport context, nor do they provide any control over this aspect of TLS. Thus, there's no way to disable this countermeasure and continue working with those nice, high-level APIs.
o You could implement your own HTTP layer on top of our TLS infrastructure (ideally CFSocketStream). Alas, this is a whole world of pain: HTTP is a lot more complex than you might think.
I'm sorry I don't have better news.
I'm not real hip on exactly what role(s) today's proxy servers can play and I'm learning so go easy on me :-) I have a client/server system I have written using a homegrown protocol and need to enhance the client side to negotiate its way out of a proxy environment.
I have an existing client and server system written in C and C++ for the speed and a small amount of MFC in the client to handle the user interface. I have written both the server and client side of the system on Windows (the people I work for are mainly web developers using Windows everything - not a choice) sticking to Berkeley Sockets as it were via wsock32 for efficiency. The clients connect to the server through a nonstandard port (even though using port 80 is an option to get out of some environments but the protocol that goes over it isn't HTTP). The TCP connection(s) stay open for the duration of the clients participation in real time conferences.
Our customer base is expanding to all kinds of networked environments. I have been able to solve a lot of problems by adding the ability to connect securely over port 443 and using secure sockets which allows the protocol to pass through a lot environments since the internal packets can't be sniffed. But more and more of our customers are behind a proxy server environment and my direct connections don't make it through. My old school understanding of proxy servers is that they act as a proxy for external HTML content over HTTP, possibly locally caching popular material for faster local access, and also allowing their IT staff to blacklist certain destination sites. Customer are complaining that my software doesn't recognize and easily navigate its way through their proxy environments but I'm finding it difficult to decide what my "best fit" solution should be. My software doesn't tear down the connection after each client request, and on top of that packets can come from either side at any time, basically your typical custom client/server system for a specific niche.
My first reaction is "why can't they just add my server's addresses to their white list" but if there is a programmatic way I can get through without requiring their IT staff to help it is politically better and arguably a better solution anyway. Plus maybe I'm still not understanding the role and purpose of what proxy servers and environments have grown to be these days.
My first attempt at a solution was to use WinInet with its various proxy capabilities to establish a connection over port 80 to my non-standard protocol server (which knows enough to recognize and answer a simple HTTP-looking GET request and answer it with a simple HTTP response page to get around some environments that employ initial packet sniffing (DPI)). I retrieved the actual SOCKET handle behind WinInet's HINTERNET request object and had hoped to use that in place of my software's existing SOCKET connection and hopefully not need to change much more on the client side. It initially seemed to be my solution but on further inspection it seems that the OS gets first-chance at the received data on this socket since when I get notified of events via the standard select(...) statement on the socket and query the size of the data available via ioctlsocket the call succeeds but returns 0 bytes available, the reads don't work and it goes downhill from there.
Can someone tell me of a client-side library (commercial is fine) will let me get past these proxy server environments with as little user and IT staff help as possible? From what I read it has grown past SOCKS and I figure someone has to have solved this problem before me.
Thanks for reading my long-winded question,
Ripred
If your software can make an SSL connection on port 443, then you are 99% of the way there.
Typically HTTP proxies are set up to proxy SSL-on-443 (for the purposes of HTTPS). You just need to teach your software to use the HTTP proxy. Check the HTTP RFCs for the full details, but the Cliffs Notes version is:
Connect to the HTTP proxy on the proxy port;
Send to the proxy:
.
CONNECT your.real.server:443 HTTP/1.1\r\n
Host: your.real.server:443\r\n
User-Agent: YourSoftware/1.234\r\n
\r\n
Then parse the proxy response, which will start with a HTTP status code, followed by HTTP headers, followed by a blank line. You'll then be talking with your destination (if the status code indicated success, anyway), and can start talking SSL.
In many corporate environments you'll have to authenticate with the proxy - this is almost always HTTP Basic Authentication, which is pretty easy - again, see the RFCs.
My company is planning on implementing a remote programming tool to configure embedded devices in the field. I assumed that these devices would have an HTTP client on them, and planned to implement some REST services for them to access. Unfortunately, I found out that they have a TCP stack but no HTTP client. One of my co-workers suggested that we try to send “soap packets” over port 80 without an HTTP client. The devices also don’t have any SOAP client. Is this possible? Would there be implications if there was a web server running on the network the devices are connected to? I’d appreciate any advice or best practices on how to implement something like this.
If your servers are serving simple files, the embedded devices really only need to send an HTTP GET request (possibly with a little extra data identifying the device, so the server can know which firmware version to send).
From there, it's pretty much a simple matter of reading the raw data coming in on the embedded device's socket -- you might need to only disregard the HTTP header on the response, or you could possibly configure your server to not send it for those requests.
you don't really need an HTTP client per-se. HTTP is a very simple text-based protocol that you can implement yourself if you need to.
That said, you probably won't need to implement it yourself. If they have a TCP stack and a standard sockets library, you can probably find a simple C library (such as this one) that wraps up HTTP or SOAP functionality for you. You could then just build that library into your application.
Basic HTTP is not a particularly difficult protocol to implement by hand. It's a text and line based protocol, save for the payload, and the servers work quite well with "primitive, ham fisted" clients, which is all a simple client needs to be.
If you can use just a subset, likely, then simply write it and be done.
You can implement a trivial http client over sockets (here is an example of how to do it in ruby: http://www.tutorialspoint.com/ruby/ruby_socket_programming.htm )
It probably depends what technology you have available on your embedded devices - if you can easily consume JSON or XML then a webservice approach using the above may work for you.