From the perspective of a wireless router on a wall in a coffee shop, can my computer be uniquely identified via an HTTP request? I think it get's my MAC adress doesn't it?
Next question, from the perspective of a remote server that I send an HTTP request to, can I be uniquely identified? In this case, the server doesn't get my MAC address, but someone else's that is past me, right? What other information could make my HTTP request uniquely identifiable? The IP address doesn't help either, because that would just narrow me down to the coffee shop I'm setting in, right? And I could always move to a different router. I suppose cookies would work here, but we can't guarantee that people won't flush their cookies. How close can we get to unique identifiability? What strategies can be used.
Your best option are HTTP Headers, you can take at this list for some headers already in the wild.
I don't know what you are trying to achieve, but I suppose you want to create an identifier for every user. So you have two options, implement this on router or gateway or on the client side.
Related
I am now facing a design question. I have made a shared library which can be used to send HTTP requests to my HTTP server. It offers GET, POST, PUT and DELETE APIs, so the user can use them to send requests directly. However, someone else can also use TCP/IP to send their own HTTP requests to my server.
My question is, how can I check if a request comes from my library or not?
Can anybody give me some suggestion?
You can use any "standard" method of authentication to verify that the requester is who they are supposed to be. Once you've determined the authenticity, provide the requester with a short term cookie that they must send with all requests. Reject any request that doesn't hold a valid cookie (except attempts to authenticate).
Technically this doesn't prevent requests from coming elsewhere other than your library. It merely restricts requests from sources that don't know your secret. But if you don't share the secret with anyone else, then you can limit yourself to only using your library which achieves the limitation to the library indirectly.
I have a 3rd party service that i've configured a webhook that triggers posting data on my url address. Now i want to restrict incoming requests to be allowed only for this service. How can i do this in Django ? Is there any trick on applying some security measures?
i'd be glad if you can provide some code snippets, please
EDITED
I can't count on the ip address of the requester, it can change in any time. I should use domain name i think
EDITED 2
I have a header called HTTP_X_REAL_IP, that contains the ip address of the service. Can i count on that header by comparing the socket ip address with properly received header ?
It appears that they don't have anything set up to easily do a test, and you aren't saying exactly what you are doing with this API, but it appears that you can query their API for information regarding a bunch of things.
What I would do is to take the information they sent you and send it back in a query to see if the information they now have matches the change they are telling you they made. If they match, it is authentic. If they don't match, discard it.
That's about the best I can tell you to do.
No, you can't use the domain name, as you don't have it when processing a request (note that normally most requesters - internet users - don't have domain names, they only have IP addresses).
The simplest solution is to add an authentication key as required parameter to your APIs. Then, you process only those API calls which provide valid authentication key.
If you would need to see what Information a Site has on you, based on the Cookie ID that they've stored on your Computer, how exactly would one do that?
You can't.
A cookie is best considered an opaque ticket, passed from the server to the client, for the client to return to the server as a means of storing state, or a reference to server-side persisted state, without the server needing to keep track of it itself.
Granted, some cookies are non-opaque, especially those that are intentionally exposed to Javascript (such as storing client-side preferences), but I'm assuming you're not interested in those.
So there is no way for a HTTP client to peer into a webserver's stored state. That's the point. Otherwise it would be insecure.
Here's a simple demonstration:
[Client] Hi Server. My name is Boris.
[Server] Hi Boris. I have assigned you visitor number 3. I have remembered your name is "Boris" and saved it in my internal database, associated with the number 3. Please refer to yourself as 3 in all future requests.
[Client] Okay, thanks, goodbye.,
(weeks pass)
[Client] Hi Server, remember me? I am visitor 3.
[Server] Yes, hello Boris.
[Client] I am visitor 3. Please remember that my hair is yellow.
[Server] Yes, I have remembered that visitor 3 also has yellow hair.
(weeks pass)
[Client] Hi Server, remember me? I am visitor 3. What color is my hair?
[Server] Your hair is yellow.
(more time passes)
[Client] Hi Server, I am visitor 3. What information do you have stored about me?
[Server] Hi visitor 3. I know your name and your hair color, but I won't tell you because I don't want to. I didn't even have to tell you that I knew those details.
So even though the cookie in this case is merely the number "3", the client has no way of knowing what the server has stored about it.
Unless the site stores things in your cookie, you're probably not able to get much. The cookie often has a userId or sessionId in it that the site uses to look up the information it knows about you. That information is stored in a database that you won't be able to get access to.
Some things you could do:
Open the developer tools in your browser and look at the internet traffic that goes back and forth. You may be able to see some information about yourself there.
Use a tool like Fiddler or BurpSuite to sniff the traffic between your computer and the site in question.
It may be wise to delete your cookies regularly or find a browser plugin like Self-Destructing Cookies if this bothers you.
I'm writing an application that listens to HTTP traffic and tries to recognize which requests where initiated by a human.
For example:
The user types cnn.com in their address bar, which starts a request. Then I want to find
CNN's server response while discarding any others requests (such as XHR, etc.)
How could you tell from the header information what means what?
After doing some research I've found that relevant responses come with :
Content-Type: text/html
Html comes with a meaningful title
status 200 ok
There is no way to tell from the bits on the wire. The HTTP protocol has a defined format, which all (non-broken) user agents adhere to.
You are probably thinking that the translation of a user's typing of just 'cnn.com' into 'http://www.cnn.com/' on the wire can be detected from the protocol payload. The answer is no, it can't.
To detect the user agent allowing the user such shorthand, you would have to snoop the user agent application (e.g. a browser) itself.
Actually, detecting non-human agency is the interesting problem (with spam detection as one obvious motivation). This is because HTTP belongs to the family of NVT protocols, where the basic idea, believe it or not, is that a human should be able to run the protocol "by hand" in a network terminal/console program (such as a telnet client.) In other words, the protocol is basically designed as if a human were using it.
I don't think header information can suffice to identify real users from bots, since bots are made to mimic real users and headers are very easy to imitate.
One thing you can do, is to track the path (sequence of clicks) followed by a user, which is most likely to be different from one made by a bot, and made some analysis on the posted information (i.e. bayesian filters).
A very easy to implement check is based on the IP source. There are databases of black listed IP addresses, see Project Honeypot - and if you are writing your software in java, here is an example on how to check an IP address: How to query HTTP:BL for spamming IP addresses.
What I do on my blog is this (using wordpress plugins):
check if an IP address is in the HTTP:BL, if it is the user is shown an html page to take action to whitelist his IP address. This is done in Wordpress by Bad Behavior plugin.
when the user submits some content, a bayesian filter verifies the content of his submission and if his comment is identified as spam, a captcha is displayed before completing the submission. This is done with akismet and conditional captcha, and the comment is also enqueued for manual approval.
After being approved once, the same user is considered safe, and can post without restrictions/checks.
Applying the above rules, I have nomore spam on my blog. And I think that a similar logic can be used for any website.
The advantage of this approach, is that most of the users don't even notice any security mechanism, since no captcha is displayed, nor anything unusual happens in 99% of the times. But still there is quite restrictive, and effective, checks going on under the hoods.
I can't offer any code to help, but I'd say look at the Referer HTTP header. The initial GET request shouldn't have a Referer, but when you start loading the resources on the page (such as JavaScript, CSS, and so on) the Referer will be set to the URL that requested those resources.
So when I type in "stackoverflow.com" in my browser and hit enter, the browser will send a GET request with no Referer, like this:
GET / HTTP/1.1
Host: stackoverflow.com
# ... other Headers
When the browser loads the supporting static resources on the page, though, each request will have a Referer header, like this:
GET /style.css HTTP/1.1
Host: stackoverflow.com
Referer: http://www.stackoverflow.com
# ... other Headers
Here is the situation, we have a site that is hosted and updated by a third party vendor. I am providing links to additional resources that are hosted on our servers. A client will access the vendor site and click on a link to gain access to our additional resources. To validate that the request came from our third party vendor I need to get the IP address of the vendors server.
My question is, is there a way to get the IP address of the vendors servers using ColdFusion? I can't use the clients IP address, I need the vendor server address the client is using.
You have to work with 3rd party to accomplish this goal, this is for sure.
I can see at least two more or less working approaches here.
(1) Append some kind of protection token to the links. Your vendor generates encrypted string or hash including some information only you two know, so you can decrypt (or generate same hash) and validate it.
Example with hashing:
moment = DateConvert("local2utc", Now());
token = Hash("SecretSaultYouBothKnow" & DateFormat(moment, "yyyy-mm-dd") & TimeFormat(moment, "-HH-mm"));
This token is passed with link and expires quickly to prevent sharing/leaking.
You can generate and validate it on your side.
It's a raw idea and there could be possible problems with validation, plus avoiding invalid links for clients (maybe skip "mm" mask as well).
Encrypted/decrypted string would work similarly. You both just need to now the secret key.
By the way, your vendor could encrypt their server IP address or other identifier for you to check it against your database and maybe apply some other actions.
(2) Your vendor could set up simple web-service for you to validate the incoming links (it could respond with 0/1 or something else simple).
Exact implementation may be different. Again, it could be some token in URL which you send back for validation.
This is similar to solution which Jason suggested: vendor could send the server-to-server request to your server on link click and then relocate to the resource. But this may be complicated because you have to be sure 1st request is already handled when client arrives.
Hope these ideas make sense.
No, there isn't. Not if the request comes directly from the client. If the vendor sends some sort of a message first you can use that to validate. Or if the vendor's server is the one making the request on behalf of the client then you could use CGI.REMOTE_ADDR. But if the vendor is just providing a link to your site, then no, you cannot be assured of the IP of the vendor's server.
The closest you could come is to check the HTTP_REFERER, as Jeremy said above, but that can be spoofed (very easily), so it wouldn't be very secure.
To access the CGI variables available to ColdFusion, you can do something like this:
<cfset ThisIP = CGI.SERVER_NAME>
There are many useful CGI variables available here:
http://www.perlfect.com/articles/cgi_env.shtml
try placing a page on your server that uses the cfhttp tag to fetch:
http://www.dslreports.com/whois
That will give you the IP address of the web server.