I'm sick of this. ALWAYS when I make a HTTP GET query from a C/C++ program using just plain sockets I get 301 Moved Permanently's. Normally I'd use libcURL, but in this case I don't want to add another library, I just need to download one flat identification file from one fixed server.
This is my current query:
GET /game/getversion.jsp?user=nightcracker&password=yeahright&version=12 HTTP/1.1\r\n
Connection: close\r\n
Host: www.minecraft.net\r\n
Accept-Encoding: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2\r\n
\r\n
I have tried EVERYTHING, and everything just gets answered with this funny message:
HTTP/1.1 301 Moved Permanently
Server: nginx/0.6.32
Date: Tue, 15 Mar 2011 02:18:11 GMT
Content-Type: text/html
Content-Length: 185
Connection: close
Location: http://www.minecraft.net/game/getversion.jsp?user=nightcracker&password=yeahright&version=12
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/0.6.32</center>
</body>
</html>
I remember this issue from before and I ragequitted before. Now I want to fix this damn bugger. So tell me SO, why do all my HTTP queries always give back a 301?
Alright, besides the issue with the Accept-Encoding, the query was fine. The problem was that I resolved in my socket code to "minecraft.net" instead of "www.minecraft.net". RAAAAH. Fixed.
I can't see anything obviously wrong since the redirected URI appears to be the same as the original GET request URI, so I would suggest downloading the command-line curl and running that in verbose mode against the same target. Perhaps it will show something in its output that can point you in the right direction. There's a chance that this is a badly-configured server or badly written JSP, so keep that in mind.
I don't know if this is the problem you have on the Minecraft server (I don't have an account) but
Accept-Encoding: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2\r\n
what the heck is that? Header fields that might go in requests include
Accept: MIME types (e.g. what you have there)
Accept-Charset: charsets (e.g. utf-8)
Accept-Encoding: encodings (e.g. gzip)
Accept-Language: languages (e.g. en)
and you seem to be mixing them up.
Well, the server is redirecting the client to another location. You just have to issue another
request to the URL coming back in the "Location" header of the 3xx respone
OOPs realized that the redirect location is the same as the original URI. DOes this URL work from the browser? If so you might try adding a User-Agent header in the request that contains the same User-Agent that the browser is sending.
You can either specify the correct URL (www.minecraft.net) or tell libcurl to follow redirects automatically:
curl_easy_setopt(curl_handle,CURLOPT_FOLLOWLOCATION,1);
Related
I am trying to make GET requests from a C++ program and every time I get a 301 Moved Permanently error. I am using an API that uses sockets and cannot seem to figure out why this error always comes up.
Here is the request that is getting made:
GET https://www.quandl.com/api/v3/datasets/EOD/AAPL.csv?sort_order=asc&auth_token=YZffVEztoepdzHNAMexz HTTP/1.1
Host: www.quandl.com
Connection: close
And here is the response to the request:
HTTP/1.1 301 Moved Permanently
Date: Sun, 12 Nov 2017 03:58:41 GMT
Content-Type: text/html
Content-Length: 182
Connection: close
Set-Cookie: __cfduid=d51b8e22f5239ed65b480d8ec37cad8251510459121; expires=Mon, 12-Nov-18 03:58:41 GMT; path=/; domain=.quandl.com; HttpOnly
Location: https://www.quandl.com/api/v3/datasets/EOD/AAPL.csv?sort_order=asc&auth_token=YZffVEztoepdzHNAMexz
Server: cloudflare-nginx
CF-RAY: 3bc6930581840ed9-EWR
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>openresty</center>
</body>
</html>
I think it may have to do with the Http-only part in the Set-cookie but am not 100% sure about that and don't know how to get rid of it. I think the url in the response after location is where the page has "moved to", however it is the exact same one as the one I am requesting so I don't understand why I am getting the error.
GET https://www.quandl.com/api/v3/datasets/EOD/AAPL.csv?sort_order=asc&auth_token=YZffVEztoepdzHNAMexz HTTP/1.1
Host: www.quandl.com
Connection: close
That's not a valid request for a https:// resource. Instead you have to create a TLS connection to the server (instead of only a TCP connection) and send the request with path-only instead of full-URL:
GET /api/v3/datasets/EOD/AAPL.csv?sort_order=asc&auth_token=YZffVEztoepdzHNAMexz HTTP/1.1
Host: www.quandl.com
Connection: close
301 isn't an error, it means the resource has changed URL's.
When you get this valid response code you can issue another request to the Location URL specified in the response.
Be careful to limit the number of times you follow a redirect because you could wind up with an infinite loop. A lot of HTTP client libraries have an option to handle this automatically.
I'm writing a program in C++ that needs to download JSON data from an HTTPS URL. The program is based on wxWidgets. That URL is for the translation service at Glosbe
So I've tried multiple different libraries including:
libcurl
Boost.Asio
the http functionality included in wxWidgets
wxCurl
Urdl
However, it always throws an error saying it can't connect, or I get a reply that says "Moved Permanently".
When i copy and paste the URL I am testing it with into a browser, it returns the JSON data perfectly.
Does anyone know the correct way to do this?
Any help would be great!
301 Moved Permanently is what the server responds when you try to access the page with HTTP instead of HTTPS. Here's a complete response I just received from the server:
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Jul 2015 20:25:01 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Location: https://en.glosbe.com/a-api
It means exactly that: "The content you are looking for is really at https://en.glosbe.com/a-api." Your browser simply adheres to the HTTP protocol by following the server's hint and automatically proceeding to request https://en.glosbe.com/a-api when you try to access http://en.glosbe.com/a-api. It works seamlessly for you as a user.
You will have to read more documentation to create HTTPS requests yourself. Each of the libraries you mentioned will have a different way of supporting HTTPS (or not support it at all). For example, have a look at http://www.boost.org/doc/libs/1_58_0/doc/html/boost_asio/overview/ssl.html, especially the "Notes" section where it says that "OpenSSL is required to make use of Boost.Asio's SSL support."
Good morning all,
I have a web app using Classic ASP, hosted on IIS 7.5. I need to create a word document and stream back to the client for display, however when the document is streamed into the response something (an IIS setting?) is causing a 401 error - and i cannot track it down.
A colleague very kindly furnished me with some C++ code which deals with the document building & streaming and an ASP page which builds the COM objects and makes the calls etc.
The document starts life as a template in an IIS virtual directory outside my ASP application. We make a copy, do some find and replace actions and then stream it into the response with the correct MIME type for the browser to handle it however it sees fit.
It seems as soon as the streaming takes place (via a call to ASPTypeLibrary::IResponsePtr piResponse->BinaryWrite()) the 401 response is sent back, until that moment the response looks fine.
The site uses Forms authentication, and i have by this point signed in, the request i see in fiddler definitely has valid session data and the rest of the site is happy with my authentication.
Any ideas? (he asks with a note of desperation)
p.s. I realise i haven't listed all the code, i can if it helps though...
The plot thickens...
I have had success using the ASP page to stream the file into the output using and ADODB.Stream object.
When this is successful Fiddler picks up two HTTP request/responses; the first request gets a 401 back, then the browser sends another request with different cookie data which returns a successful result.
When my COM object is used two requests occur, but the second request also receives a 401...
Points to some security setting to do with COM object? Something i am not adding to the response with the COM object?
As per my response to Eric, my colleague worked a bit of magic and got the thing working, i am still a little confused about why it was caused though...
The line which Magic Al changed was this one:
piResponse->AddHeader( _T("Content-Length"), (LPCTSTR)Length );
Which is called while the response is being built up and what he did was comment it out.
Apparently he noticed that the length written by the BinaryWrite was coming out 13 bytes larger than the length of the file. He tells me that this may be because it is writing out the reserved WORD blocks from the Variant it is given.
So the response header is a bit mangled and the result is an HTTP violation error in fiddler which i had overlooked and somewhere in between client and COM object the mangled 200 response is replaced with a 401.
I guess the moral of this story is that you should always pay attention to fiddler errors and to ensure your Content-Length is correct.
My new question is why 401? why not a 500? And what is likely to be throwing this out? is it coming from IIS?
This is the HTTP text fiddler registers for the exchange:
GET GET [The page address - its on localhost and is an ASP page] HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: en-GB
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
Host: localhost
Cookie: SessionUID={E41F5378-FBE2-475E-8F9A-6416AFE2BAA0}; DisplayMethod=0; ShowDataTips=1; LOGONUSER={UserName Info}ASPSESSIONIDQARAQRBD={Session ID} Authorization: Negotiate YH8GBisGAQUFAqB1MHOgMDAuBgorBgEEAYI3AgIKBgkqhkiC9xIBAgIGCSqGSIb3EgECAgYKKwYBBAGCNwICHqI/BD1OVExNU1NQAAEAAACXsgjiCwALADIAAAAKAAoAKAAAAAYBsR0AAAAPRE9DREVWLUpDV0RFVkVMT1BNRU5U
HTTP/1.1 401 Unauthorized
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
WWW-Authenticate: Negotiate oYIBYTCCAV2gAwoBAaEMBgorBgEEAYI3AgIKooIBRgSCAUJOVExNU1NQAAIAAAAWABYAOAAAABXCieJQEtMwe36vajD3zwEAAAAA9AD0AE4AAAAGAbEdAAAAD0QARQBWAEUATABPAFAATQBFAE4AVAACABYARABFAFYARQBMAE8AUABNAEUATgBUAAEAFABEAE8AQwBEAEUAVgAtAEoAQwBXAAQAOABEAGUAdgBlAGwAbwBwAG0AZQBuAHQALgBEAG8AYwB1AG0AYQB0AGkAbwBuAC4AYwBvAC4AdQBrAAMATgBEAE8AQwBEAEUAVgAtAEoAQwBXAC4ARABlAHYAZQBsAG8AcABtAGUAbgB0AC4ARABvAGMAdQBtAGEAdABpAG8AbgAuAGMAbwAuAHUAawAFACAARABvAGMAdQBtAGEAdABpAG8AbgAuAGMAbwAuAHUAawAHAAgAKRk7jhfZzQEAAAAA
Date: Thu, 13 Dec 2012 09:52:25 GMT
Content-Length: 341
Proxy-Support: Session-Based-Authentication
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Not Authorized</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Not Authorized</h2>
<hr><p>HTTP Error 401. The requested resource requires user authentication.</p>
</BODY></HTML>
------------------------------------------------------------------
GET [The page address - its on localhost and is an ASP page] HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: en-GB
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
Cookie: SessionUID={E41F5378-FBE2-475E-8F9A-6416AFE2BAA0}; DisplayMethod=0; ShowDataTips=1; LOGONUSER={UserName Info}ASPSESSIONIDQARAQRBD={Session ID}
Authorization: Negotiate oXcwdaADCgEBoloEWE5UTE1TU1AAAwAAAAAAAABYAAAAAAAAAFgAAAAAAAAAWAAAAAAAAABYAAAAAAAAAFgAAAAAAAAAWAAAABXCiOIGAbEdAAAAD4GPyFfTAkcs1KpJqG4eT0ujEgQQAQAAAPUXp1AtIpqEAAAAAA==
Host: localhost
HTTP/1.1 401 Unauthorized
Cache-Control: private
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.5
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
X-Powered-By: ASP.NET
Date: Thu, 13 Dec 2012 09:52:27 GMT
Content-Length: 6630
Proxy-Support: Session-Based-Authentication
<HTML Page telling you about the error which gets displayed when you cancel the authentication dialog.>
Getting Response is null error while receiving HTTP response.
I am developing an sample small HTTP server in C using row sockets.
There are actually 2 servers in my application one is standard Apache server which I am using for serving HTML pages and my small server will respond to only XMLHttpRequest sent from the Javascript within the HTML pages.
I am sending request from JavaScript as follows:
var sendReq = new XMLHttpRequest();
endReq.open("POST", "http://localhost:10000/", true);
sendReq.setRequestHeader('Content-Type','application/x-www-form-urlencoded');
sendReq.onreadystatechange = handleResult;
var param = "REQUEST_TYPE=2002&userName=" + userName.value;
param += "&password=" + password.value;
sendReq.send(param);
When I send this request I receive following Request in my server code:
OPTIONS / HTTP/1.1
Host: localhost:10000
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.3) Gecko/20100423 Ubuntu/10.04 (lucid) Firefox/3.6.3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Origin: http://localhost:7777
Access-Control-Request-Method: POST
I have replied to this Request as follows using socket write function:
HTTP/1.1 200 OK\n
Access-Control-Allow-Origin: *\n
Server: PSL/1.0 (Unix) (Ubuntu/Linux)\n
Access-Control-Allow-Methods: POST, GET, OPTIONS\n
Accept-Ranges: bytes\n
Content-Length: 438\nConnection: close\n
Content-Type: text/html; charset=UTF-8\n\n
I don`t know What should be the HTTP actual response to be sent on request of OPTIONS.
After this I get my Actual POST request that I have sent from JavaScript and then I respond back with
HTTP/1.1 200 OK\n\n
And then at the browser end get error Response is null.
So how to send headers/data as HTTP Response using row sockets in 'C' and how to respond to OPTIONS request. Can someone explain me by giving some example?
It's hard to understand your question, but I believe you are pointing to this as the response giving you trouble:
HTTP/1.1 200 OK\n\n
You should be including other fields, especially the Content-Length and Content-Type. If you're going to build your own HTTP server, then you should review the protocol specifications.
That said, it's not at all clear why you need to replace the HTTP server instead of using either CGI or another server side language (PHP, Java, etc). This is significantly reducing your portability and maintainability.
Finally, you appear to be transmitting the password in the request. Make sure that this is only done over some kind of encrypted (HTTPS) or else physically secured connection.
I'm not sure what you're asking, but you might find the following useful:
HTTP Made Really Easy
HTTP/1.1 rfc2616.txt
MAMA - Opera Developer Community
I found them all quite useful when I was writing a HTTP client.
This problem had occured as after processing the OPTIONS request by our server, any subsequent requests made, for some reason, were required to be responded back with "Access-Control-Allow-Origin: *" along with other normal headers and response body.
After providing this line in our responses, I always got the desired responseText/responseXML in my javascript.
I'm developing a Django app that uses python-openid. The app is running on my development server at home.
Similar to stackoverflow's login mechanism, I'd like users to login to my website using their Google credentials.
The code I've implemented to do this, works well for a couple weeks, and then stops working. I get stuck during the login process on the following Google page: https://www.google.com/accounts/o8/ud with this message: "The page you requested is invalid." It'll randomly start working again, but fails every few weeks or so.
Going through Yahoo's login worked for months, and today has stopped working with the following message: "This page has expired, go back to the original page and please try again" on this page: https://open.login.yahooapis.com/openid/op/auth
Here is the request, as captured by LiveHttpHeaders for Google:
https://www.google.com/accounts/o8/ud
POST /accounts/o8/ud HTTP/1.1
Host: www.google.com
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042513 Ubuntu/8.04 (hardy) Firefox/3.0.10
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://127.0.0.1:8000/users/login/
Content-Length:907
openid.ax.if_available=ext1&openid.mode=checkid_setup&openid.ns=http://specs.openid.net/auth/2.0&openid.realm=http://127.0.0.1:8000/accounts/login/&openid.return_to=http://127.0.0.1:8000/users/login/finish/?janrain_nonce=2009-10-05T19%3A10%3A11ZtioiRm&openid.ax.count.ext1=unlimited&openid.ax.mode=fetch_request&openid.sreg.optional=email&openid.claimed_id=http://specs.openid.net/auth/2.0/identifier_select&openid.ns.sreg=http://openid.net/extensions/sreg/1.1&openid.ns.ax=http://openid.net/srv/ax/1.0&openid.identity=http://specs.openid.net/auth/2.0/identifier_select&openid.assoc_handle=AOQobUcnzec0bpeZRztjqPrr5TQUA0aPL7SIuOPOMgWxex2HRAP09AyJ&openid.ax.required=ext0&openid.ax.type.ext0=http://schema.openid.net/namePerson&openid.ax.type.ext1=http://schema.openid.net/contact/web/default
HTTP/1.x 400 Bad Request
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
I'm not sure what's going on here, and would love some help.
It looks like the code you are using is generating a bad URL request. The real URL is https://www.google.com/accounts/o8/id, so try to fix the "ud" at the end changing it by an "id".
Hope this helps!
you can construct the uri and redirect the user to the uri with GET method. If you doing POST google expects some headers which I think it was not mentioned docs. Check the sample request. I tried with GET without python-openid it works pretty well.
You might take a look at the redirect_uri and the state inside to see if they match. I remember having the issue of having mismatched state sometimes ago with Google Login.
Btw if you use Django, I would recommend using social-app-django which is currently active and supports multiple social login options (if at some point you consider adding more social login providers).