I'm new to C++ and Boost Asio, I wanted to use Boost Asio to create a simple HTTP Get request to a website having oAthn2 (bearer token) in my program but it doesn't work and I don't know why, I have tried to libCurl and separate Http request in Clion and they worked well without any problems.
#include "netCommon/net_Common.h"
#include <chrono>
#include <thread>
using namespace boost;
typedef asio::ip::tcp ip;
int main() {
boost::system::error_code ec;
asio::io_service ios;
//The website
std::string host_name = "api-fxpractice.oanda.com";
std::string port = "443";
//resolve DNS
ip::resolver resolver_dns(ios);
ip::resolver::query query_dns(host_name,port,asio::ip::tcp::resolver::numeric_service);
ip::resolver::iterator it = resolver_dns.resolve(query_dns,ec);
//Create a socket
ip::socket sock(ios);
//Connect to endpoint
asio::connect(sock,it);
if (ec){
std::cout << " Failed to connect: " << ec.message() << std::endl;
return ec.value();
}
std::cout << "Connected successfully !!! " << std::endl;
if (sock.is_open()) {
//Create a stream buffer
std::stringstream request_stream;
request_stream << "GET https://api-fxpractice.oanda.com/v3/accounts HTTP/1.1\r\n"
"Authorization: Bearer <my token>\r\n"
"Connection: close\r\n\r\n";
//Tried to change to Host: https://api-fxpractice.oanda.com, doesn't work either
const auto request = request_stream.str();
asio::write(sock, asio::buffer(request));
using namespace std::chrono_literals;
std::this_thread::sleep_for(2000ms);
size_t bytes = sock.available();
if (bytes > 0) {
std::vector<char> vBuffer(bytes);
std::cout << "Have something to read !" << std::endl;
asio::read(sock, asio::buffer(vBuffer));
for (auto c: vBuffer) {
std::cout << c;
}
}
}
It's connected to the site but I can't get any data.
It keeps sending me either "HTTP/1.1 400 Bad Request" or "The plain HTTP request ... send to HTTPS port"
I'm not sure what I'm doing wrong here. :(
thank you guys so much.
Related
i need this solution with c++. Example txt is here "www.asd.com/try.txt" i need read from txt. I have an error Error: Host not found (non-authoritative), try again later. What is my mistake? (website and path are true "asd" is just example)
my code:
#include <iostream>
#include <string>
#include <boost/asio.hpp>
int main()
{
boost::asio::ip::tcp::iostream stream;
stream.connect("www.asd.com", "http");
if (!stream)
{
std::cout << "Error: " << stream.error().message() << "\n";
}
stream << "GET /ip.txt HTTP/1.0\r\n\r\n";
stream << "Host: www.asd.com\r\n";
stream << "Accept: */*\r\n";
stream << "Connection: close\r\n\r\n";
for (std::string line; getline(stream, line);)
{
std::cout << line << std::endl;
}
}
It means the domain (or service) cannot be resolved to an endpoint.
That domain is www.asd.com here, and the service is http. Both resolve on my machine, but it depends on your host: e.g. in some settings (containers?) there might not be any DNS resolver configured.
I'm trying to lean Boost::Asio networking library for C++ by watching this video but I stuck at making request using threads asynchronously.
The code :
#include "stdafx.h"
#include <iostream>
#include <boost/asio.hpp>
#include <boost/asio/ts/buffer.hpp>
#include <boost/asio/ts/internet.hpp>
#include <boost/system/error_code.hpp>
std::vector<char> vBuffrer(20 * 1024);
void GrabSomeData(boost::asio::ip::tcp::socket& socket) {
socket.async_read_some(boost::asio::buffer(vBuffrer.data(), vBuffrer.size()),
[&](std::error_code ec, std::size_t length)
//boost::system::error_code ec
{
if (!ec)
{
std::cout << "\n\nRead" << length << "bytes\n\n";
for (int i = 0; i < length; i++)
std::cout << vBuffrer[i];
GrabSomeData(socket);
}
});
}
int main()
{
boost::system::error_code ec;
boost::asio::io_context context;
boost::asio::io_context::work idleWork(context);
boost::asio::ip::tcp::endpoint endpoint(boost::asio::ip::make_address("13.107.21.200",ec),80);
boost::asio::ip::tcp::socket socket(context);
std::thread thrContext = std::thread([&]() {context.run(); });
std::cout << "Starting " << std::endl;
socket.connect(endpoint,ec);
if (!ec)
{
std::cout << "Connected ! " << std::endl;
}
else {
std::cout << "Fail to connect ! " << ec.message() << std::endl;
}
if (socket.is_open()) {
GrabSomeData(socket);
std::string sRequest =
"GET /index.html HTTP/1.1\r\n"
"Host: www.example.com\r\n"
"Connection: close\r\n\r\n";
socket.write_some(boost::asio::buffer(sRequest.data(), sRequest.size()), ec);
using namespace std::chrono_literals;
std::this_thread::sleep_for(2800ms);
//std::this_thread::sleep_for(1ms);
context.stop();
if (thrContext.joinable()) thrContext.join();
}
system("pause");
return 0;
}
Microsoft Visual studio give me this :
Error C2752 'asio_prefer_fn::call_traits<boost::asio::execution::any_executor<boost::asio::execution::context_as_t<boost::asio::execution_context &>,boost::asio::execution::detail::blocking::never_t<0>,boost::asio::execution::prefer_only<boost::asio::execution::detail::blocking::possibly_t<0>>,boost::asio::execution::prefer_only<boost::asio::execution::detail::outstanding_work::tracked_t<0>>,boost::asio::execution::prefer_only<boost::asio::execution::detail::outstanding_work::untracked_t<0>>,boost::asio::execution::prefer_only<boost::asio::execution::detail::relationship::fork_t<0>>,boost::asio::execution::prefer_only<boost::asio::execution::detail::relationship::continuation_t<0>>> &,void (const boost::asio::execution::detail::blocking::possibly_t<0> &,boost::asio::execution::allocator_t<std::allocator<void>>),void>': more than one partial specialization matches the template argument list boostasiotest c:\boost\boost_1_75_0\boost\asio\detail\handler_work.hpp 353
Error C2893 Failed to specialize function template 'enable_if<asio_prefer_fn::call_traits<T,void(P0,P1,PN...),void>::overload==,call_traits<T,void(P0,P1,PN...),void>::result_type>::type asio_prefer_fn::impl::operator ()(T &&,P0 &&,P1 &&,PN &&...) noexcept(<expr>) const' boostasiotest c:\boost\boost_1_75_0\boost\asio\detail\handler_work.hpp 353
Everything worked fine till I added GrabSomeData function and I have absolutely no idea how to fix it , any help would be appreciated.
PS : there is an example on the Boost website on this subject, but it is object oriented and all the pointers are referring to the class and I (think) it can't help.
Like the commenter, I cannot repro your message: it just compiles,
MSVC 19, /std:c++14, Boost 1.75.0: compiler explorer
Now, I do see other issues:
write_some may not write all the data - you will want to ensure a composed-write operation
a race condition: since you're doing GrabSomeData on a thread, you
need to synchronize access to the tcp::socket and buffer (the
shared resources).
io_context itself is thread-safe.
In this case, it's really easy to avoid, since you don't need to
start the async operation until after you sent the request:
write(socket, boost::asio::buffer(sRequest));
GrabSomeData(socket);
async_read_some has a similar problem as the write side. You will want a composed read operation that reads the expected output so read_until(socket, buf, "\r\n\r\n") and then read how much content is expected based on Content-Length header, Connection: Close and others (think chunked encoding).
You currently have no good way to store and access the response. It would be a lot easier to use a streambuf, a single composed read.
If you want to be really solid, use Beast to receive a HTTP/1.1 response (which can even be chunked) and not worry about when it's complete (the library does it for you):
auto GrabSomeData(tcp::socket& socket) {
http::response<http::string_body> res;
auto buf = boost::asio::dynamic_buffer(vBuffer);
http::read(socket, buf, res);
return res;
}
Oh, and don't do it on a thread (why was that anyways? it literally just introduced undefined behavior for no gain):
Simplified Code
Live On Coliru
Compiles on MSVC: Godbolt
#include <boost/asio.hpp>
#include <boost/beast/http.hpp>
#include <iostream>
#include <iomanip>
using boost::asio::ip::tcp;
std::vector<char> vBuffer; // TODO FIXME global variable
auto GrabSomeData(tcp::socket& socket) {
namespace http = boost::beast::http;
http::response<http::string_body> res;
auto buf = boost::asio::dynamic_buffer(vBuffer);
http::read(socket, buf, res);
return res;
}
int main() try {
boost::asio::io_context context;
std::cout << "Starting " << std::endl;
tcp::endpoint endpoint(boost::asio::ip::make_address("13.107.21.200"), 80);
tcp::socket socket(context);
socket.connect(endpoint);
std::cout << "Connected" << std::endl;
std::string const sRequest = "GET /index.html HTTP/1.1\r\n"
"Host: www.example.com\r\n"
"Connection: close\r\n"
"\r\n";
write(socket, boost::asio::buffer(sRequest));
auto response = GrabSomeData(socket);
std::cout << "Response body length: " << response.body().size() << std::endl;
std::cout << "Response headers: " << response.base() << std::endl;
std::cout << "Response body: " << std::quoted(response.body()) << std::endl;
context.run(); // run_for(10s) e.g.
} catch (boost::system::system_error const& se) {
std::cerr << "Error: " << se.code().message() << std::endl;
}
This sample printed:
Starting
Connected
Response body length: 194
Response headers: HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
X-MSEdge-Ref: 0BstXYAAAAACeQ2y+botzQISiBe2U3iGCQ0hHRURHRTE0MDgARWRnZQ==
Date: Sun, 21 Mar 2021 22:39:02 GMT
Connection: close
Response body: "<h2>Our services aren't available right now</h2><p>We're working to restore all services as soon as possible. Please check back soon.</p>0BstXYAAAAACeQ2y+botzQISiBe2U3iGCQ0hHRURHRTE0MDgARWRnZQ=="
NOTE It does indeed use chunked encoding, which you didn't seem to be anticipating.
Thanks #sehe for the answer and recommendations, it took so long but I've upgrade to New Windows10 and Visual Studio 2019, and Boost 1_76 and the problem solved!
Those errors were totally irrelevant to the code!
I've been unable to connect to Watson's Text-To-Speech service via websockets in C++ using the Boost Beast library
I've got code that successfully interfaces with echo.websocket.org on port 80, but it does not work with Watson's url. I've tried using variants of protocols (http(s), ws(s), and unspecified (which works for echo.websocket.com)) and I've tried both port 80 and 443, just to be sure.
I was able to get successful code running in Javascript, and using Firefox's inbuilt networking tool, I've verified it works over port 443. Using the exact same URL and port number gives me the following: "No such host is known."
Here is the relevant JS code that correctly establishes connection
var completeUrl = "wss://stream.watsonplatform.net/text-to-speech/api/v1/synthesize?access_token=" + accessToken + "&voice=en-US_AllisonVoice";
socket = new WebSocket(completeUrl);
The following C++ code works fine in theory, and works with echo.websocket.org on port 80, but not Watson.
#include <boost/beast/core.hpp>
#include <boost/beast/websocket.hpp>
#include <boost/asio/connect.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <cstdlib>
#include <iostream>
#include <string>
using tcp = boost::asio::ip::tcp; // from <boost/asio/ip/tcp.hpp>
namespace websocket = boost::beast::websocket; // from <boost/beast/websocket.hpp>
// Sends a WebSocket message and prints the response
int main(int argc, char** argv)
{
try
{
std::string accessToken = "XXXXX";
auto const text = "The quick brown fox jumps over the lazy dog.";
std::string baseURL = "wss://stream.watsonplatform.net/text-to-speech/api/v1/synthesize";
std::string voiceModel = "en-US_AllisonVoice";
auto const port = "443"; // port 80 for echo.websocket.org
// port 443 for watson
std::string const host = baseURL + "?access_token=" + accessToken + "&voice=" + voiceModel;
//std::string const host = "echo.websocket.org";
boost::asio::io_context ioc;
tcp::resolver resolver{ ioc };
websocket::stream<tcp::socket> ws{ ioc };
auto const results = resolver.resolve(host, port); // Problem line - "resolve: No such host is known"
std::cout << "Host resolved" << std::endl;
boost::asio::connect(ws.next_layer(), results.begin(), results.end());
ws.handshake(host, "/");
std::cout << "Connection established" << std::endl << "------------------------------" << std::endl;
ws.write(boost::asio::buffer(std::string(text)));
std::cout << "Client request: " << text << std::endl;
boost::beast::multi_buffer buffer;
ws.read(buffer);
ws.close(websocket::close_code::normal);
std::cout << "Server response: " << boost::beast::buffers(buffer.data()) << std::endl;
}
catch (std::exception const& e)
{
std::cerr << "Error: " << e.what() << std::endl;
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
If the Watson URL was wrong, it shouldn't work in Javascript. If the C++ code was wrong, it shouldn't work with echo.websocket.org. So I have no idea what the issue is.
The baseURL in your code specifies "wss" which means Secure Websockets (SSL). But your stream is delcared as a plain stream. If you want to connect to secure websocket servers, you should base your code on the websocket-client-ssl example instead:
https://github.com/boostorg/beast/blob/d43d9421a40c0251614bc45ea6dcf921a3dbaf37/example/websocket/client/sync-ssl/websocket_client_sync_ssl.cpp#L64
I'm trying to send a http GET request using the REST API of some domain.
Basically what I'm trying to do is to replace following curl request:
curl -k -H "Content-Type: application/json" -X GET
--data '{"username":"user#name.co", "password":"test"}' https:/domain.name/api/login/
with some c++ code using boost::asio.
I do not what to find all c++ code here , but some checkpoints would be great.
I found myself also working with boost in order to send some "customized" HTTP GET request - I ran the server locally on my Ubuntu machine (16.04).
In my case the request was of some proprietary API my server implemented (related to words it kept in its data base), but you can modify the queryStr variable to hold whatever query string you wish for.
Also, change the argv[1] and argv[2] when you ran the program to hold your required values (IP address, query, and port if needed - default is 80).
#include <iostream>
#include <istream>
#include <ostream>
#include <string>
#include <boost/asio.hpp>
using boost::asio::ip::tcp;
using namespace std;
int main(int argc, char* argv[])
{
cout << "main -start" << endl;
try
{
boost::asio::io_service io_service;
string ipAddress = argv[1]; //"localhost" for loop back or ip address otherwise, i.e.- www.boost.org;
string portNum = argv[2]; //"8000" for instance;
string hostAddress;
if (portNum.compare("80") != 0) // add the ":" only if the port number is not 80 (proprietary port number).
{
hostAddress = ipAddress + ":" + portNum;
}
else
{
hostAddress = ipAddress;
}
string wordToQuery = "aha";
string queryStr = argv[3]; //"/api/v1/similar?word=" + wordToQuery;
// Get a list of endpoints corresponding to the server name.
tcp::resolver resolver(io_service);
tcp::resolver::query query(ipAddress, portNum);
tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);
// Try each endpoint until we successfully establish a connection.
tcp::socket socket(io_service);
boost::asio::connect(socket, endpoint_iterator);
// Form the request. We specify the "Connection: close" header so that the
// server will close the socket after transmitting the response. This will
// allow us to treat all data up until the EOF as the content.
boost::asio::streambuf request;
std::ostream request_stream(&request);
request_stream << "GET " << queryStr << " HTTP/1.1\r\n"; // note that you can change it if you wish to HTTP/1.0
request_stream << "Host: " << hostAddress << "\r\n";
request_stream << "Accept: */*\r\n";
request_stream << "Connection: close\r\n\r\n";
// Send the request.
boost::asio::write(socket, request);
// Read the response status line. The response streambuf will automatically
// grow to accommodate the entire line. The growth may be limited by passing
// a maximum size to the streambuf constructor.
boost::asio::streambuf response;
boost::asio::read_until(socket, response, "\r\n");
// Check that response is OK.
std::istream response_stream(&response);
std::string http_version;
response_stream >> http_version;
unsigned int status_code;
response_stream >> status_code;
std::string status_message;
std::getline(response_stream, status_message);
if (!response_stream || http_version.substr(0, 5) != "HTTP/")
{
std::cout << "Invalid response\n";
return 1;
}
if (status_code != 200)
{
std::cout << "Response returned with status code " << status_code << "\n";
return 1;
}
// Read the response headers, which are terminated by a blank line.
boost::asio::read_until(socket, response, "\r\n\r\n");
// Process the response headers.
std::string header;
while (std::getline(response_stream, header) && header != "\r")
{
std::cout << header << "\n";
}
std::cout << "\n";
// Write whatever content we already have to output.
if (response.size() > 0)
{
std::cout << &response;
}
// Read until EOF, writing data to output as we go.
boost::system::error_code error;
while (boost::asio::read(socket, response,boost::asio::transfer_at_least(1), error))
{
std::cout << &response;
}
if (error != boost::asio::error::eof)
{
throw boost::system::system_error(error);
}
}
catch (std::exception& e)
{
std::cout << "Exception: " << e.what() << "\n";
}
return 0;
}
The original example I followed is here: Boost-simple-http-get-request-sync-client
boost::asio is not an application level library. So you can open a connection with it, do an SSL handshake and so on. But you cannot construct HTTP requests via boost::asio since it's designed to just send/receive data.
But, you can try to experiment with this asio HTTP client example. Probably this can be a good thing to start with.
There is an Urdl library, created by Christopher M. Kohlhoff, the author of Boost.Asio:
Urdl is a cross-platform C++ library for downloading web content using
a URL. It provides an easy-to-use extension to standard C++ iostreams
and an asynchronous interface for use with Boost.Asio.
Your request sample would be like following:
#include <urdl/http.hpp>
urdl::istream is;
// Set request options: method, content type, content
is.set_option(urdl::http::request_method("GET"));
is.set_option(urdl::http::request_content_type("application/json"));
is.set_option(urdl::http::request_content("{\"username\":\"user#name.co\", \"password\":\"test\"}"));
// open HTTP stream at URL
is.open("https:/domain.name/api/login/");
I would like to read a source code of an thml page with boost::asio.
Here is my sample code :
#include "stdafx.h"
#include <boost/algorithm/string/replace.hpp>
#include <boost/asio.hpp>
#include <vector>
#include <string>
#include <algorithm>
#include <sstream>
#include <ostream>
int _tmain(int argc, _TCHAR* argv[])
{
std::string strHost = "www.fdj.fr";
std::string strPort = "80";
std::string strUrlPath = "https://www.fdj.fr/jeux/jeux-de-tirage/keno/resultats";
std::string strUserAgent = "Fiddler";
//std::vector<std::string> header;
unsigned int TimeOut = 5000;
//do_get(strHost, strPort, strUrlPath, header, TimeOut);
try
{
using boost::asio::ip::tcp;
boost::asio::io_service io_service;
// Get a list of endpoints corresponding to the server name.
tcp::resolver resolver(io_service);
tcp::resolver::query query(strHost, "http");
tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);
tcp::resolver::iterator end;
// Try each endpoint until we successfully establish a connection.
tcp::socket socket(io_service);
boost::system::error_code error = boost::asio::error::host_not_found;
while (error && endpoint_iterator != end)
{
socket.close();
socket.connect(*endpoint_iterator++, error);
}
if (error)
throw boost::system::system_error(error);
// Form the request. We specify the "Connection: close" header so that the
// server will close the socket after transmitting the response. This will
// allow us to treat all data up until the EOF as the content.
boost::asio::streambuf request;
std::ostream request_stream(&request);
request_stream << "GET " << strUrlPath << " HTTP/1.0\r\n";
request_stream << "User-Agent: " << strUserAgent << "\r\n";
request_stream << "Host: " << strHost << "\r\n";
request_stream << "Accept: */*\r\n";
request_stream << "Connection: close\r\n\r\n";
// Send the request.
boost::asio::write(socket, request);
// Read the response status line.
boost::asio::streambuf response;
boost::asio::read_until(socket, response, "\r\n");
// Check that response is OK.
std::istream response_stream(&response);
std::string http_version;
response_stream >> http_version;
unsigned int status_code;
response_stream >> status_code;
std::string status_message;
std::getline(response_stream, status_message);
if (!response_stream || http_version.substr(0, 5) != "HTTP/")
{
std::cout << "Invalid response\n";
//return 1;
}
/*if (status_code != 200)
{
std::cout << "Response returned with status code " << status_code << "\n";
//return 1;
}*/
// Read the response headers, which are terminated by a blank line.
boost::asio::read_until(socket, response, "\r\n\r\n");
// Process the response headers.
std::string header;
while (std::getline(response_stream, header) && header != "\r")
std::cout << header << "\n";
std::cout << "\n";
// Write whatever content we already have to output.
if (response.size() > 0)
std::cout << &response;
// Read until EOF, writing data to output as we go.
while (boost::asio::read(socket, response,
boost::asio::transfer_at_least(1), error))
std::cout << &response;
if (error != boost::asio::error::eof)
throw boost::system::system_error(error);
char a;
std::cin>>a ;
}
catch (std::exception& e)
{
std::cout << "Exception: " << e.what() << "\n";
}
return 0;
}
The request sended is :
GET https://www.fdj.fr/jeux/jeux-de-tirage/keno/resultats HTTP/1.0
User-Agent: Fiddler
Host: www.fdj.fr
Accept: */*
Connection: close
If i run my programm i have :
If i run this request in fiddler ( by copy/paste) i have :
return code=200
(Accepted) and the page is loaded !
Anyone have an idea why i have this 301 error ?
I think it's due to an boost::asio parameters, but which ?
Thanks a lot,
Best regards,
Nixeus
There are two things happening here:
You are attempting to load an HTTPS URI over plain HTTP. This results in a 301 redirect that points you back to the original page over HTTPS. This indicates that the hosting server does not provide this resource over insecure HTTP.
Your HTTP implementation does not support redirects. You are making a request and delivering the first response. A full HTTP client would check for a 301 response and retry using the URI in the location header of the 301.
To fix this you will need to implement support for TLS/HTTPS and ideally HTTP redirects.