I have a website that I need to pull information from, currently I use this code:
std::stringstream Output;
IStream* Stream;
HRESULT StreamResult = URLOpenBlockingStream(0, WebsiteLink, &Stream, 0, 0);
if (SUCCEEDED(StreamResult))
{
char sBuffer[100];
unsigned long CurrentBRead;
Stream->Read(sBuffer, 100, &CurrentBRead);
while (CurrentBRead > 0U)
{
Output.write(sBuffer, (long long)CurrentBRead);
Stream->Read(sBuffer, 100, &CurrentBRead);
}
Stream->Release();
*Return = Output.str();
return true;
}
When I use this on HTTP websites, it functions properly. But when I need to access anything using HTTPS, it just hangs. After some time, I realized that there is a handshake that I must do to authenticate myself with the website, however I don't know how to do this, and all of the solutions online are just "use libcurl".
Note: I cannot use ANY 3rd party libraries
Related
I am try to upload a file to Amazon S3 using the AWS C++ SDK.
The call to CreateMultipartUpload returns successfully but the following call to UploadPart times out with the following error:
(Aws::String) m_message = "Unable to parse ExceptionName: RequestTimeout Message: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed."
I don't understand why the initiate call works but not the part upload call. There clearly isn't any network issue.
This is my code:
bool FileUploader::uploadChunk() {
Aws::S3::Model::UploadPartRequest request;
request.SetBucket("video");
request.SetKey(_key);
request.SetUploadId(_file->uploadId);
request.SetPartNumber(_file->chunksUploaded + 1);
long file_pos = _file->chunksUploaded * CHUNK_SIZE;
_input_file.seekg(file_pos, std::ios::beg);
_input_file.read(_file_buf, CHUNK_SIZE);
long n_bytes = _input_file.gcount();
if(n_bytes > 0) {
request.SetContentLength(n_bytes);
char_array_buffer buf2(_file_buf, _file_buf + n_bytes);
std::iostream *chunk_stream = new std::iostream(&buf2);
request.SetBody(std::shared_ptr<std::iostream>(chunk_stream));
Aws::S3::Model::UploadPartOutcome response = _client->UploadPart(request);
if(response.IsSuccess()) {
_file->chunksUploaded++;
_uploader->updateUploadStatus(_file);
}
return response.IsSuccess();
}
else {
return false;
}
}
The problem was my method of obtaining a stream for SetBody. I switched to using the boost library instead of a homegrown approach.
typedef boost::iostreams::basic_array_source<char> Device;
boost::iostreams::stream_buffer<Device> stmbuf(_file_buf, n_bytes);
std::iostream *stm = new std::iostream(&stmbuf);
request.SetBody(std::shared_ptr<Aws::IOStream>(stm));
This works well.
I also needed to keep track of the parts I was uploading for the call to CompleteMultipartUpload as follows:
Aws::S3::Model::CompletedPart part;
part.SetPartNumber(request.GetPartNumber());
part.SetETag(response.GetResult().GetETag());
_uploadedParts.AddParts(part);
Alternatively, you can use the TransferManager interface which will do this for you. It has an IOStream interface. In addition we provide a preallocated buffer implementation for iostream:
https://github.com/aws/aws-sdk-cpp/blob/master/aws-cpp-sdk-core/include/aws/core/utils/stream/PreallocatedStreamBuf.h
I've recently started using the Poco library (which I think is great) and I'm trying to create an Server to connect too using an ios application using socket.io - websocket's. I've managed to use an node js implementation to connect but require a C++ implementation. I've stated by instantiating the websocket within the handleRequest method but unsure to what the next steps are...
Any help would be very much appreciated..
virtual void handleRequest(HTTPServerRequest &req, HTTPServerResponse &resp)
{
char buffer[16384];
WebSocket* ws = new WebSocket(req, resp);
//ws->setKeepAlive(false);
int flags;
if (!ws->poll(500,Poco::Net::Socket::SELECT_READ || Poco::Net::Socket::SELECT_ERROR))
{
cout << ".";
}
else
{
int n = ws->receiveFrame(buffer, sizeof(buffer), flags);
if (n > 0)
{
if ((flags & WebSocket::FRAME_OP_BITMASK) == WebSocket::FRAME_OP_BINARY)
{
// process and send out to all other clients
}
}
}
}
Next steps depend on what you are trying to do. Once connected, you have an open channel that you can use for two-way data exchange between browser and server.
For details, see the WebSocketServer example.
How may I download a file in C++ with wxWidgets?
Been googling and everything and nothing shows up! Help appreciated!
Use wxHTTP class for that.
wxHTTP Example Code:
#include <wx/sstream.h>
#include <wx/protocol/http.h>
wxHTTP get;
get.SetHeader(_T("Content-type"), _T("text/html; charset=utf-8"));
get.SetTimeout(10); // 10 seconds of timeout instead of 10 minutes ...
while (!get.Connect(_T("www.google.com")))
wxSleep(5);
wxApp::IsMainLoopRunning();
wxInputStream *httpStream = get.GetInputStream(_T("/intl/en/about.html"));
if (get.GetError() == wxPROTO_NOERR)
{
wxString res;
wxStringOutputStream out_stream(&res);
httpStream->Read(out_stream);
wxMessageBox(res);
}
else
{
wxMessageBox(_T("Unable to connect!"));
}
wxDELETE(httpStream);
get.Close();
If you want more flexible solution consider using libcurl.
Depends on where you want to 'download' it from, and how the file server allows files to be downloaded. The server might use FTP, or HTTP, or something more obscure. There is no way to tell from your question which has no useful information in it.
In general, I would not use wxWidgets for this task. wxWidgets is a GUI frmaework, with some extras for various things that may or may not be helpful in your case.
From HTTP as Andrejs suggest, from FTP using wxFTP
wxFTP ftp;
// if you don't use these lines anonymous login will be used
ftp.SetUser("user");
ftp.SetPassword("password");
if ( !ftp.Connect("ftp.wxwindows.org") )
{
wxLogError("Couldn't connect");
return;
}
ftp.ChDir("/pub");
wxInputStream *in = ftp.GetInputStream("wxWidgets-4.2.0.tar.gz");
if ( !in )
{
wxLogError("Coudln't get file");
}
else
{
size_t size = in->GetSize();
char *data = new char[size];
if ( !in->Read(data, size) )
{
wxLogError("Read error");
}
else
{
// file data is in the buffer
...
}
delete [] data;
delete in;
}
http://docs.wxwidgets.org/stable/wx_wxftp.html#wxftp
You did not define what "downloading a file" means to you.
If you want to use HTTP to retrieve some content, you should use an HTTP client library like libcurl and issue the appropriate HTTP GET request.
I am starting to develop an app to access the Amazon S3 storage using the SOAP API.
I have read the documents that says the the method PutObject must be used if the file size is greater than 1 MB.
Now PutObject uses DIME attachment.
Is there a sample code or example or a fragment of code that someone can show me on how to do DIME attachement with GSOAP for the PutObject method of Amazon S3.
I want to use GSOAP because of portability and to make it generic. I do not want to use the .NET API provided by Amazon for the same reason. I want in GSOAP particularly as I have worked in GSOAP earlier.
Thanks,
david
I put together something that uploads files larger than 1MB using PutObject, it should also work for smaller files.
I share it for others who might find it useful.
Also see my previous post on using GSOAP to access S3 AMAZON AWS S3 using GSOAP C C++
The link also contains the method to generate the signature.
Here is the code for PutObject.
It uses the latest GSOAP from sourceforge.
After wsdl2h to generate the header and soapcpp2 to generate the gsoap client code the following will be the code to access the service PutObject......
Requirements : OpenSSL GSOAP Build with the compiler preprocessor directive WITH_OPENSSL. Include the library files libeay32 and ssleay32. Take the methods to generate signature from the link above.
void PutObject(char *filename)
{
AmazonS3SoapBindingProxy amazonS3Interface;
struct soap* soapPtr;
soapPtr = dynamic_cast<struct soap*>(&amazonS3Interface);
soap_init2(soapPtr, SOAP_IO_DEFAULT|SOAP_IO_KEEPALIVE, SOAP_IO_DEFAULT|SOAP_IO_KEEPALIVE);
soap_ssl_client_context(&amazonS3Interface,
SOAP_SSL_NO_AUTHENTICATION, /* for encryption w/o authentication */
/* SOAP_SSL_DEFAULT | SOAP_SSL_SKIP_HOST_CHECK, */ /* if we don't want the host name checks since these will change from machine to machine */
/*SOAP_SSL_DEFAULT,*/ /* use SOAP_SSL_DEFAULT in production code */
NULL, /* keyfile (cert+key): required only when client must authenticate to server (see SSL docs to create this file) */
NULL, /* password to read the keyfile */
NULL, /* optional cacert file to store trusted certificates, use cacerts.pem for all public certificates issued by common CAs */
NULL, /* optional capath to directory with trusted certificates */
NULL /* if randfile!=NULL: use a file with random data to seed randomness */
);
//use this if you are behind a proxy to connect to internet
amazonS3Interface.proxy_host="proxyservername"; //proxyservername
amazonS3Interface.proxy_port=4050; //proxy port
amazonS3Interface.proxy_userid="username"; //proxy authentication
amazonS3Interface.proxy_passwd="password";
amazonS3Interface.proxy_http_version="1.1"; //http ver
amazonS3Interface.dime_id_format ="uuid:09233523-345b-4351-b623-5dsf35sgs5d6-%x";
// Set callback functions
soapPtr->fdimereadopen = dime_read_open;
soapPtr->fdimereadclose = dime_read_close;
soapPtr->fdimeread =dime_read;
_ns1__PutObject preq;
_ns1__PutObjectResponse presp;
ns1__PutObjectResult res;
FILE *fp=fopen(filename,"rb");
fseek(fp, 0L, SEEK_END);
size_t sz = ftell(fp);
fseek(fp, 0L, SEEK_SET);
preq.Bucket=std::string("FGTSDrive");//bucket name to put file in
preq.AWSAccessKeyId=new std::string("ACCESSKEY");//access key here
char *sig=aws_signature("SECRETKEY","AmazonS3","PutObject",xml_datetime(),NULL);//correct secretkey here
preq.Signature=new std::string(sig);
preq.Timestamp=new time_t(time(NULL));
preq.Key=std::string(filename);//name of the key ie the filename
int result(0);
preq.ContentLength=sz; //length of the file
ns1__MetadataEntry med;
med.Name=std::string("Content-Type");
med.Value=std::string("application/zip");//change the type depending on the file extenstion
med.soap=&amazonS3Interface;
preq.Metadata.push_back(&med);
soap_set_dime(soapPtr);
result =soap_set_dime_attachment(soapPtr, (char*)fp, sz,"application/zip", NULL, 0,filename);//change the content type depending on the file extenstion
if (result != SOAP_OK) { }
result = amazonS3Interface.PutObject(&preq, &presp);
if (result != SOAP_OK) { }
amazonS3Interface.soap_stream_fault(std::cout);
}
static void *dime_read_open(struct soap *soap, void *handle, const char *id, const char *type, const char *options)
{ // we should return NULL without setting soap->error if we don't want to use the streaming callback for this DIME attachment. The handle contains the non-NULL __ptr field value which should have been set in the application.
// return value of this function will be passed on to the fdimeread and fdimereadclose callbacks. The return value will not affect the __ptr field.
std::cout <<"dime_read_open"<<std::endl;
return handle;
}
static void dime_read_close(struct soap *soap, void *handle)
{
std::cout <<"dime_read_close"<<std::endl;
fclose((FILE*)handle);
}
static size_t dime_read(struct soap *soap, void *handle, char *buf, size_t len)
{
std::cout <<"dime_read_read"<<std::endl;
return fread(buf, 1, len, (FILE*)handle);
}
Hope it helps.
Thanks,
david
My code talks to a little Java application provided by a vendor. This Java app sets up a web server at localhost:57000 which is used to control the state of 'the machine'. For the purpose of this question, I need to change the state of 'the machine' from 'off' to 'on'. To make this happen I'm supposed to HTTP PUT the following string to 'the machine' at http://localhost:57000/settings.xml:
<settings><machine_state><status>on</status></machine_state></settings>
This Curl command works perfectly:
curl -X PUT -H "Content-Type:application/xml" -d #settings.xml http://localhost:57000/settings.xml"
where the local file 'settings.xml' has the above xml string in it.
I want to do what Curl is doing with MFC's WININET classes. The following code should IMHO do exactly the same thing that curl does. Sadly, although the localhost web server returns a code 200 it ignores my xml string. What little thing am I missing?
int MyHttp::HttpPutThread() NOTHROW
{
try {
m_xml = "<settings><machine_state><status>on</status></machine_state></settings>";
m_url = "settings.xml"
CInternetSession session;
SetSessionOptions(session);
CString server = "localhost:57920";
boost::scoped_ptr<CHttpConnection> phttp(session.GetHttpConnection(server));
LPCTSTR accept = 0;//"text/xml";
boost::scoped_ptr<CHttpFile> phttpfile(phttp->OpenRequest(
"PUT", //verb
"settings.xml", //object name
0, //referer
1, //context
&accept, // accept types
0, //version
INTERNET_FLAG_EXISTING_CONNECT));
CString header = "Content-Type:application/xml\r\n";
if(phttpfile->SendRequest(header,(LPVOID)m_xml.GetBuffer(), m_xml.GetLength()))
{ // LOG_DEBUG (Same as TRACE) output are shown in comment
DWORD code(0);
phttpfile->QueryInfoStatusCode(code);
LOG_DEBUG("HttpPutThread result code: %d", code); // '200'
CString object = phttpfile->GetObject();
LOG_DEBUG("object: %s", object); // 'settings.xml'
CString statustxt;
phttpfile->QueryInfo(HTTP_QUERY_STATUS_TEXT,statustxt);
LOG_DEBUG("status text:%s", statustxt); // 'HTTP/1.0 200 OK'
CString rawheaders;
phttpfile->QueryInfo(HTTP_QUERY_RAW_HEADERS,rawheaders);
LOG_DEBUG("raw headers:%s", rawheaders); // http://localhost:57000/settings.xml
LOG_DEBUG("File url:%s",phttpfile->GetFileURL());
LOG_DEBUG("Verb:%s", phttpfile->GetVerb()); // 'PUT'
} else
{
//This does not happen
LOG_DEBUG("PUT failed in AffHttp::HttpPutThread");
}
} catch(CInternetException* pe)
{
//No exceptions are thrown
LOG_DEBUG("Exception HttpPutThread:%d", pe->m_dwError);
pe->Delete();
}
return 0;
}
Thanks in advance.
I wound up replacing the MFC classes with my own low level socket code to send exactly the same text in exactly the same order as Curl did. It seemed like the little embedded 'jetty' Java server just objected to one of the headers generated by the MFC classes.