Note: If you dont have time to read this long journey, the solution (with sourcecode) is here: http://www.coastrd.com/smtps.
For a long time sending email uing SMTP (port 25) via a remote mail server (usually at the website hosting company) was easy to do with an application. Open a TCP port 25, send "HELO ..." etc
To do this using googles email service is giving me a problem because they insist on using port 465 SMTPS ie SMTP with TLS encryption:
http://en.wikipedia.org/wiki/Transport_Layer_Security#How_it_works
In researching a way to do this with a language like C++ or a flavor of basic, i came across:
http://forums.realsoftware.com/viewtopic.php?f=2&t=29542
http://forums.realsoftware.com/viewtopic.php?f=2&t=26959&p=162671#p162671
and a Python question:
python smtp gmail authentication error (sending email through gmail smtp server)
If I am understanding this correctly, I am going to need to implement the TLS encryption in my C++ code, complete with all the hand shaking and negotiation?
From the C# question:
sending email with gmail smtp ( secure layer ) in c++
This library does not do it
http://johnwiggins.net/jwsmtp/
ADDED:
A lot of people are just installing the stunnel as a service and then configuring it to manage the an SSL connection
http://www.stunnel.org/about/
Stunnel is an OpenSSL wrapper. OpenSSL has some perfomance issues (http://josefsson.org/gnutls4win/)
"Initializing libgcrypt takes a long time on some systems, there has been reports that it can take around 10 seconds."
and requires: "libeay32.dll" 1.35MB + "libssl32.dll" 310k + "zlib1.dll" 75k
Then thre are a couple of commercial products:
http://www.chilkatsoft.com/downloads.asp
This product is mostly delivered as an Activex (COM) "dll" (requiring an installer on the users machine to 'register' the dll - another bad .net idea).
The installer loads "ChilkatMime.dll" 1.33Mb, "ChilkatCert.dll" 1.26MB, "ChilkatUtil.dll" 720k. The developers were not at all interested in cooperating on a true C .dll library that could be called from any language including C/C++/BASIC/Python etc etc. Given their attitude I am not surprised they have been the victim of code generators made by hackers.
Apart from the cheesy name and artwork, their products are reasonably priced, but the one I tried, connected on port 25 despite being told to use port 465.
By contrast, a commercial option from catalyst:
http://www.catalyst.com/products/sockettools/secure/library/index.html
is now available as component of the main socket tools product for 1/3 the price. These tools are first class! yes, you get what you pay for. The developers are responsive and open to suggestions. They offer ALL flavors of dll including a stand alone .dll that can be shipped with you product that is only 230k! For commecial solutions they win hands down.
An SLL/TLS connection can be made explicitly (as soon as the handshake begins the seesion) or implicitly (after the handshake using STARTTLS etc)
CodeIgniter is implicit for example (as are options in Python, asp, php etc)
http://codeigniter.com/forums/viewthread/84689/
Once the connection has been made, a "tunnel" exists through which a MIME session may proceed:
"EHLO " + sLocalHost + CRLF
"MAIL FROM: " + sMailFrom + CRLF
"RCPT TO: " + "me#mydomain.com" + CRLF
"DATA: Testing, Testing xyz" + CRLF
CRLF + "." + CRLF
"QUIT"
with the usual responses from the server.
Some languages handle the MIME communication for you (socket tools, codeigniter, etc) and you just feed in the email subject, body and address making it very easy to use
CryptLib is an open source solution that facilitates an SSL/TLS tunnel with a C style .dll in only 1MB (full compilation). Since the source is available, it is possible to compile a version of the dll with just the components you need that should come in somewhat less than that.
http://www.cs.auckland.ac.nz/~pgut001/cryptlib/download.html
The author was very responsive even though I got the library to work immediately and was asking about the MIME dialog. There is 330 page manual! Thank you.
THis library is not an MTA (mail transfer agent) so you must write the MIME conversation above, but it is FREE!
source code available here:
http://www.coastrd.com/smtps.
Check out http://sourceforge.net/projects/libquickmail/ .
This library can send mail with optional attachments to multiple recipients.
The SMTP transport relies on libcurl, so it supports things like authentication and TLS.
The C API is very simple to use.
Tested on Linux (GCC) and Windows (MinGW) but should work on any platform where libcurl is available.
You are correct that you'll need to enable TLS in your application. Instead of doing this on your own, I'd suggest looking into OpenSSL.
Additionally, You need to enable SMTP in your account and support SMTP authentication to send traffic through Gmail.
There is also a duplicate question that has some pointers and a C# implementation with code that might be able to help you out.
There is also a library that might be easier to use than rolling your own (although it doesn't currently have TLS support).
Related
I would like to implement the server side of a licence management software. I use C++ in LINUX OS.
When the software starts it must connect to a server that checks privileges and allows/disallow running of some features.
My question is about the implementation of the communication between client and server across internet:
The server will have a static IP on internet so is it enough to use a simple TCP/IP socket client that will connect to a TCP/IP socket server ( providing IP/PORT) ?
I am familiar with socket communication , but less with communication across internet so my question is whether this is the right approach or do I need to use a different mechanism like a http client server or other.
Regards
AFG
Here are some benefits to using HTTP as a transport:
easier to get right, more likely to work in production: Yes, you will probably have to add additional dependencies to deal with HTTP (client and server side), but it's still preferable to yet another homegrown protocol, which you have to implement, maintain, care about backwards compatibility, deal with multiplatform issues (eg. endianness), etc. In terms of implementation ease, using an HTTP based solution should be far easier in the common case (especially true if you build a REST style service API for license checking).
More help available: HTTP as the foundation of the web is one of the most widely used technologies today. Most (all?) problems you will run into are probably publicly documented with solutions/workarounds.
Encryption 'for free': Encryption is already a solved problem (HTTPS/SSL), both with regard to transport as well as with regard to what you have to implement on your end, and it's just a matter of setting it up.
Server Authentication 'for free': HTTPS/SSL doesn't only solve encryption but also server authentication, so that the client can verify whether it's actually talking to the right service.
Guaranteed to work on the internet: HTTP/HTTPS traffic is common on the internet, so you won't run into routing problems or firewalls which are hard to traverse. This might be a problem when using your own protocol.
Flexibility out of the box: You also put less constraints on clients communicating with your server, as it's very simple to build a client in many different environments, as long as they can talk HTTP (and maybe SSL), and they know how to issue the request to your server (ie. what your service API looks like).
Easy to integrate with administrative webapp: If you want to allow users to manage their accounts associated with licenses in some way (update contact info etc.), then you might even combine the license server with that application. You can also build the license administration UI part into the same app if that's useful.
And as a last remark (this puts additional constraints on your client side HTTPS/SSL implementation): you can even use client side SSL certificates, which essentially allow authenticating the client to the server. Depending on how you use them, client side certificates are harder to manage, but they can be eg. expired, or revoked, so to some extent they actually are licenses (to connect to the server).
HTTP is not a different mechanism. It is a protocol operated over TCP/IP connections.
Internet uses IP transport exclusively. You can use UDP, TCP or SCTP session (well, UDP is not much of a session) layer on top of it. TCP is the general choice.
Sockets are operating system interface. They are the only interface to network in most systems, but some systems have different interface. Nothing to do with the transport itself.
IP addresses are in practice tied to network topology, so I strongly discourage hardcoding the IP address into the server. If you have to change network provider for any reason, you won't be getting the same IP address. Use DNS, it's just one gethostbyname call.
And don't forget to authenticate the server; even with hardcoded IP it's too easy to redirect it.
I'm in the process of creating a utility to backup user's media files. The media isn't being shared etc its only a backup utility.
I'm trying to think of the best way to protect users from ISPs accusing them of downloading illegal media files by using some sort of secure connection.
The utility is written in C++ using the Qt lib and so far I've only been able to find the QtSslSocket component for secure connections. The domain already has a valid SSL certificate for the next few years.
Can anyone suggest the best way to go about implementing this from both the server and client side. i.e what does the server need to have in place and is there anything in particular the backup utility needs to implement from the client side to ensure secure transactions?
Are there any known, stable sftp or ftps servers available etc?
As far as I know, Qt doesn't have support for secure FTP transfers.
Not sure what other info. would be useful to make the question any clearer but any advice or help pointing me in the right direction will be most welcomed.
EDIT I'm also Java competent so a Java solution will work just as well...
As Martin wrote, you can wrap client. But if you don't want to do that, you can use libssh.
I searched for some sort of solution to this for a couple days and then forgot about the problem. Then today I stumbled across this little gem in the Qt-Creator source Utils::ssh, includes support for SFTP, plain-old SSH, and all sorts of goodies.
Disentangling stuff from Qt-Creator can be a pain, but having gone through this process it amounts to grabbing Botan (one of the other libs in QT-Creator) + Utils.
When it rains, it pours, I find two solutions to this problem in an hour - http://nullget.sourceforge.net/ (Requires Chinese translation), but from their summary:
NullGet is written with Qt, runs on
multiple platforms, the GUI interface
of the multi-threaded multi-protocol
HTTP download software. Use NullGet
can easily download a variety of
network protocol data stream, faster
download speeds, support for HTTP, the
protocol currently supported are:
HTTP, HTTPS, FTP, MMS, RTSP. And it
can run on most current popular
operating systems including Windows,
Linux, FreeBSD and so on.
Easiest way would be to just wrap a commandline sftp client with a Qt front end.
On the server any ftp server should do sftp pretty much out of the box.
As Synthesizerpatel says Qt Creator implements SFTP. So I have isolated the library that contains SSH and SFTP and I have created a new project named QSsh in Github (https://github.com/lvklabs/QSsh). The aim of the project is to provide SSH and SFTP support for any Qt Application.
I have written an example on how to upload a file using SFTP in examples/SecureUploader/
I hope it might be helpful
My application runs in Windows and is implemented using C++/Qt.
The application will invoke another application deployed in the Linux server which in turn will invoke some third party tools. The Linux server application will send some status updates based on the running of third party tools. Usually the third party application will run for hours and the updates will be sent at various stages. The Linux server may also has to send some files in addition to the status updates and the Windows client will also send some files required for the running of those third party tools.
I planned to implement this in libssh2 since file transfers can be done and applications can be executed as well using libssh2_channel_exec(). Updates can be sent and received through non-blocking socket transfers. Also the transfers must be secured and they are password authenticated, so I thought SSH will conform my requirements.
I also looked into Qpid of apache which implements the AMQP. The messaging seems to be a more appropriate one for my status updates since the updates are less frequent. But I am not so sure about the secured connection, password authentication and also the application invocation.
So, which one can I choose between these two? Or is there any other better option available? I am not quite used to network programming so any pointers, links regarding this are welcome..
Have you considered some web-based solutions like XML-RPC, REST, SOAP or other? Note that you can either have constant network connection and stream updates or just make your client ask for update as often as it needs.
Also, I think that building solution based on some of these protocols will give you easier coding - no need for some low-level solutions when you have great libraries. As for security part, I would consider SSL that is part of HTTPS protocol to be secure enough. Of course you can also do it hybrid style, for example SSH tunel to secure server and use SSH key authorization.
But if you are sure youwant SSH or AMQP then use first one - I think it has better security. Also, try not using username/passowrd. Instead use mentioned above keys.
Start with SSH, and then consider layering other protocols on top. You can use SSH port forwarding to create a VPN connection to a server, and maybe that will make it easier to use something like AMQP or 0MQ.
I'm not real hip on exactly what role(s) today's proxy servers can play and I'm learning so go easy on me :-) I have a client/server system I have written using a homegrown protocol and need to enhance the client side to negotiate its way out of a proxy environment.
I have an existing client and server system written in C and C++ for the speed and a small amount of MFC in the client to handle the user interface. I have written both the server and client side of the system on Windows (the people I work for are mainly web developers using Windows everything - not a choice) sticking to Berkeley Sockets as it were via wsock32 for efficiency. The clients connect to the server through a nonstandard port (even though using port 80 is an option to get out of some environments but the protocol that goes over it isn't HTTP). The TCP connection(s) stay open for the duration of the clients participation in real time conferences.
Our customer base is expanding to all kinds of networked environments. I have been able to solve a lot of problems by adding the ability to connect securely over port 443 and using secure sockets which allows the protocol to pass through a lot environments since the internal packets can't be sniffed. But more and more of our customers are behind a proxy server environment and my direct connections don't make it through. My old school understanding of proxy servers is that they act as a proxy for external HTML content over HTTP, possibly locally caching popular material for faster local access, and also allowing their IT staff to blacklist certain destination sites. Customer are complaining that my software doesn't recognize and easily navigate its way through their proxy environments but I'm finding it difficult to decide what my "best fit" solution should be. My software doesn't tear down the connection after each client request, and on top of that packets can come from either side at any time, basically your typical custom client/server system for a specific niche.
My first reaction is "why can't they just add my server's addresses to their white list" but if there is a programmatic way I can get through without requiring their IT staff to help it is politically better and arguably a better solution anyway. Plus maybe I'm still not understanding the role and purpose of what proxy servers and environments have grown to be these days.
My first attempt at a solution was to use WinInet with its various proxy capabilities to establish a connection over port 80 to my non-standard protocol server (which knows enough to recognize and answer a simple HTTP-looking GET request and answer it with a simple HTTP response page to get around some environments that employ initial packet sniffing (DPI)). I retrieved the actual SOCKET handle behind WinInet's HINTERNET request object and had hoped to use that in place of my software's existing SOCKET connection and hopefully not need to change much more on the client side. It initially seemed to be my solution but on further inspection it seems that the OS gets first-chance at the received data on this socket since when I get notified of events via the standard select(...) statement on the socket and query the size of the data available via ioctlsocket the call succeeds but returns 0 bytes available, the reads don't work and it goes downhill from there.
Can someone tell me of a client-side library (commercial is fine) will let me get past these proxy server environments with as little user and IT staff help as possible? From what I read it has grown past SOCKS and I figure someone has to have solved this problem before me.
Thanks for reading my long-winded question,
Ripred
If your software can make an SSL connection on port 443, then you are 99% of the way there.
Typically HTTP proxies are set up to proxy SSL-on-443 (for the purposes of HTTPS). You just need to teach your software to use the HTTP proxy. Check the HTTP RFCs for the full details, but the Cliffs Notes version is:
Connect to the HTTP proxy on the proxy port;
Send to the proxy:
.
CONNECT your.real.server:443 HTTP/1.1\r\n
Host: your.real.server:443\r\n
User-Agent: YourSoftware/1.234\r\n
\r\n
Then parse the proxy response, which will start with a HTTP status code, followed by HTTP headers, followed by a blank line. You'll then be talking with your destination (if the status code indicated success, anyway), and can start talking SSL.
In many corporate environments you'll have to authenticate with the proxy - this is almost always HTTP Basic Authentication, which is pretty easy - again, see the RFCs.
I am needing to implement email notifications for a C++ project. Basically a user provides all the relevant information for their email account and on certain events this component would fire off an email. Ideally I would like to find a small cross platform open source command line project that I can exec from within my project and parse the output. Something like blat but it would also support SSL connections and can be used in both Windows(XP and 2003) and Linux (Ubuntu 6.06 and 8.04)
I could also use a library if it were simple enough and licensed under a commercial friendly license, but would be open to hearing all suggestions.
Thank you very much in advance for any recommendations
(A) One option is to use XMail:
http://www.xmailserver.org/
The readme file has instructions of how to build it in Linux and Windows:
http://www.xmailserver.org/Readme.html
If you look at the forums:
http://xmailforum.homelinux.net/
or do some Google searches you should be able to find more information on how to use it.
(B) Another, possibly easier option, would be to just make your application connect to and use an external SMTP server to send your notifications.
To compose the email libmime (http://www.mozilla.org/mailnews/arch/libmime-description.html) can be helpful.
To send the mail libsmtp (http://libsmtp.berlios.de/) can be used.
All the protocol and SSL code for my email client is available in Lgi:
http://www.memecode.com/lgi.php
It's LGPL, so you could use it as a DLL/SO. However it's not packaged ready to use binaries, you'd have to build it yourself and write some glue using the SMTP and MIME code. The SSL sockets stuff uses OpenSSL and works on both Linux and Windows.
I ended up using the Perl script sendEmail. A windows binary was available and building a new binary after modifying the Perl script was not too hard to do at all. The script also had no issues running in the LTE Ubuntu environments after the required Debian packages were installed.