So, I have a desktop application and I want it to be able to check a website for new versions of itself. I am completely new to sockets (Windsocks and Berkeley), so before I invest time learning network programming I want some guidance to point me in the right direction.
The application is going to pretty much download an installation file from its website. The connection will not be secure as it doesn't matter if users can see it or not. Also the application's website will most likely be hosted # godaddy (in case somebody wants to be specific).
So my questions are; What technology should I be looking into, FTP, TCP or UDP? What are some things I should keep in mind as far as the client/server communication when it comes to file transfer with a remote server? Does anybody knows if godaddy allows this type of thing?
PS. If you think this might be a little too much to accomplish without enough theoretical/technical background, then please don't hesitate to recommend a book.
Use HTTP, and use a library to download a URL to a file. This should take 1-5 lines of code.
Why build a file transfer protocol yourself using sockets? Everything you need is built-in with HTTP. There are pre-made clients and servers available.
Related
I am looking for a way to block certain websites using C++.
The agent is installed in the End-User, and if an unauthorized site is accessed, it must be redirected to an error page.
The hosts file is not used because the user can modify it arbitrarily.
Here is a list of possible searches.
(I prefer blocking through hooking rather than developing a network driver.)
Windows Filtering Platform
Send FIN packet to server using WinPCAP
DNS Query control
Browsers target IE, Chrome, and Edge.
What are the pros and cons of the above list?
Or, Is there any possible way other than the above list?
Since there is not enough development time, there is not much learning time.
If you know the API, documentation, or code snippets you can refer to, please respond.
Thankds.
I'm building an app which upon login will connect you to certain ip addresses of which will also be running the same app.
The method of which i believe i should be using is direct tunnelling but as i say im a little new to c++, i have general coding skills, and i have sifted through a lot of forums and sites yet im still very unclear on what the best way forward is to achieve the requirement.
The reason for the connection will be to enable a secure chat, file transfer, and update software auto when connected to the program admin.
All those that have the app installed will once authorised, will be connected to admin client, then from that client all available ip's to connect to will become available to slave clients, this will increase the network size avilable to all users.
so the app needs to be able to handle ports but not via a server, instead it would be direct.
The connections also must ideally be encrypted.
Im kind of looking for what the application RetroShare does, but in text app.
(This is using C++ within Dev C++)
so just to recap, What method should i use to achieve the above?
I would take a look at SDL net to start with, its really simple to learn if you have never done any socket programming before.
for a secure connection you will probably want to start with TCP and then once you get the hang of network programming, start looking at other protocols.
Hope this helped! and good luck.
I'm in the process of creating a utility to backup user's media files. The media isn't being shared etc its only a backup utility.
I'm trying to think of the best way to protect users from ISPs accusing them of downloading illegal media files by using some sort of secure connection.
The utility is written in C++ using the Qt lib and so far I've only been able to find the QtSslSocket component for secure connections. The domain already has a valid SSL certificate for the next few years.
Can anyone suggest the best way to go about implementing this from both the server and client side. i.e what does the server need to have in place and is there anything in particular the backup utility needs to implement from the client side to ensure secure transactions?
Are there any known, stable sftp or ftps servers available etc?
As far as I know, Qt doesn't have support for secure FTP transfers.
Not sure what other info. would be useful to make the question any clearer but any advice or help pointing me in the right direction will be most welcomed.
EDIT I'm also Java competent so a Java solution will work just as well...
As Martin wrote, you can wrap client. But if you don't want to do that, you can use libssh.
I searched for some sort of solution to this for a couple days and then forgot about the problem. Then today I stumbled across this little gem in the Qt-Creator source Utils::ssh, includes support for SFTP, plain-old SSH, and all sorts of goodies.
Disentangling stuff from Qt-Creator can be a pain, but having gone through this process it amounts to grabbing Botan (one of the other libs in QT-Creator) + Utils.
When it rains, it pours, I find two solutions to this problem in an hour - http://nullget.sourceforge.net/ (Requires Chinese translation), but from their summary:
NullGet is written with Qt, runs on
multiple platforms, the GUI interface
of the multi-threaded multi-protocol
HTTP download software. Use NullGet
can easily download a variety of
network protocol data stream, faster
download speeds, support for HTTP, the
protocol currently supported are:
HTTP, HTTPS, FTP, MMS, RTSP. And it
can run on most current popular
operating systems including Windows,
Linux, FreeBSD and so on.
Easiest way would be to just wrap a commandline sftp client with a Qt front end.
On the server any ftp server should do sftp pretty much out of the box.
As Synthesizerpatel says Qt Creator implements SFTP. So I have isolated the library that contains SSH and SFTP and I have created a new project named QSsh in Github (https://github.com/lvklabs/QSsh). The aim of the project is to provide SSH and SFTP support for any Qt Application.
I have written an example on how to upload a file using SFTP in examples/SecureUploader/
I hope it might be helpful
I'm working on a project of which a large part is server side software. I started programming in C++ using the sockets library. But, one of my partners suggested that we use a standard server like IIS, Apache or nginx.
Which one is better to do, in the long run? When I program it in C++, I have direct access to the raw requests where as in the case of using standard servers I need to use a scripting language to handle the requests. In any case, which one is the better option and why?
Also, when it comes to security for things like DDOS attacks etc., do the standard servers already have protection? If I would want to implement it in my socket server, what is the best way?
"Server side software" could mean lots of different things, for example this could be a trivial app which "echoes" everything back on a specific port, to a telnet/ftp server to a webserver running lots of "services".
So where in this gamut of possibilities does your particular application lie? Without further information, it's difficult to make any suggestions, but let's see..
Web Services, i.e. your "server side" requirement is to handle individual requests and respond having done some set of business logic. Typically communication is via SOAP/XML, and this is ideal if you bave web based clients (though nothing prevents your from accessing these services via standalone clients). Typially you host these on web servers as you mentioned, and often they are easiest written in Java (I've yet to come across one that needed to be written in C++!)
Simple web site - slightly different to the above, respods to HTML get/post requests and serves up static or dymanic content (I'm guessing this is not what you're after!)
Standalone server which responds to something specific, here you'd have to implement your own "messaging"/protocols etc. and the server will carry out a specific function on incoming request and potentially send responses back. Key thing here is that the server does something specific, and is not a generic container (at which point 1 makes more sense!)
So where does your application lie? If 1/2 use Java or some scripting language (such as Perl/ASP/JSP etc.) If 3, you can certainly use C++, and if you do, use a suitable abstraction, such as boost::asio and Google Protocol buffers, save yourself a lot of headache...
With regards to security, ofcourse bugs and security holes are found all the time, however the good thing with some of these OS projects is that the community will tackle and fix them. Let's just say, you'll be safer using them than your own custom handrolled imlpementation, the likelyhood that you'll be able to address all the issues that they would have encountered in the years they've been around is very small (no disrespect to your abilities!)
EDIT: now that there's a little more info, here is one possible approach (this is what I've done in the past, and I've jused Java most of the way..)
The client facing server should be something reliable, esp. if it's over the internet, here I would use a proven product, something like Apache is good or IIS (depends on which technologies you have available). IMHO, I would go for jBoss AS - really powerful and easily customisable piece of kit, and integrates really nicely with lots of different things (all Java ofcourse!) You could then have a simple bit of Java which can then delegate to your actual Server processes that do the work..
For the Server procesess you can use C++ if that's what you are comfortable with
There is one key bit which I left out, and this is how 1 & 2 talk to each other. This is where you should look at an open source messaging product (even more higher level than asio or protocol buffers), and here I would look at something like Zero MQ, or Red Hat Messaging (both are MQ messaging protocols), the great advantage of this type of "messaging bus" is that there is no tight coupling between your servers, with your own handrolled implementation, you'll be doing lots of boilerplate to get the interaction to work just right, with something like MQ, you'll have multiplatform communication without having to get into the details... You wil save yourself a lot of time and bother if you elect to use something like that.. (btw. there are other messaging products out there, and some are easier to use - such as Tibco RV or EMS etc, however they are commercial products and licenses will cost a lot of money!)
With a messaging solution your servers become trivial as they simply handle incoming messagins and send messages back out again, and you can focus on the business logic...
my two pennies... :)
If you opt for 1st solution in Nim's list (web services) I would suggest you to have a look at WSO's web services framework for C++ , Axis CPP and Axis2/C web services framework (if you are not restricted to C++). Web Services might be the best solution for your requirement as you can quickly build them and use either as processing or proxy modules on the server side of your system.
I need to have as part of a desktop application a file server which should respond as fast as possible to file transfer requests (from remote clients, usually located on the same LAN). There will be many file requests for small sized files. The server should be able to provide both upload and download services.
I am not tight to any particual technology so I am open to any programming language, toolkits, libraries as long as they can run on Windows.
My initial take is to go with a C/C++ implementation using Windows Sockets or use the services provided by libraries such as Boost (asio or such). I have also thought of Erlang but that I'll have to learn and so the performance benefits should justify the increased development time due to having to learn the language.
LATER EDIT: I appreciate the answers that say use FTP or HTTP or basically anything that has been already created but considering you still want to write one from scratch, what would you do?
Why not just go with FTP? You should be able to find an adequate server implementation in any language, and client access libraries too.
It sounds like a lot of wheel-reinvention. Granted, FTP is not ideal, and has a few odd spots, but ... it's there, it's standard, well-known, and already very widely implemented.
For frequent uploads of small files, the fastest way would be to implement your own proprietary protocol, but that would require a considerable amount of work - and also it would be non-standard, meaning future integration would be difficult unless you are able to implement your protocol in any client you'll support. If you choose to do it anyway, this is my suggestion for a simple protocol:
Command: 1 byte to identify what'll be done: (0x01 for upload request, 0x02 for download request, 0x11 for upload response, 0x12 for download response, etc).
File name: can be fixed-size or prefixed with a byte for the length (assuming the name is less than 255 bytes)
Checksum, MD5 for instance (if upload request or download response)
File size (if upload request or download response)
payload (if upload request or download response)
This could be implemented on top of a simple TCP socket. You can also use UDP, avoiding the cost of establishing a connection but in this case you have to deal with retransmission control.
Before deciding to implement your own protocol, take a look at HTTP libraries like libcurl, you could make your server use standard HTTP commands like GET for download and POST for upload. This would save a lot of work and you'll be able to test the download with any web browser.
Another suggestion to improve performance is to use as the file repository not the filesystem, but something like SQLite. You can create a single table containing one char column for the file name and one blob column for the file contents. Since SQLite is lightweight and does an efficient caching, you'll most of the time avoid the disk access overhead.
I'm assuming you don't need client authentication.
Finally: although C++ is your preference to give you raw native code speed, rarely this is the major bottleneck in this kind of application. Most probably will be disk access and network bandwidth. I'm mentioning this because in Java you'll probably be able to make a servlet to do exactly the same thing (using HTTP GET for download and POST for upload) with less than 100 lines of code. Use Derby instead of SQLite in this case, put that servlet in any container (Tomcat, Glassfish, etc) and it's done.
If all the machines are running on Windows on the same LAN, why do you need a server at all? Why not simply use Windows file sharing?
I would suggest not to use FTP, or SFTP, or any other connection oriented technique. Instead, go for a connectionless protocol or technique.
The reason is that, if you require lots of small files to be uploaded or downloaded, and the response should be as fast as possible, you want to avoid the cost of setting up and destroying connections.
I would suggest that you look at either using an existing implementation or implementing your own HTTP or HTTPS server/service.
Your bottlenecks are likely to come from one of the following sources:
Harddisk I/O - The WD velociraptor is supposed to have a random access speed of about 100MB/s. Also, it is important whether you set it up as RAID0,1,5 or what nots. Some read fast but write slow. Trade-offs.
Network I/O - Assuming that you have the fastest harddisks in a fast RAID setup, unless you use Gbit I/O, your network will be slow. If your pipes are big, you still need to supply it with data.
Memory cache - The in-memory file-system cache will need to be big enough to buffer all the network I/O so that it does not slow you down. That will require large amounts of memory for the kind of work you're looking at.
File-system structure - Assuming that you have gigabytes worth of memory, then the bottleneck will most likely be the data-structure that you use for the file-system. If the file-system structure is cumbersome it will slow you down.
Assuming that all the other problems are solved, then do you worry about your application itself. Notice, that most of the bottlenecks are outside your software control. Therefore, whether you code it in C/C++ or use specific libraries, you will still be at the mercy of the OS and hardware.
Sounds like you should use an SFTP (SSH) server, it's firewall/NAT safe, secure, and already does what you want and more. You could also use SAMBA or windows file sharing for an even more simple implementation.
Why not use something existing, for example a normal Web server handles a lot of small files (images) very well and fast.
And lots of people already spent time in optimizing the code.
And the second benefit is that the transfer is done with HTTP which is an established protocol. And is easily switched to SSL if you need more security.
For the uploads, they are also no problem with a script or custom module - with the same method you can also add authorization.
As long as you don't need to dynamically seek the files i guess this would be one of the best solutions.
It's a new part to an existing desktop application? What's the goal of the server? Is it protecting the files that are uploaded/downloaded and providing authentication and/or authorisation? Does it provide some kind of structure for the uploads to be stored in?
One option may be to install Apache HTTP Server on the machine and serve the file via that. Use POST to upload and GET to download.
If the clients are within a LAN could you not just share a drive?