I'm just starting to look into how to integrate nmap, an open source security product, into some c++ code. If anyone's tried this, and has some ideas on the best approach, I'd certainly appreciate it.
Thanks for the responses. Specifically, I'd like to run a port scan (ipv6). I would definitely prefer non-gpl solutions such as a command line or sockets interface. However, I'm also this point I'm looking for the fastest solution/s, as we're up against some stringent timelines, and we can backload implemententing the non-gpl solution if necessary.
(note: I am not a lawyer, and none of this should be taken as legal advice)
You should probably note that Nmap considers a product that parses its output to be a derived work, according to the licensing chapter in the manual, and thus fall under the GPL licensing obligations. The GPLv2 does not define what a derived work is, instead letting that be up to the courts and according to definitions in copyright law. The usual interpretation is that any form of linking, other than linking to system libraries included in the operating system, makes the linked work a derived work, while separate process that talk over pipes or the network are not necessarily derived works, though as mentioned in the GPL FAQ, "if the semantics of the communication are intimate enough, exchanging complex internal data structures, that too could be a basis to consider the two parts as combined into a larger program." That seems to be the interpretation that the Nmap developers are taking.
Anyhow, assuming that you don't need to worry about the GPL, you probably want to look at the output options for Nmap; in particular, -oX for XML output and -oG for "greppable" output. If you need more control over what Nmap does, you should look into the Nmap Scripting Engine, a Lua scripting engine in Nmap that gives you all kinds of control.
Are you looking to use specific pieces of functionality? An easy way I've found of using nmap in other languages is to have it spit out xml using the -oX switch. There is a DTD (and numerous ways to covert this to an xsd for your favourite binding tool) what you can use to consume the data.
Given that you're writing in C++ though it probably would be easy enough to link against directly if you needed.
You could always read it in pipe. According to tradition that is acceptable even if non-GPL accesses GPL.
"Integrate" doesn't really say enough. What kind of things do you want to do? Depending on the level of "integration" it might be enough to run nmap as a separate process and capture its output. The advantage there is that you can update the version of nmap without rebuilding your app. If you require tighter coupling, then it depends on what functionality you need and "library-izing" nmap, but be warned that it's GPL code and this kind of integration would require source distribution of your app..
Related
Using RedGate's Reflector, you may easily get the full source for an application. But to reduce the chances of an algorithm being found out, there are obfuscators to reduce this likelihood.
My questions are:
How effective are obfuscators?
What is safer in terms of hiding your algorithms: C++ without .NET libraries or obfuscated .NET programs.
Are there any others way to make .NET source code even more secure?
If you definitely want to secure something, go for C++, as in the .NET World there is a powerful deobfuscator named de4dot (see here: https://github.com/0xd4d/de4dot) that deobfuscates what most obfuscators on the market produce, event the ones it does not explicitely know.
However, this will only raise the bar, as even in the c++ world, there are powerful tools also (IDA: http://www.hex-rays.com/products/ida/index.shtml).
There are other solutions, like mixed code assemblies where you can have the part you want to hide in native code and the rest in managed. see here for more: Mixed (Native and Managed) Assemblies
"What is safer in terms to get source code: c++ program without .net libraries or .net obfuscated program." Without .net ofc...
Obfuscated .net .java still easy to decompile. There are pro obfuscators, which makes the code not recompilable those are slow down the hack process a bit.
Have you have heard if something can be closed that can be opened?
even if is writen is Assembly...
Usually beginner programmers are scared about this kind of theft. I would suggest to first create a software part, which worth to be stolen for others ( not for you )
Because .NET is designed to be self-descriptive, using an obfuscator will only hinder their progress. Although decompilers will have reduced readability, anyone understanding MSIL will have a better chance. Even C++ applications will be decompilable at some stage, as the program ultimately gets executed step by step in memory. C++ applications will take longer to work it out, but if a hacker knows Assembler (which they probably would if they are decompiling your application to gain access to the algorithm), its just a matter of time.
Obfuscation is really to make it as difficult as possible in a reasonable timespan, rather than making it impossible. The same principal lies with encryption. Encryption isn't impossible to break, it just takes so long that the context may not be of any use in 70-80 years time.
There are 2 alternatives I can think of apart from the ones covered here:
Host the algorithm at a remote location
Host the algorithm in a hardware component - very very expensive
The first option would be more suited if you have a network connection available. The processing is done on a separate server and the algorithm is not exposed to the public. This is how activation codes work nowadays. A serial code of some sort is generated, encrypted with a public key encryption and sent to a source which will decrypt and validate the data. The response is also encrypted and sent back.
Also, digitally signing your application and your dependencies will also assist as hackers could not plug-in components very easily. If they tried to use a DLL in place of one of your old ones (to fake a call to a service and return "success"), your code would check the digital signature before using the DLL.
So in summary, obfuscating will slow down the process but not prevent it. The only way I can think of is to host that algorithm at a secure location and send requests to it. Yes there is the problem of the hosting scenario, DoS etc, but your algorithm is protected, which is what you wanted.
Try something link this. However it will only obfuscate the strings. To obfuscate function calls, variables and other elements, look for commercial products and services for that purpose.
How effective are obfuscators?
I find ConfuserEx's name, constants, and control flow protection quite effective making .NET code difficult to read.
Using unicode for the Name protection can render class/method/etc names into unreadable unicode.
The constants protection encode constant strings like debug log string which provide excellent hints for hacker to guess what the code is doing.
The control flow scramble your code into a lot of switch-cases.
See below for an example of ConfuserEx'ed C# code:
What is safer in terms of hiding your algorithms: C++ without .NET libraries or obfuscated .NET programs.
If .NET assembly is not obfuscated, then it's like giving away the source code.
Comparing obfuscated .NET and native x86/x64 code, IMO, the difficulty to read them is about the same. Native x86 code can be disassembled into quite readable C code using software like IDA-Pro. Also there are people who can read and understand x86 language really fast.
Are there any others way to make .NET source code even more secure?
There is this Microsoft .NET Native(still in early stage with limitations) that compiles C# code into native x86/x64 code. This is not actually a protection but just people read x86 code slower.
I would like to ask about the available (free or not) Static and Dynamic code analysis tools that can be used to C++ applications ESPECIALLY COM and ActiveX.
I am currently using Visual Studio's /analyze compiler option, which is good and all but I still feel there is lots of analysis to be done.
I'm talking about a C++ application where memory management and code security is of utmost importance.
I'm trying to check for problems relating to security such as memory management, input validation, buffer overflows, exception handling... etc
I'm not interested in, say, inheritance depth or lines of executable code.
Without a doubt you want to use Axman. This is by far the best ActiveX/Com security testing tool available, and its open source. This was one of the leading tools used in the Month Of Browser Bugs by H.D. Moore, who is also the creator of Metasploit. I I have personally used Axman to find vulnerabilities and write exploit code.
Axman uses TypeLib to identify all of the components that makeup a COM . This is a type relfection, and it means that Source code is not required. Axman uses reflection to automatically generate fuzz test cases against a COM.
There is a security tool category called the fuzzers which were used in the recent Pwn2Own 2010 contest in Vancouver. The winning guy said that he's not going to tell software makers which bug he found but instead how to create a good fuzzer that will allow them to find the bugs. This was covered by computerworld.
Basically, it finds every place that the software can take input and tries to inject random data until the application crashes. Starting from there, the user attempts to understand what went wrong and develops an effective attack.
I don't know any particular fuzzers but there are many kinds of them for various uses (buffer overflows vs sql injections, 2 very different problems, 2 different fuzzers)
We use Coverity Prevent which is a very sophisticated static analysis tool that stores defects in a database that has a web interface. It works for C, C++, and Java.
We also use open source tools like Valgrind.
Start here are work your way you http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis
I don't mean to be rude when I say "google it". Personally (ymmv) I learn much more along the way from googling than just having someone give me "the" answer.
Also, when I look for tools, I will go to to SourceForge and look for - in this case - "static code analysis"
Btw, check out Valgrind
I know that a lot of c/c++ XML library questions have been asked already (I tried to read through all of them before getting to this).
Here are the things I'm going to need in my own project:
Excellent performance
SAX2
Validation
Open source
Cross platform
I was going to use Xerces-C, but I see that a simple SAX2 setup with nothing going on in the filter is taking 5 seconds to run. (Perhaps I'm doing something wrong here?)
I would like to use libxml++, but as I tried to get it set up on my MacBook, there were some crazy dependencies that took me all the way back to gtk-doc, at which point I sort of tabled the idea.
So now I'm at libxml2. Is this the way to go? Have I missed an important option, bearing in mind the five requirements above? I don't mind using a (good) c-library like libxml2, but a c++ interface would be nice. (I don't like Xerces-C's API very much.)
I am willing to bend on the SAX2 requirement if comparable functionality is available.
Having spent a goodly amount of time on this same problem, it was my conclusion that libxml2 is the best option available under your guidelines. The C interface is not too difficult to use and it's very fast.
There are some other good options for commercial libraries, but most of the other comparable open-source options are either painfully slow or are mired in a deep, annoying vat of dependency soup.
You say you need these things in your project, but don't give any idea of the pipeline. For example, we had a whole load of static XML files which needed to be loaded quickly, but only validated rarely. So validated using a separate process in batch (using RelaxNG as it was human writable markup ) and loaded the XML using expat. The system also used XMPP, so checked streaming input, but that didn't require validating against a schema (partly because it was streamed, and mostly because most of the possible errors were not expressible in a schema).
If you need a whole host of other facilities, you can consider Qt, which has good XML support. Be warned though, it's WAY more than an XML processing library; it's a full blown application framework with support for GUIs, networking and a whole host of other things.
Qt
You can also try Poco. It's another application framework, but not as huge as Qt (i.e. no GUI-related things etc.)
Poco
Lastly, if you don't mind a C library, you can use Expat. It's not SAX per se, but writing code using Expat is somewhat like SAX. It has C++ wrappers, but they're not officially part of the project IIRC, and may not be as well-maintained or designed. I'm not too sure though.
Expat
Hope this helps!
EDIT: I misread your original post: not too sure about the validation features of these libraries, I've never used them before.
I'm looking to add full text indexing to a Linux desktop application written in C++. I am thinking that the easiest way to do this would be to call an existing library or utility. This article reviews various open source utilities available for the Gnome and KDE desktops; metatracker, recoll and stigi are all written in C++ so they each seem reasonable. But I cannot find any notable documentation on how to use them as libraries or through an API. I could, instead, use something like Clucene or Xapian, which are generic full text indexing libraries. They seem more straightforward but if I used them, I'd have to implement my own indexing daemon, an unappealing prospect.
Also, Xesam seems to be the latest thing, does anyone have any evidence that it works?
So, does anyone have experience using any of the applications or libraries? How did you use it and what documentation was useful?
I used CLucene, which you mentioned (and also Lucene.NET), and found it to be pretty good.
There's also Strigi which AFAIK works with Xesam and is the default used in KDE.
After further looking around, I found and worked with Recol. It believe that it has the best C++ interface to a full text search engine, in this case Xapian.
It is important to realize that clucene and Xapian are both highly complex libraries designed primarily for multi-user server applications. Cutting them down to a level appropriate for a client-system is not easy. If I remember correctly, Strigi has a complex, pure C interface which isn't adapted.
Clucene also doesn't seem to be that actively maintained currently and Xapian seems to be maintained. But the thing is the existence of recol, which allows you to index particular files without the massive, massive setup that raw Xapian or clucene requires - creating your own "stemming" set is not normally desirable, etc.
There are plenty of logging/trace systems for letting your program output data or strings or state as it runs. Most of these let you print arbitrary strings which you can view live or after your program runs.
I noticed an ad here on SO for Smartinspect which seems to take this to a higher level, giving stack traces for each log, fancier options like plotting graphs and data values which change over time, and a lot of polish to the basic idea of a simple list of output text strings.
Since I use C++, Smartinspect won't even work for me.
A little googling finds tons of logging frameworks, but nothing that seems to do anything more than text dumps. Are their fancier tools (similar to Smartinspect?) that do more? Commercial or open source is fine, and multiplatform is a big plus.
I know this is not the answer you are most probably looking for but I would suggest that such a framework will be very hard (if not impossible) to find for C++. Doing something like dumping the stack cannot be done in a portable way as it can in a language like Java, which not only shares a common runtime accross all platforms, but provides powerful introspection capabilities too.
I don't program in Java, but my guess is that it can provide a stack-trace in the same way as Python: the stack is probably just another object in the runtime which can be inspected and manipulated.
C++ on the other hand has none of these niceities: its meant to be a close-to-the-metal language that basically adds object-orientism to C (I'm sure others will come up with much more elanorate explanations of C++'s benefit's over C but thats another discussion).
In short, C++ is not rich enough at the level required to provide the kind of features you require in a generic way. There may be some platform-specific code that could get some of this info at defined points for you out there, but it certainly wont be standards compliant, cross-platform C++.
With regards to graphs etc, that sounds much more like post-processing, which you should either be able to find something for, or more likely, you can perhaps output your log messages in a format which can be interpreted by some of these existing tools.
Other things you could look at would be integrating with syslogd, for which again, there may be richer analysis tools for (this would provide you with a capability along the lines of the one advertised for SmartInspect - that is TCP/IP based logging).
NB: a lot of what I said here about C++ comes from previous experiences trying to find decent frameworks in C++ to do tweaky, introspective type things (such as proper mock objects etc).
I wrote a article about dumping the stack in C/C++ with Windows and Unix/Linux at DDJ some years ago. Maybe it helps you:
See http://www.ddj.com/architect/185300443
If you can restrict yourself to a certain platform you can add stack traces to your logs manually. We use e.g. the glibc functionality to get stack traces on Linux to attach stack traces into our exception class. There is similar functionalyty available on Windows, but as mentioned these infrastructures are not portable.