I want to store a lot of configuration data pertaining to cluster, process, IP addresses etc. I have worked on one such product earlier where LDAP was used for this purpose. Although it was PITA to configure it the first time, I liked the transactional LDAP part which helps in dynamic reloading of the configuration when there is a change. It can be done with a flat file using inotify, but that is not as good as transactional LDAP. But, as I said, the configuration was a real pain, and also I don't want to borrow the same idea of LDAP in this product.
So can anyone give me an idea about which will be the next best replacement, which makes entering configuration easy and also that can help in dynamic configuration and notify my process whenever there is a change in the configuration file and exactly what changed (directly or indirectly)?
I am planning to develop my product in C++ and C.
The configuration can be edited by an Admin, or if he is too lazy he can automate it using some script. Also through cli, but not by a running process, that will land me up in concurrency and locking issues.
My program is a daemon, some sort of cluster manager running on multiple nodes.
There is no wrapper provided for user to edit configuration.
I am only looking for Linux/Solaris platform.
You have not really given enough background information for a good answer to be given. So, here are some of the unasked questions, the answers to which will influence your choice:
How is the configuration file edited? By your process, or by hand-editing, or by some other program?
How is the main program running - in the foreground with a user interacting, or in the background as a daemon?
If you expect people to hand-edit the configuration, then you can provide a wrapper script for doing so which sends a signal (conventionally SIGHUP) to the daemon to tell it to reread its configuration file.
If your main program is going to guide the user through the editing, then you really don't need to tell the program when the editing is complete. It already knows.
You mention Linux in the tags; can we assume that Windows portability is not an issue?
As to configuration file formats, you can go with the vogue (and bloat) of using XML. However, although that is a good tool for programs communicating, it is not very good for people to edit. You should look at E S Raymond's "The Art of UNIX Programming" which is a good general read and has a chapter on different configuration file formats. You should probably adopt one of the schemes outlined there. Which scheme is best depends in part on what information you have to capture in your configuration file.
If you're going to embed an interpreter (Perl, Lua, Tcl/Tk, ...) into your program, you might use that language to handle the configuration file...or you might not.
Related
I have given a project to develop Algorithmic trading system using c++ and quickFix library, I search on google about quickFix library but didn't find any useful information.
Can anybody give me some information , from where should I start?
You provide very little detail in your question, so I can only guess at a helpful approach. I have done what you are starting, in Python, and can give you some orientation. All the links Karl mentioned are crucial, (you should pay special attention to the quickfix documentation on the config file) to which I would add FIXIMATE.
To do something like this in QF you need to answer a number of questions.
Logon. Figure out how to logon. Try to get a data dictionary from your counterparty. You don't want to be forced to modify your DD too much.
Interface. How will you tell QF to logon, logoff, exit terrible positions, and so forth? I use a command line tool (cmd2) that gives me this ability. Other people code GUI windows.
Message Cracking. Some versions of QF come with a cracker but if you don't have it in C++ you will have to write your own so you can parse the incoming messages.
Data Management. How will you save incoming market data, both in RAM and to disk for analysis later? How will you represent and monitor your positions, your working orders, your audit trail? Familiarize yourself with the ScreenLogFactory and FileLogFactory in QF.
Auxiliary Functions. You will need a lot of functions you will write yourself to help at all stages. Save them all in one place and organize them into categories for easy access.
Monitoring. How will you know if something goes wrong (or right) when you are not in front of your computer monitoring the algo? I launch a completely separate process which consumes messages via a queue and sends me texts and emails.
Risk. You don't want your machine sending 1000 orders to market in the blink of an eye. You need to code some checks that will veto bad orders as a final stage before they go out. Also some code that will tell you if you are in a position when you are supposed to be flat. This part is very important.
Strategy. You will need the ability to quickly ingest data, analyze it, and generate signals. For flexibility you should not design your strategy into your system, but you should design a strategy object which can support any strategy you come up with. Then you deploy those objects within your system.
Order handling. Your algo needs to know when and how to enter orders, cancel them, move stops, etc. It will need to deal with partial fills, and be able to support multiple order types.
This is just the beginning, off the top of my head. It is a long road to do all by yourself with no help. Very interesting though, and rewarding.
You can find the QuickFIX downloads on the quickfixengine.org website here: http://www.quickfixengine.org/. From there, you can download either the source code or download pre-built packages for Visual Studio 2010, 2012 and 2013.
Documentation for QuickFIX can be found in their documentation area here: http://www.quickfixengine.org/quickfix/doc/html/. The documentation includes compilation/installation instructions and a "Getting Started" section which discusses setting up a project and writing your first QuickFIX application.
If you wish to know more about the FIX protocol, you are advised to look at the FIX website here: http://www.fixtradingcommunity.org. There are specifications on that website that will give you information on the types of messages supported by FIX and how they should be used.
I hope you are doing good and i really appreciate your help here for my query.
We have our system T3000 written in C++ (http://www.temcocontrols.com/ftp/software/9TstatSoftware.zip and codes are available here https://github.com/temcocontrols/T3000_Building_Automation_System).
I am trying to integrate 'BIRT reporting tool' in my C++ application. I want to create report based on the data available in our T3000 system. I think BIRT is embeddable (??). We don't need to compile and change the project, just need to be able to call it from T3000.exe mainly.
My thinking is we may put one menu label in existing T3000 and try to display report in user single click.
Can you please help me to solve my issue with 'BIRT' ? I really appreciate your answer.
Regards
Raju
Well, the answer depends on what your definition of "embeddable" is.
BIRT is written in pure Java.
I could think of 3 different ways:
Of course it is possible to integrate Java code into an existing C/C++ program (see Embed Java into a C++ application?).
You could just use the BIRT runtime engine and generate the report as PDF or HTML from the command line (that means, basically you call the java executable from your program with several arguments). See Birt - How to run report engine on the console? and http://eclipser-blog.blogspot.de/2008/02/automatic-generation-of-birt-reports.html for more information.
You could run a Java web server like Tomcat in a second process and then start your report by calling a http URL (e.g. you could use the included Servlet example). See http://www.eclipse.org/birt/documentation/integrating/viewer-usage.php
Similar to 3. (see below)
Some notes:
The second option is slow, due to the Java and BIRT engine startup overhead (this may take several seconds). With the first and third option, the startup overhead is or can be minimized to only once (and for each report).
For the second and third option it may be necessary to modify the existing code of the example programs to suit your needs.
The first option is probably the best for an industry-quality solution, but it is also the most difficult to develop.
Anyway, Java skills are necessary IMHO.
If you plan to run this on a SOC instead of a PC, take performance into account.
Is a Java-based solution well-suited for this kind of hardware? BIRT needs quite a lot of RAM and CPU (for a SOC). Hardware like the Raspi 3 should handle this quite easily, I reckon.
I integrated the BIRT runtime into an existing Python application (all this running on an application server) in a fourth way: I wrote a listener program that listens on a TCP socket for BIRT tasks. It uses a pool of worker processes (written in Java) which in turn use the BIRT report engine to generate the output. The client program (here: written in Python) opens a TCP connection to the listener and uses this socket to tell it which report to generate (including report parameters and destination file name). The listener program then in turn chooses a worker process for the task and gives the task to the worker process.
So, basically, this fourth option is similar to the third one, with two differences:
The communication is socket-based (instead of http), allowing bi-di communication.
The architecture is multi-processes instead of multi-threading. We choose this because very large reports could cause out-of-memory errors for otherwise unrelated reports that just happen to run at the same time. It's the same basic architecture Oracle chose for their reports server.
However, developing the programs took months.
HVB: I have to give you more than a simple thanks for the explanation above, this info will save us time I am sure. Raju will be sharing our experience after we get into the project a little deeper so others can benefit.
I've been looking into centralising my computer game saves to make it easier to backup and restore as well as putting them up on the cloud via dropbox but there in so may places that it makes it quite difficult. I noticed the Windows 7 and Vista now support Symbolic links so I've been playing around with that but I was wonder the follow:
Is it possible (code example or a point in the right direction) for an application (vb.net or C++) to spoof a file or folder?
E.g. Application A (a game like Diablo III or Civilization V) attempts to read or right from file A (the game save), application B (the save repository) detects this read/write request and pipes the request through itself preforming the request on file B (the actual game save in another location). Application A is in no way altered and treats the file normally.
Note: I realise there are many simple ways of preforming the same task in essence such as monitoring the use of Application A or periodically checking file A and copying it if it has been altered since the last check etc but all these methods have draw backs and less interested in making it work than if it is possible.
It is entirely possible to do this through a file system filter driver. For information about these, take a look here:
http://msdn.microsoft.com/en-us/windows/hardware/gg462968
Filter drivers can hook into CreateFile operations and redirect the create to a different place if you want, but they are much harder to write as compared to normal applications. They run in kernel mode and must obey the limitations of drivers.
You can "fake" special folders, like control panel does, but I don't think you can create anything accessible/writeable (in an easy way). I might be wrong though. I had the same idea once too (as a compatibility step for some company stuff), but couldn't find anything supporting an easy way to do it. It seems like it might be easier to be done on Unix systems (but that's obviously no option here). Also, I wouldn't expect any nice or easy solutions for .net.
Only approach I could think about right now, would be highjacking the according API calls (e.g. FileOpen) to reroute/manipipulate them (similar to what root kits do), but I wouldn't say that's a good idea, considering it might be detected as possible malware or cheats by things like punkbuster or antivirus solutions.
Yes or no depending on (using your terms) the level of abstraction that Application A is using.
If Application A is performing a CreateFile wto start access and passing a fixed filesystem path then Application B would need to emulate a file system and do so in the kernel.
On the other hand if Application A were to user HTTP with RESTful URLs then the HTTP server could answer all requests from files or by dynamically creating the content.
So the question can only be answered in specific by knowing the details of Application A.
I've recently found this blog entry on a tool that writes XSS attacks directly to the database. It looks like a terribly good way to scan an application for weaknesses in my applications.
I've tried to run it on Mono, since my development platform is Linux. Unfortunately it crashes with a System.ArgumentNullException deep inside Microsoft.Practices.EnterpriseLibrary and I seem to be unable to find sufficient information about the software (it seems to be a single-shot project, with no homepage and no further development).
Is anyone aware of a similar tool? Preferably it should be:
cross-platform (Java, Python, .NET/Mono, even cross-platform C is ok)
open source (I really like being able to audit my security tools)
able to talk to a wide range of DB products (the big ones are most important: MySQL, Oracle, SQL Server, ...)
Edit: I'd like to clarify my goal: I'd like a tool that directly writes the result of a successful XSS/SQL injection attack into the database. The idea is that I want to check that every place in my app does correct output encoding. Detecting and avoiding the data getting there in the first place is an entirely different thing (and might not be possible when I display data that's written to the DB by a third-party application).
Edit 2: Corneliu Tusnea, the author of the tool I linked to above, has since released the tool as free software on codeplex: http://xssattack.codeplex.com/
I think metasploit has most of the attributes you are looking for. It may even be the only one that has all of what you specify, since all the others I can think of are closed source. There are a few existing modules that deal with XSS and one in particular that you should take a peek at: HTTP Microsoft SQL Injection Table XSS Infection. From the sounds of that module it is capable of doing exactly what you are wanting to do.
The framework is written in Ruby I believe, and is supposed to be easy to extend with your own modules which you may need/want to do.
I hope that helps.
http://www.metasploit.com/
Not sure if this is what you're after, its a parameter fuzzer for HTTP/HTTPS.
I haven't used it in a while, but IIRC it acts a proxy between you and the web application in question - and will insert XSS/SQL Injection attack strings into any input fields before deeming whether the response was "interesting" or not, thus whether the application is vulnerable or not.
http://www.owasp.org/index.php/Category:OWASP_WebScarab_Project
From your question I'm guessing it is a type of fuzzer you're looking for, and one specifically for XSS and web applications; if I'm right - then that might help you!
Its part of the Open Web Application Security Project (OWASP) that "jah" has linked you to above.
There are some Firefox plugins to do some XSS testing here:
http://labs.securitycompass.com/index.php/exploit-me/
A friend of mine keeps saying, that php-ids is pretty good. I haven't tried it myself, but it sounds as if it could approximately match your description:
Open Source (LGPL),
Cross Platform - PHP is not in your list, but maybe it's ok?
Detects "all sorts of XSS, SQL Injection, header injection, directory traversal, RFE/LFI, DoS and LDAP attacks" (this is from the FAQ)
Logs to databases.
I don't think there is such a tool, other than the one you pointed us to. I think there's a good reason for that: It's probably not the best way to test that each and every output is properly encoded for the applicable context.
From reading about that tool it seems the premise is to insert random xss vectors into the database and then you browse your application to see if any of those vectors succeed. This is rather a hit and miss methodology, to say the least.
A much better idea, I think, would be to perform code reviews.
You may find it helpful to have a look at some of the resources available at http://owasp.org - namely the Application Security Verification Standard (ASVS), the Testing Guide and the Code Review Guide.
I know there are startup folders and certain registry keys I need to look into. But how to do that using Windows API? I'm interested to know for Windows XP and Vista. Thanks for your time.
There is no single API to get all the programs that run while the system is starting up. Consider all the things that Autoruns shows. Updates to that program occasionally allow it to show new classes of programs, and since those are updates to the program and not to the OS, it's obviously not some API that's changing to include more things. Rather, it's the program that's gradually expanding its notion of what an "auto-run program" is.
Work out what you wish to consider to be a "startup process," and then you can determine what APIs or other techniques you can combine to find out what all those processes are.
Among the places to look include the following:
The "run" registry key, like AJ mentioned.
The "startup" menu, as given by various constants used with ShGetFolderPath and some of its related functions.
Tasks scheduled to run on login.
Services.
Each bullet point includes at least one link to a page on MSDN that should get you started. If you need more, consider asking some more specific questions.
I am not sure such an API exists. Nevertheless you can write a function to do that.
The startup process names and location are stored in following location in registry:
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run
<Name> <Location of EXE>
You can write small function to open the above registry and read the startup process details.