I'm looking for common patterns of implementing ad-hoc querying capabilites graphically. I've looked at SQL query builders in Access and TOAD, but I'm interested if anyone is aware of products that have build such a tool against a domain specific data warehouse (e.g. clinical databases).
Thanks,
Beyond Tableau (mentioned by Arthur), I would suggest either Qlikview or Spotfire, both of which allow for ad-hoc graphical querying in in-memory databases. These applications are much more powerful than something like Crystal or Jasper reports.
I have no specific answer but there are reporting tools that help you do stuff that I think you might be interested in.
One pay one that I tried out myself and liked quiet a bit was Tableau It is pay software and the server can be expensive, but I liked the desktop app. You will have to know enough database to figure out how to draw data out of it. Once that is done though you can reuse it. Once you draw out the dataset though you can 'play' with it graphically.
You can get into more complicated Reporting tools like Crystal Reports, Jasper Report and I think IBM has something that deals with 'Cubes' or whatever. You can look up all that by looking into Business Intelligence software. (I hate that name)
The problem with having domain specific stuff is that databases can be different. And even if you use a common vendor tool then the the query tool would have to be built specific to the db.
So maybe this doesn't answer your direct question but hope it is a little helpful.
Related
I want to build a software solution for datafeed aggregation. I dont want to use datafeed aggregation services like Rakuten PopShops or Datafeedr. I am looking for a guidance on how to build such solution from a software architecture point of view.
Which architectural design patterns should I use ?
Which technologies should I use ?
How to face the problem of non-normalized data formats and APIs from affiliate networks and merchants ?
Do you know some book on such topic ?
Since I am comming from a Java Enterprise world, these technologies seems to be a usable components of the solution for me:
Apache Camel,
ElasticSearch,
NoSQL database (MongoDB),
Akka,
ZeroMQ, RabbitMQ,
Spring Framework,
Typesafe Reactive Platform and a lot of other tools.
Unfortunately there is no single answer to your question. I would say stop thinking about solutions/patterns at this point. Try to figure out what exactly you want to do from a requirements perspective. What kind of data you want to aggregate, where to get the data from, data scrubbing rules, legal issues etc. Once you have that nailed down, take the easiest path to implement it. Technologies you have used in the past/comfortable with. Then add in other technologies once you find out that your existing solutions will not work.
Currently we have a data service that is consumed by various clients inside our organization; most of them use the PDF report to view the data. The problem we are facing is that the PDF generation is obsolete and built over a technology that is becoming hard to maintain.
So what we want to achieve now is basically two goals:
Encapsulate all data access in a SOA manner, publishing services for RAW data, PDF report and maybe some others like "Excel RAW"
Give our business users the ability to load and change the templates for the PDF reports, without asking for a "development" (basically that they can be as independent as possible)
There are two main issues; the report has to be "pretty", and by that I mean that our users ask for details such as image resolution, almost pixel accuracy for text/image positioning, and that sort of stuff. The server/library that we choose has to be able to achieve this requirement.
The other is that our technology stack currently is limited to a JAVA/LINUX platform, and while we could evaluate other platforms (for instance a product developed in .NET), a solution in Java EE/LINUX would be preferable.
Any suggestions?
P.S.: The data is stored in an Oracle database.
Certainly one prickly problem you have here is providing the ability to allow your users provide "pretty"/pixel-perfect report. Depending on what type of people your users are (technical staff / developers, business staff, general users) it may not be possible to find a system that allows you to offload significant work-effort to your users. It is just a hard domain.
If your reports are spreadsheet style, you might wish to examine systems like Business Objects, Coognos or Yellow Fin. These systems require a significant set up in terms of creating models to report against, but they can provide tools for users to design their own reports through a web interface. These systems usually stand separate to your main application though there are certainly ways to integrate them (though for exposing services to your own customers it might be difficult to get it to work exactly as you would like).
If your reports are document-style (as opposed to spreadsheet-style), you could look at Docmosis which is intended for integration with applications (please note I work for the company that created Docmosis). Docmosis allows your application to produce PDF reports from DOC/ODT/DOCX documents which act as templates for population from databases / Java objects / text etc. The templates can be provided/modified/uploaded by your users. It integrates with Java and linux environments so your technology environment is well suited. For many applications it provides automatically the layout based on the template that is desired.
With regards to the provision of SOA services to your users, it sound like a fine approach depending on the users you have (does a service approach provide them something easy to use?). Because your customers are internal I'm sure you already have determined the suitability of services.
Hope that helps.
Web services and web APIs have managed to increase the accessibility of the information stored and catalogued on the internet. They have also opened up a vast array of enterprise power functionality for smaller thin client applications.
By taping into these services developers can provide functionality that would have taken them months perhaps years to set up. They can combine them into single applications that make life generally easier for its users.
Whether displaying information about the music being played, finding items of interest in the locale of the user or just simply tweeting and blogging from the same application - the possibilities are growing everyday.
I want to know about the most interesting or useful services that are out there, especially ones that most of us may not have heard about yet. Do you maintain an API or service? or do you have a clever mash up that provides even more benefits than the originals?
YQL - Yahoo provide a tool that lets you query many different API's across the web, even for sites that don't provide an API as such.
From the site:
The Yahoo! Query Language is an
expressive SQL-like language that lets
you query, filter, and join data
across Web services.
...
With YQL, developers can access and
shape data across the Internet through
one simple language, eliminating the
need to learn how to call different
APIs.
The World Bank API is pretty cool. Google uses it in search results. My favourite implementations are the cartograms at worldmapper.
(source: worldmapper.org)
It's very niche, but I happen to think the OpenCongress API is amazing.
Less niche: Google Translate has an API which will guess the language of something. You'd be AMAZED how frequently this comes in handy (even though it's not as tweakable as you'd like and is not trained on small samples).
I was just about to have a stab at using the SoundCloud API
I know many people who already use for sharing their musical masterpieces and its a pretty good site. Hopefully the api will be as well!
I like the RESTful API for weather.com. It's free and very useful for the new age of location-aware apps: https://registration.weather.com/ursa/xmloap/step1
It does require registration, but they don't spam you or anything - it's just to provide you a key to use the API.
Ah yes - here's another one I've been meaning to check out but haven't tried yet
The BBC offer a bunch of apis/feeds that look very promising
http://ideas.welcomebackstage.com/data
They include apis for accessing schedule data for both TV and Radio listings along with all kinds of news searches. It even looks like they'll be offering some sort of geo-location service soon so it will be interesting to see what that has to offer
Another interesting one for liberal brits! ;)
The Guardian news paper have their own api
http://www.guardian.co.uk/open-platform
MuiscBrainz
Excellent service for music mashups.
Not so many knows that Last.FM initial database was scraped from this service.
The United States Postal Service offers a web service that does address standardization. Quite useful in reducing clutter and cleaning data before it gets put into your database.
A customer of mine happend to ask me to come up with a technical solution
for the following scenario:
basis is a crm tool where the customers (i.e. the ones who bought the system)
are not wanted to access the database for queries directly (in terms of table level access)
there's plenty of reporting in the software, but sometimes it's just not
exactky what's needed, so some sort of interface for external reporting tools is
required.
--> beeing 'state of the art' my customer want's to use web services for this;
which IMHO is definitly not the way to do this; I'd favorize the creation
of VIEWS which the crm customers can use to feed their reporting tools via ODBC
for example.
So, what's you guys opinion on this, how would you approach this scenario? Any best practices?
How about empowering the customer to be able to write custom modules or plugins to the CRM software? If the built in reports don't suffice, they can quickly roll out their plugin to accomplish their objectives. IMO, exposing APIs is a much better approach. Also. there's nothing wrong with customers accessing the database tables directly. But the latter approach may be more cumbersome to some customers.
Your mileage may vary.
I thing that web-services is a nice solution. Using VIEWS - is using db authorization. While using WS you can be more secure :-)
Currently using SAS as measurement engine and Business Objects as display layer. Looking to develop a new, faster, slicker solution. Has anyone developed or purchased a campaign measurement reporting system? This solution should measure everything from email stats, web stats, customer activity, lift, ROI, etc.
Ok.. I'm researching and finding nada... We are working with a team from India and they want to re-write everything from scratch.. Any solutions out there at all?
If you are already using SAS, have you looked at their Marketing Automation software?
Update:
Just saw a press release from SAS about a new "Software as a Service" Campaign Management solution. Might be worth checking out for this.
When I was a consultant, we either rolled our own or used SAS (or a combination of the two).
Another vote for roll your own, it's mad that this area is so under served. The expense of building your own solution from the ground up, and the hassle of managing a remote team makes me think you may get further by integrating some existing tools.
Google Analytics for web usage has an API, there are many web log tools, you then need to bolt in the customer figures from your end of things.
I really doubt you could do much better than SAS in this area. Especially if you pick up some of thier specialist packages.
You could have a look at R which is a pretty slick open source statistics package. Unfortunately its not used very much for marketing; most of the examples and freely available code is geared towards biochemistry, genetics etc.