A customer of mine happend to ask me to come up with a technical solution
for the following scenario:
basis is a crm tool where the customers (i.e. the ones who bought the system)
are not wanted to access the database for queries directly (in terms of table level access)
there's plenty of reporting in the software, but sometimes it's just not
exactky what's needed, so some sort of interface for external reporting tools is
required.
--> beeing 'state of the art' my customer want's to use web services for this;
which IMHO is definitly not the way to do this; I'd favorize the creation
of VIEWS which the crm customers can use to feed their reporting tools via ODBC
for example.
So, what's you guys opinion on this, how would you approach this scenario? Any best practices?
How about empowering the customer to be able to write custom modules or plugins to the CRM software? If the built in reports don't suffice, they can quickly roll out their plugin to accomplish their objectives. IMO, exposing APIs is a much better approach. Also. there's nothing wrong with customers accessing the database tables directly. But the latter approach may be more cumbersome to some customers.
Your mileage may vary.
I thing that web-services is a nice solution. Using VIEWS - is using db authorization. While using WS you can be more secure :-)
Related
I want to build a software solution for datafeed aggregation. I dont want to use datafeed aggregation services like Rakuten PopShops or Datafeedr. I am looking for a guidance on how to build such solution from a software architecture point of view.
Which architectural design patterns should I use ?
Which technologies should I use ?
How to face the problem of non-normalized data formats and APIs from affiliate networks and merchants ?
Do you know some book on such topic ?
Since I am comming from a Java Enterprise world, these technologies seems to be a usable components of the solution for me:
Apache Camel,
ElasticSearch,
NoSQL database (MongoDB),
Akka,
ZeroMQ, RabbitMQ,
Spring Framework,
Typesafe Reactive Platform and a lot of other tools.
Unfortunately there is no single answer to your question. I would say stop thinking about solutions/patterns at this point. Try to figure out what exactly you want to do from a requirements perspective. What kind of data you want to aggregate, where to get the data from, data scrubbing rules, legal issues etc. Once you have that nailed down, take the easiest path to implement it. Technologies you have used in the past/comfortable with. Then add in other technologies once you find out that your existing solutions will not work.
Currently we have a data service that is consumed by various clients inside our organization; most of them use the PDF report to view the data. The problem we are facing is that the PDF generation is obsolete and built over a technology that is becoming hard to maintain.
So what we want to achieve now is basically two goals:
Encapsulate all data access in a SOA manner, publishing services for RAW data, PDF report and maybe some others like "Excel RAW"
Give our business users the ability to load and change the templates for the PDF reports, without asking for a "development" (basically that they can be as independent as possible)
There are two main issues; the report has to be "pretty", and by that I mean that our users ask for details such as image resolution, almost pixel accuracy for text/image positioning, and that sort of stuff. The server/library that we choose has to be able to achieve this requirement.
The other is that our technology stack currently is limited to a JAVA/LINUX platform, and while we could evaluate other platforms (for instance a product developed in .NET), a solution in Java EE/LINUX would be preferable.
Any suggestions?
P.S.: The data is stored in an Oracle database.
Certainly one prickly problem you have here is providing the ability to allow your users provide "pretty"/pixel-perfect report. Depending on what type of people your users are (technical staff / developers, business staff, general users) it may not be possible to find a system that allows you to offload significant work-effort to your users. It is just a hard domain.
If your reports are spreadsheet style, you might wish to examine systems like Business Objects, Coognos or Yellow Fin. These systems require a significant set up in terms of creating models to report against, but they can provide tools for users to design their own reports through a web interface. These systems usually stand separate to your main application though there are certainly ways to integrate them (though for exposing services to your own customers it might be difficult to get it to work exactly as you would like).
If your reports are document-style (as opposed to spreadsheet-style), you could look at Docmosis which is intended for integration with applications (please note I work for the company that created Docmosis). Docmosis allows your application to produce PDF reports from DOC/ODT/DOCX documents which act as templates for population from databases / Java objects / text etc. The templates can be provided/modified/uploaded by your users. It integrates with Java and linux environments so your technology environment is well suited. For many applications it provides automatically the layout based on the template that is desired.
With regards to the provision of SOA services to your users, it sound like a fine approach depending on the users you have (does a service approach provide them something easy to use?). Because your customers are internal I'm sure you already have determined the suitability of services.
Hope that helps.
For personal and university research reasons I am thinking of building a simple CRM using a service oriented architecture. Its meaning is just to explain the architecture itself, not commercial use.
I was thinking of implementing a CRM that offers a simple analytics service and customer care (user storing, personal comments, and few other things).
The architecture that I'm designing defines:
- WebGUI (a client of the other services)
- AnalyticsService (a service that receives data, analyzes and collect it)
- CustomerCareService (a service that uses RESTful APIs to apply CRUD operations).
Each service has it own database, being completely independent from others. They expose a public interface. The interface of course must provide some sort of authentication, to deny unautorized requests.
The advantages I'd like to explain in this kind of architecture is the possibility to have all things indepentent and the ability to combine them to offer new services (for example if there was an OrderService to handle orders it would be easy to combine it with Customer using the public APIs). The big advantage to me is that it'd be easy enough to build other clients that use these services.
I don't know what is some good Authentication method, that could be easy to implement, I'm also not sure about how to make this APIs (use XML or plain REST APIs with GET/POST data). I've worked with Amazon, PayPal and other company APIs, they seem to use REST services (paypal uses an ugly _cmd GET parameter while Amazon uses better URI) to know what to do, but reading something about SOAs it appears that people also use XML. Of course I also need to take into account that the web interface must be able to recognize the logged in user, get the permissions (token or whatever else) and use it with services to show information.
So I'm not sure SOA is the kind of architecture I'm really building up... is it SaaS instead of SOA?
I think it would be better to use RESTful applications, with JSON or something like that to implement it (I'm not a big fan of XML, I find it to be too verbose).
For clarity I'm listing here my questions:
Is this kind of architecture called SOA or SaaS (or both)?
What is a good implementation for what I want to obtain? (please explain it as more detailed as possible)
What sort of authentication is more suitable for a client (user token vs OAuth or similiar)
Do you have some suggestion for this kind of project?
I've about 3 months to do it, so I cannot do something real complex (beside the fact that it would not be realistic for a single programmer).
I know Python (WSGI frameworks), Ruby on Rails, C/C++ and other languages (.net excluded) and I'd like to develop it under a Linux environment (MySQL or Postgres, or even a NoSQL if you have any suggestion for the right choice), I could also combine several languages being these services independent programs.
What I'd like here is to have some good point of view and some good suggestion.
Thanks!
I would define SaaS as a Business model rather than an architecture; however like all business domain requirements it will influence systems architecture but it, itself is not. What you have defined is essential a Service Oriented Architecture.
Your statement "independent and the ability to combine them to offer new services" is the essential non-functional design requirement that suggests SOA.
Good implementation for SOA is about having well defined and flexible interfaces, with very clear delineation of responsibilities. However it is difficult subject to be prescriptive about. The proof is in the eating; does it provide that flexible reuse. My suggestion is spend time reading SOA design pattern resources, and understand the defining characteristics with regard to the appropriate context for use. Then apply the Single Responsibility principle appropriate level of abstraction. c.f. (Domain) Space Based Architecture is kind of SOA meta-pattern.
In regard to Authorisation, I recommend following the service approach, use a distribute directory services system like open LDAP, and note that is entirely reasonable for service provides and users to have their own credentials and you can use Public-Private keys for signing messages.
The main suggestion is study and learn from experience of others:
http://www.soapatterns.org/
http://martinfowler.com/eaaCatalog/
SOA doesn't forces to use XML.
Currently web technologies dominate, and define future.
So we in my company selected JSON RESTful services as foundation. And SOA as principles.
There is no sense to suggest languages, because the purpose of SOA and good implementation is
- to enable any language or framework to be used
(FYI we use Java with Spring MVC-based web-services, Node.js, PHP)
Web services and web APIs have managed to increase the accessibility of the information stored and catalogued on the internet. They have also opened up a vast array of enterprise power functionality for smaller thin client applications.
By taping into these services developers can provide functionality that would have taken them months perhaps years to set up. They can combine them into single applications that make life generally easier for its users.
Whether displaying information about the music being played, finding items of interest in the locale of the user or just simply tweeting and blogging from the same application - the possibilities are growing everyday.
I want to know about the most interesting or useful services that are out there, especially ones that most of us may not have heard about yet. Do you maintain an API or service? or do you have a clever mash up that provides even more benefits than the originals?
YQL - Yahoo provide a tool that lets you query many different API's across the web, even for sites that don't provide an API as such.
From the site:
The Yahoo! Query Language is an
expressive SQL-like language that lets
you query, filter, and join data
across Web services.
...
With YQL, developers can access and
shape data across the Internet through
one simple language, eliminating the
need to learn how to call different
APIs.
The World Bank API is pretty cool. Google uses it in search results. My favourite implementations are the cartograms at worldmapper.
(source: worldmapper.org)
It's very niche, but I happen to think the OpenCongress API is amazing.
Less niche: Google Translate has an API which will guess the language of something. You'd be AMAZED how frequently this comes in handy (even though it's not as tweakable as you'd like and is not trained on small samples).
I was just about to have a stab at using the SoundCloud API
I know many people who already use for sharing their musical masterpieces and its a pretty good site. Hopefully the api will be as well!
I like the RESTful API for weather.com. It's free and very useful for the new age of location-aware apps: https://registration.weather.com/ursa/xmloap/step1
It does require registration, but they don't spam you or anything - it's just to provide you a key to use the API.
Ah yes - here's another one I've been meaning to check out but haven't tried yet
The BBC offer a bunch of apis/feeds that look very promising
http://ideas.welcomebackstage.com/data
They include apis for accessing schedule data for both TV and Radio listings along with all kinds of news searches. It even looks like they'll be offering some sort of geo-location service soon so it will be interesting to see what that has to offer
Another interesting one for liberal brits! ;)
The Guardian news paper have their own api
http://www.guardian.co.uk/open-platform
MuiscBrainz
Excellent service for music mashups.
Not so many knows that Last.FM initial database was scraped from this service.
The United States Postal Service offers a web service that does address standardization. Quite useful in reducing clutter and cleaning data before it gets put into your database.
I'm looking for common patterns of implementing ad-hoc querying capabilites graphically. I've looked at SQL query builders in Access and TOAD, but I'm interested if anyone is aware of products that have build such a tool against a domain specific data warehouse (e.g. clinical databases).
Thanks,
Beyond Tableau (mentioned by Arthur), I would suggest either Qlikview or Spotfire, both of which allow for ad-hoc graphical querying in in-memory databases. These applications are much more powerful than something like Crystal or Jasper reports.
I have no specific answer but there are reporting tools that help you do stuff that I think you might be interested in.
One pay one that I tried out myself and liked quiet a bit was Tableau It is pay software and the server can be expensive, but I liked the desktop app. You will have to know enough database to figure out how to draw data out of it. Once that is done though you can reuse it. Once you draw out the dataset though you can 'play' with it graphically.
You can get into more complicated Reporting tools like Crystal Reports, Jasper Report and I think IBM has something that deals with 'Cubes' or whatever. You can look up all that by looking into Business Intelligence software. (I hate that name)
The problem with having domain specific stuff is that databases can be different. And even if you use a common vendor tool then the the query tool would have to be built specific to the db.
So maybe this doesn't answer your direct question but hope it is a little helpful.