I'm using Hortonworks and I want to set up a REST webservice. The data is stored in HBase and I want to secure it with KNOX and Ranger.
I'm not sure if it is possible to configure it with NiFi, so I can call the REST API and get data from the HBase (with Hive). It is important, that the connection is secured with KNOX and I can also use Ranger for the data security.
Can I use NiFi for it?
What is the best solution/architecture to fit the use case?
It would be also great, if you can also add some related advanced informations, so I can go deeper into this subject.
thank you in advanced and best regards
n3
From NiFi's perspective it would only be interacting with your REST service so it would come down to how you authenticate to your REST service. The InvokeHTTP processor can perform basic authentication with a username and password, or can provide certificate information through an SSL context.
Everything to do with Ranger, Knox, and HBase is all hidden behind your REST service.
Related
We are working on a solution built on top of WSO2 APIM.
Problem Statement:
We want to add custom capability in existing APIM. Right now if we consume a single api endpoint it gives us meaningful statics some of them are:
API Usage API
API Usage per Application
Top Users per Application
Resource Usage per Application
But what we need is to store some/all incoming request data into database and extract meaningful information over time.
Example:
Suppose we have a api endpoint like below:
HTTP-Verb: POST
URL: http://localhost:9444/subscriptions/1/create_subscription
Params: name:sample_name, type:sample_type, user_id:12345
What we want?
It could be WSO2, OpenAM or any other opensource API manager.
We need to store parameter values e.g. sample_name, sample_type
and user_id in database.
Observations?
Looks like WSO Stream Processor can provide this functionality.
As it has a solution for HTTP Analytics. But we are not certain
to proceed with this option.
Another possible candidate is to Extend API Manager. But docs doesn't have anything to target what we wanted to do.
I am not sure but we can use RequestInterceptor?
Anyone has a experience with this kind of problem please let me know.
There are two problems you have to solve here,
1. Capture the required data from the APIM side
2. Publish to Analytics Server
3. Perform Analytics on received events
As you have observed WSO2 SP can be used for 3.
For 1 & 2, please see blog (https://medium.com/#naduni_pamudika/how-to-publish-custom-attributes-from-wso2-api-manager-to-wso2-analytics-38dd83e8aea2) for extending the publishers to capture and publish additional data.
You can also take a look at OpenIG, it is easy to extend and you can implement your own filter to store request data in database. Another approach is develop your own gateway based on Netflix Zuul or Spring Cloud Gateway
I've been suggested to use token-based authentication, in order to secure my webservices, and to create another filter that verifies tokens, apart from auth. The idea is to use the auth filter for log-in, which I have already done in the backend, and to create a new filter for webservices. Could someone recommend a good tutorial on how to do this, or give me an example?
This is what i use, specifically for mobile app web services:
https://github.com/lucadegasperi/oauth2-server-laravel
Which is an OAuth2 server package for Laravel. It includes all you need to authenticate, generate / validate tokens, throttle, and protect your endpoints.
The OAuth2 spec has a bit of a learning curve, but is definitely worth it.
I'm designing an architecture where the web interface is a client (developed using a front-end js framework) and all requests are routed to several webservices.
All communication will happen using standard HTTP responses and JSON entities.
Now I'm facing the authentication mechanism.
My service will, of course, have several users, and I need to restrict access to users' resources.
Users will 1) signin to the web client (/admin) and then 2) the client-side js will perform several AJAX requests on webservices on user behalf.
Should I create a persistent session between the client/server and then pass some reference alongside each request or authenticate each single request using a stateless approach? How could I authenticate the web-client requests for the current user without adding too much overhead or complexity to my system?
I'm looking at passport-local and passport-localapikey but it's not very clear to me if I should authorize my client or the user itself (meaning should I have only one pair of credentials for all users when performing web-service request or one pair per user?)
A simple example (explained I don't need to copy-paste code) would be very appreciated. At this stage I'd prefer the solution introducing less complexity but granting a good security in order to be able to set it up very quickly.
PS. I could also take into account creating a distinct service handling authentication in order to create a common API to be shared between client and server, but that seems a bit over-engineering to me.
Thanks,
If you're already using Express as a framework for Node.js, you can use it's built in session handling. It is capable of using any sort of session store including memory, redis, mongo, etc.
There's a good example here: http://blog.modulus.io/nodejs-and-express-sessions
I am currently working on a single-page web application. The web app will make calls to a REST-like API for authentication and data storage. We are currently in the middle of securing the application, and have worked out a strategy securing the site so only registered users can gain access. But one thing we also want to do is securing the API from others to write their own applications, or access it in any other way than through our web application. The problem from my view is that the API will be open for everybody and not only for my web application.
Anyone who knows how to do this, or who can point me in the right direction. Because right now, don't have a clue.
Considered using certificates and validation?
Your API should only be accessible, if the session of the client is authorized. That's pretty much anything you could do.
There are complex approaches like using client- and server-side encryption or something really basic: render a secret in your webpage that validates the user again on every request.
You could check the headers, where the original request comes from. And so on...
But as most of that is public in a users browser, anyone could read it and adopt it in a third party app.
So save yourself and the people that really want to do a third party app some time and provide a public API :)
Simplest way will be to use OAuth 2.0 ( supports both authentication and authorization) which you need.
Also ensure you secure the data on wire using TLS (HTTPS) and any of the options below
1. HTTP Digest
2. OAuthn 2.0
3. Certificates ( Shared secret)
Stick to HTTPS+Oauth2 for now.
You could lock down your you API to accept requests from known IP's. Also depending on how your network infrastructure is designed, your web application can sit in a DMZ and your API on an internal network accessible only by servers in your network, one of which will include your backend API (This article here info https://www.digitalocean.com/community/tutorials/5-common-server-setups-for-your-web-application has some tips). For better security, a secure network design in addition to an application security framework implementation like OAuth2 and HTTPS (mentioned above). For API's, I've found that resource based authorization works better than role based authorization. Lastly, constant review of your security setup is vital as things change all the time. A good approach to this is Threat Modelling described by OWASP here https://www.owasp.org/index.php/Application_Threat_Modeling
i am designing a database for web service authentication system, based on the KERBEROS working (using tokens instead of tickets). anybody help me to understand the data modeling for kerberos system. or where can i get some resources about the data schema for kerberos like authentication methods.
Do not reinvent the wheel. Use Kerberos as-is. Period.