I am developing an application which is used by multiple clients and the main goal of the app is to be used online. But a specific requirement is, it should be able to work offline too (only for a single client(s) in an emergency situation and for short time - 24 hours maximum). now I am using Django REST framework for backend and Jquery/AJAX frontend for GET and PUT etc. requests which update PostgreSQL DB on the backend. Now little research on the internet suggests the I should use PouchDB on the frontend and/or CouchDB on the backend. But my questions are:
Is it really possible?
If yes, then which DB should be used for the backend database?
When the offline client become available online, how we can synchronize the data generated online?
Can we cache some data on clients machine for offline availability purpose?
Is it still possible to use PostgreSQL for the backend? (I really want to use it !)
Related
I'm looking for advice choosing a back-end architecture for a web app. In the app, users upload tabular data from multiple files. Their data is then processed, aggregated and visualized. Data is private and each user has their own dashboard.
I believe Cube.js is an excellent choice for the dashboard, but I am wondering what back-end web framework I should integrate it with. I have experience of Django, but would use Express if it had significant advantages.
Thanks for any advice!
Cube.js is designed to run as a microservice. It means there's no requirement for you to use Node.js or Express as a stack for the rest of your backend implementation. However usually you'd need to have separate backend for storing data about users, reports and dashboards if those are dynamic. Cube.js server handles only analytic queries.
Cube.js dashboard templates are designed to work with GraphQL. So any backend web framework where there's a solid support of GraphQL will be a good choice. Most notable ones in Node.js world are apollo-server, Prisma, Hasura. Seems like Python has it's own champion as well: https://github.com/graphql-python/graphene.
I am currently running WSO2 Analytics on a windows server but I want implement the analysing part somehow that a client can connect to the server and do some processing like visualization on its own rather than all processing being done on the server. Is this something possible on WSO2 platform?
Thanks
You can setup database you want ( see the documentation ). For production usage I woudn't even recomment using the bundled H2 database. WSO2 analytics supports number of databases by default, I believe Oracle is one of them.
As stated in the comments - you can create a client or service which reads the data from the database and displays them its own way.
most challenging part for me is that how the client uses the information from the database?
This is already on your own (outside scope of this question). You've asked if your client can access the analytics (result) data - yes you can. How to do that is up to you. (depending what the client is, ..)
For example at our client they are building data APIs which are directly consumable by different frontend libraries creating nicer charts and reports.
I am planning to use Firebase database and want to know how it fits in to the following scenario.
lets say I have a browser app, android app / iOS which uses Web Services to get / insert data, web services talks to the Data Base and returns data to the client.
This way I have to write code once in my web services and all the clients use that to retrieve and insert data to the database.
If I want to use Firebase, will I be following the same approach of having webservices between the client's and the Firebase DB.
I have done some sample Firebase examples where it it gets data from database directly without web services and in this approach we have to write our logic on each client (Web browser/ android app/ iOs app).
I have looked into this article
https://firebase.googleblog.com/2013/03/where-does-firebase-fit-in-your-app.html?showComment=1480073224245#c464815735109872173
The Pattern 2 has the server concept but that does not look appropriate in my scenario.
Can I have my web service and Firebase database and get data Synchronization capabilities.
Correct me if I am wrong and please suggest the approach I need to take.
Thank you for your valuable suggestions in advance.
Thanks & regards,
Rao Burugula
The article you link gives you the most common options for integrating Firebase into your app. Pattern 2 is the easiest way to use the Firebase Database and run your own server-side code:
In this model the Firebase Database sits between the app on the user's device and your back-end code. By using this model, you can still get all the benefits of the realtime synchronization, security rules and scalability, but also have back-end code that runs in a trusted environment.
Of course you can also go for a more traditional three-tier model, where your app server sites between the devices and the database. But in that case the Firebase database won't have direct interaction with your app anymore, so you'll have to take care of the realtime aspects of the synchronization (if you want those) in your own code.
I also recommend reading the Google Cloud documentation on using the Firebase Database and App Engine's Flexible Environments. The architecture described there is the same, but a bit more up-to-date:
I'm thinking of using firebase to write a mobile app using PhoneGap and the HTML5 Application Cache.
Lets suppose each user has a list of TODO items. If the app is started when the phone is offline, will it be able to load data from the previous session and sync when a connection is established? If so I'm wondering how this is implemented because I couldn't find a reference to localStorage in firebase.js.
The short answer is: not yet.
Once an app has connected to Firebase, the client will cache data locally and be able to access data where there is an outstanding "on" callback even after a network connection is lost. However, this data is not persisted to disk, the "offline mode" will only work while the app is still running.
Full offline support will be coming in the future.
edit 2016 : full offline support is now possible for native iOS and Android apps: https://www.firebase.com/blog/2015-05-29-announcing-mobile-offline-support.html
An alternative to Firebase that solves this problem for JS apps is CouchDb (server) <=> PouchDb (JS client). If you've implemented a nice clean client side service layer then porting to PouchDb should be fairly straight forward since both are NoSQL/JSON databases. CouchDb also supports indexed map/reduce views.
PouchDb is a Javascript API that implements a fully offline CouchDb client. It can auto detect and use either local storage, IndexDb or WebSQL to permanently persist local data while online or offline. The PouchDb API can be used to access either your local or remote databases (just change the URL) and wire up full syncing or filtered syncing between the two. There are many useful PouchDb plugins, code samples and a small wrapper library to support AngularJS's Q promises API.
Using PouchDb, you can safely start up your app while offline and then days later restart your app and sync all your CUD data changes to the server. This can result in update collisions so CouchDb supports record versioning that is designed to detect and track this. Consequently, you'll likely need server side logic to resolve these collisions. This is unavoidable for distributed systems with offline synchronization and a key feature of CouchDb. I'm not sure that Firebase supports this MVCC feature.
PouchDb is basically a reimplementation of Apache CouchDb including it's synchronization protocol. Both CouchDb and PouchDb are well tested, free and open source. Being open source means that a CouchDb server can also be deployed as an Intranet service - optionally syncing to an external cloud service. There are a number of CouchDb hosting providers.
IBM's Cloudant hosting team recently added their BigCouch clustering features to Apache CouchDb 2.0 project so now you can scale from Micro Db (PouchDb) => Single Server => Multi-Master (Replicated) => Big Couch Clustered / Geo Clustered. Unlike MongoDb, CouchDb safely supports single server deployment.
NOTE: PouchDb can also sync to CouchBase using the same protocol but Couchbase !== CouchDb. It's a commercial product.
Another cool trick is that PouchDb can be run inside a NodeJS server as a replacement for CouchDb. I think it's not (yet) ready for production use but very handy for unit testing. See express-pouchdb.
Links:
Apache CouchDb Project
Couch Db - The Definitive Guide - Free book.
Pouch Db
AngularJS wrapper for PouchDB
Couchbase
CouchDb Hosters (Updated July 2020)
Compare CouchDb Hosters
Cloudant
Hyve
DIY
How To Install CouchDB and Futon on Ubuntu 12.04 - Digital Ocean
Secure CouchDB by using SSL/HTTPS
Docker + CouchDb:
Dockerizing a CouchDB Service
GitHub: Yet Another Dockerized CouchDB
Dockerized CouchDB 1.6.0 with stud SSL terminator
Security Model
One issue you'll need to consider when migrating to CouchDb is that it has a more limited access control model. This is partly due to it's replication algorithm. This blog post covers this in detail (better than the real definitive guide).
Matt Woodward's Definitive Guide to CouchDB Authentication and Security
CouchDB Security Overview
superlogin is an excellent NodeJS Passport/Authentication API that wrappers CouchDb allowing PouchDb offline apps to support social OAuth logins or username/password accounts. Includes a fully working demo and client side ng-superlogin Angular API.
Adapters
In practice you'll probably want to use WebSQL for your PouchDB storage as it performs much better. - Here's the full details on the storage adapters
PouchDb Extras
There's a staggering array of new cool "Christmas Tree" goodies always poping out the open source door from the prolific PouchDb community.
One the best features of PouchDb is all the open source plugins (37) and UI framework adapters (12).
Very old question, yet problem still persists.
I have been using firebase only for a few days and I couldn't find a satisfactory solution to this problem myself, so I thought I would share what I ended up doing.
On app start I check if I am online navigator.onLine. If not, I read from local storage and restore the firebase ref from it.
export const firebaseFromLocalStorage = (local, storeRef) => {
// assuming data is array
const localData = JSON.parse(localStorage.getItem(local)) || []
localData.map(obj => {
const key = obj['.key']
delete obj['.key']
storeRef
.child(key)
.set(obj)
})
}
Then proceed as normal. When the app goes online it will sync its local state with the server just like when it starts online and loses connection momentarily.
Off course this implies that localStorage needs to be kept in sync with your firebase ref. I do this at each get operation.
Hope this helps
I am working with a legacy web application based off of Turbogears 1.1 (CherryPy 2.3) and I would like to integrate it with a Django 1.4 web application. What I would like to do ideally is find some way for both applications to share authentication/session state so that the experience is seamless to the user.
Both applications can run on the same server and technically can access the same mysql database instance.
Initial thoughts are that this could be achieved by:
Storing session data in a shared database
Use the Django application as a 'master' that would issue requests via http to the turbogears application
Running the Django application from within Cherrpy via the internal CherryPyWSGIServer
Any other suggestions would be welcome!
I would suggest looking into creating a custom Django auth and session backend which reuses the existing Turbogears data. You will likely also find it necessary to use Django 1.5's configurable user model.