I'm thinking of using firebase to write a mobile app using PhoneGap and the HTML5 Application Cache.
Lets suppose each user has a list of TODO items. If the app is started when the phone is offline, will it be able to load data from the previous session and sync when a connection is established? If so I'm wondering how this is implemented because I couldn't find a reference to localStorage in firebase.js.
The short answer is: not yet.
Once an app has connected to Firebase, the client will cache data locally and be able to access data where there is an outstanding "on" callback even after a network connection is lost. However, this data is not persisted to disk, the "offline mode" will only work while the app is still running.
Full offline support will be coming in the future.
edit 2016 : full offline support is now possible for native iOS and Android apps: https://www.firebase.com/blog/2015-05-29-announcing-mobile-offline-support.html
An alternative to Firebase that solves this problem for JS apps is CouchDb (server) <=> PouchDb (JS client). If you've implemented a nice clean client side service layer then porting to PouchDb should be fairly straight forward since both are NoSQL/JSON databases. CouchDb also supports indexed map/reduce views.
PouchDb is a Javascript API that implements a fully offline CouchDb client. It can auto detect and use either local storage, IndexDb or WebSQL to permanently persist local data while online or offline. The PouchDb API can be used to access either your local or remote databases (just change the URL) and wire up full syncing or filtered syncing between the two. There are many useful PouchDb plugins, code samples and a small wrapper library to support AngularJS's Q promises API.
Using PouchDb, you can safely start up your app while offline and then days later restart your app and sync all your CUD data changes to the server. This can result in update collisions so CouchDb supports record versioning that is designed to detect and track this. Consequently, you'll likely need server side logic to resolve these collisions. This is unavoidable for distributed systems with offline synchronization and a key feature of CouchDb. I'm not sure that Firebase supports this MVCC feature.
PouchDb is basically a reimplementation of Apache CouchDb including it's synchronization protocol. Both CouchDb and PouchDb are well tested, free and open source. Being open source means that a CouchDb server can also be deployed as an Intranet service - optionally syncing to an external cloud service. There are a number of CouchDb hosting providers.
IBM's Cloudant hosting team recently added their BigCouch clustering features to Apache CouchDb 2.0 project so now you can scale from Micro Db (PouchDb) => Single Server => Multi-Master (Replicated) => Big Couch Clustered / Geo Clustered. Unlike MongoDb, CouchDb safely supports single server deployment.
NOTE: PouchDb can also sync to CouchBase using the same protocol but Couchbase !== CouchDb. It's a commercial product.
Another cool trick is that PouchDb can be run inside a NodeJS server as a replacement for CouchDb. I think it's not (yet) ready for production use but very handy for unit testing. See express-pouchdb.
Links:
Apache CouchDb Project
Couch Db - The Definitive Guide - Free book.
Pouch Db
AngularJS wrapper for PouchDB
Couchbase
CouchDb Hosters (Updated July 2020)
Compare CouchDb Hosters
Cloudant
Hyve
DIY
How To Install CouchDB and Futon on Ubuntu 12.04 - Digital Ocean
Secure CouchDB by using SSL/HTTPS
Docker + CouchDb:
Dockerizing a CouchDB Service
GitHub: Yet Another Dockerized CouchDB
Dockerized CouchDB 1.6.0 with stud SSL terminator
Security Model
One issue you'll need to consider when migrating to CouchDb is that it has a more limited access control model. This is partly due to it's replication algorithm. This blog post covers this in detail (better than the real definitive guide).
Matt Woodward's Definitive Guide to CouchDB Authentication and Security
CouchDB Security Overview
superlogin is an excellent NodeJS Passport/Authentication API that wrappers CouchDb allowing PouchDb offline apps to support social OAuth logins or username/password accounts. Includes a fully working demo and client side ng-superlogin Angular API.
Adapters
In practice you'll probably want to use WebSQL for your PouchDB storage as it performs much better. - Here's the full details on the storage adapters
PouchDb Extras
There's a staggering array of new cool "Christmas Tree" goodies always poping out the open source door from the prolific PouchDb community.
One the best features of PouchDb is all the open source plugins (37) and UI framework adapters (12).
Very old question, yet problem still persists.
I have been using firebase only for a few days and I couldn't find a satisfactory solution to this problem myself, so I thought I would share what I ended up doing.
On app start I check if I am online navigator.onLine. If not, I read from local storage and restore the firebase ref from it.
export const firebaseFromLocalStorage = (local, storeRef) => {
// assuming data is array
const localData = JSON.parse(localStorage.getItem(local)) || []
localData.map(obj => {
const key = obj['.key']
delete obj['.key']
storeRef
.child(key)
.set(obj)
})
}
Then proceed as normal. When the app goes online it will sync its local state with the server just like when it starts online and loses connection momentarily.
Off course this implies that localStorage needs to be kept in sync with your firebase ref. I do this at each get operation.
Hope this helps
Related
I am developing an application which is used by multiple clients and the main goal of the app is to be used online. But a specific requirement is, it should be able to work offline too (only for a single client(s) in an emergency situation and for short time - 24 hours maximum). now I am using Django REST framework for backend and Jquery/AJAX frontend for GET and PUT etc. requests which update PostgreSQL DB on the backend. Now little research on the internet suggests the I should use PouchDB on the frontend and/or CouchDB on the backend. But my questions are:
Is it really possible?
If yes, then which DB should be used for the backend database?
When the offline client become available online, how we can synchronize the data generated online?
Can we cache some data on clients machine for offline availability purpose?
Is it still possible to use PostgreSQL for the backend? (I really want to use it !)
I am currently running WSO2 Analytics on a windows server but I want implement the analysing part somehow that a client can connect to the server and do some processing like visualization on its own rather than all processing being done on the server. Is this something possible on WSO2 platform?
Thanks
You can setup database you want ( see the documentation ). For production usage I woudn't even recomment using the bundled H2 database. WSO2 analytics supports number of databases by default, I believe Oracle is one of them.
As stated in the comments - you can create a client or service which reads the data from the database and displays them its own way.
most challenging part for me is that how the client uses the information from the database?
This is already on your own (outside scope of this question). You've asked if your client can access the analytics (result) data - yes you can. How to do that is up to you. (depending what the client is, ..)
For example at our client they are building data APIs which are directly consumable by different frontend libraries creating nicer charts and reports.
I had few questions that might be easy for others, but that I couldn't wrap my head around.
In developing a "PRODUCTION LEVEL" full-stack web application(node.js/react/webpack),
1) Where do you set up your database? (when developing, I'm using apache couchdb running on my localhost, but when deployed, is cloud database(cloudant) the only solution? or am i missing something?)
2) Is it recommended to deploy my server(node.js) to either digital ocean/aws/heroku, AND set up a third party database else where? (in my case, I'm have to use either Digital Ocean/Aliyun(Chinese Web Service), but they don't seem to have a database package that comes with couchDB) - What is the practical solution for production level application?
3) If cloud database is the practical solution, What Do I do if there is no database storage center for CouchDB located in China? is there a cloud database storage that universally saves all noSQL data regardless of your type of DB? (mongoDB, couchDB, etc.,)
4) AWS/Heroku provides add-ons where you can connect cloud database to my application, does this make the speed of my application faster? For Digital Ocean, it shows article about setting up CouchDB with their service, but does that mean that database will be available for my users to access? or is that just for development purposes
5) Where and How does "Docker" come in to play to help in my situation?
Sincerely,
I cannot say for CouchDB, but I have hosted multiple web applications on AWS using their RDS Database (MySQL). The service you choose (AWS/DO/Heroku), depends on your application and your requirements (pricing etc).
I don't think AWS has a package for MongoDB, but there is a third party service MongoLabs, which can host the MongoDB Database, I bet there would be some out there for CouchDB too.
Or if you cannot get a third party hosting, consider installing the database on your server itself. Getting a VPS from either DO or AWS and setting it up yourself could be an option in that case. The link you mentioned in your last paragraph would help you here. And yes, if you use that and let node connect to it, you can use it just like any other cloud based database, just that it would be on your server.
I haven't used Docker, so cannot say if and how that could help
UPDATE: (reply to comment)
A VPS is a server in the cloud. You don't set up the database on your local computer, no one can access that. You set up your database on the VPS (in the cloud) and then everyone can access that.
A VPS is like your own clean copy of server (ubuntu/fedora) in the cloud, so you can pretty much do anything on it, like your local computer. So basically your database would also be in the cloud.
There are actually 2 ways you could do that.
Get a VPS, install your database and set up your node.js server on the same VPS. Your node application would access the database on the same VPS.
Get a VPS specially for the database, and set up your node.js on another VPS, this would separate the database and node app on two different servers.
To answer part of your question... if you set up a CouchDB server on Digital Ocean (or on AWS, Azure, Google Cloud etc) it will be available to your production users, not just you. You will want to set up security/firewall to limit who can access your server of course.
Cloudant provide CouchDb as a service, in other words you would not have to install the software or manage a server.
With Digital Ocean/AWS/Azure/Google it is down to you to manage the virtual server and the database/other software on it. You can install CouchDb on any of these services and you can install both NodeJS and CouchDb on the same virtual server if you wish.
Bitnami have a CouchDb package that you can use to deploy CouchDb on to several of the major hosting companies, which makes the setup process easier.
I see that AWS and Azure have data centres in China, but at the moment Digital Ocean do not as far as I am aware. I hope this helps
I am planning to use Firebase database and want to know how it fits in to the following scenario.
lets say I have a browser app, android app / iOS which uses Web Services to get / insert data, web services talks to the Data Base and returns data to the client.
This way I have to write code once in my web services and all the clients use that to retrieve and insert data to the database.
If I want to use Firebase, will I be following the same approach of having webservices between the client's and the Firebase DB.
I have done some sample Firebase examples where it it gets data from database directly without web services and in this approach we have to write our logic on each client (Web browser/ android app/ iOs app).
I have looked into this article
https://firebase.googleblog.com/2013/03/where-does-firebase-fit-in-your-app.html?showComment=1480073224245#c464815735109872173
The Pattern 2 has the server concept but that does not look appropriate in my scenario.
Can I have my web service and Firebase database and get data Synchronization capabilities.
Correct me if I am wrong and please suggest the approach I need to take.
Thank you for your valuable suggestions in advance.
Thanks & regards,
Rao Burugula
The article you link gives you the most common options for integrating Firebase into your app. Pattern 2 is the easiest way to use the Firebase Database and run your own server-side code:
In this model the Firebase Database sits between the app on the user's device and your back-end code. By using this model, you can still get all the benefits of the realtime synchronization, security rules and scalability, but also have back-end code that runs in a trusted environment.
Of course you can also go for a more traditional three-tier model, where your app server sites between the devices and the database. But in that case the Firebase database won't have direct interaction with your app anymore, so you'll have to take care of the realtime aspects of the synchronization (if you want those) in your own code.
I also recommend reading the Google Cloud documentation on using the Firebase Database and App Engine's Flexible Environments. The architecture described there is the same, but a bit more up-to-date:
Good Day Everyone,
Excuse the newbie question, I am new to Microsoft Sync Framework. I've done extensive research on the Internet in order to find my way in this puzzle with the different versions of Sync Framework, of SQL Server, of Sync Services for ADO.NET, of SyncAgent vs SyncOrchestrator, etc. etc. and what should be used in which type of scenario. Unfortunately, after about a week of struggling all day long with how to code my ASP.NET 2.0 C# web application right, I am still lost.
My current situation is this: I am developing applications for a large Department and I cannot expect to get approval for installing new stuff on the server side. I am stuck with SQL Server 2008 and (I believe), the server has Sync Framework 1.0 installed on it. However, I have the freedom to install later versions on the client computers that will connect to the server. These will have SQL Server 2008 Express (NOT Compact) and will each run the web app. in their localhost IIS. The synchronization model is centralized in that the clients will only connect to the server for bidirectional synchronization (in a star-shaped network topology, do we call this the hub-spoke model?) but will not connect to each other (no peer-to-peer collaboration).
I have prepared both sides of the database for synchronization (enabled Change Tracking, put GUID data type for Primary Keys, etc. etc.)
The core of the synchronization, the program that makes the interface between the two nodes to synchronize, seem to be exclusively the web application on the client side. Right?
QUESTION: If I want to use Sync Framework v2.0 or v2.1, can I just ignore what version is installed on the server? In other words, is the Sync Framework on the server side even doing something? The SQL Server does not have the web application installed on it.
Unfortunately I could not find answer to this rather simple question on the web!
Thanks very much for your help! Have a great day!
Kindest Regards,
Zyxy
No, you don't need to install sync framework on the central sever. All you need from the central database is a connection string. As long as you can access the central database with a login that has sufficient rights, then you don't need to install anything on it. The sync application can run from anywhere as long as it is able to connect to the central server.
depends how you build your application.
if all sync code is on your app and you simply point it to connect the central sql server, then no need to have sync framework on the server.
if however you decide to use WCF such that you have a service component of the sync on the server side, then you need sync framework on the server. you client will have a proxy for the server side service and part of the synchronization will be executed server side on the wcf service.
with regards to SQL Express on the client side, SQL Ce is the only supported client database is you use VS Local Database Cache Project item or if you manually code against the SyncAgent/SQLCeClientSyncProvider/DbServerSyncProvider.
if you use SQL Express, you will have to use the newer SyncOrchestrator/SqlSyncProvider combo but that doesnt use the built-in SQL Change tracking.