I have a linear regression model build using spark mlib. Now i want to export the model and use the model via a web service.
I was looking into PMML was not able to find a proper source as how can I use it. Do I need to install any specific PMML library or how can it be done.
The same model i tried exporting as a pickle file using joblib and deployed it as a web service via flask. I want to do something similar for the model developed in Spark mlib.
Can anyone please help me out or point me to correct source?
For a simple REST web service using PMML you could have a look at https://github.com/openscoring/openscoring. It uses JAVA though. For Python+Flask I have no idea but PMML is nothing more than a XML file which needs to be evaluated. So you could try to write your own Python logic to do it. I've also found this https://github.com/maxkferg/pmml-scoring-engine but it doesn't have much stars.
It is noted though that PMML is quite limited in its usage. Not all models are supported e.g. ALS. Another approach is to deploy a PySpark application via its binaries. I blogged about this here. We deployed it using Cloud Foundry but you can use the buildpack to deploy it on Heroku as well if you need a public endpoint. Hope this helps.
Related
I am new to cloudantdb and django. Is there any way to integrate cloudantdb with danjgo framework. Is it possible to connect cloudant with django?
As you may be aware, Cloudant is built on Apache CouchDB, so if you can't find references for Cloudant, it's usually worth also searching for CouchDB. I don't really use Django, but a quick search throws up the following references:
http://leok.me/2013/05/02/what-you-need-to-know-couchdb-django/
https://lethain.com/an-introduction-to-using-couchdb-with-django/
http://eflorenzano.com/blog/2008/11/10/using-couchdb-django/
https://djangopackages.org/grids/g/couchdb/
Most if not all solutions on the web are very outdated.
The solution used by IBM in a capstone project on Coursera was:
Use SQLite backend in your Django app
Create Cloud Functions for communicating with Cloudant
Create REST API definitions in your Django app that utilize the Cloud Functions to communicate with Cloudant
I don't know if it's the best solution, but it works. I do feel like SQLite is redundant at this point but without it, I couldn't get the app to work.
I am trying to create a web application using python on the google cloud platform. I have followed Google's quick-start guide and completed it.
App currently just prints out text
However, I am struggling to figure out a way to create a gui. I am aware of the python modules tk and tkinter and I have tried to use those with no success. I also saw the warning in the Google Cloud Console about how installed modules only exist for the current instance of the app - but even when installing the module directly before deploying the app - it still throws an import error.
I did some research and followed this guide: https://cloud.google.com/appengine/docs/standard/python/tools/using-libraries-python-27
Assuming I didn't miss any steps - would this guide solve my problem? Or is there a better way to create a GUI?
Any suggestions are encouraged. Thanks
App Engine is web oriented service. Read about it in the overview and the appropriate gui there would be a web based one.
If you want to have a remote server that will server graphical applications look closer at Google Cloud Compute Engine where you can have a VM instance, where you can install and run any application.
I would like to know which versions of django can I use with google cloud sql. I am currently trying 1.8 but I guess it is not supported.
Is there any documentation on supported version?
Thank you
So, after contacting google cloud support to be sure about the possibilities, I will answer my question:
You can use any version of Django in appengine, you can install it to a lib module and then add the file appengine_config.py with the lines:
from google.appengine.ext import vendor
vendor.add('lib')
If you want to use the django mysql backend you can do so and you can use Google Cloud SQL first OR second generation. I am currently using second generation. There is a free trial and good documentation > https://cloud.google.com/appengine/docs/python/cloud-sql/django
I have developed one example project in django1.4 & python 2.7, I want to deploy it on google app engine,
but how to configure my project as per App Engine we didn't get.
We have a site running on google app engine, but it is including with all html,js.
How do we configure a database on google app engine to deploy our django project?
Possibly the best option is to use Django Non-Rel. It's the only way (that I know of) to use the Django ORM (the django database interface) on Google App Engine without using Google's costly cloud SQL service. To do this, you'll need to use a customized version of Django and import several more libraries. It's a small project to get it up and running, but it's worth the effort. More information can be found on this website:
http://django-nonrel.org/
Note, that even though django-nonrel allows you to use the Django database interface, it will not allow you to use certain SQL features, such as joins. If you need joins, then your best option would be to use Google App Engine + Google Cloud SQL. Documentation for that is here.
Regarding the comments:
Yes, it can run on windows, I run it on Windows.
Also, the site allbuttonspressed.com is old and out of date, use the
one above for information.
How much of a pain is it to run a Django App on App Engine? Also, does the Datastore work as-is with Django?
I spent some time trying to answer the same question... it seems to me that the most difficult thing to transfer to GAE are django's models... in the sense that they require various modifications and rethinking, mainly because GAE's backend is not a standard relational DB, but google's BigTable. I found a nice intro to this here:
http://www.youtube.com/watch?v=rRCx9e38yr8
Anyways, it's worth downloading one of those 'patches' and have a go with it!
For me the best solution is the 'app-engine-patch'. I downloaded the sample project and it worked straightaway! (Mind that you need to have GAE's SDK installed separately) A killer-feature for me is the fact that the django-admin and many other classic django functionalities have been ported too!!!
http://code.google.com/p/app-engine-patch/
The documentation is still quite minimal in my opinion, but it's good enough to get you going. It'll help you to skim though the official GAE docs though!
Just Yesterday (depending on your time zone), Google released a new SDK for Python on Appengine that supports django 1.0 out of the box.
You need to use django-nonrel (source).
You will still find loads of issues:
Many2Many relations not supported
Fake joins increase number of queries
App Engine doesn't allow any python lib with socket or C dependencies (sentry, lxml...)
You can try to get early access to CloudSQL.
Otherwise you are not constraint to use App Engine, you can think about using:
Heroku
Gondor
Cheaper and more control with support requirement files like pip.
31.01.2012, Google released App Engine 1.6.2 that supports Django out-of-the box.
App Engine includes version 0.96 of Django out-of-the-box, but it is quite crippled.
App Engine Helper and app-engine-patch supposedly fix this problem to some degree, but I haven't tried either myself.
http://code.google.com/appengine/articles/appengine_helper_for_django.html
http://code.google.com/appengine/articles/app-engine-patch.html
The amount of pain depends on how much existing code you want to reuse. Unfortunately because of the Datastore does not support SQL, you often cannot just take any django-pluggable app and use it on your GAE project.
App-engine-patch http://code.google.com/p/app-engine-patch/ looks to be ahead of the other django helpers in bringing the standard applications (Sites, ContentTypes, Flatpages) over to GAE. I have used app-engine-patch on several gae projects, and once you understand how to port a django-sql model to a django-gae model and converting sql to datastore queries things can be done very quickly - but there is always a learning curve.
appengine-helper tries to bridge the Datastore gap by providing a model so you don't have to change your model superclasses, but I've found that you end up having to change ManyToMany relationships and any sql anyway, so the advantage ends up being minimal. ae-patch has a roadmap to try to provide an ae-datastore backend, but it probably won't happen for a while.
Google has now launched their Cloud SQL storage. That is actually MySQL 5.5 in the cloud. IMO that's a very nice way to migrate your Django app into the cloud. They have a free trial up to June 1, 2013.
If you need some tips how to set up your Django project for Appengine and Cloud SQL I've written a tutorial for that.