Django Rest Api with Pandas - django

I am using django-rest-framework to make an api of my data. I am making an app that takes into account user data and remove outliers from that data using Pandas. I am able to present my data on frontend using django templates but somehow I am not able to make an API containing the statistical data using django-rest-framework . Can someone explain it and please help me to rectify my errors and also provide the necessary code
Here is my code
class Data(models.Model):
Name = models.CharField(max_length=30,null=True,blank=True)
Age = models.IntegerField(null=True,blank=True)
Weight = models.FloatField(null=True,blank=True)
Height = models.FloatField(null=True,blank=True)
Sugar = models.FloatField(null=True,blank=True)
def __unicode__(self):
return self.Name
My Serializer Class
class DataSerializer(serializers.ModelSerializer):
class Meta:
model = Data
fields = '__all__'
my views.py
def my_view(request):
con = sqlite3.connect("db.sqlite3")
df = pd.read_sql_query("SELECT * from visualapp_health", con)
a = df.fillna(0)
a['x-Mean'] = abs(a['Age'] - a['Age'].mean())
a['1.96*std'] = 1.96*a['Age'].std()
a['Outlier'] = abs(a['Age'] - a['Age'].mean()) > 1.96*a['Age'].std()
con.close()
return render(request, 'visual.html', {'visual': a})
I am able to get all the Data when using Django templates but somehow I don't able to understand how to make an API of all the data using django-rest- framework.

I finally got it I used django-pandas library and it worked and there is no need to connect to the database just convert your django queryset to pandas dataframe.

Related

Python Marshmallow & SQL-Alchemy - update/store complex objects

Hello community!
So I a a student and relatively new to python, implementing a CRUD-API right now, based on SQL-Alchemy, restx and Flask. For the conversion of the database-objects, I use Marshmallow. The database is a legacy database which can not be changed. In the database, there are some attributes that are encoded, for example a list of labs is encoded as string with the lab names in it. I use converters to cover them, as shown here:
gene_bp = Blueprint('gene', __name__)
api = createApi(gene_bp)
class FooDto(SQLAlchemyAutoSchema):
class Meta:
model = Foo
include_fk = True
include_relationships = True
load_instance = True
bar = decimal_converter.DecimalEncoder()
The Endpoint does look like this:
#api.route('/api/<string:id>')
#api.doc()
class FooRoute(Resource):
def get(self, id):
foo = get_foo_or_abort(id)
if foo:
return createSuccessResponse(FooDto().dump(foo))
The reading-part of the API is no problem with this, but now I struggle with the write/update part. This is my code until now:
#api.route('/api/<string:id>')
#api.doc()
class FooRoute(Resource):
#api.doc(params={'dto': 'Item to update', 'id': 'object id'})
def put(self, id):
fooToStore: FooDto = api.payload
if not fooToStore:
abort(400)
if fooToStore['id'] != id:
abort(400)
# todo: payload to database object
# todo: save payload to database
db.session.commit()
return updatedFoo
So as you can see, there are two ToDos open:
How do i convert the DTO back to a database object, without manual mapping? I understand that there is this method
fooToUpdate = dto.load(geneToStore)
but how do I add the needed complexity for encoded attributes like the lab list, or foreign keys to it?
What is the best practice in storing the data to the database?
Thank you all very much!

Selectable list of users in Django?

models.py
class EmployeeReportRequestForm(forms.Form):
EMPLOYEECHOICE = []
employees = User.objects.filter(group_name='rep')
for employee in employees:
EMPLOYEECHOICE.append([employee.get_username(), employee.get_full_name()])
employee_choice = forms.CharField(max_length=50, widget=forms.Select(choices=EMPLOYEECHOICE))
start_date = forms.DateField(widget=forms.SelectDateWidget())
end_date = forms.DateField(widget=forms.SelectDateWidget())
Trying make a form that allows someone to make a selection from a list of users in a particular group, I figured this would work, but it is not. The most info I'm able to get error wise is
"django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet."
I'm assuming my issue is trying to query the User database from within a Model and that I need to run the lines of code that generate the EMPLOYEECHOICE list in a view and then somehow pass that to the Model? Or just define the widget to be used in the View?
models.py
class EmployeeRequestForm(forms.Form):
def __init__(self):
self.choice_list = [('test', 'test'),]
self.employees = User.objects.filter(groups__name='rep')
for self.x in self.employees:
self.choice_list.append([self.x.get_username(), self.x.get_full_name()])
self.CHOICES = self.choice_list
super (EmployeeRequestForm, self).__init__()
self.fields['employee_choice'].widget = forms.Select(choices=self.CHOICES)
employee_choice = forms.CharField(max_length=100)
start_date = forms.DateField(widget=forms.SelectDateWidget())
end_date = forms.DateField(widget=forms.SelectDateWidget())
Figured out what I was doing wrong, needed to define the choices within the init method. Not sure if this is the best way to do it, but it works.

how to render datatable inside ajax-modal in openstack horizon dashboard?

i'm beginner in Python Django. according to openstack's customizing horizon dashboard tutorial i've successfully added new panel, tab with datatable. i also added table actions in table class which open ajax-modal. but instead of render form inside that i need to render datatable which should feel up by ajax-response. each row contains form input element (e.g text,radio). but i couldn't figure out how to render datatable inside ajax-modal.
please have a look on tables.py
class AddSwitch(tables.LinkAction):
name = "addswitch"
verbose_name = _("Add Switch")
url = "horizon:project:sdncontroller:addswitch"
classes = ("ajax-modal", "btn-create",)
class Switches(tables.DataTable):
dpid = tables.Column("dpid",verbose_name=_("DPID"))
address = tables.Column('address', verbose_name=_("Address"))
vendor = tables.Column('vendor', verbose_name=_("Vendor"))
packets = tables.Column('packets', verbose_name=_("Packets"))
bytes = tables.Column('bytes', verbose_name=_("Bytes"))
flows = tables.Column('flows', verbose_name=_("Flows"))
connectedsince = tables.Column('connectedsince', verbose_name=_("ConnectedSince"))
detail= tables.Column('details', verbose_name=_("Detail"))
class Meta:
name = "Switches"
verbose_name = _("Switches")
table_actions = (AddSwitch,)
also i've created workflows.py and create class for AddSwitch
class AddSwitch(workflows.Workflow):
slug = "addswitch"
name = _("Add Switch")
finalize_button_name = _("Add")
success_message = _('Added switch "%s".')
failure_message = _('Unable to add switch "%s".')
success_url = "horizon:project:sdncontroller:index"
default_steps = (AddSwitchStep,)
def format_status_message(self, message):
name = self.context.get('name')
return message % name
def handle(self, request, context):
try:
#api.lbaas.pool_create(request, **context)
return True
except Exception:
return False
this is the point where i stuck. i don't how to code and where to code for rendering datatable and that too fill up dynamically through ajax-response.
Thanks, I hope someone who could lead me into this.
You forgot to mention the "columns" attribute in the Class Meta. Please follow the mechanism currently used by Horizon to render the "Instances" data table. You can find the detailed step by step tutorial to create and render a data table here: http://docs.openstack.org/developer/horizon/topics/tutorial.html
Hope it helps

Can't Return JSON object using MongoEngine Pymongo with Django?

So I'm trying to return a JSON object for a project. I've spent a few hours trying to get Django just returning the JSON.
Heres the view that we've been working with:
def json(request, first_name):
user = User.objects.all()
#user = User.objects.all().values()
result = simplejson.dumps(user, default=json_util.default)
return HttpResponse(result)
Here's my model:
class User(Document):
gender = StringField( choices=['male', 'female', 'Unknown'])
age = IntField()
email = EmailField()
display_name = StringField(max_length=50)
first_name = StringField(max_length=50)
last_name = StringField(max_length=50)
location = StringField(max_length=50)
status = StringField(max_length=50)
hideStatus = BooleanField()
photos = ListField(EmbeddedDocumentField('Photo'))
profile =ListField(EmbeddedDocumentField('ProfileItem'))
allProfile = ListField(EmbeddedDocumentField('ProfileItem')) #only return for your own profile
This is what it's returning:
[<User: User object>, <User: User object>] is not JSON serializable
Any thoughts on how I can just return the JSON?
With MongoEngine 0.8 or greater, objects and querysets have a to_json() method.
>>> User.objects.to_json()
simplejson.dumps() doesn't know how to "reach into" your custom objects; the default function, json_util.default must just be calling str() or repr() on your documents. (Is json_util custom code you've written? If so, showing its source here could prove my claim.)
Ultimately, your default function will need to be able to make sense of the MongoEngine documents. I can think of at least two ways that this might be implemented:
Write a custom default function that works for all MongoEngine documents by introspecting their _fields attribute (though note that the leading underscore means that this is part of the private API/implementation detail of MongoEngine and may be subject to change in future versions)
Have each of your documents implement a as_dict method which returns a dictionary representation of the object. This would work similarly to the to_mongo method provided on documents by MongoEngine, but shouldn't return the _types or _cls fields (again, these are implementation details of MongoEngine).
I'd suggest you go with option #2: the code will be cleaner and easier to read, better encapsulated, and won't require using any private APIs.
As dcrosta suggested you can do something like this, hope that will help you.
Document definition
class MyDocument(Document):
# Your document definition
def to_dict(self):
return mongo_to_dict_helper(self)
helper.py:
from mongoengine import StringField, ListField, IntField, FloatField
def mongo_to_dict_helper(obj):
return_data = []
for field_name in obj._fields:
if field_name in ("id",):
continue
data = obj._data[field_name]
if isinstance(obj._fields[field_name], StringField):
return_data.append((field_name, str(data)))
elif isinstance(obj._fields[field_name], FloatField):
return_data.append((field_name, float(data)))
elif isinstance(obj._fields[field_name], IntField):
return_data.append((field_name, int(data)))
elif isinstance(obj._fields[field_name], ListField):
return_data.append((field_name, data))
else:
# You can define your logic for returning elements
return dict(return_data)

Serialize in django witha query set that contains defer

i have i little problem, and that is how can serialize a django query with defer ?
I have this model :
class Evento(models.Model):
nome=models.CharField(max_length=100)
descricao=models.CharField(max_length=200,null=True)
data_inicio= models.DateTimeField()
data_fim= models.DateTimeField()
preco=models.DecimalField(max_digits=6,decimal_places=2)
consumiveis= models.CharField(max_length=5)
dress_code= models.CharField(max_length=6)
guest_list=models.CharField(max_length=15)
local = models.ForeignKey(Local)
user= models.ManyToManyField(User,null=True,blank=True)
def __unicode__(self):
return unicode('%s %s'%(self.nome,self.descricao))
my query is this :
eventos_totais = Evento.objects.defer("user").filter(data_inicio__gte=default_inicio,
data_fim__lte=default_fim)
it works fine i think (how can i check if the query has realy defer the field user ? ) but when i do:
json_serializer = serializers.get_serializer("json")()
eventos_totais = json_serializer.serialize(eventos_totais,
ensure_ascii=False,
use_natural_keys=True)
it always folow the natural keys for user and local, i need natural keys for this query because of the fields local. But i do not need the field user.
To serialize a subset of your models fields, you need to specify the fields argument to the serializers.serialize()
from django.core import serializers
data = serializers.serialize('xml', SomeModel.objects.all(), fields=('name','size'))
Ref: Django Docs