I need to implement a Django RESTful service where I can run an algorithm and get the result.
On the server I need to have a CSV file with the records for the algorithm. A service, let's say /train will allow me to train a Random Forest with the data in the CSV file, and finally another service /predict will receive the parameters and send me the result. The problem is that I have this running as a script on my computer and I don't know how to structure it for a web application.
I have already done RESTful APIs in Django but this problem is different I think.
I won't need models?
What about the serializers?
My idea is to send a GET request to /predict with the parameters needed for the Random Forest and return the algorithm result.
Any suggestions? Or a public repo with a similar problem?
let say you have
train_view for '/train' with POST request.
result_view for /predict with GET request
Do you need models ?
I think you need that since in request /predict you are going to apply logic on the data you have given in request /train, so create model.
Do you need serializers
Since you have model, you can write modelserializer
Related
I wanna do the following inside a Django REST ModelViewSet:
receive data from an HTTP POST request
break the data into several pieces
pass the pieces to different ViewSets that should handle authentication, etc. based on the original request
Background is the following:
I have a working REST implementation with normal ModelViewSets for all endpoints and now I'm implementing a syncing protocol for a Progressive Web App that should work completely offline.
The idea is that the normal ViewSets keep working and I implement the custom logic only once in the ViewSets. But I want to be able to collect request data offline and send all at once (also to guarantee order). Then the "sync" endpoint breaks the data apart and passes it to the respective views. I'm only missing a copy of the request object with adjusted data...
What I tried to far:
deepcopy of the request object does not work
I have a hard time figuring out how to construct a request from scratch that mimics the original
I have an App composed by back-end: Python with Django and Django REST, and front-end composed of React.
Right now I have Excel files with data, which I import with python in json format to the back-end, so they are available for a fetch in the front-end via REST-url like here.
I am now translating my data into a web-based-database to be queried into my app.
But I have questions regarding the structure of my app with this change.
I have url-based queries for my new database.
Should I continue to import the queries in the back-end REST framework and, from there, to React?
Or should I use the url-based queries directly inside my React, substituting the REST url calls?
You can get an idea by referring this url.
https://www.andreasreiterer.at/connect-react-app-rest-api/
this describing about how to bind data using REST APIs in react.
I have found some sources that presented me two ways of solving the problem
Case 1:
Have the JSON Query importation on server side in your back-end and pass this data to your API (REST in my case).
Basic source: https://www.valentinog.com/blog/tutorial-api-django-rest-react/
Pros:
Do not need to change the structure for the rest of my application. The data layer continues the same, as before I was working with an Excel file and now I just change to a JSON Query.
The connection between server-client continues to be straight forward
Credential systems can be applied more easily for the data will be stored in your API
Cons
Harder to implement
Connection between python and url queries must have individual settings (url-queries are usually browser-based, and some queries can't be performed in python)
Harder to debug
Case 2: Query data with a native fetch Javascript method and handle the data in client side.
Basic Sources:https://www.robinwieruch.de/react-fetching-data/
https://blog.hellojs.org/fetching-api-data-with-react-js-460fe8bbf8f2
Pros:
Faster and easier to implement
Easier to debug
Javascript handles queries in a simpler way than python
Cons:
Credential system can't be applied
Less secure/robust method
Double connection between client and server (client-queries and client-API), because the API would still be maintained to store local information.
I am looking to make an endpoint which would expect an Outbound message from Say SalesForce. The end point would consume the data (most likey update DB with the data). I was hoping to have some questions answered/clarified.
1) would django rest framework be the way to go?
2) since i dont have much control, I believe the auth-key/token etc would come in as a query string param. Does rest framework allow authentication this way?
Thanks for you time and help.
Regards
Tanmay
Most definitely. As long as you know the format of the incoming data from SalesForce, maybe JSON?
1) DRF would most definitely be the way to go. You can easily write an endpoint that accepts a POST request and handles the data preprocessing before saving it to DB
2) Yes, you can do so. You can take a look at writing a Custom Authentication class for that endpoint here.
ember-data understands json-api natively ,if we have to integrate ember-data and its save() and find() methods with postgrest style REST calls, where do we need to do changes ?
Do we need to modify client side in ember or some server side logic for mapping with ember-data requirements.
So postgrest REST api calls look like these to get films and their title and competition.name from related table ->
http://localhost:3001/film?select=title,competition{name}
http://localhost:3001/users?select=email::text&id=eq.2&sign_in_count=eq.16
There are two questions here:
Is it better to transform to/from JSON API at the server or the client?
If transforming in the client, where do transforms to/from JSON API occur?
Is it better to transform to/from JSON API at the server or the client?
The first question is really a matter of preference. I personally prefer having the server emit and accept JSON API format because it allows you to ship fewer lines of JavaScript to the client and there's a tendency for multiple clients to communicate with the same server, so standardizing that makes for faster client application development.
For example, you might have two Ember clients (one general user-facing, one admin-facing), an iOS client, and perhaps another server all requesting to your PostgREST server.
That said, you can also think of the format that PostgREST uses as its own spec and have all the clients adhere to that.
If transforming in the client, where do transforms to/from JSON API occur?
Which brings us to question 2: How do you make Ember Data communicate with a server that does not use the JSON API standard?
This occurs in two places: The Adapter and the Serializer.
The Adapter translates requests for data into the appropriate URL where the data can be found (or added) and initiates requests.
So, "give me the photo with the ID of 22" (store.find('photo', 2)), will ask the adapter (assuming Photo #2 isn't already loaded), "hey, someone wants Photo #2, go fetch it please".
The Adapter will make the request and hand the response over to its Serializer.
The Serializer is responsible for translating the data that comes back into a format that Ember Data understands: JSON API.
Both Adapter and Serializer have methods you can implement to customize their behaviors.
In your case with PostgREST, the best places to start would be implementing your own findRecord on the Adapter and implementing your own normalizeResponse on the Serializer.
The docs for both explain the actions you need to take and what type of value you should return from each method.
These are two of the most basic interfaces. They don't provide a lot of functionality out of the box, but will help you become familiar with how these two objects interact.
Once you're comfortable with this basic interaction, check out the sample RestAdapter and RestSerializer for ideas on how to rely on some of the conventions Adapters and Serializers have to offer to clean up any custom code you've written.
How can I log the requests and responses that the Django server handles?
I have a Django server application has tables and relations. Many tables are exposed through several API endpoints. There are also several custom functions that take GET/POST requests with params and run a function in python and return a result.
As the server admin, I would like to log all the incoming requests and the outgoing responses for each of these requests. It is essential to me that I capture the GET/POST params made with each requests and the data that is sent out with each response. What would be the best way for me to implement this?
I would suggest you DON'T do this through Django, all you're going to do is add to the request work load. You're going to make your application slower, all so you can have it in "one place", remember, "the right tool for the job".
I highly suggest you log to a standard logger file/syslog, and then use something like GrayLog/Loggly/Splunk/Logstash/etc. -- you're going to thank me in the long run, you're not going to grind your Django server and/or DB to the ground.
If you think I'm kidding, think of this: you're more likely going to put this in a middleware right? BEFORE auth, right? If I wanted to make your life a living hell all I have to do is while true; do curl https://yourdomain.com/doesnt-need-to-exist &; done i'm going to fill your DB up in a couple of days, not to mention degrade your service as a whole. Do everyone a favor and don't use the db, specially a prod db to store logs
django-wiretap is a package that stores the requests and responses in the database via models.