I had to build a dashboard , so dash seemed pretty easy. But i faced lot of issues integrating with flask or django.
So I had to rebuild the dashboard with django framework with plotly.js , while using plotly dash #app.callback() was very fast and quick in updating the graphs. To mock the similar behaviour I tried using ajax in django and plotly.js. Even though the job gets done using ajax, i see there is a lag in performance, it takes 3-4 secs to render the updated graph.
Is there a better or efficient way to achieve similar performance of dash callbacks in django/ajax ?
Just because every time i have to read the csv file during ajax call , hinders my performance ?
sample backend code for ajaxcall
def itemsAjax(request):
if request.is_ajax():
itemsList = request.POST.get('items')
#itemsList = ast.literal_eval(itemsList)
#itemsList = [n.strip() for n in itemsList]
itemsList=itemsList.split(',')
daterange = request.POST.get('daterange').split('-')
franchise = request.POST.get('franchise')
startDate = dt.strptime(daterange[0].strip(),'%m/%d/%Y')
endDate = dt.strptime(daterange[1].strip(),'%m/%d/%Y')
df = at.readData()
flag = ut.determineFlag(startDate,endDate)
df = at.filter_df_daterange(df,startDate,endDate)
itemTrend_df = at.getDataForItemsTrend(df,startDate,endDate,flag)
itemTrend_plt = [{'type':'scatter',
'x' : itemTrend_df[itemTrend_df['S2PName']==item]['S2BillDate'].tolist(),
'y' : itemTrend_df[itemTrend_df['S2PName']==item]['totSale'].tolist(),
#'text' : itemTrend_df[itemTrend_df['S2PName']==item]['Hover'],
'mode' : 'markers+lines',
'name' : item
}for item in itemsList]
return JsonResponse({'itemTrend_plt':itemTrend_plt})
else:
return HttpResponse(' Not authorised!')
I'm have some trouble with the DatetimeTickFormatter object from Bokeh. Plots randomly won't generate/update.
I have been searching and found this post at Bokeh, where it states that Plot not shown if DatetimeTickFormatter partially defined. I need some help with how to define it properly. Currently I'm doing this:
from bokeh.models.formatters import DatetimeTickFormatter
from bokeh.plotting import figure
DTF = DatetimeTickFormatter()
DTF.hours = ["%H:%M"]
DTF.days = ["%d/%m/%Y"]
DTF.months = ["%d/%m/%Y"]
DTF.years = ["%d/%m/%Y"]
and
p = figure()
p.xaxis.formatter = DTF
How should I define DTF “properly”?
Firstly, I am sorry if the title is long. I am working on face detection using python. I am trying to write a script where it will notify user when there is same picture or almost same picture/faces detected between two directories/folder.
Below is the script that I wrote so far.
import cv2
import glob, requests
def detect1():
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_alt.xml')
for img in glob.glob('/Users/Ling/Pythonfiles/Faces/*.jpg'):
cv_img = cv2.imread(img)
gray = cv2.cvtColor(cv_img, cv2.COLOR_BGR2GRAY)
faces1 = face_cascade.detectMultiScale(gray, 1.3, 5)
for (x,y,w,h) in faces1:
cv2.rectangle(cv_img,(x,y),(x+w,y+h),(255,0,0),2)
def detect2():
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_alt.xml')
for image in glob.glob('/Users/Ling/Pythonfiles/testfolder/*.jpg'):
cv_image = cv2.imread(image)
gray = cv2.cvtColor(cv_image, cv2.COLOR_BGR2GRAY)
faces2 = face_cascade.detectMultiScale(gray, 1.3, 5)
for (x,y,w,h) in faces2:
cv2.rectangle(cv_image,(x,y),(x+w,y+h),(255,0,0),2)
def notify():
if detect2 == detect1:
key = "<yourkey>"
sandbox = "<yoursandbox>.mailgun.org"
recipient = "<recipient's email>"
request_url = 'https://api.mailgun.net/v2/{0}/messages'.format(sandbox)
request = requests.post(request_url, auth=('api', key),
data={
'from': '<sender's email',
'to': recipient,
'subject': 'face detect',
'text': 'common face detected'
})
print 'Status: {0}'.format(request.status_code)
print 'Body: {0}'.format(request.text)
There is no error but there is no notification either. I have a folder with 10 pictures of random faces I downloaded it from Google Image(just for learning purpose)and another folder with 2 picture of people that their face is same as the some of the picture in the previous folder. The picture with the same face is in different angle.
I wrote the script by referring to tutorial from https://pythonprogramming.net/haar-cascade-face-eye-detection-python-opencv-tutorial/
and add some line to send the notification if the program detect the same face from both folder.
My question is how do I exactly notify the user if there are same faces detected. I believe this code is incomplete and hoping that someone can give me suggestion on what to add/edit or what I should not write in this script.
Thank you in advance.
I don't know if I understand you correctly, but I think your looking for face recognition not only a face detection.
The Haar Feature-based Cascade Classifier learned very generell "How a face should look like". It detects the positions of a learned object/shape in a given input image and returns the bounding boxes.
So if you want to know if the detected face matches with a known face you need to train a recognizer. OpenCV has 3 build-in face recognizer: EigenFaceRecognizer, FisherfaceRecognizer, LBPHFaceRecognizer (Local Binary Patterns Histograms Face Recognizer).
use them with e.g. recognizer = cv2.createLBPHFaceRecognizer()
You need a training set for your users. Maybe your trainings folder could look like:
1_001.jpg, 1_002.jpg, 1_003.jpg, 2_001.jpg 2_002.jpg, ..., n_xyz.jpg
where n is the label (user id -> unique for each user) and xyz is maybe a description or a sequence number.
Update:
I used the Faces94 benchmark dataset for testing. Therefore I packed them into the folder trainingSamples and two of them (same person but different face) into the folder testFaces relative to my python script.
To rename all images in a folder matching with the pattern above I used a bash command rename
eg. asamma.[1-20].jpg to 001_[1-20].jpg
rename 's/^asamma./001_/' *
import cv2
import numpy as np
import os
class FaceRecognizer:
def __init__(self):
self.cascadeClassifier = cv2.CascadeClassifier('haarcascade_frontalface_alt.xml')
self.faceRecognizer = cv2.face.createLBPHFaceRecognizer()
if os.path.isfile('faceRecognizer.xml'):
self.faceRecognizer.load('faceRecognizer.xml')
else:
images = []
labels = []
for file in os.listdir('trainingSamples/'):
image = cv2.imread('trainingSamples/'+file, 0)
images.append(image)
labels.append(int(file.split('_')[0]))
## if you don't have pre-cropped profile pictures you need to detect the face first
# faces = self.cascadeClassifier.detectMultiScale(image)
# for (x, y, w, h) in faces
# images.append(image[y:y+h, x:x+w])
# labels.append(int(file.split('_')[0]))
self.faceRecognizer.train(images, np.array(labels))
self.faceRecognizer.save('faceRecognizer.xml')
def predict(self, image, filename):
user, confidence = self.faceRecognizer.predict(image)
if confidence < 100.0:
print('found user with id {} in picture {} with a confidence of {}'.format(user, filename, confidence))
## if you don't have pre-cropped profile pictures you need to detect the face first
# faces = self.cascadeClassifier.detectMultiScale(image)
# for (x, y, w, h) in faces
# user, confidence = self.faceRecognizer.predict(image[y:y+h, x:x+w])
# # confidence of 0.0 means perfect recognition (same images)
# if confidence < 100.0:
# print('found user with id {} in picture {} with a confidence of {}'.format(user, filename, confidence))
faceRecognizer = FaceRecognizer()
for file in os.listdir('testFaces/'):
image = cv2.imread('testFaces/'+file, 0)
faceRecognizer.predict(image, file)
The code produces the output:
found user with id 4 in picture 004_20.jpg with a confidence of 27.836526552656732
found user with id 1 in picture 001_6.jpg with a confidence of 22.473253497606876`
So it correctly recognize user 4 and user 1.
The code is tested with OpenCV 3.1-dev on Ubuntu 15.10 using Python 3.4.3 and Python 2.7.9.
Hi so I am using matplot lib in my django project. And I am able to embed a graph in my template/webpage. I need some help now on being able to animate a graph.
The graph is stored as a csv file with the location in my model. I want the graphs to be built dynamically. Now I have that. But I want them to be an animation. How would I go about doing this? I have to use matplotlib btw.
I assume make a video? I've seen people embed videos, but I was hoping to using the actual animation features of matplotlib such as:
http://pythonprogramming.net/python-matplotlib-live-updating-graphs/
Here is how I render the graph in my view:
def render_graph(request, graph_id=None):
graph = get_object_or_404(models.Graph, pk=graph_id) #grabs actual graph
data = np.genfromtxt(graph.data, delimiter=',', names=['x','y']) #extract data from cvs after x and y
fig = Figure()
ax = fig.add_subplot(111)
ax.plot(data['x'], data['y'], label=graph.name)
canvas=FigureCanvas(fig)
response = HttpResponse(content_type='image/png')
canvas.print_png(response)
return(response)
have you considered using d3 (d3js.org)? Might give better results.
I'm trying to grab all of the img links in the slideshow on a Reuters article. I was wondering if someone could explain to me why this only grabs the first image and no others?
Here's the article for reference: http://www.reuters.com/article/2014/04/11/us-cocoa-gold-westafrica-insight-idUSBREA3A0DP20140411
links = soup.find_all("div", {'id': 'frame_fd1fade'})
for link in links:
for img in link.find_all('img', src=True):
img = img["src"].split("src=")[-1]
print img
In the page source for the article, there's only one div with id="frame_fd1fade". Within it, there is one img tag, which contains the first picture. You'll have to look into the mechanism that the page uses to change pictures and use that somehow to get your images.
Try running this to see how many instances of frame_fd1fade there are in the source:
import urllib
import re
f = urllib.urlopen("http://www.reuters.com/article/2014/04/11/us-cocoa-gold-westafrica-insight-idUSBREA3A0DP20140411")
cnt = 0
for line in f:
if re.search("frame_fd1fade", line):
cnt += 1
print "cnt =", cnt