I am new to django and SQL queries. I am trying some annotation with django. but unable to get results
+-----------------------+-----------+---------------------+
| email | event | event_date |
|-----------------------+-----------+---------------------|
| hector#example.com | open | 2017-01-03 13:26:13 |
| hector#example.com | delivered | 2017-01-03 13:26:28 |
| hector#example.com | open | 2017-01-03 13:26:33 |
| hector#example.com | open | 2017-01-03 13:26:33 |
| tornedo#example.com | open | 2017-01-03 13:34:53 |
| tornedo#example.com | 1 | 2017-01-03 13:35:22 |
| tornedo#example.com | open | 2016-09-05 00:00:00 |
| tornedo#example.com | open | 2016-09-17 00:00:00 |
| sparrow#example.com | open | 2017-01-03 16:05:36 |
| tornedo#example.com | open | 2017-01-03 20:12:15 |
| hector#example.com | open | 2017-01-03 22:06:47 |
| sparrow#example.com | open | 2017-01-09 19:46:26 |
| sparrow#example.com | open | 2017-01-09 19:47:59 |
| sparrow#example.com | open | 2017-01-09 19:48:28 |
| sparrow#example.com | delivered | 2017-01-09 19:52:24 |
+-----------------------+-----------+---------------------+
I have a table like this which contains email activity. I want to find who opened recently and also i want to count of each event happened. I want results exactly like
email | open | click | delivered | max_open_date
hector#example.com 4 <null> 1 2017-01-03 22:06:47
sparrow#example.com 3 <null> 1 2017-01-09 19:48:28
tornedo#example.com 4 1 <null> 2017-01-03 20:12:15
my model looks:
class EmailEvent(models.Model):
event = models.TextField(blank=True, null=True)
email = models.TextField(blank=True, null=True)
event_date = models.DateTimeField(blank=True, null=True)
i tried the following code. it giving correct count for open, click, delivered but giving wrong result for max_open_date. but i don't know why
EmailEvent.objects.values('email').annotate(
max_open_date=Case(When(event='open', then=Max('event_date')))),
open=Sum(Case(When(event='open',then=1),output_field=IntegerField())),
click=Sum(Case(When(event='click',then=1),output_field=IntegerField())),
open=Sum(Case(When(event='open',then=1),output_field=IntegerField())),
delivered=Sum(Case(When(event='delivered',then=1),output_field=IntegerField())),
)
Help me to get exact results i want. sorry for my bad english. Thanks!
I do not use django, but probably you need something like this:
max_open_date=Max(Case(When(event='open', then='event_date')))
Related
I have a software application which communicates with a Graphic User Interface using MQTT protocol.
The first software (let's call it app) publish messages and the second software (let's call it gui) subscribes to topics.
App software
The app software is a C++ application which embeds a MQTT client based on mosquittopp.
It sends messages on different topics in order to notify gui software.
Gui software
The gui software is a Qt application which embeds a MQTT client also based on mosquittopp.
When a message is received on a topic, a signal messageReceived is emitted.
The signal messageReceived is connected to a slot Message_received_from_topic.
All messages from all topics are processed into Message_received_from_topic slot.
Presenting the issue
A gui screen is composed of several fields. A field is updated when a message is received on the associated topic.
When gui stays on the same screen, receiving messages through MQTT allow to update fields without reloading the full screen.
However when changing to a screen which contains many fields, there are some glitches that appear because the screen is refreshed before all fields are generated.
In order to avoid those glitches, I want to lock screen refreshing before updating all fields at least once and then unlock refresh.
That's why not receiving MQTT messages in the right order is problematic.
Issue analysis
The issue I'm facing is that messages are not received in order by gui.
Here is a table that present how messages are sent and received.
+----------------------------------------------+----------------------------------------------+----------------------------------------------+
| Sent by **app** | On broker (using mosquitto_sub) | Received by **gui** |
+---------------------------+------------------+---------------------------+------------------+---------------------------+------------------+
| Topic | Message | Topic | Message | Topic | Message |
+---------------------------+------------------+---------------------------+------------------+---------------------------+------------------+
| widget/refresh | 0 | widget/refresh | 0 | widget/refresh | 0 |
| widget/input/value/9 | {"label":"2020"} | widget/input/value/9 | {"label":"2020"} | widget/refresh | 1 |
| widget/input/value/10 | {"label":"02"} | widget/input/value/10 | {"label":"02"} | widget/input/value/9 | {"label":"2020"} |
| widget/input/value/11 | {"label":"20"} | widget/input/value/11 | {"label":"20"} | widget/input/position/9 | 1 |
| widget/input/value/12 | {"label":"15"} | widget/input/value/12 | {"label":"15"} | widget/input/selection/9 | true |
| widget/input/value/13 | {"label":"06"} | widget/input/value/13 | {"label":"06"} | widget/input/value/10 | {"label":"02"} |
| widget/input/position/12 | 0 | widget/input/position/12 | 0 | widget/input/position/10 | 0 |
| widget/input/selection/12 | false | widget/input/selection/12 | false | widget/input/selection/10 | true |
| widget/input/position/13 | 0 | widget/input/position/13 | 0 | widget/input/value/11 | {"label":"20"} |
| widget/input/selection/13 | false | widget/input/selection/13 | false | widget/input/position/11 | 0 |
| widget/input/position/9 | 0 | widget/input/position/9 | 0 | widget/input/selection/11 | true |
| widget/input/selection/9 | true | widget/input/selection/9 | true | widget/input/value/12 | {"label":"15"} |
| widget/input/position/10 | 0 | widget/input/position/10 | 0 | widget/input/position/12 | 0 |
| widget/input/selection/10 | true | widget/input/selection/10 | true | widget/input/selection/12 | false |
| widget/input/position/11 | 0 | widget/input/position/11 | 0 | widget/input/value/13 | {"label":"06"} |
| widget/input/selection/11 | true | widget/input/selection/11 | true | widget/input/position/13 | 0 |
| widget/input/position/9 | 1 | widget/input/position/9 | 1 | widget/input/selection/13 | false |
| widget/refresh | 1 | widget/refresh | 1 | | |
+---------------------------+------------------+---------------------------+------------------+---------------------------+------------------+
Here messages are received in the wrong order, I'm also missing one message on topic "widget/input/position/9"
All publications and subscriptions are using QoS 2.
Questions
Does anybody knows why messages are not received in the right order by gui when they are correctly received using mosquitto_sub ?
Is there any configuration to apply to mosquitto broker in order to keep message order ?
I am using a window function to get the difference in the values of a column (downloads) between two dates. I'd also like to get the product of that difference multiplied by the size of the file to get the bytes downloaded for the period.
With the help of this community, I am able to get the number of downloads but cannot find the correct syntax to get the product of downloads * size.
Table 'files'
+---------------+------------------------+------+-----------+------------+
| site | full_path | size | downloads | date_stamp |
+---------------+------------------------+------+-----------+------------+
| Lawrenceville | lr1/dir1/subdir1/file1 | 1000 | 7 | 2019-08-08 |
| Lawrenceville | lr1/dir1/subdir1/file1 | 1010 | 9 | 2019-08-15 |
| Lawrenceville | lr1/dir1/subdir1/file2 | 1213 | 5 | 2019-08-08 |
| Lawrenceville | lr1/dir1/subdir1/file2 | 2000 | 5 | 2019-08-15 |
| Lawrenceville | lr1/dir2/subdir1/file1 | 2213 | 5 | 2019-08-15 |
| Rennes | rr1/dir1/subdir1/file3 | 200 | 3 | 2019-08-08 |
| Rennes | rr1/dir1/subdir1/file3 | 201 | 4 | 2019-08-15 |
+---------------+------------------------+------+-----------+------------+
SELECT site, sum(diff) FROM (SELECT site, downloads - lag(downloads, 1) OVER (PARTITION BY site, full_path ORDER BY date_stamp) AS diff FROM files WHERE date_stamp IN ('2019-08-15', '2019-08-08')) group by site
produces this:
+---------------+-----------+
| site | downloads |
+---------------+-----------+
| Lawrenceville | 2 |
| Rennes | 1 |
+---------------+-----------+
I have tried:
SELECT site, sum(diff), sum(sum(diff)*bytes) FROM (SELECT site, downloads - lag(downloads, 1), size OVER (PARTITION BY site, full_path ORDER BY date_stamp) AS diff, bytes FROM files WHERE date_stamp IN ('2019-08-15', '2019-08-08')) group by site
sqlite3.OperationalError: near "(": syntax error
Ideally I want this output:
+---------------+-----------+----------+
| site | downloads | bytes |
+---------------+-----------+----------+
| Lawrenceville | 2 | 2020 |
| Rennes | 1 | 201 |
+---------------+-----------+----------+
Lawrenceville had 2 downloads of file lr1/dir1/subdir1/file1 which is 1010 bytes (on 2019-08-15). File lr1/dir1/subdir1/file2 had no downloads for that period. It would be nice to include files lr1/dir1/subdir1/file2 and lr1/dir2/subdir1/file1 but they get excluded by the window function. I can get them with a separate query.
Rennes has 1 download of file rr1/dir1/subdir1/file3
If your current query works then you only need max() window function in the subquery:
SELECT site, sum(diff) downloads, sum(diff) * size bytes
FROM (
SELECT
site,
downloads - lag(downloads, 1) OVER (PARTITION BY site, full_path ORDER BY date_stamp) AS diff,
max(size) OVER (PARTITION BY site, full_path) AS size
FROM files
WHERE date_stamp IN ('2019-08-15', '2019-08-08')
)
group by site
See the demo.
Results:
| site | downloads | bytes |
| ------------- | --------- | ----- |
| Lawrenceville | 2 | 2020 |
| Rennes | 1 | 201 |
I find it best to use an example, so here we go:
Say I have a table with chores and a table with a weekly schedule like this:
CHORES:
|----+---------------+----------+-------|
| id | name | type | hours |
|----+---------------+----------+-------|
| 1 | clean kitchen | cleaning | 4 |
|----+---------------+----------+-------|
| 2 | clean toilet | cleaning | 3 |
etc
SCHEDULE:
|------+---------------+---------------+-----|
| week | monday | tuesday | etc |
|------+---------------+---------------+-----|
| 1 | clean kitchen | clean toilet | etc |
|------+---------------+---------------+-----|
| 2 | clean toilet | clean kitchen | etc |
etc
I want to make sure that for one week, you can't have duplicate cells, so this wouldn't be allowed:
SCHEDULE:
|------+---------------+--------------+-----|
| week | monday | tuesday | etc |
|------+---------------+--------------+-----|
| 1 | clean toilet | clean toilet | etc |
etc
What would I have to do in my models.py to get this behaviour?
Try django unique-together in model meta option.
https://docs.djangoproject.com/en/1.11/ref/models/options/#unique-together
I'd better user ManyToMany through another table like that:
SCHEDULE:
------+------------------------+
| week | chores |
|------+------------------------+
| 1 | many to many to chores |
|------+------------------------+
| 2 | many to many to chores |
And trough table like that
THROUGH TABLE:
|---------+---------------+---------------+
| week_id | day of week | chores_id |
|---------+---------------+---------------+
| 1 | Monday | clean toilet |
|---------+---------------+---------------+
| 1 | Tuesday | clean kitchen |
And in that table make unique together for week_id and chores_id
I'm new to Django, and I'm trying to create a simple application that keep track on different server configurations in a SQlite database. I've created 2 database models:
from django.db import models
class Server(models.Model):
name = models.CharField(max_length=250)
class Config(models.Model):
server = models.ForeignKey(Server, on_delete=models.CASCADE)
configuration = models.CharField(max_length=250)
config_version = models.IntegerField()
Here are the 2 models sample data:
Server:
| id | name |
| ------ | ------ |
| 1 | Server1 |
| 2 | Server2 |
| 3 | Server3 |
Config:
| id | configuration | config_version | server |
| ------ | ------------- | -------------- | ------ |
| 1 | srv1_cfg1 | 1 | 1 |
| 2 | srv2_cfg1 | 1 | 2 |
| 3 | srv2_cfg2 | 2 | 2 |
| 4 | srv2_cfg3 | 3 | 2 |
| 5 | srv3_cfg1 | 1 | 3 |
| 6 | srv1_cfg2 | 2 | 1 |
| 7 | srv1_cfg3 | 3 | 1 |
I would like to query the Config table, and get only rows with the maximum value of "config_version" field for each server id, like:
Desired result:
| id | configuration | config_version | serverid | servername |
| ------ | ------------- | -------------- | -------- | ---------- |
| 4 | srv2_cfg3 | 3 | 2 | Server2 |
| 5 | srv3_cfg1 | 1 | 3 | Server3 |
| 7 | srv1_cfg3 | 3 | 1 | Server1 |
I've tried many different options to construct the correct query, but so far I cannot get what I want. My best result is to query the Server table:
Server.objects.annotate(maxver=Max('config__config_version'))
But it seems I cannot get access to the Config table objects, so I guess I need to query the Config table with some filtering?
I can do this with a raw SQL query, but I would strongly prefer to do it the "Django" way. Any help will be much appreciated.
After some more struggle with this, I've came with a solution that works for me. I'm sure it is not optimal, but at least seems to works:
from django.db.models import Max, F
s1 = Config.objects.annotate(maxver=Max('server__config__config_version'))
config_list = s1.filter(config_version=F('maxver'))
If there is a better way to do this, I would love to know it.
I'm using Doctrine 2 with my ZF2 project, but i'm getting some weird problem with my server CPU and memory. And my server simply crashes.
I'm getting a lot of sleep state querys and they seem not to get cleaned.
mysql> show processlist;
+---------+--------------+-----------+------------------+----------------+------+--------------------+------------------------------------------------------------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+---------+--------------+-----------+------------------+----------------+------+--------------------+------------------------------------------------------------------------------------------------------+
| 2832346 | leechprotect | localhost | leechprotect | Sleep | 197 | | NULL |
| 2832629 | db_user | localhost | db_exemple | Sleep | 3 | | NULL |
| 2832643 | db_user | localhost | db_exemple | Sleep | 3 | | NULL |
| 2832646 | db_user | localhost | db_exemple | Sleep | 3 | | NULL |
| 2832664 | db_user | localhost | db_exemple | Sleep | 154 | | NULL |
| 2832666 | db_user | localhost | db_exemple | Sleep | 153 | | NULL |
| 2832669 | db_user | localhost | db_exemple | Sleep | 152 | | NULL |
| 2832674 | db_user | localhost | db_exemple | Sleep | 7 | | NULL |
| 2832681 | db_user | localhost | db_exemple | Sleep | 1 | | NULL |
| 2832683 | db_user | localhost | db_exemple | Sleep | 4 | | NULL |
| 2832690 | db_user | localhost | db_exemple | Sleep | 149 | | NULL |
(.......)
Also, it seems php GC is not cleaning all the objects from memory, or even kill processes.
Is there a way to disable the cache system? Would it improve the use of my resorces=
Most my querys are similar to:
$query = $this->createQueryBuilder('i');
$query->innerJoin('\Application\Relation', 'r', 'WITH', 'r.child = i.id');
$query->innerJoin('\Application\Taxonomy', 't', 'WITH', 't.id = r.taxonomy');
$query->where('t.type = :type')->setParameter('type', $relation);
$query->groupBy('i.id');
$items = $query->getQuery()->getResult(2);
Thanks in advance.
Firstly check the mysql's wait_timout variable. From the documentation:
Wait_timeout : The number of seconds the server waits for activity on
a noninteractive connection before closing it.
In normal flow (which not using persistent connections), php closes the connection automatically after script execution. To ensure there are no sleeping threads; at the end of your script simply close the connection:
$entityManager->getConnection()->close();
If these queries are running in a big while/for loop, you might want to read doctrine 2 batch processing documentation.