I am using multiple APIs and saving them to the database. I have one model called Station (it has a DateTime field and some other fields) and every API is for one station. These APIs come from devices that measure some variables and they get updated every 3 minutes.
I wrote a background task that calls a saveToDB function and stores them in the database. For example:
1. station A "some variables" 2022/10/1 13:10
2. station B "some variables" 2022/10/1 13:10
3. station A "some variables" 2022/10/1 13:13
4. station B "some variables" 2022/10/1 13:13
Now I need to take an average of every station every 10 minutes, 2 hours, week, month, and year.
There are 30 stations. How can I do this?
If your question is what the django code would look like to make these calculations, your should read up here on aggregation. Jump down to the "values()" section of the page. The code to group by station and calculate the average of all entries would be:
Station.objects.values('station_id').annotate(myAverage=Avg('some_variable'))
This will group all entries by station.
However, you can simplify by using a nested loop to isolate each station and run the average over each 10 minute interval. Not sure exactly what the conditions for which 10 minute intervals you need, but let's say you want each 10 minutes from yesterday. Something like:
from datetime import datetime, timedelta
from django.db.models import Avg
from .models import Station
def test():
# get yesterday's date at midnight
yesterday_at_midnight = datetime.today().replace(hour=0, minute=0, second=0, microsecond=0) - timedelta(days=1)
# add 10 minutes to the yesterday_at_midnight variable
ten_minutes_after_midnight = yesterday_at_midnight + timedelta(minutes=10)
# set the start time to yesterday_at_midnight
start_time = yesterday_at_midnight
# set the end time to ten_minutes_after_midnight
end_time = ten_minutes_after_midnight
# loop over each station in the database
for s in Station.objects.all():
# loop over the next 143 10 minute intervals (24 hours - 1 interval)
for i in range(143):
# find all the Station objects that fall within the current 10 minute interval
stations = Station.objects.filter(station_id=s.station_id, created_at__gte=start_time, created_at__lt=end_time).aggregate(Avg('some_variable'))
# do something with this QuerySet
print(stations)
# increment the start and end times by 10 minutes
start_time += timedelta(minutes=10)
end_time += timedelta(minutes=10)
Related
I am having a bit of tough time wrapping my head around this. I have a column based on response time in hours and our company's SLA (service level agreement) is that all incoming inquires should be answered within 2 days (the response time in hours is total hours spent on responding to inquiry).
The problem is that our company operates with winter time (7 h 45 min) and summer time (7 h). My dataset consist both and I want Power BI to differietiate winter and summer time when I try to compute SLA. In winter time 1 working day = 7 h 45 min and in summer time = 7 h. I have just used the average of summer/winter time = 7 h 30 min. The SLA column consist 3 data types, "Innen en arbeidsdag", "Innen to arbeidsdager" and "over 2 arbeidsdager".
I have used the following syntax:
SLA = SWITCH(TRUE(),Response time in hours>15, "Over to arbeidsdager", esponse time in hours>7.5, "Innen to arbeidsdager", Response time in hours<=7.5, "Innen en arbeidsdag")
How can I use DATESBETWEEN in this syntax to tell Power BI that Response Time YTD column from 15th May to 15th September is summer time, Working day = 7 h?
Just as an idea, I wouldn't use DATESBETWEEN. If you want to input dates directly to logically categorize your work seasons, try this:
SWITCH(TRUE()
,'Date'[Date] >= DATE(2022,5,15)
&& 'Date'[Date] <= DATE(2022,9,15)
,7
,BLANK()--You can carry on with logic for other seasons
)
I am using this syntax on a date table, but you can do this with any table that has a date column.
My dataset is called 'Masterdata' and my columns looks like this:
Svartid i t SLA
6,12 Innen en arbeidsdag
11,73 Innen to arbeidsdager
20,42 Over to arbeidsdager
1,07 Innen en arbeidsdag
etc etc
My syntax so far:
SLA = SWITCH(TRUE(),Masterdata[Svartid i t]>15, "Over to arbeidsdager", Masterdata[Svartid i t]>7.5, "Innen to arbeidsdager", Masterdata[Svartid i t]<=7.5, "Innen en arbeidsdag")
So how can implement
SWITCH(TRUE(),'Date'[Date] >= DATE(2022,5,15) && 'Date'[Date] <= DATE(2022,9,15),7,BLANK()
In my syntax?
I would like to sort my model's data on a week number for example when user logins user will see current week number and associated data and user should be able to select previous week number and get all past data as well also week starts on Sunday and ends on Saturday (considering 1 week)
Select Week :[2021 - 30 ] <- week number
You can filter it such way:
from datetime import datetime, timedelta
itemsList = Model_Name.objects.filter(user=request.user, created_date__gte=datetime.now()-timedelta(days=7))
I have got similar problem in my task and I solved it in simple steps:
#django #django-models #django-views #problem-solving
def get_weeks(request):
week = now().isocalendar()
if Event.objects.filter(start_date__week=week.week).exists():
instances=Event.objects.filter(start_date__week=week.week)
I have a huge list(pandas dataframe) that looks like this
user userID
Date
1/1/2018 Annual 12345
1/3/2018 Annual 12345
1/5/2018 One Time
1/11/2018 One Time
1/12/2018 One Time
1/13/2018 Annual 98765
.
.
2/1/2018 Annual 12345
2/3/2018 Annual 12345
2/5/2018 One Time
2/11/2018 One Time
2/12/2018 One Time
2/13/2018 Annual 98765
This is a list of history of user activities. Every time someone uses this service, it is recorded. There are annual membership holders and one time users.
What I want to do is counting number of new annual membership purchases per month.
Membership is valid for one year so I assume if a membership is purchased on 1/1/2017, userID 11111 is valid until 12/31/2017. In the example list above, user 12345 used the service twice but the second one shouldnt count because user 12345 purchased annual membership on 1/1/2018. Similarly, user 12345s activity on 2/1/2018 shouldnt count as a new membership purchase because it was purchased on 1/1/2017.
And it is also assumed that annual membership was purchased when they used their first service as an annual membership holder.(userID 12345 purchased his/her membership on 1/1/2018)
EDIT
example
import numpy as np
import pandas as pd
from random import randint
from random import randrange
from datetime import timedelta
from datetime import datetime
start = datetime.strptime('1/1/2017', '%m/%d/%Y')
end = datetime.strptime('12/31/2017', '%m/%d/%Y')
def random_date(start, end):
delta = end - start
int_delta = (delta.days * 24 * 60 * 60) + delta.seconds
random_second = randrange(int_delta)
return start + timedelta(seconds=random_second)
userIDs = []
dates = []
userType = []
for i in range(10000):
userIDs.append( randint(100, 999))
dates.append( random_date(start, end) )
userType.append( randint(1, 2) )
df = pd.DataFrame({'ID': userIDs, 'date':dates, 'type': userType})
df['date'] = pd.to_datetime(df['date'])
df.set_index('date', inplace = True)
You could try groupings (by year and userID, then by year and month), but working with the expiry date would require many maneuvers. I believe a more mechanical solution can handle the problem pretty straightforward.
from dateutil.relativedelta import relativedelta
count = {} # month's number of subscriptions
since = {} # member's date of subscription
for i, r in df[df.type==1].sort_values('date').iterrows():
if r.ID in since and r.date < since[r.ID] + relativedelta(years=1):
continue # valid member, not counting
since[r.ID] = r.date
ym = r.date.year, r.date.month
count[ym] = count.setdefault(ym, 0) + 1
I prefer not to consider date as index, because two members should be able to adhere at the same time.
Printing count in order gives something like:
(2017, 1) 94
(2017, 2) 7
(2018, 1) 76
(2018, 2) 20
(2018, 3) 5
(2019, 1) 50
(2019, 2) 39
(2019, 3) 10
(2019, 4) 2
My question is like this, I define a data source access to real-time data per second (has), I will for this batch of data per hour for an average calculation, and then to have this group of data per hour average accumulative sum, finally put the cumulative value of an hour, to another data source.Such as the mean value is 10 1 hour, I will give 10 to a data source, the second hour average is 20, then I will add 1 hour on average 10 output 30, so on, to the end of the 24 hours a day, the second day in from the first hour is calculated.Problem now is how can I, cache the last time I calculate the cumulative values, how to solve the similar problem?
eg: enter image description here
Just like the picture above, in the column of the avg data is I through instream_ # window.externalTimeBatch (datetime, 1 hour) can get, at the back of the column is the result of the output per hour, I think.
like this:enter image description here
Every compute a result is output
You can use Event Table to cache the data in relation database like postgres/mysql/oracle.
Fist define table from database :
#From(eventtable='rdbms', datasource.name='db_event', table.name='_tag_time_value')
define table tEvent (tag string, datetime long , value double);
and if the stream time reach , you can insert overwrite 0.0 into the event table
How to get model filter on date ranges.
In my project using employee doj, i need to deign a table.
like employee who have joined less than three months , 3-6month, 6-12 months, 12-24 months.
Depart < 3month 3-6months 6-12months 12-24months
----------------------------------------- ---- -----
A dep 4 6 6 8
------------------------------------------------------
How to do this filter in django.
I have gone through this link ,but its confusing .
http://www.nerdydork.com/django-filter-model-on-date-range.html
Thanks in Advance
The range filter works like this:
MyModel.objects.filter(date__range=(range_start, range_end))
You can use that in conjunction with datetime.timedelta to get month periods. For example:
from datetime import datetime, timedelta
now = datetime.now()
# <3 months
three_months_ago = now - timedelta(months=3)
MyModel.objects.filter(date__range=(three_months_ago, now))