Can I make QuestDB to add column data to existing rows? - questdb

I use QuestDB and populate data from ILP messages. I want to see data coming from different sensors on the same row when tags match. For example I send first
sensors,location=ny temperature=22 1465839830100400200
Table sensors is automatically created then with symbol column location and double temperature. After some time I can have another message like
sensors,location=ny humidity=35.2 1465839830100400200
If I don't do anything, humidity field is simply ignored. If I add humidity field to the table manually as
alter table sensors add column humidity double
before sending second message I still get back 2 rows, one of which has temperature and empty humidity and another one has humidity and empty temperature
| timestamp | location | temperature | humidty |
| ---------- | -------- | ----------- | ------- |
| 2016-06-13 | ny | 22.0 | |
| 2016-06-13 | ny | | 35.2 |
How can I make temperature and humidity come to the same row even when they are sent independently?

This may not be the most performant but it's easy:
SELECT timestamp, location, max(temperature), max(humidity) from sensors;

Related

Single Filter for PowerBI

I have 2 tables in powerbi, one contains all transactions to and from people (each client identified with an id, where "I" can be either the receiver or sender of $) and the other is the detail for each client.
Table 1 would look something like
| $ | sender id | receiver id |
|---|-----------| ------------|
| 10| 1 | 2 |
| 15| 1 | 3 |
| 20| 1 | 2 |
| 15| 3 | 1 |
| 10| 3 | 1 |
| 25| 2 | 1 |
| 10| 1 | 2 |
The second table contains sender id and name:
| id | name |
|----|-------|
| 1 | "me" |
| 2 | John |
| 3 | Susan |
The expected result is something like (not necesarily in a table, just to show)
| $ sent | $ received | Balance|
|--------|------------|--------|
| 55 | 45 | +10 |
And in a filter have "John" and "Susan" so when i Select one of them i could see $ sent, $received and balance for each of them.
The problem of course is that i end up with one active and one inactive relationship so if i apply such a filter i end up with 0 in sender/receiver and the whole value in the other (depending which is made active and inactive) and if i make another table that's "id sender"+"name sender" then i cant filter all at once.
Is it possible to do this?
I hope this is kinda understandable
You will need to add 2 columns to your user table
received = CALCULATE(SUM(T1[$]), Filter(T1, UserTable[id] = T1[reveicer id]))
The same you can do for send. Now in your visual, use the new columns.
Enjoy!
after going around a bit I found a way to solve this, probably not the most orthodox way to do it, but it works.
What I did is to add 2 columns to my sales table, one was labeled "movement" and in sql it is just a 'case' where when the receiver is 'me' its "Charged" and when the receiver is 'not-me' its "Payment", then i added a column with a case so it would always bring me the 'not-me' id, and i used that for may relationship with my users table.
Then I just added filters in my cards making one a "Payment" card and the other a "Charged" card.
This is all following the previous example, it was actually just a bit more tricky as I could actually have a payment from me to myself, but thats just another "case" for when it was 'me-me'
Hope this is understandable, english is not my first language and the information i actually used is partially confidential so i had to make the above example.
thanks all and have a nice day.

Add column to existing table in rds

I have table in RDS which consists two columns id and user activity at some time exactly values active/away.I get user activity every day so I need to add user activity column every day to that table.Any ideas how to do it?Now I have table with first two columns in RDS,but I am in stuck with how to add columns to that table
+-------------+------------+------------+
| id | 2020-08-13 | 2020-08-14 |
-----------------------------------------
| 12345 | active | away |
You could use an alter table ... add column, but this is not the right way to solve the problem.
In a relational database, you add additional rows for repeated data, not additional columns. So your table should look like this:
+-------------+-------------+------------+
| id | status_date | status |
------------------------------------------
| 12345 | 2020-08-13 | active |
| 12345 | 2020-08-14 | away |
Then you add a new row using an insert.

Adding a measure which finds the next row value for every row (similar to SQL Lead window function)

will be very grateful if you could share your experience and advice on the following problem in Power BI:
3 Tables given in the data model:
calendar dimension table
fact table on sessions
fact table on spending
| CW | Total cost | Sessions | Expected Column 1 | Expected Column 2 |
+----+-------------+-----------+-------------------+-------------------+
| 1 | 1200 | 50 | | |
| 2 | 1500 | 60 | 1200 | 50 |
| 3 | 1700 | 48 | 1500 | 60 |
| 4 | 1150 | 36 | 1700 | 48 |
| 5 | 900 | 29 | 1150 | 36 |
+----+-------------+-----------+-------------------+-------------------+
CW column indicates the calendar week and it is from calendar table. Sessions and Total cost are from sessions and spending tables respectively. Data is aggregated and visualized on calendar week level.
Problem: I need to create measures to derive Expected column 1 and expected column 2 based on total cost and sessions columns. Basically getting next values for each row similar to lead window function.
I have checked power BI community and there are several ideas (for example here https://community.powerbi.com/t5/Desktop/DAX-Query-to-Find-Next-Value/td-p/833896).
But these solution assume all columns are from the same table, however in the above described case
all 3 columns are from different tables.
Will the be possible to get expected columns 1 and 2 and how? Many thanks in advance!

How can I visualize timeseries data aggregated by more than one dimension on AWS insights?

I'd like to use cloudwatch insights to visualize a multiline graph of average latency by host over time. One line for each host.
This stats query extracts the latency and aggregates it in 10 minute buckets by host, but it doesn't generate any visualization.
stats avg(latencyMS) by bin(10m), host
bin(10m) | host | avg(latencyMS)
0m | 1 | 120
0m | 2 | 220
10m | 1 | 130
10m | 2 | 230
The docs call this out as a common mistake but don't offer any alternative.
The following query does not generate a visualization, because it contains more than one grouping field.
stats avg(myfield1) by bin(5m), myfield4
aws docs
Experementally, cloudwatch will generate a multi line graph if each record has multiple keys. A query that would generate a line graph must return results like this:
bin(10m) | host-1 avg(latencyMS) | host-2 avg(latencyMS)
0m | 120 | 220
10m | 130 | 230
I don't know how to write a query that would output that.
Parse individual message for each host then compute their stats.
For example, to get average latency for responses from processes with PID=11 and PID=13.
parse #message /\[PID:11\].* duration=(?<pid_11_latency>\S+)/
| parse #message /\[PID:13\].* duration=(?<pid_13_latency>\S+)/
| display #timestamp, pid_11_latency, pid_13_latency
| stats avg(pid_11_latency), avg(pid_13_latency) by bin(10m)
| sort #timestamp desc
| limit 20
The regular expressions extracts duration for processes having id 11 and 13 to parameters pid_11_latency and pid_13_latency respectively and fills null where there is no match series-wise.
You can build from this example by creating the match regular expression that extracts for metrics from message for hosts you care about.

ALL() isn't working to "remove a filter" in DAX; relationship issue?

Basic premise:
'Orders' are comprised of items from multiple 'Zones'.
Customers can call in for 'Credits' (refunds) on botched 'Orders'.
There is a true many-to-many relationship here, because one order could have multiple credits called in at different times; similarly, a customer can call in once regarding multiple orders (generating only one credit memo).
'Credits' granularity is at the item level, i.e.
CREDIT | SO | ITEM | ZONE | CREDAMT
-------------------------------------------------------
42 | 1 | 56 | A | $6
42 | 1 | 52 | A | $8
42 | 1 | 62 | B | $20
42 | 2 | 56 | A | $12
'Order Details' granularity is at the zone level, i.e.
SO | ZONE | DOL_AMT
-------------------------------
1 | A | $50
1 | B | $20
1 | C | $100
2 | A | $26
I have a 'Zone' filter table that helps me sort things better and roll up into broader categories, i.e.
ZONE | TEMP | SORT
-------------------------------
A | DRY | 2
B | COLD | 3
C | DRY | 1
What I need:
I want a pair of visuals for a side by side comparison of order total by zone next to credit total by zone.
What's working:
The 'Credits' component is easy, CreditTotal = abs(sumx(Credits,Credits[CREDAMT])) with Zone as a legend item.
I have a ORDER component that works when the zone is in the credit memo
Order $ by Zone =
CALCULATE (
SUM ( 'Order Details'[DOL_AMT] ),
USERELATIONSHIP ( 'Order Details'[SO], Credits[SO] ),
ALL ( Credits[CreditCategory] )
)
My issue:
Zones that didn't have a credit against them won't show up. So instead of
CREDIT | ZONE | ORDER $ BY ZONE
----------------------------------
42 | A | $76
42 | B | $20
42 | C | $100
I get
CREDIT | ZONE | ORDER $ BY ZONE
----------------------------------
42 | A | $76
42 | B | $20
I have tried to remove this filter by tacking on ALL(Zones[Zone]) and/or ALL('Order Details'[Zone]), but it doesn't help, presumably because it is reporting "all zones" actually found in the 'Credits' table. I'm hoping there's some way to ask it to report all zones in the 'Order Details' table based upon SOs in the 'Credits' table.
In case it helps, here's how the relationships are structured; as an aside, I've tried mixing and matching various combinations of active/inactive, single vs. bidirectional filtering, etc., but the current configuration is the only one that seems to remotely work as desired.
I'm grateful for any suggestions; please let me know if anything is unclear. Thank you.
I was able to get it to work by using 'Order Details'[Zone] rather than Zones[Zone] in the table visual and this measure:
Order $ by Zone =
CALCULATE (
SUM ( 'Order Details'[DOL_AMT] ),
USERELATIONSHIP ( 'Order Details'[SO], Credits[SO] )
)
Notice that regardless of your measure, there is no row in Credits corresponding to zone C, so it doesn't know what to put in the CREDIT column unless you tell it exactly how.
If you remove the CREDIT dimension column, then you don't need to swap tables as I suggested above. You can just use the measure above and then write a new measure for the CREDIT column instead:
CreditValue =
CALCULATE(
VALUES(Credits[CREDIT]),
ALL(Credits),
Credits[SO] IN VALUES('Order Details'[SO])
)