is there a way to delete a view rows matching a query in QuestDB?
I can't find any statement allowing me that.
This would be the best option:
delete from mytable where columnvalue==2;
Thanks!
In QuestDb Update and Delete statement are not supported. At least now. The ways to delete data are:
Drop a partition
Write a copy of the table without the rows you want to delete, drop table and then rename the table to the one you wanted. Something like
Create table mytablecopy AS (
SELECT * FROM mytable where columnvalue != 2
) Timstamp(...) PARTITION BY ...;
DROP TABLE mytable;
RENAME table mytablecopy TO mytable;
These are costly workarounds for exceptional cases.
Updates are allowed in questdb now. In my opinion a much better option is to have an extra column in all your tables called something like isDeleted and use the the update query to maintain what is deleted and whats not. Another note here would be to add indexing on this column for efficiency.
see this for more details : https://questdb.io/docs/develop/update-data#postgres-compatibility
See the example below how to use the update query :
"use strict"
const { Client } = require("pg")
const start = async () => {
const client = new Client({
database: "qdb",
host: "127.0.0.1",
password: "quest",
port: 8812,
user: "admin",
options: "-c statement_timeout=300000"
})
await client.connect()
const createTable = await client.query(
"CREATE TABLE IF NOT EXISTS trades (ts TIMESTAMP, date DATE, name STRING, value INT) timestamp(ts);"
)
console.log(createTable)
for (let rows = 0; rows < 10; rows++) {
// Providing a 'name' field allows for prepared statements / bind variables
let now = new Date().toISOString()
const query = {
name: "insert-values",
text: "INSERT INTO trades VALUES($1, $2, $3, $4);",
values: [now, now, "node pg prep statement", rows],
}
await client.query(query)
}
const updateData = await client.query(
"UPDATE trades SET name = 'update example', value = 123 WHERE value > 7;"
)
console.log(updateData)
await client.query("COMMIT")
const readAll = await client.query("SELECT * FROM trades")
console.log(readAll.rows)
await client.end()
}
start()
.then(() => console.log("Done"))
.catch(console.error)
Related
Here is the photo explanation
I want to show the NewLinePrice in "Other Service", the value will subtract the value of "Graphic Design", so the first column will show 1152245 and so on.
So far i've tried to defined the new column "NewLinePrice", the following is the formula but not work
NewLinePrice =
Var category = 'Group-dtl'[ProductCategoryName]
var graphic_line_price = CALCULATE(SUM('Group-dtl'[LinePrice]),FILTER('Group-dtl', 'Group-dtl'[ProductCategoryName] = "Graphic Design"))
var graphic_line_price_temp = IF(category = "Graphic Design", 'Group-dtl'[LinePrice], 0)
//var graphic_line_price = 1
Var pre_result = IF(category = "Other Service", 'Group-dtl'[LinePrice] - graphic_line_price, BLANK())
Var result = IF(pre_result > 0, pre_result, BLANK())
return result
Anyone have ideas how to do that?
I spend some time to find the answer for your question, at the end I discover that to achiever your outcome, you cannot perform the calculation within the original but to create a new table, the reason is unknown, however at least it is achievable, see my answer below and accept if help, appreciate my hardworking :)
This is my original table name [Sheet1]
First I create a new table based on the original table
Table =
ADDCOLUMNS(VALUES(Sheet1[Product Name]),
"Sales",CALCULATE(SUM(Sheet1[Amount]),
FILTER(ALL(Sheet1),Sheet1[Product Name]=EARLIER(Sheet1[Product Name]))))
From the new table, I add new column using the following formula to return different value only for "Other Service"
New Line1 =
Var ServiceValue = CALCULATE(MAX(Sheet1[Amount]),Sheet1[Product Name] = "Other Service")
Var graphicValue = CALCULATE(MAX(Sheet1[Amount]),Sheet1[Product Name] = "Graphic Design")
Var charge = ServiceValue - graphicValue
return
if('Table'[Product Name] = "Other Service", charge,'Table'[Sales])
Here is new table with updated value:
I am using JS SDK with DynamoDB to fetch data.
I am able to fetch data from my table using simple query with partition key and sort key.
My sort key sk has records -
Year#Batch#Rate
If I pass var sk = "2006#CSE#90"; it returns all of records matching this,
Requirement - How can I get all products with year 2006 , Batch CSE AND Rate =>90
readItem_pbro(){
console.log("inside pbro");
var table2 = "pbro";
var pk = "1";
var sk = "2006#CSE#90";
var params2 = {
TableName: table2,
Key:{
"pk": pk,
"sk": sk
}
};
Edit 1 :: Created a different column for score/rate as score. It is numeric.
Now my query in JS is -
but I am getting error - ValidationException: The provided key element does not match the schema
readItem_score_filter(){
console.log("inside pbro");
var table2 = "pbro";
var pk = "1"; // string
var sk = "2006#CSE"; // string
var score = 90; //number
var params2 = {
TableName: table2,
Key:{
"pk": pk,
"sk": sk,
FilterExpression:'score >=:score',
}
};
what is wrong in my FilterExpression.
Edit 2 :: Added Key condition Expression but issue still remains the same
Error: ValidationException: The provided key element does not match the schema
Here is my complete function now:
readItem_score_filter(){
console.log("inside pbro");
var table2 = "pbro";
var pk = "1"; //string
var sk = "2006#CSE"; // string
var score = 90; //number
var params2 = {
TableName: table2,
Key:{
"pk": pk,
"sk": sk,
"score": score,
KeyConditionExpression: 'pk = :pk AND sk=:sk',
FilterExpression: "score >=:score",
}
};
this.user.docClient.get(params2, function(err, data) {
if (err) {
console.log(err);
} else {
console.log(data);
}
});
}
Screenshot of table attached incase you need to see::
If "2006#CSE#90" this is the value of sort key column then you cant do anything at Dynamodb level..
comparings like this can be done through regular expressions but DynamoDB doesn't support Regular Expressions.
you simply need to get results and then seperate these values and compare ..
Updated :- use different column for score.
And use Filter Expression to get records having rate more than 90.
I dont know python , but still am trying here
var params2 = {
TableName: "pbro",
KeyConditionExpression: "pk = :pk AND sk =:sk",
FilterExpression: "score >= :score"
};
I'm playing with the New Data API for Amazon Aurora Serverless
Is it possible to get the table column names in the response?
If for example I run the following query in a user table with the columns id, first_name, last_name, email, phone:
const sqlStatement = `
SELECT *
FROM user
WHERE id = :id
`;
const params = {
secretArn: <mySecretArn>,
resourceArn: <myResourceArn>,
database: <myDatabase>,
sql: sqlStatement,
parameters: [
{
name: "id",
value: {
"stringValue": 1
}
}
]
};
let res = await this.RDS.executeStatement(params)
console.log(res);
I'm getting a response like this one, So I need to guess which column corresponds with each value:
{
"numberOfRecordsUpdated": 0,
"records": [
[
{
"longValue": 1
},
{
"stringValue": "Nicolas"
},
{
"stringValue": "Perez"
},
{
"stringValue": "example#example.com"
},
{
"isNull": true
}
]
]
}
I would like to have a response like this one:
{
id: 1,
first_name: "Nicolas",
last_name: "Perez",
email: "example#example.com",
phone: null
}
update1
I have found an npm module that wrap Aurora Serverless Data API and simplify the development
We decided to take the current approach because we were trying to cut down on the response size and including column information with each record was redundant.
You can explicitly choose to include column metadata in the result. See the parameter: "includeResultMetadata".
https://docs.aws.amazon.com/rdsdataservice/latest/APIReference/API_ExecuteStatement.html#API_ExecuteStatement_RequestSyntax
Agree with the consensus here that there should be an out of the box way to do this from the data service API. Because there is not, here's a JavaScript function that will parse the response.
const parseDataServiceResponse = res => {
let columns = res.columnMetadata.map(c => c.name);
let data = res.records.map(r => {
let obj = {};
r.map((v, i) => {
obj[columns[i]] = Object.values(v)[0]
});
return obj
})
return data
}
I understand the pain but it looks like this is reasonable based on the fact that select statement can join multiple tables and duplicated column names may exist.
Similar to the answer above from #C.Slack but I used a combination of map and reduce to parse response from Aurora Postgres.
// declarative column names in array
const columns = ['a.id', 'u.id', 'u.username', 'g.id', 'g.name'];
// execute sql statement
const params = {
database: AWS_PROVIDER_STAGE,
resourceArn: AWS_DATABASE_CLUSTER,
secretArn: AWS_SECRET_STORE_ARN,
// includeResultMetadata: true,
sql: `
SELECT ${columns.join()} FROM accounts a
FULL OUTER JOIN users u ON u.id = a.user_id
FULL OUTER JOIN groups g ON g.id = a.group_id
WHERE u.username=:username;
`,
parameters: [
{
name: 'username',
value: {
stringValue: 'rick.cha',
},
},
],
};
const rds = new AWS.RDSDataService();
const response = await rds.executeStatement(params).promise();
// parse response into json array
const data = response.records.map((record) => {
return record.reduce((prev, val, index) => {
return { ...prev, [columns[index]]: Object.values(val)[0] };
}, {});
});
Hope this code snippet helps someone.
And here is the response
[
{
'a.id': '8bfc547c-3c42-4203-aa2a-d0ee35996e60',
'u.id': '01129aaf-736a-4e86-93a9-0ab3e08b3d11',
'u.username': 'rick.cha',
'g.id': 'ff6ebd78-a1cf-452c-91e0-ed5d0aaaa624',
'g.name': 'valentree',
},
{
'a.id': '983f2919-1b52-4544-9f58-c3de61925647',
'u.id': '01129aaf-736a-4e86-93a9-0ab3e08b3d11',
'u.username': 'rick.cha',
'g.id': '2f1858b4-1468-447f-ba94-330de76de5d1',
'g.name': 'ensightful',
},
]
Similar to the other answers, but if you are using Python/Boto3:
def parse_data_service_response(res):
columns = [column['name'] for column in res['columnMetadata']]
parsed_records = []
for record in res['records']:
parsed_record = {}
for i, cell in enumerate(record):
key = columns[i]
value = list(cell.values())[0]
parsed_record[key] = value
parsed_records.append(parsed_record)
return parsed_records
I've added to the great answer already provided by C. Slack to deal with AWS handling empty nullable character fields by giving the response { "isNull": true } in the JSON.
Here's my function to handle this by returning an empty string value - this is what I would expect anyway.
const parseRDSdata = (input) => {
let columns = input.columnMetadata.map(c => { return { name: c.name, typeName: c.typeName}; });
let parsedData = input.records.map(row => {
let response = {};
row.map((v, i) => {
//test the typeName in the column metadata, and also the keyName in the values - we need to cater for a return value of { "isNull": true } - pflangan
if ((columns[i].typeName == 'VARCHAR' || columns[i].typeName == 'CHAR') && Object.keys(v)[0] == 'isNull' && Object.values(v)[0] == true)
response[columns[i].name] = '';
else
response[columns[i].name] = Object.values(v)[0];
}
);
return response;
}
);
return parsedData;
}
I have power bi client filter code below:
const basicFilter: pbi.models.IBasicFilter = {
$schema: "http://powerbi.com/product/schema#basic",
target: {
table: "Store",
column: "Count"
},
operator: "In",
values: [1,2,3,4],
filterType: pbi.models.FilterType.BasicFilter
}
in my scenario a table can have multiple columns, so if I want to filter by multiple columns of the table then how can I do? In the above code only one column like Count is working, but how to configure for multiple columns?
You must define a filter for each of your conditions and pass an array with all your filters in ReportConfiguration.filters property:
var embedConfig = {
...
filters: [basicFilter1, basicFilter2, filter3]
};
or to report.setFilters method:
report.setFilters([basicFilter1, basicFilter2, filter3])
.catch(errors => {
// Handle error
});
I have a dynamodb table that backs a shopping cart. The schema is CartKey then a List of Maps that contain a CartItemId. Is there a way to update a cart item, which is nested in the list of maps, based on the CartKey and a CartItemId.
Thanks
I'm in search for a solution to the same issue. Unfortunately I don't think one is available.
In mature document-based DBs (such as MongoDB) you should be able to specify a queried index (see https://docs.mongodb.org/manual/reference/operator/update/positional/#up.S), but DynamoDB doesn't support that.
The next best thing is to query the Cart document with the entire CartItems array, iterate it to find the index of your CartItem and do a conditional write. (For example: update the document and set CartItems[7].Quantity to 4 only if CartItems[7].ProductId is "WSK-1234")
Yes you need to do a read before a write and perform some client-side searching, but at least you can be sure you aren't updating the wrong item.
I would change your data model from a list of maps, to a map of maps where the keys are CartItemId's.
Example document:
{
CartKey : 'Cart-123',
items : : {
CartItemId1 : { quantity : 1, productId: "pid-123" },
CartItemId2 : { quantity : 4, productId: "pid-987" }
}
}
Then you can perform update expressions to specific CartItems.
UpdateExpression: "set items.CartItemId1.quantity = 2"
I did something similar with a map of maps and it worked for me. Hopefully this will be helpful.
$RegID = "abracadabra";
$tableName="DefaultDelivery";
$marshaler = new Marshaler();
$requested_delivery = '{"Packet0":{"PacketNo":"2","Quantity":"1000ml","Type":"Toned Milk"},"Packet2":{"PacketNo":"4","Quantity":"250ml","Type":"Toned Milk"}}';
$eav = $marshaler->marshalJson('
{
":RequestedDelivery" : '.$requested_delivery.'
}
');
$key = $marshaler->marshalJson('
{
"RegistrationID" : "'.$RegID.'"
}
');
$params = [
'TableName' => "$tableName",
'Key' => $key,
'ExpressionAttributeValues' => $eav,
'UpdateExpression' => 'SET RequestedDelivery = :RequestedDelivery',
'ReturnValues' => 'UPDATED_NEW'
];
try {
$result = $client->updateItem($params);
echo "SUCCESS";
}
catch (DynamoDbException $e){
echo "Unable to update Item : \n";
}