Querying MongoDB with a regular expression in Rust - regex

I am trying to implement the bucket pattern as solution to a previous question.
In the example they issue an update with a selector that uses a regular expression:
db.history.updateOne({ "_id": /^7000000_/, "count": { $lt: 1000 } },
{
"$push": {
"history": {
"type": "buy",
"ticker": "MDB",
"qty": 25,
"date": ISODate("2018-11-02T11:43:10")
} },
"$inc": { "count": 1 },
"$setOnInsert": { "_id": "7000000_1541184190" }
},
{ upsert: true })
I'm trying to do the same in Rust, but the query is interpreting my regex as a string literal and not evaluating the regex.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct RepetitionBucketUpdate {
#[serde(with = "serde_regex")]
id: Regex,
device_id: Uuid,
session_id: Uuid,
set_id: Uuid,
exercise: String,
level: String,
count: mongodb::bson::Bson,
}
impl From<JsonApiRepetition> for RepetitionBucketUpdate {
fn from (value: JsonApiRepetition) -> Self {
let id = format!("^{}_", value.device_id.to_string().replace("-", ""));
let re = Regex::new(&id).unwrap();
RepetitionBucketUpdate {
id: re,
device_id: value.device_id,
session_id: value.session_id,
set_id: value.set_id,
exercise: value.exercise,
level: value.level,
count: mongodb::bson::bson!( { "$lt": BUCKET_RECORD_LIMIT }),
}
}
}
let update = bson::doc! {
"$push": {
"repetitions": mongodb::bson::to_bson(&repetition_update).unwrap(),
},
"$inc": { "count": 1 },
"$setOnInsert": { "id": oid }
};
let options = mongodb::options::UpdateOptions::builder()
.upsert(true)
.build();
collection.update_one(query, update, options).await.map_err(CollectorError::DbError)?;
If I println! the update parameters I see:
query: Document({"id": String("^6fcd683c20d5415da1341e7d2f780749_"), "device_id": String("6fcd683c-20d5-415d-a134-1e7d2f780749"), "session_id": String("8388e24d-e680-46f4-9205-b9e43e39a17a"), "set_id": String("53d5a3ec-5962-402d-8e8a-41e9c5e3e01f"), "exercise": String("Bench Press"), "level": String("WheelsWithinWheels"), "count": Document(Document({"$lt": Int32(1000)}))})
update: Document({"$push": Document(Document({"repetitions": Document(Document({"number": Int32(88), "rom": Double(69.42), "duration": Double(666.0), "time": Int64(10870198172412)}))})), "$inc": Document(Document({"count": Int32(1)})), "$setOnInsert": Document(Document({"id": String("6fcd683c20d5415da1341e7d2f780749_1600107371599537000")}))})
options: UpdateOptions {
array_filters: None,
bypass_document_validation: None,
upsert: Some(
true,
),
collation: None,
hint: None,
write_concern: None,
}
It's not matching, and it's inserting every update as a new document rather than bucketing subsequent updates.
I can successfully issue a regex based query from the mongodb shell. How do I query with a regex using Rust and mongodb as in the example?

I figured this out, so for anyone who has this problem in the future:
The mongodb driver does not use the Regex crate, instead the bson crate defines a Regex struct.
My usage changed from the above (see question) to:
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct RepetitionBucketUpdate {
pub id: mongodb::bson::Bson,
pub device_id: Uuid,
session_id: Uuid,
set_id: Uuid,
exercise: String,
level: String,
count: mongodb::bson::Bson,
}
let id = format!("^{}_", value.device_id.to_string().replace("-", ""));
let re = mongodb::bson::Regex {
pattern: id,
options: String::new(),
};
And voilà, ça marche!

Related

MongoDB find document with Date field using a part of Date

i want to search a date like the following:
09-11
03-22
and it will search in the available documents and bring the matched documnet.
an available document example :
2022-09-11T15:31:25.083+00:00
how can i do this?
i tried following query but that didn't work:
db.users.find({ createdAt: new RegExp('09-11') }) // Null
You can do it with aggregate query:
$toString - to convert date to string
$regexMatch - to apply regex search
db.collection.aggregate([
{
"$match": {
"$expr": {
"$regexMatch": {
"input": {
"$toString": "$createdAt"
},
"regex": "09-11"
}
}
}
}
])
Working example
Using aggregate you can extract $dayOfMonth and $month from initial date, filter using$match and after $project the initial document by excluding calculated day and month from the document.
db.users.aggregate([
{
$addFields: {
month: {
$month: "$createdAt"
},
day: {
$dayOfMonth: "$createdAt"
},
}
},
{
$match: {
month: 9,
day: 11
}
},
{
$project: {
month: 0,
day: 0
}
}
])

#aws/dynamodb-data-mapper: How to write ConditionExpression in QueryOptions filter area?

I would like to fetch data from DynamoDB database by using [AWS-DynamoDB-Data-Mapper][1]. I am using QueryIterator and to provide indexName, limit, filter purpose, I am using [QueryOptions][2]. In QueryOptions, I would like to use filter?: ConditionExpression. Where I would like to use contains() function, but I am not able to form ConditionExpression in the below example -
{
"Id": {"N": "456" },
"ProductCategory": {"S": "Sporting Goods" },
"Price": {"N": "650" }
}
const paginator = dbMapper.query(
Products,
{
Id: 456,
Price: between(500, 1000)
},
{
indexName: 'indexByPriceId',
limit: 10,
scanIndexForward: false,
filter: contains(ProductCategory, 'goods')
}
);
It returns an syntax error in this line filter: contains(ProductCategory, 'goods'), it says ContainsPredicate is not a ConditionExpression.
Please help me out by providing me the ConditionExpression in QueryOptions.
[1]: https://github.com/awslabs/dynamodb-data-mapper-js
[2]: https://awslabs.github.io/dynamodb-data-mapper-js/packages/dynamodb-data-mapper/interfaces/queryoptions.html#projection

How do I insert an optional field as null using AppSync Resolvers and Aurora?

I have an optional String field, notes, that is sometimes empty. If it's empty I want to insert null, otherwise I want to insert the string.
Here is my resolver -
{
"version" : "2017-02-28",
"operation": "Invoke",
#set($id = $util.autoId())
#set($notes = $util.defaultIfNullOrEmpty($context.arguments.notes, 'null'))
"payload": {
"sql":"INSERT INTO things VALUES ('$id', :NOTES)",
"variableMapping": {
":NOTES" : $notes
},
"responseSQL": "SELECT * FROM things WHERE id = '$id'"
}
}
With this graphql
mutation CreateThing{
createThing() {
id
notes
}
}
I get -
{
"data": {
"createRoll": {
"id": "6af68989-0bdc-44e2-8558-aeb4c8418e93",
"notes": "null"
}
}
}
when I really want null without the quotes.
And with this graphql -
mutation CreateThing{
createThing(notes: "Here are some notes") {
id
notes
}
}
I get -
{
"data": {
"createThing": {
"id": "6af68989-0bdc-44e2-8558-aeb4c8418e93",
"notes": "Here are some notes"
}
}
}
which is what I want.
How do I get a quoteless null and a quoted string into the same field?
TL;DR you should use $util.toJson() to print the $context.arguments.notes correctly. Replace your $notes assignment with
#set($notes = $util.toJson($util.defaultIfNullOrEmpty($context.arguments.notes, null)))
Explanation:
The reason is VTL prints whatever the toString() method returns and your call to
$util.defaultIfNullOrEmpty($context.arguments.notes, 'null') will return the string "null", which will be printed as "null".
If you replace with $util.defaultIfNullOrEmpty($context.arguments.notes, null) then it will return a null string. However, VTL will print $notes because that is the way it handles null references. In order to print null, which is the valid JSON representation of null, we have to serialize it to JSON. So the correct statement is:
#set($notes = $util.toJson($util.defaultIfNullOrEmpty($context.arguments.notes, null)))
Full test:
I'm assuming you started with the RDS sample provided in the AWS AppSync console and modified it. To reproduce, I updated the content field in the Schema to be nullable:
type Mutation {
...
createPost(author: String!, content: String): Post
...
}
type Post {
id: ID!
author: String!
content: String
views: Int
comments: [Comment]
}
and I modified the posts table schema so content can also be null there: (inside the Lambda function)
function conditionallyCreatePostsTable(connection) {
const createTableSQL = `CREATE TABLE IF NOT EXISTS posts (
id VARCHAR(64) NOT NULL,
author VARCHAR(64) NOT NULL,
content VARCHAR(2048),
views INT NOT NULL,
PRIMARY KEY(id))`;
return executeSQL(connection, createTableSQL);
}
This is the request template for the createPost mutation:
{
"version" : "2017-02-28",
"operation": "Invoke",
#set($id = $util.autoId())
"payload": {
"sql":"INSERT INTO posts VALUES ('$id', :AUTHOR, :CONTENT, 1)",
"variableMapping": {
":AUTHOR" : "$context.arguments.author",
":CONTENT" : $util.toJson($util.defaultIfNullOrEmpty($context.arguments.content, null))
},
"responseSQL": "SELECT id, author, content, views FROM posts WHERE id = '$id'"
}
}
and response template:
$util.toJson($context.result[0])
The following query:
mutation CreatePost {
createPost(author: "Me") {
id
author
content
views
}
}
returns:
{
"data": {
"createPost": {
"id": "b42ee08c-956d-4b89-afda-60fe231e86d7",
"author": "Me",
"content": null,
"views": 1
}
}
}
and
mutation CreatePost {
createPost(author: "Me", content: "content") {
id
author
content
views
}
}
returns
{
"data": {
"createPost": {
"id": "c6af0cbf-cf05-4110-8bc2-833bf9fca9f5",
"author": "Me",
"content": "content",
"views": 1
}
}
}
We were looking into the same issue. For some reason, the accepted answer does not work for us. Maybe because it's a beta feature and there is a new resolver version (2018-05-29 vs 2017-02-28, changes here: Resolver Mapping Template Changelog).
We use this for the time being using NULLIF():
{
"version": "2018-05-29",
"statements": [
"INSERT INTO sales_customers_addresses (`id`, `customerid`, `type`, `company`, `country`, `email`) VALUES (NULL, :CUSTOMERID, :TYPE, NULLIF(:COMPANY, ''), NULLIF(:COUNTRY, ''), :EMAIL)"
],
"variableMap": {
":CUSTOMERID": $customerid,
":TYPE": "$type",
":COMPANY": "$util.defaultIfNullOrEmpty($context.args.address.company, '')",
":COUNTRY": "$util.defaultIfNullOrEmpty($context.args.address.country, '')",
":EMAIL": "$context.args.address.email"
}
}

How to exclude substring in Elasticsearch regexp

I'm trying to write an elasticsearch regexp that excludes elements that have a key that contains a substring, let's say in the title of books.
The elasticsearch docs suggest that a substring can be excluded with the following snippet:
#&~(foo.+) # anything except string beginning with "foo"
However, in my case, I've tried to create such a filter and failed.
{
query: {
constant_score: {
filter: {
bool: {
filter: query_filters,
},
},
},
},
size: 1_000,
}
def query_filters
[
{ regexp: { title: "#&~(red)" } },
# goal: exclude titles that start with "Red"
]
end
I've used other regexp in the same query filter that have worked, so I don't think there's a bug in the way the regexp is being passed to ES.
Any ideas? Thanks in advance!
Update:
I found a workaround: I can add a must_not clause to the filter.
{
query: {
constant_score: {
filter: {
bool: {
filter: query_filters,
must_not: must_not_filters,
},
},
},
},
size: 1_000,
}
def must_not_filters
[ { regexp: { title: "red.*" } } ]
end
Still curious if there's another idea for the original regex though

Regex in Mongodb for ISO Date field

How can I pick all the dates with time value as 00:00:00 despite the date value? Regex doesn't work for me.
{
"_id" : ObjectId("59115a92bbf6401d4455eb21"),
"name" : "sfdfsdfsf",
"create_date" : ISODate("2013-05-13T02:34:23.000Z"),
}
something like :
db.myCollection.find({"create_date": /*T00:00:00.000Z/ })
You need to first convert created date into string of time, and if time is 00:00:00:000, then include the document.
db.test.aggregate([
// Part 1: Project all fields and add timeCriteria field that contain only time(will be used to match 00:00:00:000 time)
{
$project: {
_id: 1,
name: "$name",
create_date: "$create_date",
timeCriteria: {
$dateToString: {
format: "%H:%M:%S:%L",
date: "$create_date"
}
}
}
},
// Part 2: match the time
{
$match: {
timeCriteria: {
$eq: "00:00:00:000"
}
}
},
// Part 3: re-project document, to exclude timeCriteria field.
{
$project: {
_id: 1,
name: "$name",
create_date: "$create_date"
}
}
]);
From MongoDB version >= 4.4 we can write custom filters using $function operator.
Note: Donot forget to chage the timezone to your requirement. Timezone is not mandatory.
let timeRegex = /.*T00:00:00.000Z$/i;
db.myCollection.find({
$expr: {
$function: {
body: function (createDate, timeRegex) {
return timeRegex.test(createDate);
},
args: [{ $dateToString: { date: "$create_date", timezone: "+0530" } }, timeRegex],
lang: "js"
}
}
});