how to apply conditions on JSON response in Jmeter? - regex

Response is as follows,
I have to extract all 'deviceResponseStatus.id' where status="PENDING"
how to implement it using JSON Path or REGEX in Jmeter?
{
"apiResponseStatus": "SUCCESS",
"deviceResponseList": [
{
"apiResponseStatus": "SUCCESS",
"id": 23,
"ownership": "CORPORATE",
"deviceName": "Demimbu",
"deviceType": "MOBILE",
"osType": "ANDROID",
"inTotalStorage": 0,
"inAvailableStorage": 0,
"exTotalStorage": 0,
"exAvailableStorage": 0,
"osVersion": "4.2",
"buildVersion": "1.01.08",
"status": "PENDING",
"isDeleted": false,
"policyIdNames": {
"id": 3,
"name": "Test Policy 3"
},
"deviceGroupIdNames": {
"id": 1,
"name": "Default Group"
},
"userIdName": {
"id": 1,
"name": "Randy Lewis"
},
"lastConnected": 1423810405000,
"createdByIdName": {
"id": 1,
"name": "Randy Lewis"
},
"locked": false,
"traceOn": false,
"userEmail": "mdm.user1#gmail.com",
"enrollmentDate": 1423790255000,
"simNumbers": [
"1236547896"
],
"imeiCodes": [],
"mobileNumbers": [
"4558585858858"
],
"createDate": 1421927250000,
"lastModified": 1423810406000,
"rooted": false,
"roaming": false,
"compliant": false
},
{
"apiResponseStatus": "SUCCESS",
"id": 24,
"ownership": "CORPORATE",
"deviceName": "Photobean",
"deviceType": "MOBILE",
"osType": "ANDROID",
"inTotalStorage": 20941258752,
"inAvailableStorage": 8024924160,
"exTotalStorage": 0,
"exAvailableStorage": 0,
"bluetoothMacAddress": "8C:3A:E3:BE:C2:04",
"wirelessMacAddress": "8c:3a:e3:51:bb:e3",
"model": "LG-D686",
"osVersion": "4.4.2",
"buildVersion": "1.01.09",
"status": "PENDING",
"isDeleted": false,
"policyIdNames": {
"id": 3,
"name": "Test Policy 3"
},
"deviceGroupIdNames": {
"id": 1,
"name": "Default Group"
},
"userIdName": {
"id": 1,
"name": "Randy Lewis"
},
"make": "LGE",
"lastConnected": 1423648831000,
"createdByIdName": {
"id": 1,
"name": "Randy Lewis"
},
"locked": false,
"traceOn": false,
"userEmail": "mdm.user1#gmail.com",
"enrollmentDate": 1423628675000,
"simNumbers": [
"404909008623970"
],
"imeiCodes": [
"359004053152889",
"359004053152871"
],
"mobileNumbers": [
"638847339387"
],
"createDate": 1422435029000,
"lastModified": 1423648831000,
"rooted": true,
"roaming": false,
"compliant": false
},
{
"apiResponseStatus": "SUCCESS",
"id": 25,
"ownership": "CORPORATE",
"deviceName": "Livetube",
"deviceType": "MOBILE",
"osType": "ANDROID",
"inTotalStorage": 0,
"inAvailableStorage": 0,
"exTotalStorage": 0,
"exAvailableStorage": 0,
"osVersion": "4.2",
"buildVersion": "1.01.08",
"status": "APPROVAL_PENDING",
"isDeleted": false,
"policyIdNames": {
"id": 3,
"name": "Test Policy 3"
},
"deviceGroupIdNames": {
"id": 1,
"name": "Default Group"
},
"userIdName": {
"id": 1,
"name": "Randy Lewis"
},
"lastConnected": 1423810522000,
"createdByIdName": {
"id": 1,
"name": "Randy Lewis"
},
"locked": false,
"traceOn": false,
"userEmail": "mdm.user1#gmail.com",
"enrollmentDate": 1423790371000,
"simNumbers": [
"1236547896"
],
"imeiCodes": [],
"mobileNumbers": [
"4558585858858"
],
"createDate": 1422435030000,
"lastModified": 1423810522000,
"rooted": false,
"roaming": false,
"compliant": false
},
{
"apiResponseStatus": "SUCCESS",
"id": 26,
"ownership": "CORPORATE",
"deviceName": "Riffwire",
"deviceType": "MOBILE",
"osType": "IOS",
"inTotalStorage": 0,
"inAvailableStorage": 0,
"exTotalStorage": 0,
"exAvailableStorage": 0,
"status": "PENDING",
"isDeleted": false,
"policyIdNames": {
"id": 3,
"name": "Test Policy 3"
},
"deviceGroupIdNames": {
"id": 1,
"name": "Default Group"
},
"userIdName": {
"id": 1,
"name": "Randy Lewis"
},
"createdByIdName": {
"id": 1,
"name": "Randy Lewis"
},
"locked": false,
"traceOn": false,
"userEmail": "mdm.user1#gmail.com",
"simNumbers": [],
"imeiCodes": [],
"mobileNumbers": [],
"createDate": 1422435030000,
"lastModified": 1423216312000,
"rooted": false,
"roaming": false,
"compliant": false
},
Also, Please suggest any good option if any.

You question is JMeter specific, but i believe it's actually regular Json Path expression. For example, in SoapUI it could be done with
$.deviceResponseList[?(#.status=='PENDING')].id
Probably this could help in some way

JMeter provides JSON Path Extractor via JMeter Plugins Extras With Libs Set
I believe that this is the best way to accomplish your goal.
See Using the XPath Extractor in JMeter guide (scroll down to "Parsing JSON" chapter) for the plugin installation instructions and some sample JSON Path queries.

Related

Can I partially color a bar based on percentage complete on a temporal axis?

I am trying to create a Gantt chart and I want to color a single task with two colors, based on a percentage complete. Say, make the complete part green and the remaining part orange.
How can I achieve this?
Below is a sample code, also available in the editor here.
{
"data": {
"values": [
{"Description": "Task 1", "Start": "2023-01-05", "End": "2023-01-10", "Percentage complete": 0},
{"Description": "Task 2", "Start": "2023-01-01", "End": "2023-01-15", "Percentage complete": 75},
{"Description": "Task 3", "Start": "2023-01-01", "End": "2023-01-03", "Percentage complete": 100}
]
},
"layer": [
{
"mark": "bar",
"encoding": {
"y": {
"field": "Description",
"type": "ordinal",
"stack": null
},
"x": {
"field": "Start",
"type": "temporal"
},
"x2": {
"field": "End",
"type": "temporal"
}
}
}
]
}
The expected result should look like this.
I tried looking at folding, transforming, and scale. But as I am new to Vega-lite, to no avail.
You have two options.
Reshape your data upstream. Your partially coloured bars should be rendered as two bars stacked - one for incomplete and one for complete.
Use Reactive Geometry as described here. This may need Vega rather than VL.
Here it is using reactive geometry.
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"background": "white",
"padding": 5,
"width": 200,
"style": "cell",
"data": [
{
"name": "source_0",
"values": [
{
"Description": "Task 1",
"Start": "2023-01-05",
"End": "2023-01-10",
"Percentatecomplete": 0
},
{
"Description": "Task 2",
"Start": "2023-01-01",
"End": "2023-01-15",
"Percentatecomplete": 0.75
},
{
"Description": "Task 3",
"Start": "2023-01-01",
"End": "2023-01-03",
"Percentatecomplete": 1
}
]
},
{
"name": "data_0",
"source": "source_0",
"transform": [
{"type": "formula", "expr": "toDate(datum[\"Start\"])", "as": "Start"},
{"type": "formula", "expr": "toDate(datum[\"End\"])", "as": "End"},
{
"type": "filter",
"expr": "(isDate(datum[\"Start\"]) || (isValid(datum[\"Start\"]) && isFinite(+datum[\"Start\"])))"
}
]
}
],
"signals": [
{"name": "y_step", "value": 20},
{
"name": "height",
"update": "bandspace(domain('y').length, 0.1, 0.05) * y_step"
}
],
"marks": [
{
"name": "layer_0_marks",
"type": "rect",
"style": ["bar"],
"from": {"data": "data_0"},
"encode": {
"update": {
"fill": {"value": "#4c78a8"},
"x": {"scale": "x", "field": "Start"},
"x2": {"scale": "x", "field": "End"},
"y": {"scale": "y", "field": "Description"},
"height": {"signal": "max(0.25, bandwidth('y'))"}
}
}
},
{
"type": "rect",
"from": {"data": "layer_0_marks"},
"encode": {
"update": {
"x": {"field": "x"},
"y": {"field": "y"},
"fill": {"value": "red"},
"width": {"signal": "(datum.x2 - datum.x) * datum.datum.Percentatecomplete"},
"height": {"signal": "max(0.25, bandwidth('y'))"}
}
}
}
],
"scales": [
{
"name": "x",
"type": "time",
"domain": {"data": "data_0", "fields": ["Start", "End"]},
"range": [0, {"signal": "width"}]
},
{
"name": "y",
"type": "band",
"domain": {"data": "data_0", "field": "Description", "sort": true},
"range": {"step": {"signal": "y_step"}},
"paddingInner": 0.1,
"paddingOuter": 0.05
}
],
"axes": [
{
"scale": "x",
"orient": "bottom",
"gridScale": "y",
"grid": true,
"tickCount": {"signal": "ceil(width/40)"},
"domain": false,
"labels": false,
"aria": false,
"maxExtent": 0,
"minExtent": 0,
"ticks": false,
"zindex": 0
},
{
"scale": "x",
"orient": "bottom",
"grid": false,
"title": "Start, End",
"labelFlush": true,
"labelOverlap": true,
"tickCount": {"signal": "ceil(width/40)"},
"zindex": 0
},
{
"scale": "y",
"orient": "left",
"grid": false,
"title": "Description",
"zindex": 0
}
]
}

How do you change the font formatting of a single axis label value

I have a text field ['Region'] with four values {Global, AMER, APAC, EMEA}. I have a simple bar chart and want the label to be bold only for 'Global' but have been unable to do so.
I tried:
{
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"data": { "values": [
{"Region": "Global", "Months": 1, "RegionRank":1},
{"Region": "AMER", "Months": -1, "RegionRank":2},
{"Region": "APAC", "Months": 3, "RegionRank":3}, {"Region": "EMEA", "Months": 2, "RegionRank":4}
]},
"layer": [
{
"mark": {
"type": "bar",
"filled": false,
"stroke": "gray",
"strokeDash": [3.6],
"strokeWidth": 3
}
},
{
"mark": {
"type": "text",
"fontSize": 14,
"xOffset": {
"expr": "datum['Months']<0 ? -15:15"
}
},
"encoding": {
"text": {
"field": "Months",
"format": "+.0f"
}
}
}
],
"encoding": {
"y": {
"field": "Region",
"type": "nominal",
"axis": {
"offset": 10,
"title": null,
// problem section
"labelFontWeight": {
"condition": {"test": "datum['Region'] == 'Global'","value":"bold"},
"value":"normal"
}
},
"sort": {
"op": "min",
"field": "RegionRank",
"order": "ascending"
}
},
"x": {
"field": "Months",
"type": "quantitative",
"axis": {"title": null}
}
}
}
but there is no change to the label font weight.
What is interesting is that I switched the result conditions, making the '==Global' result "normal" and the else "bold" and it changed font weight on all the labels to bold. That leads me to believe that my condition is never being evaluated as true. Any ideas why? Is what I want to do - changing a signal value in an axis label - possible?
You mean like this?
{
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"data": {
"values": [
{"Region": "Global", "Months": 1, "RegionRank": 1},
{"Region": "AMER", "Months": -1, "RegionRank": 2},
{"Region": "APAC", "Months": 3, "RegionRank": 3},
{"Region": "EMEA", "Months": 2, "RegionRank": 4}
]
},
"layer": [
{
"mark": {
"type": "bar",
"filled": false,
"stroke": "gray",
"strokeDash": [3.6],
"strokeWidth": 3
}
},
{
"mark": {
"type": "text",
"fontSize": 14,
"xOffset": {"expr": "datum['Months']<0 ? -15:15"}
},
"encoding": {"text": {"field": "Months", "format": "+.0f"}}
}
],
"encoding": {
"y": {
"field": "Region",
"type": "nominal",
"axis": {
"offset": 10,
"title": null,
"labelFontWeight": {
"condition": {"test": "datum.label == 'Global'", "value": "bold"},
"value": "normal"
}
},
"sort": {"op": "min", "field": "RegionRank", "order": "ascending"}
},
"x": {"field": "Months", "type": "quantitative", "axis": {"title": null}}
}
}

aws data migration service not picking any source table

I created aws dms (data migration service) task but task is not picking any source schema table.
Both source and target end points are successful. I first tried Sybase to Oracle migration then to trouble-shoot Oracle to Oracle migration task.
In both cases it didn't pick any table.
I have tried by giving specific table name and also % in but it is not picking any table in both cases.
Both source and target end points use admin account and also source table has select to public grant.
Kindly suggest. I have attached jsons and images.
Thanks in advance.
Attached below is json for table mapping and complete task:
{
"rules": [
{
"rule-type": "selection",
"rule-id": "1",
"rule-name": "1",
"object-locator": {
"schema-name": "SRC_ORCL",
"table-name": "%"
},
"rule-action": "include"
}
]
}
Task Json
{
"TargetMetadata": {
"TargetSchema": "dbadmin",
"SupportLobs": true,
"FullLobMode": false,
"LobChunkSize": 0,
"LimitedSizeLobMode": true,
"LobMaxSize": 32,
"InlineLobMaxSize": 0,
"LoadMaxFileSize": 0,
"ParallelLoadThreads": 0,
"ParallelLoadBufferSize": 0,
"BatchApplyEnabled": false,
"TaskRecoveryTableEnabled": false,
"ParallelLoadQueuesPerThread": 0,
"ParallelApplyThreads": 0,
"ParallelApplyBufferSize": 0,
"ParallelApplyQueuesPerThread": 0
},
"FullLoadSettings": {
"TargetTablePrepMode": "DROP_AND_CREATE",
"CreatePkAfterFullLoad": false,
"StopTaskCachedChangesApplied": false,
"StopTaskCachedChangesNotApplied": false,
"MaxFullLoadSubTasks": 8,
"TransactionConsistencyTimeout": 600,
"CommitRate": 10000
},
"Logging": {
"EnableLogging": false,
"LogComponents": [
{
"Id": "TRANSFORMATION",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "SOURCE_UNLOAD",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "IO",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "TARGET_LOAD",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "PERFORMANCE",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "SOURCE_CAPTURE",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "SORTER",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "REST_SERVER",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "VALIDATOR_EXT",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "TARGET_APPLY",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "TASK_MANAGER",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "TABLES_MANAGER",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "METADATA_MANAGER",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "FILE_FACTORY",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "COMMON",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "ADDONS",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "DATA_STRUCTURE",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "COMMUNICATION",
"Severity": "LOGGER_SEVERITY_DEFAULT"
},
{
"Id": "FILE_TRANSFER",
"Severity": "LOGGER_SEVERITY_DEFAULT"
}
],
"CloudWatchLogGroup": null,
"CloudWatchLogStream": null
},
"ControlTablesSettings": {
"historyTimeslotInMinutes": 5,
"ControlSchema": "TGT_ORCL",
"HistoryTimeslotInMinutes": 5,
"HistoryTableEnabled": true,
"SuspendedTablesTableEnabled": true,
"StatusTableEnabled": true
},
"StreamBufferSettings": {
"StreamBufferCount": 3,
"StreamBufferSizeInMB": 8,
"CtrlStreamBufferSizeInMB": 5
},
"ChangeProcessingDdlHandlingPolicy": {
"HandleSourceTableDropped": true,
"HandleSourceTableTruncated": true,
"HandleSourceTableAltered": true
},
"ErrorBehavior": {
"DataErrorPolicy": "LOG_ERROR",
"DataTruncationErrorPolicy": "LOG_ERROR",
"DataErrorEscalationPolicy": "SUSPEND_TABLE",
"DataErrorEscalationCount": 0,
"TableErrorPolicy": "SUSPEND_TABLE",
"TableErrorEscalationPolicy": "STOP_TASK",
"TableErrorEscalationCount": 0,
"RecoverableErrorCount": -1,
"RecoverableErrorInterval": 5,
"RecoverableErrorThrottling": true,
"RecoverableErrorThrottlingMax": 1800,
"ApplyErrorDeletePolicy": "IGNORE_RECORD",
"ApplyErrorInsertPolicy": "LOG_ERROR",
"ApplyErrorUpdatePolicy": "LOG_ERROR",
"ApplyErrorEscalationPolicy": "LOG_ERROR",
"ApplyErrorEscalationCount": 0,
"ApplyErrorFailOnTruncationDdl": false,
"FullLoadIgnoreConflicts": true,
"FailOnTransactionConsistencyBreached": false,
"FailOnNoTablesCaptured": false
},
"ChangeProcessingTuning": {
"BatchApplyPreserveTransaction": true,
"BatchApplyTimeoutMin": 1,
"BatchApplyTimeoutMax": 30,
"BatchApplyMemoryLimit": 500,
"BatchSplitSize": 0,
"MinTransactionSize": 1000,
"CommitTimeout": 1,
"MemoryLimitTotal": 1024,
"MemoryKeepTime": 60,
"StatementCacheSize": 50
},
"PostProcessingRules": null,
"CharacterSetSettings": null,
"LoopbackPreventionSettings": null,
"BeforeImageSettings": null
}
It worked, After I made below changes
i) Sybase source end point to use database as pubs2.
ii) In task specified % in both Schema and table name.

Combining "bool" and "terms" query in ElasticSearch v6.2 (on AWS)

I am trying to debug an ElasticSearch query and I could use some help with figuring out what I am doing wrong here.
The problem I am having is when I add a "terms" query on a field, no hits are found when I expect results back.
This is the query WITHOUT the "terms" part:
GET /activity/doc/_search
{
"query":{
"bool":{
"must": [
{
"range": {
"ageMax": {
"gte": 20
}
}
},
{
"range": {
"ageMin": {
"lte": 28
}
}
},
{
"range":{
"activityDate":{
"gte":"2019-06-12T16:23:12.709Z"
}
}
},
{
"geo_distance":{
"distance":"50.0km",
"location.gps":{
"lon":-122.406417,
"lat":37.785834
}
}
}
]
}
}
}
This is the result I get back:
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 4,
"hits": [
{
"_index": "activity",
"_type": "doc",
"_id": "f35980fe-94cb-4c4a-9ee7-84dbace823b0",
"_score": 4,
"_source": {
"ageMax": 68,
"__typename": "Activity",
"photo": {
"bucket": "vevivo8106a3b4577d41ec943f5ff2d7536d38-develop",
"region": "eu-west-1",
"Key": "Facebook_137538237374224/FA5A7B52-48E6-4816-85FD-06AD04721FBF.jpg",
"url": "https://vevivo8106a3b4577d41ec943f5ff2d7536d38-develop.s3.eu-west-1.amazonaws.com/public/Facebook_137538237374224/FA5A7B52-48E6-4816-85FD-06AD04721FBF.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAQUVMMI25HVHW4O5W%2F20190611%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20190611T202205Z&X-Amz-Expires=900&X-Amz-Security-Token=AgoGb3JpZ2luENj%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCWV1LXdlc3QtMSKAAoIYw2UKtl9sRiRSxJ6OpnAhQ1GT4WfSzCWQybWB1ufO%2BGUTP4GFNuB7nmq5Y3wOvhmGdMbCY54Do5GGA1GpW3c0OzUtzu%2FKm5AreF8gLSwcqYBoVCiiPlEHhsJ%2FUINKCdwcFp%2BNWs5czT%2Fj%2BrPa8yqBkQxVbxAc%2BoMxadBhvARlPoYcqdR25vnbaoDewiS%2BFE7UjbvF0HvLcu8G2S6Dgy9r1w0tnZIzj512WsOwj0AsM5MFr7ut1xUdOuyJq8sC4BV8xa8FR7VKFEdYpYJyfId%2B0sTQZcv%2FbOHKDCvdRTvDtzez3GXj6nrEon5mRG81cJdYlRMRWaoEBvbG1Mn0pesqrwUIrf%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARACGgwwNDQzODQ5Mjk0NjYiDG%2FxkEMhjuwECcH1%2FyqDBUf9AxuOCXR%2B5MtZfA%2BceNx7gzsai7LmfFpPQcYqd0xv5M0VzkUiPCD5wUP417qDzX5KC9Zft%2BX9C2RKYJoxd0%2Bav%2BsQoN62Mld0wuDRykISJTd5Qknq8FFvWKznD%2BTh2jr%2BWD2LYwhw3gt07p9LZwi9BKV67ktzo5rK77XTLfCFKDZBDiyEUWyrBea1%2BZt45p%2F5O6S7D7yxW0GFYXXrwLfbg2FGJikXDVwmcHnw5s8oh6b9UFNJUZzqllXYA7Tg8S0nkixZuu1O4OBU6mht7YhPng2%2FZLy0k%2FyoaLggHc3lbPpQJYSj1XIXqbZlbS5wSKZyivkUuBJAMFNLcRgj%2BpA0TYCGTwF2zZgxwvWm3H%2B7NDvH%2FrnQajW7ANna6HhS1WZ4fEtnFWNWxArjCJoQk5hQPnkyWyeQda9aAMBDr72hhRTw8PTZkim6nSKxwM4UnQ4jElopt0UbXSKQxusZJFo%2F0UZAgdWg00TdjIgVs3q%2BZ9CMS5jt%2BTbatrCDWDIXKIEuMrqqKanrPDfq60hV0I2BP8poTGT3RczVE7tbfeMLui3E5jGvP9xkoCZfWt9GXZvQWTXinilVonPkHRVGa7sqhygcYh9TmhO18eKiBk7mwO1cwIKlRfTZJBr%2F1xu3IP7oKBG8AOwjSJJx0fDpRLUMCWH%2BC%2BHNiAqjRpShCvX3OFOFRygMa50C0ocdlGrEtKDZBXa2%2BSW4WR0LEeozMeWjJTPF9iA%2FE8GyCscdnx2bMAhccJupAplL7UhCMTdRCJ2SalnNJR7Q49LvG4ryG8OdxhhYZD4n51wVVWlB7fqsP%2Bki8feH77jokiofq3eOU9jFk3SaxbfkzB8%2FcVhtzCrS%2BSswvZaA6AU%3D&X-Amz-Signature=399ae23fe3f01338d1bf79b918c9ce51ffa91c8f705d62ea3b516b0341b60578&X-Amz-SignedHeaders=host"
},
"dateModified": "2019-06-11T20:21:52.870Z",
"version": 1,
"usersWatching": 0,
"createdAt": "2019-06-11T20:22:43.215Z",
"likesCount": 0,
"textData": "Watch a dance movie?",
"enrolledUsers": 0,
"activityDate": "2019-06-19T20:12:44.000Z",
"ageMin": 18,
"dateCreated": "2019-06-11T20:21:52.870Z",
"peopleRequested": 2,
"commentsCount": 0,
"location": {
"address": {
"zipcode": "94108",
"country": "United States",
"city": "San Francisco",
"street": "Stockton St",
"state": null
},
"gps": {
"lon": -122.406417,
"lat": 37.785834
}
},
"enrollmentRequests": 0,
"id": "f35980fe-94cb-4c4a-9ee7-84dbace823b0",
"activityCreatorId": "Facebook_137538237374224",
"category": "Movies::Ballet",
"updatedAt": "2019-06-11T20:22:43.215Z"
}
},
{
"_index": "activity",
"_type": "doc",
"_id": "ce9ab1ee-8fa8-42dd-aeb8-c1a9f58ab6b3",
"_score": 4,
"_source": {
"ageMax": 68,
"__typename": "Activity",
"photo": {
"bucket": "vevivo8106a3b4577d41ec943f5ff2d7536d38-develop",
"region": "eu-west-1",
"Key": "Facebook_137538237374224/FF8E51D9-279B-4EC2-9461-55E2CBFC637A.jpg",
"url": "https://vevivo8106a3b4577d41ec943f5ff2d7536d38-develop.s3.eu-west-1.amazonaws.com/public/Facebook_137538237374224/FF8E51D9-279B-4EC2-9461-55E2CBFC637A.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAQUVMMI25F4QOD3PE%2F20190611%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20190611T203152Z&X-Amz-Expires=900&X-Amz-Security-Token=AgoGb3JpZ2luENj%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCWV1LXdlc3QtMSKAAhy6dUpwqL9847F4NppRj%2FsFwXtJgJQk%2BK6fsSlAzfUwJGEXEDhGrOEs45HiOpHAEN5YoEskPEb6YpfUdLpXanO5TaYp%2F4Q4uiMMTcR9PVPg%2BVFUylTYJQ%2BskQDcqJw%2BPqppZiWvMMEzJNkB335B6gyqLgsJWyl0okgKLknVVKTJAntsGiqfX%2FvVFk94aoMP0Ubv3ymXyxZ9dxqA5Mqe6EbNoxteQMdLQqoZPfXiGQmvDjgfpZph2SCkkOSwp1slGF0vCjOIztj%2B4Rsfq9jfI14Ks6th25SHOZjeB0HEx497KgFyYQFp41ke8u4WsJ91alv8fGpyMhId2b8v%2F%2BwkNsgqrwUIrv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARACGgwwNDQzODQ5Mjk0NjYiDOiqDscm7ZoQvO8lWCqDBZB56K6XqJ565o1bnZc%2Bh5x9JHeAU8kOcaDvSfa8syzOCQTSW%2BORfmPJzKDij5DN%2B4QFSvycY%2BuU%2FTQqfB6F3bbfD8rLr1HsVBFaKzwIOA6RSEDolCJb3Q2ZyazfFPuzOof3BlEFKYrUoqO1x5Ur0K3z8DRsst7TyAMBPxiYnN%2BZMzsxjFXRz7ps24suwvO3Urz0zogJWGP%2FxSvX%2F6386bTo8X9oIPAVhwdjEebzYATCfe02IRPUxXGQHyZ3qVF5Ccy7zioLW2iNVUQawCN26lfcoUq79xt%2F9mygFM36Xyr3cC0EH5dV5hIj6f%2F9GDvjb%2FLex0c3XPBtzgDDdJoLPgU4UFdIyhYlHNrUZN40vpLZqJIBKQkjyUcoSkzW0dhCx0DukjimLScwwUPxEtSb%2BKqk94zbsZUJjnSBzcGf2DjbCYbDs5EZqg0pm8iVCFtAJODIDLktwZUhl%2BCrv9JYv8epzQgIBDBUIPjeudOMLodahv%2BfFwHl9lRLNUZkDPGgmhIby62J5y2pkoJE353mKPs%2BDvqkmqTMy2377DH1IOsrPRmkgWldPpv4uFD5jKN3r9xxQv4LUswzD%2FfPSBG2pgvaUgeE2YcWg3o6CcW4KtUOPLucHLw66lkgcnQ8M9feo3j35z%2B9lSblNTd6rCuVgrxa9zuRzNxpRl37ZSut%2B7VPy%2FcKN05mHsZfvIBVcPXF4SdQwgnH2%2BZz4fNEU75MRLyqud7rKQ47h2pamKL%2F%2BUqPn8tNlAAZNMpQHh1KAyIrE35WBbIMgAZSVRUeXcvHVONTJrMbeVC%2FZ0EYAfnCR8uxBsGWPIcsObXN%2FJAGbLGcMW5QB%2Bu1MTSI0qbLoxtzxK9r9lQwzZ6A6AU%3D&X-Amz-Signature=5bdfec0ae588f234af3f8e6dd75c3c7ed8f85fdb3c333c895d02c6f63bf0a548&X-Amz-SignedHeaders=host"
},
"dateModified": "2019-06-11T20:31:31.646Z",
"version": 1,
"usersWatching": 0,
"createdAt": "2019-06-11T20:32:51.687Z",
"likesCount": 0,
"textData": "Anyone for a dance movie?",
"enrolledUsers": 0,
"activityDate": "2019-06-21T20:31:23.000Z",
"ageMin": 18,
"dateCreated": "2019-06-11T20:31:31.646Z",
"peopleRequested": 2,
"commentsCount": 0,
"location": {
"address": {
"zipcode": "94108",
"country": "United States",
"city": "San Francisco",
"street": "Stockton St",
"state": null
},
"gps": {
"lon": -122.406417,
"lat": 37.785834
}
},
"enrollmentRequests": 0,
"id": "ce9ab1ee-8fa8-42dd-aeb8-c1a9f58ab6b3",
"activityCreatorId": "Facebook_137538237374224",
"category": "Movies::Romance",
"updatedAt": "2019-06-11T20:32:51.687Z"
}
},
{
"_index": "activity",
"_type": "doc",
"_id": "309db646-903c-471e-b045-b1f55ae6cff0",
"_score": 4,
"_source": {
"ageMax": 68,
"__typename": "Activity",
"photo": {
"bucket": "vevivo8106a3b4577d41ec943f5ff2d7536d38-develop",
"region": "eu-west-1",
"Key": "Facebook_137538237374224/18BB874D-F59C-4924-8764-75A25020C61C.jpg",
"url": "https://vevivo8106a3b4577d41ec943f5ff2d7536d38-develop.s3.eu-west-1.amazonaws.com/public/Facebook_137538237374224/18BB874D-F59C-4924-8764-75A25020C61C.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAQUVMMI25F4QOD3PE%2F20190611%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20190611T203026Z&X-Amz-Expires=900&X-Amz-Security-Token=AgoGb3JpZ2luENj%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCWV1LXdlc3QtMSKAAhy6dUpwqL9847F4NppRj%2FsFwXtJgJQk%2BK6fsSlAzfUwJGEXEDhGrOEs45HiOpHAEN5YoEskPEb6YpfUdLpXanO5TaYp%2F4Q4uiMMTcR9PVPg%2BVFUylTYJQ%2BskQDcqJw%2BPqppZiWvMMEzJNkB335B6gyqLgsJWyl0okgKLknVVKTJAntsGiqfX%2FvVFk94aoMP0Ubv3ymXyxZ9dxqA5Mqe6EbNoxteQMdLQqoZPfXiGQmvDjgfpZph2SCkkOSwp1slGF0vCjOIztj%2B4Rsfq9jfI14Ks6th25SHOZjeB0HEx497KgFyYQFp41ke8u4WsJ91alv8fGpyMhId2b8v%2F%2BwkNsgqrwUIrv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARACGgwwNDQzODQ5Mjk0NjYiDOiqDscm7ZoQvO8lWCqDBZB56K6XqJ565o1bnZc%2Bh5x9JHeAU8kOcaDvSfa8syzOCQTSW%2BORfmPJzKDij5DN%2B4QFSvycY%2BuU%2FTQqfB6F3bbfD8rLr1HsVBFaKzwIOA6RSEDolCJb3Q2ZyazfFPuzOof3BlEFKYrUoqO1x5Ur0K3z8DRsst7TyAMBPxiYnN%2BZMzsxjFXRz7ps24suwvO3Urz0zogJWGP%2FxSvX%2F6386bTo8X9oIPAVhwdjEebzYATCfe02IRPUxXGQHyZ3qVF5Ccy7zioLW2iNVUQawCN26lfcoUq79xt%2F9mygFM36Xyr3cC0EH5dV5hIj6f%2F9GDvjb%2FLex0c3XPBtzgDDdJoLPgU4UFdIyhYlHNrUZN40vpLZqJIBKQkjyUcoSkzW0dhCx0DukjimLScwwUPxEtSb%2BKqk94zbsZUJjnSBzcGf2DjbCYbDs5EZqg0pm8iVCFtAJODIDLktwZUhl%2BCrv9JYv8epzQgIBDBUIPjeudOMLodahv%2BfFwHl9lRLNUZkDPGgmhIby62J5y2pkoJE353mKPs%2BDvqkmqTMy2377DH1IOsrPRmkgWldPpv4uFD5jKN3r9xxQv4LUswzD%2FfPSBG2pgvaUgeE2YcWg3o6CcW4KtUOPLucHLw66lkgcnQ8M9feo3j35z%2B9lSblNTd6rCuVgrxa9zuRzNxpRl37ZSut%2B7VPy%2FcKN05mHsZfvIBVcPXF4SdQwgnH2%2BZz4fNEU75MRLyqud7rKQ47h2pamKL%2F%2BUqPn8tNlAAZNMpQHh1KAyIrE35WBbIMgAZSVRUeXcvHVONTJrMbeVC%2FZ0EYAfnCR8uxBsGWPIcsObXN%2FJAGbLGcMW5QB%2Bu1MTSI0qbLoxtzxK9r9lQwzZ6A6AU%3D&X-Amz-Signature=5c5f8e0237e7dc3725617a82db97dfd37c87fe3872a85eabb883100f41aa26e1&X-Amz-SignedHeaders=host"
},
"dateModified": "2019-06-11T20:30:14.449Z",
"version": 1,
"usersWatching": 0,
"createdAt": "2019-06-11T20:31:25.613Z",
"likesCount": 0,
"textData": "Romance Movie Anyone?",
"enrolledUsers": 0,
"activityDate": "2019-06-14T20:31:17.000Z",
"ageMin": 18,
"dateCreated": "2019-06-11T20:30:14.449Z",
"peopleRequested": 1,
"commentsCount": 0,
"location": {
"address": {
"zipcode": "94108",
"country": "United States",
"city": "San Francisco",
"street": "Stockton St",
"state": null
},
"gps": {
"lon": -122.406417,
"lat": 37.785834
}
},
"enrollmentRequests": 0,
"id": "309db646-903c-471e-b045-b1f55ae6cff0",
"activityCreatorId": "Facebook_137538237374224",
"category": "Movies::Romance",
"updatedAt": "2019-06-11T20:31:25.613Z"
}
},
{
"_index": "activity",
"_type": "doc",
"_id": "3a7c629e-803d-4fb9-8b7c-5fac08255649",
"_score": 4,
"_source": {
"ageMax": 62,
"__typename": "Activity",
"photo": {
"bucket": "vevivo8106a3b4577d41ec943f5ff2d7536d38-develop",
"region": "eu-west-1",
"Key": "d70053b2-cbf2-47ea-959e-6b081e00dac9/ECEFB5DB-0F32-4142-A26C-10B7ED120452.jpg",
"url": "https://vevivo8106a3b4577d41ec943f5ff2d7536d38-develop.s3.eu-west-1.amazonaws.com/public/d70053b2-cbf2-47ea-959e-6b081e00dac9/ECEFB5DB-0F32-4142-A26C-10B7ED120452.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAQUVMMI25EVY5X32Z%2F20190612%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20190612T125107Z&X-Amz-Expires=900&X-Amz-Security-Token=AgoGb3JpZ2luEOn%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCWV1LXdlc3QtMSKAAhT4eq9r4GURlO%2B6hQW0PAMKVmFBUtHaI14%2Fz3i0p3WBjCHnlT49PoOXHaUgZKUVWqDQYKsSWthth%2FfV5k%2BRN76Nce%2BZKNfa9Gzpy%2BCUHVY34koYZSPo%2FFLC%2Be5ox3RBBgrpkB%2BzPHkuc4KNyPKIoSr%2BOHolUhRMZIOmRL84lzzttazBpgwIqxacKo6DQha2k%2FJTh5v%2FqDUXxLwr8Bj0DMRVx7PZg4MxLFSAZ0lShUa7H%2BpHKxkA%2F9wFcTPCK32HuvAub32O1qn1N8zBJxqZhLa5YtBA0vydq%2BIUHPsYePDryc0jTmz4MNVcQsrTPddOPkDoD1qX%2BI1pXEStBPV3EPkqrwUIvv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARACGgwwNDQzODQ5Mjk0NjYiDDCSdZ%2BxIQDybtgqASqDBQC%2BvI1sG5B9CqFZNNSl9qiRMnqbvQJ3WU57qsiz4kJftgMBU0nMdhl97p6mDXhJBvqC2vTgnnGiTbrRMr7%2BvNbNzbigkUO0exM%2B7bqxt6ij%2B8gz%2FdLd8T5Faj8j6AzvKGtwHxaGvaG0tCxvJGD%2BNYzOJxS8mae%2Fvf6qgpH%2BzQaf057NaZ9KsNlFZfCDD2CpBT%2FmB05tnzPmLDy5amzyuCyLWZgi7fzB6mqcKBmN0DktWb6RPRyaNFFi6PaJwHIDF%2BL1B3jx%2BmOXMon2Xwki8j9WugGWhQvMAl%2Fug82QoX1bxHegymeesIG052D6e%2F5XieeDwUZZ5IQinTx3eqs3m%2BqFlkxdIRb3JoPRldVm8hNUHG59mEI01r16%2B5SjTJ%2B4yvVR3x2%2BkGV2DY%2FjV5K8PueUHsIlN2hMGepG%2F7Rwg5fMdgg0YX3zEHZuyVdGvcQtXJbpmpYXxsXmRBkhIt3n59%2FtypAKHP1yLzV2Gxp2aYwUGiDVvn%2BsPHlfFL2FM4eil8g6%2FQVmSjZlw3Wa9Ke5HhIq5RV8la4TyOa7ogoAmoiVTlvqNvYgWlWrxl9zij4MgNwZ3S5Z79svhUNS8zHPDCtHru%2FXj81AUaTDsFSU%2BcVH%2BfKZuYVp3xWFAMSWhD3RQ5RT%2B8zCSKfvcI7hnI2i%2BhvAFFdqD8yPKBQ0A17LyYBc%2FAG91JDvw8lOM8fbmHQiMcmnd62V%2FE9RoQ0OAf3mrobMulrwxLKoQd%2F%2F9Hf8G7fVZZFP5jnIDVkIZScY8pZkzjYgftscGjagStFm4UtjM3KKWkBW5kb3zQREEFl9lUNrna4I89rVwxGxHcj0yVso18If2VPv932VzI2b455B2cbLGnswh%2BqD6AU%3D&X-Amz-Signature=72ac408bd3e51780c2ce3a44843365ca70550d07e0d7eb1e8781968ca5bf854e&X-Amz-SignedHeaders=host"
},
"dateModified": "2019-06-12T12:50:51.986Z",
"version": 1,
"usersWatching": 0,
"createdAt": "2019-06-12T12:52:28.461Z",
"likesCount": 0,
"textData": "Movie night?",
"enrolledUsers": 0,
"activityDate": "2019-06-14T12:50:14.000Z",
"ageMin": 18,
"dateCreated": "2019-06-12T12:50:51.986Z",
"peopleRequested": 1,
"commentsCount": 0,
"location": {
"address": {
"zipcode": "94108",
"country": "United States",
"city": "San Francisco",
"street": "Stockton St",
"state": null
},
"gps": {
"lon": -122.406417,
"lat": 37.785834
}
},
"enrollmentRequests": 0,
"id": "3a7c629e-803d-4fb9-8b7c-5fac08255649",
"activityCreatorId": "d70053b2-cbf2-47ea-959e-6b081e00dac9",
"category": "Movies::Ballet",
"updatedAt": "2019-06-12T12:52:28.461Z"
}
}
]
}
}
When I add the "terms" query in the filter like this:
GET /activity/doc/_search
{
"query":{
"bool":{
"must": [
{
"range": {
"ageMax": {
"gte": 20
}
}
},
{
"range": {
"ageMin": {
"lte": 28
}
}
},
{
"range":{
"activityDate":{
"gte":"2019-06-12T16:23:12.709Z"
}
}
},
{
"geo_distance":{
"distance":"50.0km",
"location.gps":{
"lon":-122.406417,
"lat":37.785834
}
}
}
],
"filter": {
"terms":
{
"category": ["Movies::Ballet"]
}
}
}
}
}
I expect to get back only the documents with category 'Movies::Ballet' but I get no hits..
I tried adding the "terms" query to the "must" array but same result.
I appreciate any help with figuring out where I am going wrong
I was finally able to get the query I want to work by changing 2 things:
Changed the category field for my test records to "Movies_Ballet" instead of "Movies::Ballet". I guess the "::" is a special character I was not aware of (someone please correct me if I am wrong here)
Changed the terms query to lowercase like:
"filter": {
"terms": {
"category": [
"movies_romance",
"TWO"
]
}
The expected result is returned.
hope this helps someone save some time

using mongodb case insentive regex with case insentive index

is mongo regex ignoring my index? I have a case insentive index, but by the look of things my regex search recognize it and ignores it.
db.getCollection("myCol").find({ value: /^mysearchVal/i }}).explain(...)
I have 95, 708 docs total.
output:
{
"queryPlanner": {
"plannerVersion": 1,
"namespace": "myDb.myCol",
"indexFilterSet": false,
"parsedQuery": {
"Value": {
"$regex": "^mysearchVal",
"$options": "i"
}
},
"winningPlan": {
"stage": "FETCH",
"filter": {
"Value": {
"$regex": "^mysearchVal",
"$options": "i"
}
},
"inputStage": {
"stage": "IXSCAN",
"keyPattern": {
"Value": 1
},
"indexName": "value_case_insensitive_and_unique",
"collation": {
"locale": "en",
"caseLevel": false,
"caseFirst": "off",
"strength": 2,
"numericOrdering": false,
"alternate": "non-ignorable",
"maxVariable": "punct",
"normalization": false,
"backwards": false,
"version": "57.1"
},
"isMultiKey": false,
"multiKeyPaths": {
"Value": []
},
"isUnique": true,
"isSparse": false,
"isPartial": false,
"indexVersion": 2,
"direction": "forward",
"indexBounds": {
"Value": [
"[\"\", {})",
"[/^mysearchVal/i, /^mysearchVal/i]"
]
}
}
},
"rejectedPlans": []
},
"executionStats": {
"executionSuccess": true,
"nReturned": 1,
"executionTimeMillis": 1447,
"totalKeysExamined": 95708,
"totalDocsExamined": 95708,
"executionStages": {
"stage": "FETCH",
"filter": {
"Value": {
"$regex": "^mysearchVal",
"$options": "i"
}
},
"nReturned": 1,
"executionTimeMillisEstimate": 1270,
"works": 95709,
"advanced": 1,
"needTime": 95707,
"needYield": 0,
"saveState": 785,
"restoreState": 785,
"isEOF": 1,
"invalidates": 0,
"docsExamined": 95708,
"alreadyHasObj": 0,
"inputStage": {
"stage": "IXSCAN",
"nReturned": 95708,
"executionTimeMillisEstimate": 596,
"works": 95709,
"advanced": 95708,
"needTime": 0,
"needYield": 0,
"saveState": 785,
"restoreState": 785,
"isEOF": 1,
"invalidates": 0,
"keyPattern": {
"Value": 1
},
"indexName": "value_case_insensitive_and_unique",
"collation": {
"locale": "en",
"caseLevel": false,
"caseFirst": "off",
"strength": 2,
"numericOrdering": false,
"alternate": "non-ignorable",
"maxVariable": "punct",
"normalization": false,
"backwards": false,
"version": "57.1"
},
"isMultiKey": false,
"multiKeyPaths": {
"Value": []
},
"isUnique": true,
"isSparse": false,
"isPartial": false,
"indexVersion": 2,
"direction": "forward",
"indexBounds": {
"Value": [
"[\"\", {})",
"[/^mysearchVal/i, /^mysearchVal/i]"
]
},
"keysExamined": 95708,
"seeks": 1,
"dupsTested": 0,
"dupsDropped": 0,
"seenInvalidated": 0
}
},
"allPlansExecution": []
},
"ok": 1.0
}
the output shows 95,708 keys and docs examined, 1 doc returned. really? did the index apply in this case or am I missing a point or two?
Case insensitive regular expression queries generally cannot use
indexes effectively. The $regex implementation is not collation-aware
and is unable to utilize case-insensitive indexes.
https://docs.mongodb.com/manual/reference/operator/query/regex/#index-use