Couchbase custom reduce function - mapreduce

I have some documents in my Couchbase with the following template:
{
"id": 102750,
"status": 5,
"updatedAt": "2014-09-10T10:50:39.297Z",
"points1": 1,
"points2": -3,
"user1": {
"id": 26522,
...
},
"user2": {
"id": 38383,
...
},
....
}
What I want to do is to group the documents on the user and sum the points for each user and then show the top 100 users in the last week. I have been circling around but I haven't come with any solution.
I have started with the following map function:
function (doc, meta) {
if (doc.user1 && doc.user2) {
emit(doc.user1.id, doc.points1);
emit(doc.user2.id, doc.points2);
}
}
and then tried the sum to reduce the results but clearly I was wrong because I wasn't able to sort on the points and I couldn't also include the date parameter

you need to see my exemple I was able to group by date and show the values with reduce. but calculate the sum I did it in my program.
see the response How can I groupBy and change content of the value in couchbase?

I have solved this issue by the help of a server side script.
What I have done is I changed my map function to be like this:
function (doc, meta) {
if (doc.user1 && doc.user2) {
emit(dateToArray(doc.createdAt), { 'userId': doc.user1.id, 'points': doc.points1});
emit(dateToArray(doc.createdAt), { 'userId': doc.user2.id, 'points': doc.points2});
}
}
And in the script I query the view with the desired parameters and then I group and sort them then send the top 100 users.
I am using Node JS so my script is like this: (the results are what I read from couchbase view)
function filterResults(results) {
debug('filtering ' + results.length + ' entries..');
// get the values
var values = _.pluck(results, 'value');
var groupedUsers = {};
// grouping users and sum their points in the games
// groupedUsers will be like the follwoing:
// {
// '443322': 33,
// '667788': 55,
// ...
// }
for (var val in values) {
var userId = values[val].userId;
var points = values[val].points;
if (_.has(groupedUsers, userId)) {
groupedUsers[userId] += points;
}
else
groupedUsers[userId] = points;
}
// changing the groupedUsers to array form so it can be sorted by points:
// [['443322', 33], ['667788', 55], ...]
var topUsers = _.pairs(groupedUsers);
// sort descending
topUsers.sort(function(a, b) {
return b[1] - a[1];
});
debug('Number of users: ' + topUsers.length + '. Returning top 100 users');
return _.first(topUsers, 100);
}

Related

How to highlight a column programmatically in AMCharts 4?

In AMCharts version 3, there is a demo showing how to highlight a particular column.
Is this possible using AMCharts version 4? For example, in the Simple Column demo, highlight the UK column based on its value (ie, where country = 'UK').
I tried modifying the example at https://stackoverflow.com/a/54358490/906814 but I can't get a handle on the columns in order to assess their values and then apply the active state highlight (JSFiddle).
// copied from https://stackoverflow.com/a/54358490/906814 but not working yet
var activeState = series.columns.template.states.create("active");
activeState.properties.fill = am4core.color("#E94F37");
series.columns.each(function(column) {
alert("column") // no alert is seen
column.setState("active");
column.isActive = true;
})
There are two approaches you can take.
1) Use an adapter on the column's fill and stroke and check the column value before modifying the color, e.g.
series.columns.template.adapter.add('fill', function(fill, target) {
if (target.dataItem && target.dataItem.categoryX == "UK") {
return "#ff0000";
}
return fill;
});
series.columns.template.adapter.add('stroke', function(stroke, target) {
if (target.dataItem && target.dataItem.categoryX == "UK") {
return "#ff0000";
}
return stroke;
})
Demo
2) Use a property field and set the stroke and fill from your data:
chart.data = [
// ...
{
"country": "UK",
"value": 1122,
"color": "#ff0000"
},
// ...
];
// ...
series.columns.template.propertyFields.fill = "color";
series.columns.template.propertyFields.stroke = "color";
Demo

AWS RDS Data API executeStatement not return column names

I'm playing with the New Data API for Amazon Aurora Serverless
Is it possible to get the table column names in the response?
If for example I run the following query in a user table with the columns id, first_name, last_name, email, phone:
const sqlStatement = `
SELECT *
FROM user
WHERE id = :id
`;
const params = {
secretArn: <mySecretArn>,
resourceArn: <myResourceArn>,
database: <myDatabase>,
sql: sqlStatement,
parameters: [
{
name: "id",
value: {
"stringValue": 1
}
}
]
};
let res = await this.RDS.executeStatement(params)
console.log(res);
I'm getting a response like this one, So I need to guess which column corresponds with each value:
{
"numberOfRecordsUpdated": 0,
"records": [
[
{
"longValue": 1
},
{
"stringValue": "Nicolas"
},
{
"stringValue": "Perez"
},
{
"stringValue": "example#example.com"
},
{
"isNull": true
}
]
]
}
I would like to have a response like this one:
{
id: 1,
first_name: "Nicolas",
last_name: "Perez",
email: "example#example.com",
phone: null
}
update1
I have found an npm module that wrap Aurora Serverless Data API and simplify the development
We decided to take the current approach because we were trying to cut down on the response size and including column information with each record was redundant.
You can explicitly choose to include column metadata in the result. See the parameter: "includeResultMetadata".
https://docs.aws.amazon.com/rdsdataservice/latest/APIReference/API_ExecuteStatement.html#API_ExecuteStatement_RequestSyntax
Agree with the consensus here that there should be an out of the box way to do this from the data service API. Because there is not, here's a JavaScript function that will parse the response.
const parseDataServiceResponse = res => {
let columns = res.columnMetadata.map(c => c.name);
let data = res.records.map(r => {
let obj = {};
r.map((v, i) => {
obj[columns[i]] = Object.values(v)[0]
});
return obj
})
return data
}
I understand the pain but it looks like this is reasonable based on the fact that select statement can join multiple tables and duplicated column names may exist.
Similar to the answer above from #C.Slack but I used a combination of map and reduce to parse response from Aurora Postgres.
// declarative column names in array
const columns = ['a.id', 'u.id', 'u.username', 'g.id', 'g.name'];
// execute sql statement
const params = {
database: AWS_PROVIDER_STAGE,
resourceArn: AWS_DATABASE_CLUSTER,
secretArn: AWS_SECRET_STORE_ARN,
// includeResultMetadata: true,
sql: `
SELECT ${columns.join()} FROM accounts a
FULL OUTER JOIN users u ON u.id = a.user_id
FULL OUTER JOIN groups g ON g.id = a.group_id
WHERE u.username=:username;
`,
parameters: [
{
name: 'username',
value: {
stringValue: 'rick.cha',
},
},
],
};
const rds = new AWS.RDSDataService();
const response = await rds.executeStatement(params).promise();
// parse response into json array
const data = response.records.map((record) => {
return record.reduce((prev, val, index) => {
return { ...prev, [columns[index]]: Object.values(val)[0] };
}, {});
});
Hope this code snippet helps someone.
And here is the response
[
{
'a.id': '8bfc547c-3c42-4203-aa2a-d0ee35996e60',
'u.id': '01129aaf-736a-4e86-93a9-0ab3e08b3d11',
'u.username': 'rick.cha',
'g.id': 'ff6ebd78-a1cf-452c-91e0-ed5d0aaaa624',
'g.name': 'valentree',
},
{
'a.id': '983f2919-1b52-4544-9f58-c3de61925647',
'u.id': '01129aaf-736a-4e86-93a9-0ab3e08b3d11',
'u.username': 'rick.cha',
'g.id': '2f1858b4-1468-447f-ba94-330de76de5d1',
'g.name': 'ensightful',
},
]
Similar to the other answers, but if you are using Python/Boto3:
def parse_data_service_response(res):
columns = [column['name'] for column in res['columnMetadata']]
parsed_records = []
for record in res['records']:
parsed_record = {}
for i, cell in enumerate(record):
key = columns[i]
value = list(cell.values())[0]
parsed_record[key] = value
parsed_records.append(parsed_record)
return parsed_records
I've added to the great answer already provided by C. Slack to deal with AWS handling empty nullable character fields by giving the response { "isNull": true } in the JSON.
Here's my function to handle this by returning an empty string value - this is what I would expect anyway.
const parseRDSdata = (input) => {
let columns = input.columnMetadata.map(c => { return { name: c.name, typeName: c.typeName}; });
let parsedData = input.records.map(row => {
let response = {};
row.map((v, i) => {
//test the typeName in the column metadata, and also the keyName in the values - we need to cater for a return value of { "isNull": true } - pflangan
if ((columns[i].typeName == 'VARCHAR' || columns[i].typeName == 'CHAR') && Object.keys(v)[0] == 'isNull' && Object.values(v)[0] == true)
response[columns[i].name] = '';
else
response[columns[i].name] = Object.values(v)[0];
}
);
return response;
}
);
return parsedData;
}

How do I write information from classes into lists in typescript?

How do I write informations from classes into lists in typescript?
class UserDeposit {
username: string;
userdeposit: number;
constructor(username: string, userdeposit: number) {
this.username = username;
this.userdeposit = userdeposit;
}
getUsername(): string {
return this.username;
}
getDepositValue(): number {
return this.userdeposit;
}
}
I want to create a list, where I want to store all the information like this:
let deposit: UserDeposit[] = [
{ "username": "Test", "deposit": 20 },
{ "username": "Test2", "deposit": 20 },
...........,
];
In my console output I want to get the number of players and the total value of deposit like this (pseudo code):
deposit.countUser + " User joined the Jackpot. The total value is " + deposit.totalValue
2 User joined the Jackpot. The total value is 40$
I am a newbie with TypeScript.
Can someone help me?
Your class is fine.
In this way you can add new elements to your list:
// initialize list:
let deposit: UserDeposit[] = [];
// add new elements
deposit.push(new UserDeposit('test1', 10));
deposit.push(new UserDeposit('test2', 20));
deposit.push(new UserDeposit('test3', 30));
And here comes the output:
console.log('Users: ' + deposit.length + '\nTotal Deposit Value: ' + deposit.reduce((a, b) => a + b.getDepositValue(), 0));
The console should print:
Users: 3
Total Deposit Value: 60

Using getFillteredRows in google charts with additional property

I have a row structure like this
c:[
{ v: 'somevalue'},
{ v: 'somevalue'},
{
v: 'somevalue',
link: 'abc.com'
}
]
now I need all the rows which has link property present in 3rd column, is it possible using getFillteredRows function ?
first, to use cell properties correctly, the structure would resemble the following...
c:[
{ v: 'somevalue'},
{ v: 'somevalue'},
{
v: 'somevalue',
p: {
link: 'abc.com'
}
}
]
to get or set the properties, use the following methods...
getProperty(rowIndex, columnIndex, name)
setProperty(rowIndex, columnIndex, name, value)
adding in getFilteredRows (spelling - one L in filter)...
use the test function, to find all the rows which has link property present in 3rd column
var rowsFound = data.getFilteredRows([{
column: 2,
test: function (value, row, column, table) {
return (table.getProperty(row, column, 'link') !== null);
}
}]);

How do I hide values past the x-axis in chartjs 2.0?

How do I hide values past the x-axis in chartjs 2.0? You will notice the chart juts past the -60 mark. The x-axis uses a time scale and I have the max and min values set.
Here's my chart configuration:
{
"type":"line",
"data":{
"datasets":[
{
"label":"Scatter Dataset",
"data":[
{
"x":"2016-09-16T16:36:53Z",
"y":88.46153846153845
},
...
{
"x":"2016-09-16T16:37:54Z",
"y":88.3076923076923
}
],
"pointRadius":0,
"backgroundColor":"rgba(0,0,255,0.5)",
"borderColor":"rgba(0,0,255,0.7)"
}
]
},
"options":{
"title":{
"display":true,
"text":"Water Level Over Last 60 Seconds"
},
"animation":false,
"scales":{
"xAxes":[
{
"type":"time",
"position":"bottom",
"display":true,
"time":{
"max":"2016-09-16T16:37:54Z",
"min":"2016-09-16T16:36:54.000Z",
"unit":"second",
"unitStepSize":5
},
"ticks":{
callback: function(value, index, values) {
return "-" + (60 - 5 * index);
}
}
}
],
"yAxes":[
{
"display":true,
"ticks":{
}
}
]
},
"legend":{
"display":false
}
}
}
You can achieve this using Chart.js plugins. They let you handle events occuring while creating, updating or drawing the chart.
Here, you'll need to affect before the chart is initialised :
// We first create the plugin
var cleanOutPlugin = {
// We affect the `beforeInit` event
beforeInit: function(chart) {
// Replace `ticks.min` by `time.min` if it is a time-type chart
var min = chart.config.options.scales.xAxes[0].ticks.min;
// Same here with `ticks.max`
var max = chart.config.options.scales.xAxes[0].ticks.max;
var ticks = chart.config.data.labels;
var idxMin = ticks.indexOf(min);
var idxMax = ticks.indexOf(max);
// If one of the indexes doesn't exist, it is going to bug
// So we better stop the program until it goes further
if (idxMin == -1 || idxMax == -1)
return;
var data = chart.config.data.datasets[0].data;
// We remove the data and the labels that shouldn't be on the graph
data.splice(idxMax + 1, ticks.length - idxMax);
data.splice(0, idxMin);
ticks.splice(idxMax + 1, ticks.length - idxMax);
ticks.splice(0, idxMin);
}
};
// We now register the plugin to the chart's plugin service to activate it
Chart.pluginService.register(cleanOutPlugin);
The plugin is basically a loop through the data to remove the values that shouldn't be displayed.
You can see this plugin working in a live example on jsFiddle.
For instance, the following chat with a min set to 2 and a max to 6 ...
... would give the following result :