I am writing a map/reduce script to search for a custom record and create a CSV file from its results.
Firstly, in the getInputData() creating a search and returning its results.
After that comes the map function where I am using JSON.parse to parse the result values.
I have an array declared globally as var data = '' in which I am pushing the values I get in map function.
But when all the looping ends in map() and it enters summarize(), the data array becomes empty again.
I want that data array to be called in summarize() where I can use file.create() function to create the csv file.
I used return data; in map() but it is showing as empty when in summarize().
You have to use context.write at the end of map to send data to the next stage.
context.write({
key: key,
value: value
});
Adding a simple script to show how to access values in the summarize stage:
/**
*#NApiVersion 2.x
*#NScriptType MapReduceScript
*/
define([],
function () {
function getInputData() {
return [{
id: 1,
value: 'abc'
}, {
id: 2,
value: 'def'
}];
}
function map(context) {
var value = JSON.parse(context.value);
context.write({
key: value.id,
value: value.value
});
}
function summarize(context) {
var values = [];
context.output.iterator().each(function (key, value) {
log.debug({
title: 'Output for key ' + key,
details: value
});
values.push(value);
return true;
});
log.debug({
title: 'Summary',
details: JSON.stringify(values)
});
}
return {
getInputData: getInputData,
map: map,
summarize: summarize
}
});
Related
I try update my list after item remove by this article
but get Invariant Violation error.
my mutation:
const deleteFn = useMutation<FeaturedPlaylistGroupDelete, FeaturedPlaylistGroupDeleteVariables>(deleteQuery, {
update: (cache, mutationResult) => {
console.log('mutationResult', mutationResult)
const data = cache.readQuery({ query: featuredPlaylistsGroupsQuery })
console.log('cache', cache)
console.log('cacheData', data)
cache.writeQuery({
query: featuredPlaylistsGroupsQuery,
data: data.filter((item) => item.id !== mutationResult.data.featuredPlaylistGroupDelete.id),
})
},
})
featuredPlaylistsGroupsQuery:
export const featuredPlaylistsGroupsQuery = gql`
query FeaturedPlaylistGroups(
$active: Boolean
$noCategory: Boolean
$dateFrom: String
$dateTo: String
$title: String
$regions: [String!]
$categories: [String!]
) {
featuredPlaylistGroups(
active: $active
noCategory: $noCategory
dateFrom: $dateFrom
dateTo: $dateTo
regions: $regions
title: $title
categories: $categories
) {
active
category {
title
}
datetime
id
region
title
}
}
`
deleteQuery:
const deleteQuery = gql`
mutation FeaturedPlaylistGroupDelete($id: String!) {
featuredPlaylistGroupDelete(id: $id) {
active
categoryId
category {
title
}
datetime
id
region
title
}
}
`
error:
Invariant Violation: Can't find field
featuredPlaylistGroups({}) on object {
...
When you use readQuery, what's returned is what would have been returned in the data part of the response for that query. This is always an object. So for a query like
query {
foo
bar
}
You get an object like
{
"foo": "FOO",
"bar": "BAR"
}
When you call readQuery using your featuredPlaylistsGroupsQuery, you'll get an object with a single property named featuredPlaylistGroups. So your code should look more like:
const cached = cache.readQuery({ query: featuredPlaylistsGroupsQuery })
const featuredPlaylistGroups = cached.featuredPlaylistGroups.filter(item => {
return item.id !== mutationResult.data.featuredPlaylistGroupDelete.id
})
const data = {
...cached,
featuredPlaylistGroups,
}
cache.writeQuery({
query: featuredPlaylistsGroupsQuery,
data: data,
})
However, this still will not work because featuredPlaylistsGroupsQuery takes a number of variables. We need those variables in order to read and write from the cache, since each combination of variable that has been queries is stored separately in the cache. So you will either need to keep track of the variables used and call readQuery/writeQuery on all used combinations, or use something like apollo-link-watched-mutation
here's my case: I'm trying to make a query on a table (table name HCI.LocCatApp) using a value sent by API as KeyConditionExpression, and I'm storing the results (which must be numbers not strings) in an array, and I want to use each value from this array as a FilterExpression to scan another table (table name HCI.Category) .. So what I need is to loop on the array values, take each of them as FilterExpression and perform the scan operation. I'm currently trying to use IN but I'm not sure if it's even supported or not.
And keep in mind that the array is being filled during the runtime. And the callback can be performed only once.
here's my code:
'use strict'
var AWS = require('aws-sdk');
var mydocumentClient = new AWS.DynamoDB.DocumentClient();
exports.handler = function (event, context, callback) {
var params = {
TableName: 'HCI.LocCatApp',
KeyConditionExpression : 'LocID = :lid',
ExpressionAttributeValues: {
":lid": event.LocID
},
ProjectionExpression: 'CatID'
};
var catIDs = [];
var catIDsObject = {};
var index = 0;
mydocumentClient.query(params, function (err, data){
if (err) {
callback(err, null);
}else{
data.Items.forEach(function(item){catIDs.push(item.CatID)});
//callback(null, catIDs);
}
})
catIDs.forEach(function(value){
index ++;
var catIDsKey = ":catID"+index;
catIDsObject[catIDsKey] = value;
})
var params2 = {
TableName: 'HCI.Category',
FilterExpression : "CatID IN (:cIDs)",
ExpressionAttributeValues : {
':cIDs' : catIDs
}
};
mydocumentClient.scan(params2, function (err, data){
if (err) {
callback(err, null);
}else{
callback(null, data);
}
})
}
For some reason, the current code runs successfully but it doesn't find any matches, even if I fill in the values manually in the array, there's still no results, the IN operation doesn't seem to work.
And many thanks in advance
In your code catIds is an array of IDs (strings probably).
When you pass it to FilterExpression, you are assuming that it will be converted to a) string b) to a string in correct format.
FilterExpression : "CatID IN (:cIDs)",
ExpressionAttributeValues : {
':cIDs' : catIDs
}
I cannot try this myself at the moment, but I'm assuming this is where the query fails. IN operator expects a comma separated list of values to compare to, in parenthesis. So, after the array is inserted to query, it should be like this
FilterExpression : "CatID IN (cat1, cat2, cat2)",
But most probably it contains extra set of [ and ], and maybe even the array to string conversion causes it to something like [Object object] etc.
One solution would be to use Array.join to concatenate all the elements from the array to single string before passing it to FilterExperession. Something like this
FilterExpression : "CatID IN (:cIDs)",
ExpressionAttributeValues : {
':cIDs' : catIDs.join()
}
'use strict';
module.exports = function (City) {
City.GetCurrentPopulation = function (req) {
var population;
City.app.models.Pupulation.find({where{id:req.id}}, //This line //gives me an error that cannot read property 'find' of undefined
function(req.res){
population=res.population;
});
response='Population for ' +req.cname ' is' +population;
req(null, response);
};
City.remoteMethod(
'GetCurrentPopulation', {
http: {
path: '/GetCurrentPopulation',
verb: 'GetCurrentPopulation'
},
returns: {
arg: 'startdate',
type: 'string'
}
}
);
There is a model city and i want to access another model like "population.find(some filters)" How to do this?
I have a remote method written in city model. Where i am trying to access population record as
var countryp=population.find(where{id:4});
var currentpopulation=countryp.Totalpopulation;
It gives an error population.find is not a function.
Please suggest way to do this.
City.app.models.Population can only work if you defined some relation between City & Population models. Otherwise it wont work that way. If there is no relation to the other model. You need to get a reference to the app object using
Try like this:
var app = require('../../server/server');
module.exports = function (City) {
var Population = app.models.Population;
City.GetCurrentPopulation = function(req) {
Population.find({where{id:req.id}}, function (err) {
if (err) {
return console.log(err);
} else {
// do something here
});
}
You can refer to the documentation here https://loopback.io/doc/en/lb3/Working-with-LoopBack-objects.html#using-model-objects
How can I return a record that matches some custom id (not the regular id) from my collection?
// record schema
{
id: 123, // assigned id from mirage
dId: 'DEVICE0001', // device id I want to use to pick
value: 'some content'
}
To select with something like this:
// app/mirage/config.js
this.get("/something/:device_did", function(db, request){
var did = request.params.device_did;
// select using my device id
return db.devices.firstWhere( { 'dId': did });
});
The API docs only reference a 'find()' function that acts on the id, and a 'where()' that gets an array of records.
select the first element from the models array!
var device = db.devices.where({'dId' : did });
return device.models[0];
So from what I understand in Couchbase is that one can sort keys* by using
descending=true
but in my case I want to sort by values instead. Consider the Twitter data in json format, my question is What it the most popular user mentioned?
Each tweet has the structure of:
{
"text": "",
"entities" : {
"hashtags" : [ ... ],
"user_mentions" : [ ...],
"urls" : [ ... ]
}
So having used MongoDB before I reused the Map function and modified it slightly to be usable in Couchbase as follows:
function (doc, meta) {
if (!doc.entities) { return; }
doc.entities.user_mentions.forEach(
function(mention) {
if (mention.screen_name !== undefined) {
emit(mention.screen_name, null);
}
}
)
}
And then I used the reduce function _count to count all the screen_name occurrences. Now my problem is How do I sort by the count values, rather than the key?
Thanks
The short answer is you cannot sort by value the result of you view. You can only sort by key.
Some work around will be to either:
analyze the data before inserting them into Couchbase and create a counter for the values you are interested by (mentions in your case)
use the view you have to sort on the application size if the size of the view is acceptable for a client side sort.
The following JS code calls a view, sorts the result, and prints the 10 hottest subjects (hashtags):
var http = require('http');
var options = {
host: '127.0.0.1',
port: 8092,
path: '/social/_design/dev_tags/_view/tags?full_set=true&connection_timeout=60000&group=true',
method: 'GET'
}
http.request(
options,
function(res) {
var buf = new Buffer(0);
res.on('data', function(data) {
buf += data;
});
res.on('end', function() {
var tweets = JSON.parse(buf);
var rows = tweets.rows;
rows.sort( function (a,b){ return b.value - a.value }
);
for ( var i = 0; i < 10; i++ ) {
console.log( rows[i] );
}
});
}
).end();
In the same time I am looking at other options to achieve this
I solved this by using a compound key.
function (doc, meta) {
emit([doc.constraint,doc.yoursortvalue]);
}
url elements:
&startkey=["jim",5]&endkey=["jim",10]&descending=true