Meteor regex find() far slower than in MongoDB console - regex

I've been researching A LOT for past 2 weeks and can't pinpoint the exact reason of my Meteor app returning results too slow.
Currently I have only a single collection in my Mongo database with around 2,00,000 documents. And to search I am using Meteor subscriptions on the basis of a given keyword. Here is my query:
db.collection.find({$or:[
{title:{$regex:".*java.*", $options:"i"}},
{company:{$regex:".*java.*", $options:"i"}}
]})
When I run above query in mongo shell, the results are returned instantly. But when I use it in Meteor client, the results take almost 40 seconds to return from server. Here is my meteor client code:
Template.testing.onCreated(function () {
var instance = this;
// initialize the reactive variables
instance.loaded = new ReactiveVar(0);
instance.limit = new ReactiveVar(20);
instance.autorun(function () {
// get the limit
var limit = instance.limit.get();
var keyword = Router.current().params.query.k;
var searchByLocation = Router.current().params.query.l;
var startDate = Session.get("startDate");
var endDate = Session.get("endDate");
// subscribe to the posts publication
var subscription = instance.subscribe('sub_testing', limit,keyword,searchByLocation,startDate,endDate);
// if subscription is ready, set limit to newLimit
$('#searchbutton').val('Searching');
if (subscription.ready()) {
$('#searchbutton').val('Search');
instance.loaded.set(limit);
} else {
console.log("> Subscription is not ready yet. \n\n");
}
});
instance.testing = function() {
return Collection.find({}, {sort:{id:-1},limit: instance.loaded.get()});
}
And here is my meteor server code:
Meteor.publish('sub_testing', function(limit,keyword,searchByLocation,startDate,endDate) {
Meteor._sleepForMs(200);
var pat = ".*" + keyword + ".*";
var pat2 = ".*" + searchByLocation + ".*";
return Jobstesting.find({$or:[{title:{$regex: pat, $options:"i"}}, { company:{$regex:pat,$options:"i"}},{ description:{$regex:pat,$options:"i"}},{location:{$regex:pat2,$options:"i"}},{country:{$regex:pat2,$options:"i"}}],$and:[{date_posted: { $gte : endDate, $lt: startDate }},{sort:{date_posted:-1},limit: limit,skip: limit});
});
One point I'd also like to mention here that I use "Load More" pagination and by default the limit parameter gets 20 records. On each "Load More" click, I increment the limit parameter by 20 so on first click it is 20, on second click 40 and so on...
Any help where I'm going wrong would be appreciated.

But when I use it in Meteor client, the results take almost 40 seconds to return from server.
You may be misunderstanding how Meteor is accessing your data.
Queries run on the client are processed on the client.
Meteor.publish - Makes data available on the server
Meteor.subscribe - Downloads that data from the server to the client.
Collection.find - Looks through the data on the client.
If you think the Meteor side is slow, you should time it server side (print time before/after) and file a bug.
If you're implementing a pager, you might try a meteor method instead, or
a pager package.

Related

Trying to get two postman.setNextRequest (not chained) or two Actions in Workspace

I’m quite new to postman (and coding) and was trying to find and piece together many snippets of scripts to make it work the way I want.
What I want is very simple: I have a list of IDs that I want to make a POST in each of them, get one of the responseBody as a variable and do another POST. I think I’m close but I can’t manage to get it to work.
I’ve tried:
Two POST request in the same Collection and running the collection.
In the first request I have a POST to
https://APIADDRESS/?order_id{{orderid}}&contract[copy_order_data]=true
On the Pre-request Script tab:
var orderids = pm.environment.get(“orderids”);
if (!orderids) {
orderids = [“bc46bf79-2846-44ed-ac4d-78c77c92ccc8”,“81aacc33-1ade-41a3-b23e-06b03b526b8f”];
}
var currentOrderId = orderids.shift();
pm.environment.set(“orderid”, currentOrderId);
pm.environment.set(“orderids”, orderids);
On the Tests tab:
var orderids = pm.environment.get(“orderids”);
if (orderids && orderids.length > 0) {
var jsonData = JSON.parse(responseBody);
postman.setEnvironmentVariable(“invoice.id”, jsonData.invoice.id);
postman.setNextRequest(“Create invoice”);
} else {
postman.setNextRequest(null);
}
invoice.id is a environment variable populated with the response body of the first action/post and then using the variable on the second action/post.
And then the second request would be a POST to
https://APIADDRESS/invoices/{{invoice.id}}/finalize.json
Of course this doesn’t work. Either it doesn't run the second request in the collection or it doesn't do the loope to more than 1 ID on the list.
So I thought that putting the second POST inside the first one would solve it. But I had no luck.
Can please someone help me?
I have tried mentioned use case with sample API's provided by POSTMAN.
Can you try it?
First POST Method Request : https://postman-echo.com/post
Pre-request Script of first POST method
var orderids = pm.environment.get("orderids");
if(!orderids ){
orderids = ["bc46bf79-2846-44ed-ac4d-78c77c92ccc8","81aacc33-1ade-41a3-b23e-06b03b526b8f"];
}
var currentOrderId = orderids.shift();
pm.environment.set("orderid", currentOrderId);
pm.environment.set("orderids", orderids);
Tests Tab of first POST Method
var orderids = pm.environment.get("orderids");
if (orderids && orderids.length > 0) {
var jsonData = JSON.parse(responseBody);
postman.setEnvironmentVariable("invoice.id", jsonData.headers.host);
postman.setNextRequest("Test1");
} else {
postman.setNextRequest(null);
}
Second POST Method Reqeust: https://postman-echo.com/post?key={{invoice.id}}
After executing the above collection it will set orederids and invoice.id value in environment variables and then it will call next POST Method.
Hope this will help you.
Thanks #HalfBloodPrince, from the Postman Echo it worked but in my case it doesn't :S
What I manage to get it working was using a Json file as a list of Orderids.
In that case I've separated all requests.
Request1 - https://APIADDRESS/?order_id{{orderid}}&contract[copy_order_data]=true
Tests tab:
var jsonData = JSON.parse(responseBody);
postman.setEnvironmentVariable("invoice.id", jsonData.invoice.id);
Request2 - https://APIADDRESS/invoices/{{invoice.id}}/finalize.json
That way everything is in a neat and organized way.
Thanks

'Content-Length: 0', Javascript, XMLHttpRequest, Django REST API

I wish I can find some assistance with my code.
I have set up a Database on mySQL, and connected it to Django REST. Those both work as expected, and I can access the REST with Firefox REST Client with it returning the correct tables from the database.
I have started working for user interface with html and javascript and I have encountered a problem I am unable to solve. I am a student, and this is part of my school work, but unfortunately my teachers are unavailable at the moment due summer vacations and I am eager to continue my project. Hence I am askin for Your assistance.
As I have tested the Django REST through Firefox REST Client, I am sure the database and REST service is not at fault so here we come to my code.
I seem to be able to get connection to the REST Service, giving me code 200 and state 4 (pictures linked underneath)
ReadyStateChange + ReadyState console.logs
Picture 2 shows that my GET request gets stuck on OPTIONS, instead of executing the correct request.
200 OPTIONS
However I am unable to pull data out, giving me 'Content-Length: 0'.
Originally I thought the issue would be cross-domain request problem until my fellow student said he does not think it is, however he was unable to find solution for my code either.
I am trying to find reason and workaround for this error, and if you guys do have idea why this is happening I would deeply appriciate your help!
Here is my code:
<div id="demo"></div>
<script>
loadData() //function kutsu
function loadData(){
if (window.XMLHttpRequest) {
// code for modern browsers
xmlhttp = new XMLHttpRequest();
} else {
// code for old IE browsers
xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
}
var url = "http://127.0.0.1:8000/vamkrs/";
xmlhttp.open("GET", url, true);
xmlhttp.withCredentials = true;
xmlhttp.setRequestHeader("Authorization", "Basic " + btoa("username:password"));
xmlhttp.send();
setTimeout(xmlhttp.onreadystatechange = function() {
console.log(this.status);
console.log(this.readyState);
if (this.readyState == 4 && this.status == 200) {
var myData = JSON.parse(this.responseText);
document.getElementById("demo").innerHTML = myData.responseText(); }
},1500);
/*
xmlhttp.onreadystatechange = function(){
console.log(this.status);
if (this.readyState == 4 && this.status == 200) {
var myData = JSON.parse(this.responseText);
document.getElementById("demo").innerHTML = myData.responseText();}
}; */
}
</script>
Ps. Sorry, English is not my native language so some spelling mistakes might have been made
Pss. First time posting here, I apologize if mistakes were made on the post

Meteor subscriptions using deps

I'm trying to implement a searching feature in a Meteor application that re-subscribes/publishes a collection on every search, so there is only the exact Collection necessary in the client. I'm creating a reactive variable searchString, then changing it to the text in the search box on every search, then splitting the string into tags:
// Client
var searchString = "";
var searchStringDep = new Deps.Dependency;
var getSearchString = function(){
searchStringDep.depend();
return searchString;
}
var handle = Deps.autorun(function(){
var tags = getSearchString().split(" ");
tags = _.map(tags, function(tag){
return tag.replace(/[^\w]/g, "");
}).filter(function(t){
return t.toLowerCase();
});
Meteor.subscribe('results', tags);
});
Template.library.events({
'submit form': function(ev){
ev.preventDefault();
searchString = ev.target.search.value;
searchStringDep.changed();
}
})
Then, publishing a new Collection on the server, based on the tags:
// Server
Meteor.publish('results', function(tags){
regTags = _.map(tags, function(tag) { return new RegExp(tag)});
return Samples.find({tags: {$in: regTags}})
});
So I'm trying to match on regexes, but am having a weird issue where the subscription only changes when I add another tag, but changing existing tags fails.
So if the first searchString was tag1 and the second tag1 tag2, it works fine.
But if the first is tag1 and the second is tag2, the Collection doesn't update.
Any help is appreciated...I'm a beginner to Meteor, so if there is a better way to do what I'm trying to do, all suggestions are welcome. Thanks so much
'change #search': function(){
Meteor.subscribe('sampleResults', $('#search').val()); // or if you want on submit it's the same idea
}
and publish like
Meteor.publish('sampleResults, function(text){
return Samples.find({tags: {$regex: text}});
}
A few things:
1) Meteor has a very nice way of setting up reactive variables with the ReactiveVar component. I would suggest using that rather than creating another dependency for a variable.
2) The name that you are subscribing to: results is different than what is published on the server sampleResults and that can cause issues.
3) If you are on Meteor >= 0.9.1 you should be using Tracker and not Deps. You can use Deps if you want, but the new updated API is Tracker and is probably more stable. See the changelog for more details on that.
4) You don't have to set your Deps.autorun function equal to a variable. So you can have it as:
Tracker.autorun(function() {
// Code here
});

jmeter - counting cookie values

I'm playing around with jmeter to validate a bug fix.
Server logic sets the cookie "mygroup" it can be either "groupa" or "groupb". I want to be able fire a series of requests and be able to see there is a proper balanced distribution between the values of this cookie. Ie make 100 requests and 50 times the cookie will be set to "groupa" and "groupb".
I'm a bit stuck on this. I currently have the following. I can see the cookies being set in the results tree but I'd like to be able to display the a table with the version and the number of requests of each.
Thread Group
HTTP Cookie Manager
HTTP Request
View Results Tree
Within the results tree I can see Set-Cookie: mygroup="groupa" and also sometimes mygroup="groupb" how do I tabulize this ??
You can have cookies values exported as JMeter variables by setting:
CookieManager.save.cookies=true
in user.properties.
Add a Cookie Manager to your Test Plan.
In this case you will have the var COOKIE_mygroup set by JMeter.
You can then count it like this using a JSR223 Sampler + Groovy (add groovy-all-version.jar in jmeter/lib folder:
String value = vars.get("COOKIE_mygroup");
Integer counterB = vars.getObject("counterB");
Integer counterA = vars.getObject("counterA");
if(counterA == null) {
counterA = new Integer(0);
vars.putObject("counterA", counterA);
}
if(counterB == null) {
counterB = new Integer(0);
vars.putObject("counterB", counterB);
}
if(value.equals("groupa")) {
counterA = counterA+1;
vars.putObject("counterA", counterA);
} else {
counterB = counterB+1;
vars.putObject("counterB", counterB);
}
Asyou have only one thread, at end of loop you can then compare the 2 values or just display the value:
add a debug sampler
add a view tree result
Run test plan, in view result tree click on debug sampler , select response tab and you should have your values

How can I force the __utmz cookie to be set on the first page load

I am using a sweet little function to track visitors to my site and dump the info to Salesforce. However many form submissions have (none) set for many values because (as I understand it) the cookie is not set until the second page load.
I have tested this and that seems to be accurate, the problem is many people fill out a form on the first page, and I don't get any info through these submissions.
I am loading GA as follows:
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-29066630-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script');
ga.src = ('https:' == document.location.protocol ? 'https://ssl' :
'http://www') + '.google-analytics.com/ga.js';
ga.setAttribute('async', 'true');
document.documentElement.firstChild.appendChild(ga);
})();
</script>
And then running a php function to parse the __utmz cookie apart:
function tracking_cookie() { // ver 1.5
$trackarr = split('[|]',$_COOKIE["__utmz"]);
$conversion = $_COOKIE["Conversion"];
for($i=0;$i<count($trackarr);$i++){
$keyvalues=split('[=]',$trackarr[$i]);
$key=substr($keyvalues[0],-6);
switch ($key){
case "utmcsr":
$cookie['SearchEngine'] = $keyvalues[1];break;
case "utmccn":
$cookie['SearchCampaign'] = $keyvalues[1];break;
case "utmcmd":
$cookie['SearchType'] = $keyvalues[1];break;
case "utmcct":
$cookie['AdText'] = $keyvalues[1];break;
case "utmctr":
$cookie['Keyword'] = $keyvalues[1];break;
case "mgclid":
$cookie['isPPC'] = $keyvalues[1];break;
}
}
I have more code running after this. Any ideas on how to force the cookie to load the first time around?
A cookie isn't readable until the second page load - because they get sent in the request.
What you could do is on the initial page load (and no _utmz cookie is found in PHP) append another JS file / invoke some JS that will run an ajax command back to your server after the page has loaded.
This will act as the second page load and should have access to the new cookie (and the user wouldn't have left the page)
Have you taken a look at this Force.com Toolkit for Google Analytics? May be more reliable to get the information right from Google. Without knowing what else you are tracking or how you are storing it in Salesforce it is hard to give more detail.
https://github.com/mavens/Force.com-Toolkit-for-Google-Analytics
There is also some discussion on Google Groups - http://groups.google.com/a/googleproductforums.com/forum/#!category-topic/analytics/discuss-tracking-and-implementation-issues/bVR84di07pQ
You did say, "Many form submissions have none", so does that mean you do get data from time to time? Can you narrow it down to a browser?