I performed my first query with a grouping function using the following code:
let dc = new TypedDataContext()
open MathNet.Numerics.Statistics
let newData = query { for x in dc.MyData do
where (x.ID = "number of type string")
groupBy x.Code into g
let average = query { for x in g do
averageBy x.Billed_Amt }
select (g, average) }
|> Seq.toList
System.Console.WriteLine(newData)
I am now wanting to calculate the standard deviation of the billed amounts in each group. However, when I try to reference the column, 'Billed_Amt,' like so
let sd = newData.Billed_Amt.StandardDeviation()
I receive the following error: "constructor or member 'Billed_Amt' is not defined."
I also tried newData.g.Billed_Amt.StandardDeviation(), in case I needed to reference the groups first, but I got the same error message referring to 'g'.
How do I overcome this?
I will say that I notice that the 'list' I created with the query claims to have 63 items. These would be the different group keys, not the rows of data themselves.
I feel like the research I have done online to solve this problem has just sent me in circles. Most of the resources online do not dumb it down enough for newbies like me. Any help would be appreciated.
Thank you!
Related
I am creating a site where clients can construct a price quote.
I have used Tabulator to show the data in a table.
The Tabulator data is in its own .JS file "buildTabulator.js", while the Quote scripts are in another .JS file "quote.js".
I am able to extract the data and get it mostly doing what I want so far.
But, when I try to create an IF ELSE statement in the quote.js file to determine which price should be displayed based on the unit of measure (UOM) of the item, it will not work correctly. It always shows the EACH or EA price of the item which is in the IF part of the IF ELSE statement.
More specifically, if the user adds an item to the quote that is sold by the EACH it should show one price, but if they choose an item that is sold by the CASE it should display a different price.
Please see the attached screenshots and code below and advise if you can...
I am hoping this is something simple that a noob like me has missed by mistake.
First screenshot is showing the code in VSC.
Second screenshot is showing an example in the console log of the browser. This example is using an item that is sold by the CASE and should show the case price of the item but instead shows a "-" because the EACH price of this item is a dash since its not sold by the each.
In the log it shows CS for CASE which is correct but the "-" should actually be 0.2937 which is shown in the table above the console.
Please let me know if there is any more information I can provide or any questions you may have.
Thank you so much in advance!!
CODE FROM "buildTabulator.js"
cellClick: function (e, cell) {
globalThis.itemImage =
"imgsQuote/" + cell.getRow().getData().IMAGE + ".png";
globalThis.itemCode = cell.getRow().getData().CODE;
globalThis.itemDescription = cell.getRow().getData().DESCRIPTION;
globalThis.itemBx = cell.getRow().getData().BX;
globalThis.itemCs = cell.getRow().getData().CS;
globalThis.itemUom = cell.getRow().getData().UOM;
globalThis.itemCost = cell.getRow().getData().COST;
globalThis.itemBox = cell.getRow().getData().BOX;
globalThis.itemHalf = cell.getRow().getData().HALF;
globalThis.itemLess = cell.getRow().getData().LESS;
globalThis.itemCase = cell.getRow().getData().CASE;
globalThis.itemBxWt = cell.getRow().getData().BXWT;
globalThis.itemCsWt = cell.getRow().getData().CSWT;
// globalToLocal();
setItemPrice();
},
CODE FROM "quote.js"
function setItemPrice() {
console.log(globalThis.itemUom);
var itemPrice;
if ((globalThis.itemUom = "EA")) {
itemPrice = globalThis.itemBox;
} else {
itemPrice = globalThis.itemCase;
}
console.log(itemPrice);
}
after another 3 hours or so of searching I found an answer that works:
setting variable with ternary expression
I'm trying to create "Sale Rep" summaries by "Shop", where I can simply filter a column by the rep's name, them populate a total sales for each shop next to the relevant filter result.
I'm using this to filter all the Stores by Scott:
=(filter(D25:D47,A25:A47 = "Scott"))
Next, want to associate the Store/Account in F to populate with the corresponding value of E inside of G. So, G25 should populate the value of E25 ($724), G26 with E26 ($822), and F27 with E38 ($511.50)
I don't know how to write the formula correctly, but something like this is what I'm trying to do: =IF(F25=D25:D38),E25 I know that's not right, and it won't work in a fill down. But I'm basically trying to look for and copy over the correct value match of D and E inside of G. So, Misty Mountain Medicince in F27 will be matched to the value of E38 and populated in G27.
The filter is what's throwing me off, because it's not a simple fill down. And I don't know how to match filtered results from one column to a matched value in another.
Hope the screenshot helps. Screenshot of table:
Change Field Rep: Scott to Scott and you might apply:
=query(A25:E38,"select D,E where A='"&F24&"'")
// Enter the following into G25 and copy down column G
=(filter(E25:E47, D25:D47 = F25))
or
// Enter the following into G25 will expand with content in F upto row 47
=ArrayFormula(IF(F25:F47 <> 0, VLOOKUP(F25:F47, D25:E47, 2, FALSE),))
This is a follow-up to this question:
flatMap and `Ambiguous reference to member` error
There I am using the following code to convert an array of Records to an array of Persons:
let records = // load file from bundle
let persons = records.flatMap(Person.init)
Since this conversion can take some time for big files, I would like to monitor an index to feed into a progress indicator.
Is this possible with this flatMap construction? One possibility I thought of would be to send a notification in the init function, but I am thinking counting the records is also possible from within flatMap ?
Yup! Use enumerated().
let records = // load file from bundle
let persons = records.enumerated().flatMap { index, record in
print(index)
return Person(record)
}
I am trying to capture image url's from inside tweets.
REGISTER 'hdfs:///user/cloudera/elephant-bird-pig-4.1.jar';
REGISTER 'hdfs:///user/cloudera/elephant-bird-core-4.1.jar';
REGISTER 'hdfs:///user/cloudera/elephant-bird-hadoop-compat-4.1.jar';
--Load Json
loadJson = LOAD '/user/cloudera/tweetwall' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad') AS (json:map []);
B = FOREACH loadJson GENERATE flatten(json#'tweets') as (m:map[]);
tweetText = FOREACH B GENERATE FLATTEN(m#'text') as (str:chararray);
intermediate date looks like this:
(#somenameontwitter your nan makes me laugh with some of the things she comes out with like http://somepics.com/my.jpg)
then I try to do the following to get only the image url back :
x = foreach tweetText generate REGEX_EXTRACT_ALL(str, '((http)(.*)(.jpg|.bmp|.png))');
dump x;
but that doesn't seem to work. I have also been trying with filter to no avail.
Even when trying the above with .* it returns empty results () or (())
I'm not good with regex and pretty new to Pig so it could be that I'm missing something simple here that I'm just not seeing.
update
example input data
{"tweets":[{"created_at":"Sat Nov 01 23:15:45 +0000 2014","id":5286804225,"id_str":"5286864225","text":"#Beace_ your nan makes me laugh with some of the things she comes out with blabla http://t.co/b7hjMWNg is an url, but not a valid one http://www.something.com/this.jpg should be a valid url","source":"\u003ca href=\"http:\/\/twitter.com\/download\/iphone\" rel=\"nofollow\"\u003eTwitter for iPhone\u003c\/a\u003e","truncated":false,"in_reply_to_status_id":52812992878592,"in_reply_to_status_id_str":"522","in_reply_to_user_id":398098,"in_reply_to_user_id_str":"3","in_reply_to_screen_name":"Be_","user":{"id":425,"id_str":"42433395","name":"SAINS","screen_name":"sa3","location":"Lincoln","profile_location":null,"description":"","url":null,"entities":{"description":{"urls":[]}},"protected":false,"followers_count":92,"friends_count":526,"listed_count":0,"created_at":"Mon May 25 16:18:05 +0000 2009","favourites_count":6,"utc_offset":0,"time_zone":"London","geo_enabled":true,"verified":false,"statuses_count":19,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"EDECE9","profile_background_image_url":"http:\/\/abs.twimg.com\/images\/themes\/theme3\/bg.gif","profile_background_image_url_https":"https:\/\/abs.twimg.com\/images\/themes\/theme3\/bg.gif","profile_background_tile":false,"profile_image_url":"http:\/\/pbs.twimg.com\/profile_images\/52016\/DGDCj67z_normal.jpeg","profile_image_url_https":"https:\/\/pbs.twimg.com\/profile_images\/526\/DGDCj67z_normal.jpeg","profile_banner_url":"https:\/\/pbs.twimg.com\/profile_banners\/424395\/13743515","profile_link_color":"088253","profile_sidebar_border_color":"D3D2CF","profile_sidebar_fill_color":"E3E2DE","profile_text_color":"634047","profile_use_background_image":true,"default_profile":false,"default_profile_image":false,"following":false,"follow_request_sent":false,"notifications":false},"geo":null,"coordinates":null,"place":null,"contributors":null,"retweet_count":0,"favorite_count":1,"entities":{"hashtags":[],"symbols":[],"user_mentions":[{"screen_name":"e_","name":"\u2601\ufe0f effy","id":3998,"id_str":"398","indices":[0,15]}],"urls":[]},"favorited":false,"retweeted":false,"lang":"en"}]}
Try this and let me know if this works
x = foreach tweetText generate REGEX_EXTRACT(str,'.*(http://.*.[jpg|bmp|png])',1);
DUMP x;
I managed to get it working (though I doubt it is totally optimal)
x = foreach tweetText generate REGEX_EXTRACT(str,'(http://.*(.jpg|.bmp|.png))',1) as image;
filtered = FILTER x BY $0 is not null;
dump filtered;
so the initial problem was just the regex (and my lack of knowledge on the subject).
Thanks for the assistance sivasakthi jayaraman!
I have a list of pk's and I would like to get the result in the same order that my list is defined... But the order of the elements is begging changed. How any one help me?
print list_ids
[31189, 31191, 31327, 31406, 31352, 31395, 31309, 30071, 31434, 31435]
obj_opor=Opor.objects.in_bulk(list_ids).values()
for o in obj_oportunidades:
print o
31395 31435 31434 30071 31309 31406 31189 31191 31352 31327
This object should be used in template to show some results to the user... But how you can see, the order is different from the original list_ids
Would have been nice to have this feature in SQL - sorting by a known list of values.
Instead, what you could do is:
obj_oportunidades=Opor.objects.in_bulk(list_ids).values()
all_opor = []
for o in obj_oportunidades:
print o
all_opor.append(o)
for i in list_ids:
if i in all_opor:
print all_opor.index(i)
Downside is that you have to get all the result rows first and store them before getting them in the order you want. (all_opor could be a dictionary above, with the table records stored in the values and the PKeys as dict keys.)
Other way, create a temp table with (Sort_Order, Pkey) and add that to the query:
Sort_Order PKey
1 31189
2 31191
...
So when you sort on Sort_Order and Opor.objects, you'll get Pkeys it in the order you specify.
I found a solution in: http://davedash.com/2010/02/11/retrieving-elements-in-a-specific-order-in-django-and-mysql/ it's suited me perfectly.
ids = [a_list, of, ordered, ids]
addons = Addon.objects.filter(id__in=ids).extra(
select={'manual': 'FIELD(id,%s)' % ','.join(map(str,ids))},
order_by=['manual'])
This code do something similiar to MySQL "ORDER BY FIELD".
This guy: http://blog.mathieu-leplatre.info/django-create-a-queryset-from-a-list-preserving-order.html
Solved the problem for both MySQL and PostgreSQL!
If you are using PostgreSQL go to that page.