running out of memory plotting with Google Charts and very sparse data - google-visualization

We have very sparse data that we are attempting to plot with Google Charts. There are 16 different vectors and each has about 12,000 points. The points are times. All of the times are different. My reading of the API is that I need to create a row where each element corresponds to a different vector. So that's a set of 192,000, where the first element in each row is the time and all of the other elements are null except for the one that has data there, for a total of 3,072,000 elements. When we give this to Google Charts, the browser dies.
The problem with using arrayToDataTable is that our array is sparse. Likewise, arrayToDataTable doesn't work.
My question: is there a more efficient way to do this? Can I plot each data value independently, rather than all at the same time?

It turns out that the answer to this question is to do server-side data reduction in the form of binning. The individual rows each have their own timestamp, but because we are displaying this in a graph with a width of at most 2000 pixel, it makes sense to bin on the server into 2000 individual rows, each one with 16 columns. Then the total array is 32,000 elements, which appears well within the limits of the browser.

Related

PowerBI - Show lines on a map from one point to another

We got several OLAP Cubes in PowerBI Datasets.
One of the cubes has a dimension "dim_location" which contains columns for latitude and longitude. But each dataset has 2 pairs of values, let's call them start_latitude, start_longitude and end_latitude, end_longitude.
I got a fact table connected to that dim_location and want to show some of the measures on a map.
It works perfectly fine with both the map visual and the ArcGIS visual, if I use either the end or the start coordinates. I can show the values as circles with changing size or changing color dependent on the value of a measure. So far so good.
But what I instead want to accomplish is to show a line on the map for each dataset. Each line shall go from start point to end point, color dependent on measure value.
Is there a way to offer the coordinates in the cube dimension in some string syntax that will create a shape, like a polygon with only 2 points, which would result in a line, which can then be shown on the map?
As stated before everything works fine on the map and the ArcGIS visual with one point (lat/lon) per dataset. Tried to find help online for some polygon syntax but came up empty.

Is it possible to set the number of rows in a Power BI Small Multiples grid to a variable?

I want a visual comparison of stacked histograms using the same x-axis scale. In this situation, I don't always have the same number of histograms for each dataset.
I would like to do a small multiples grid in Power BI that is N rows by 1 column; where N is the number of instruments that are collecting data for the client site selected in a Slicer. Some clients may be collecting data with one instrument. Other clients may use 3 or more instruments.
Is there a way to set the number of rows to a variable in a small multiples grid or would it be possible to embed a chart in a matrix?
The default is a 2×2 grid of small multiples, but you can adjust the number of rows and columns to up to 6×6. Any multiples that don’t fit on that grid will load in as you scroll down.
You can adjust the style and position of the small multiple titles in the Small multiple title card:
And you can change the dimensions of the grid in the Grid layout card:

QT infinite view on model

I am looking for a way to create an infinite view on a model that is not initialized completely. I would like to create something similar to an Excel spreadsheet, and all I came in was to start with an initialized model (e.g. 100x100 empty cells, maybe working on a database that has empty values), and then just dynamically add next rows/columns (and update view) once we are close to the end of a scrollbar.
But I am wondering if it is the best solution - I think I would definitively benefit from a model that's filled in only partially - by that, I mean store information in the model only about filled cells, and let view handle showing 'empty cells' (which would have been created once we - for example - click them).
I know it would be necessary to store XY positions and cell data (instead of only a 2D container with data), but I would like to try different solutions:
a) have a pointer-like container which would contain a list of filled cells with their positions on a 2D plane
b) have a 2D container with size (x,y), where x and y would mean the 'last filled cell' in a given dimension
And for both solutions, I would like to dynamically allocate more place once data is written.
So there is my question - how can it be achieved with QT model/view programming, if it is even possible to show 'ghost cells' without a model filled with empty data? It would be also nice if I could get a brief explanation of how it is done in apps like excel etc.
Well, your table will never be truly infinite unless you implement some indexing with numbers with infinite digit count and in that case, you will probably not be able to use Qt classes.
But I think you should choose some big enough number to define the maximum. It can be a really large number... if you are on a 64-bit machine, then your 'infinite' table can have 9,223,372,036,854,775,807 rows and the same number of columns. This big number happens to be the maximum of signed 64-bit int. And int is used for indexing with QModelIndex in QAbstractItemModel. So you can have a total of 8.5070592e+37 cells in your two-dimensional 'Excel' table. If this table size is not big enough for you then I do not know what is. Just for comparison, there are approximately 7e+27 atoms in the average human body, maybe a bit more after the covid lockdowns because people were eating instead of practicing sports. :) So you can count all atoms of all people on this planet (say there are a bit less than 10e+10 people altogether). But you will need to buy a bit bigger computer for this task.
So if you decide to go this way, then you can easily override QAbstractTableModel and display it in QTableView. Of course, you cannot save the underlying data in a two-dimensional array because you do not have enough memory. But you have to choose some other method. For example a QHash<QPoint, QString> where QPoint will represent the coordinates and QString the value (you can choose any other type instead of a string of course). Then when you will want to get the value for the given coordinates, you just look up the value in the hash table. The number of data points you will be able to hold depends only on your memory size. This solution is very simple, I guess it will be some 30 rows of code, not more.

3D lookup table to discretize the volume

I have a depth camera that returns measured distance values of the volume in millimeters. It's needed to create a 3D lookup table to store all possible distance values for each pixel in the image. So I am getting an array of the size 640x480x2048. This approach is very memory consuming and if I use integers in C++ it takes about 2.5 GB of RAM. Additionally, I also have some parameters for each item in the volume, so all together it reaches maximum capacity of my 4GB memory.
My question is: Is there any good experience how I can optimally store and manage above described data set?
P.S Please don't consider the option of file storage. It doesn't fit me.
Thanks in advance

How do I specify multiple data sets to an XY-scatter plot using the Google Chart API?

Why doesn't this Google Chart API URL render both data sets on this XY scatter plot?
http://chart.apis.google.com/chart?cht=lxy&chd=t:10,20,30,40,50,60,70,80,90,100,110,120,130,140,150,160,170,180,190,200|0.10,0.23,0.33,0.44,0.56,0.66,0.79,0.90,0.99,1.12,1.22,1.33,1.44,1.56,1.68,1.79,1.90,2.02,2.12,2.22|0.28,0.56,0.85,1.12,1.42,1.68,1.97,2.26,2.54,2.84,3.12,3.40,3.84,4.10,4.53,4.80,5.45,6.02,6.40,6.80&chco=3072F3,ff0000,00aaaa&chls=2,4,1&chs=320x240&chds=0,201,0,7&chm=s,FF0000,0,-1,5|s,0000ff,1,-1,5|s,00aa00,2,-1,5
I've read the documentation over and over again, and I can't figure it out.
First a point of clarification. You talk about a "XY scatter plot", but these are actually 2 distinct chart types in the Google Chart API. Your URL refers to cht=lxy parameter which is an XY line chart.
The first problem with your URL is your data parameter (chd). Since it is an XY line chart, data sets must be defined in pairs but I see an odd number of data sets (3).
Christian D's response is incorrect. There is no percentage requirement.
You may be better off using a wrapper API which abstracts away many of these ugly details.
I think it actually does render both data sets, but you can only se one of them because there's only one scale on the y axis. (In other words, 0.10 is too small to show.)
And, you should really be using percentages. 100 is the highest accepted value:
Where chart data string consists of positive floating point numbers from zero (0.0) to one hundred (100.0)