I am working through the Adventure Works data mining examples on the Microsoft website. In it, we are going to train a model using all sales data globally, then use the data for a region and bike model as inputs. Wouldn't this just predict incorrectly, ignoring specific trends for that area for that bike model?
What would be the advantage of doing this?
I think the idea that is that, when developing a learner, global data encompasses regional data. If you're building some sort of classifier and hope to run it at a regional level, you only need to use the regional-specific data, no?
Every model needs to be trained with relevant data.
The confusing part is that perhaps I'm not understanding the differential of "regional" data. Ultimately, the global data should definitely be relevant to your predictive model.
Related
I have created a new prediction model based on a dataset that was given to me. It predicts a nominal (binary) class attribute (positive/negative) based on a number of numerical attributes.
Now I have been asked to use this prediction model to predict classes for a new dataset. This dataset has all the same attributes except for the class column, which does not exist yet. How do I apply my model to this new data? I have tried adding an empty class column to my new dataset and then doing the following:
Simply loading the new dataset in WEKA's explorer and loading the model. It tells me there is no training data.
Opening my training set in WEKA's explorer and then opening my training model, then choosing my new data as a 'supplied test set'. It runs but does not output any predictions.
I should note that the model works fine when testing on the training data for cross validation. It also works fine with a subset of the training data I separated ages ago for test/eval use. I think it may be a problem with how I am adding a new class column, maybe?
For making predictions, Weka requires the two datasets, training and the one for making predictions, to have the exact same structure, down to the order of labels. That also means, that you need to have a class attribute with the correct labels present. In terms of values for your class attribute, simply use the missing value (denoted by a question mark).
See the FAQ How do i make predictions with a trained model? on the Weka wiki for more information on how to make predictions.
Let's say I have some text data that has already been labeled in SageMaker. This data could have either been labeled by humans or an ner model. Then let's say I want to have a human go back over the dataset, either to label new entity class or correct existing labels. How would I set up a labeling job to allow this? I tried using an output manifest from another labeling job, but all of the documents that were already labeled cannot be accessed by workers to re-label.
Yes, this is possible you are looking for Custom Labelling worklflows you can also apply either Majority Voting (MV) or MDS to evaluate the accuracy of the job
I want to detect only person detection by using YOLOv3. it detecting well. but it also searching and matching for other objects. That's why it takes some more time to detect person.
Now I want to create my own data set based on person. I have downloaded 85 thousand cropped images from Google API http://www.rapdataset.com/
But for train these images, it need to be labeled and annotated. thousands of images, how can I do? it takes much time to annotate. Is there any way to annotate these images programmatically like in python?
You can try this one to help you, or you have the alternative to use the already annotated set of COCO, and choose persons only of it, like in this question.
So I am a newbie to Google BigQuery ML and was wondering if the auto.arima automatically makes my time series data stationary ?
Suppose, I have a data that is not stationary and if I give the data as is to the auto arima model using Google BigQuery ML, will it first makes my data stationary before taking it as input ?
but that's part of the modeling procedure
From the documentation that explains What's inside a BigQuery ML time series model, it does appear that auto.ARIMA will make the data stationary.
However, I would not expect it to alter the source data table; it won't make that stationary, but in the course of building candidate models it may alter the input data prior to actual model fitting (transforms; e.g. box-cox, make stationary, etc.)
Imagine an applications that displays data from a sqlite database.
The app is making use of model/view programming.
It can have multiple views acting in parallel on different subsets of the same data (subsets made by filtering the required data types).
(Sidenote: I am using Qt, so there is no controller part, of course, but I did not find a more suitable tag.)
I am not sure which approach to take:
1a. Load all database data into one single model
1b. Then apply the model to all views, filtering the data inside the view with a proxy model
2a. One model for each view, but filtering done inside sqlite database.
Pros/Cons:
Idea 1:
(+) one model, makes use of model/view advantages (e.g. updating all connected views)
(-) memory usage could get huge because all data is loaded into a model, but only a subset is shown
Idea 2:
(+) theoeretically lower memory usage because only the filtered data is loaded from the database
(-) the views can have filters that could lead to intersecting data, meaning the same data would be stored in more than one model -> perhaps practically even bigger memory usage than in Idea 1
The data being loaded here is just case metadata, e.g. title, description, datetime and so on. Bigger data like images, files are not being loaded here. So as the database could indeed grow big (big for this kind of application, say 200 gb for power users), this does not affect the topic of the present question, because the metadata is much, much smaller and is proportional to overall data count, not data size.
Do you have practical experience with such a configuration and can suggest which one to use? It seems to me that Idea 1 is the way to go, but I am not sure about it.
In my experience, the less data is loaded from the database into memory, the better. It is not just the memory usage, but also startup time. If the data is delivered over the network, loading a few gigabytes can take forever.
So I would go for a variant of your second solution, where each table view has its own model. The model is an implementation of QAbstractItemModel that lazily fetches only the rows that currently need to be displayed. The models could, however, share a common cache. This will also make sure that they all display the same data where it intersects.