I'm trying to implement a simple linear regression with ML.NET. I have followed Microsoft's tutorial, except that I'm loading data from an IEnumerable rather than from a file. However, I always get zero as a prediction. Investigating back, I found that the model's parameters are empty (see screenshot below).
This is the simplest program with which I was able to reproduce the problem.
class Data
{
public float Revenue { get; set; }
public float Customers { get; set; }
public float IsWeekend { get; set; }
}
class DataPrediction
{
[ColumnName("Score")]
public float RevenueEstimate { get; set; }
}
static async Task Main(string[] args)
{
List<Data> trainData = new List<Data>();
trainData.Add(new Data { IsWeekend = 0, Customers = 20, Revenue = 138 });
trainData.Add(new Data { IsWeekend = 0, Customers = 18, Revenue = 106 });
trainData.Add(new Data { IsWeekend = 0, Customers = 26, Revenue = 142 });
trainData.Add(new Data { IsWeekend = 0, Customers = 21, Revenue = 131 });
trainData.Add(new Data { IsWeekend = 0, Customers = 33, Revenue = 146 });
trainData.Add(new Data { IsWeekend = 1, Customers = 92, Revenue = 287 });
trainData.Add(new Data { IsWeekend = 1, Customers = 113, Revenue = 312 });
List<Data> testData = new List<Data>();
testData.Add(new Data { IsWeekend = 0, Customers = 27, Revenue = 136 });
testData.Add(new Data { IsWeekend = 0, Customers = 22, Revenue = 109 });
testData.Add(new Data { IsWeekend = 1, Customers = 87, Revenue = 256 });
MLContext mlContext = new MLContext();
IDataView trainDataView = mlContext.Data.LoadFromEnumerable(trainData);
IDataView testDataView = mlContext.Data.LoadFromEnumerable(testData);
var pipeline = mlContext.Transforms.CopyColumns("Label", "Revenue")
.Append(mlContext.Transforms.Concatenate("Features", "Customers", "IsWeekend"))
.Append(mlContext.Regression.Trainers.FastTree())
;
var model = pipeline.Fit(trainDataView);
var predictions = model.Transform(testDataView);
var metrics = mlContext.Regression.Evaluate(predictions, "Label", "Score");
var predictionFunction = mlContext.Model.CreatePredictionEngine<Data, DataPrediction>(model);
var prediction = predictionFunction.Predict(new Data { IsWeekend = 1, Customers = 108 });
}
Here is what I get if I break at the closing brace:
Of course, prediction.RevenueEstimate is 0, and the metrics show an average absolute error and a RMS error that are similar to the label's average.
It seems to me that this is a very essential program, and I'm using the FastTree LR trainer just like Microsoft suggests in its tutorial. What am I doing wrong?
PS: Decorating the data classes with ColumnName attributes has no effect.
Related
Hello! I am unable to test the written apex class.. Could you help me with this task?
Through the component, we transfer the ID of the CSV file sent to the server to the apex class, after which our apex class creates stations if there were none before, and also updates the list of sensors based on the records of the CSV file (insert/ update).
Controller Class
public inherited sharing class lwnReadCsvFileController {
#AuraEnabled
public static list<Sensor__c> readCSVFile(Id idContentDocument){
list<Sensor__c> lstSensToInsert = new list<Sensor__c>();
if(idContentDocument != null) {
// getting File Data based on document id
ContentVersion objVersion = [SELECT Id, VersionData FROM ContentVersion WHERE ContentDocumentId =:idContentDocument];
// split the file data
list<String> lstCSVLines = objVersion.VersionData.toString().split('\n');
//creating Basic Stations records
List<Base_Station__c> createdBSRec = [SELECT Name, Id From Base_Station__c];
List<Base_Station__c> newBSRec = new List<Base_Station__c>();
Set<String> uniqueSet = new Set<String>();
for ( Integer i = 1; i < lstCSVLines.size(); i++ ) {
integer coincidences = 0;
list<String> csvRowData = lstCSVLines[i].split(',');
for ( Base_Station__c rec : createdBSRec ) {
if (csvRowData[0] == rec.Name) {
coincidences++;
}
}
if ( coincidences == 0 ) {
uniqueSet.add(csvRowData[0]);
}
}
List<String> uniquelist = new List<String>(uniqueSet);
for (integer i = 0; i < uniquelist.size() ; i++) {
Base_Station__c newBS = new Base_Station__c(Name = uniquelist[i]);
NewBSRec.add(newBS);
}
upsert newBSRec;
//creating Sensor records
for(Integer i = 1; i < lstCSVLines.size(); i++){
Sensor__c objRec = new Sensor__c();
list<String> csvRowData = lstCSVLines[i].split(',');
string tempBase = csvRowData[0];
objRec.Name = csvRowData[1];
objRec.Base_Station__c = [SELECT Id From Base_Station__c Where Name = :tempBase ].id;
objRec.Sensor_ID__c = integer.valueof(csvRowData[1]);
objRec.Status__c = csvRowData[2];
objRec.Sensor_Model__c = csvRowData[3];
lstSensToInsert.add(objRec);
}
if(!lstSensToInsert.isEmpty()) {
List<Sensor__c> createdSenRec = [SELECT Name, Sensor_ID__c From Sensor__c];
for (Sensor__c sens : lstSensToInsert ) {
integer coincidences = 0;
for (Sensor__c rec : createdSenRec ) {
if (sens.Sensor_ID__c == rec.Sensor_ID__c) {
sens.id = rec.id;
update sens;
coincidences++;
}
}
if ( coincidences == 0 ) {
insert sens;
}
}
}
}
return lstSensToInsert;
}
}
Test Class 100% coverage
#isTest
public class IwnReadCsvFileControllerTest {
public static String str = 'BASE STATION,SENSOR ID,STATUS,SENSOR MODEL \n' +
'Leeds,1,Enabled ,R8 \n' +
'Glasgow Central,2,Enabled,R8';
#isTest
public static void testReadCSVFile(){
Base_Station__c newBS = new Base_Station__c(Name = 'Leeds');
upsert newBS;
Sensor__c newSensor = new Sensor__c (Name = '1', Sensor_Id__c = 1, Status__c = 'Enabled', Base_Station__c = newBs.id);
insert newSensor;
ContentVersion contentVersionInsert = new ContentVersion(
Title = 'Test',
PathOnClient = 'Test.csv',
VersionData = Blob.valueOf(str),
IsMajorVersion = true
);
insert contentVersionInsert;
Id getId = [Select ContentDocumentId From ContentVersion Where Id =:contentVersionInsert.id and isLatest=true].ContentDocumentId;
List<Sensor__c> result = lwnReadCsvFileController.readCSVFile(getId);
}
}
Your unit test class is passing a ContentVersion Id:
List<Sensor__c> result = lwnReadCsvFileController.readCSVFile(contentVersionInsert.Id);
but your class is treating this Id as a ContentDocument Id:
public static list<Sensor__c> readCSVFile(Id idContentDocument){
if(idContentDocument != null) {
ContentVersion objVersion = [
SELECT Id, VersionData
FROM ContentVersion
WHERE ContentDocumentId =:idContentDocument
];
Your test class needs to query for the ContentDocumentId of the newly-inserted ContentVersion and pass that Id into your class under test.
I am trying to get some CloudWatch metrics using .NET SDK. The code that is seen below returns some data points but after returning about 20 data points it raises an Exception of InvalidNextToken.
private static void Main(string[] args)
{
Console.Clear();
var creds = new StoredProfileAWSCredentials();
var c = new AmazonCloudWatchClient(creds, RegionEndpoint.EUCentral1);
Task<GetMetricDataResponse> t = null;
string nextToken = null;
do
{
var req = new GetMetricDataRequest
{
EndTimeUtc = DateTime.UtcNow,
MaxDatapoints = 10,
StartTimeUtc = DateTime.UtcNow.AddHours(-1),
ScanBy = new ScanBy("TimestampDescending"),
NextToken = nextToken,
MetricDataQueries = new List<MetricDataQuery>
{
new MetricDataQuery
{
Id = "a" + Guid.NewGuid().ToString().Replace("-", ""),
MetricStat = new MetricStat
{
Stat = "Maximum",
Metric = new Metric
{
MetricName = "CPUUtilization",
Dimensions = new List<Dimension>
{
new Dimension
{
Name = "InstanceId",
Value = "i-04f27d16c91c70119"
}
},
Namespace = "AWS/EC2"
},
Period = 60,
Unit = StandardUnit.Percent
}
}
}
};
t = c.GetMetricDataAsync(req);
t.Wait();
var usage = t.Result;
if (usage.MetricDataResults.Any())
foreach (var r in usage.MetricDataResults)
foreach (var rValue in r.Values)
Console.WriteLine(Math.Round(rValue * 100));
nextToken = t.Result.NextToken;
} while (!string.IsNullOrEmpty(nextToken));
Console.ReadKey();
}
The exact Exception message is:
InvalidNextTokenException: The service returned an error with Error
Code InvalidNextToken and HTTP Body:
Sender
InvalidNextToken 795050a1-3bcd-4a44-9794-5becd0c4f5cf
For nextToken to work, you need to send the exact same request, with only the token changing between calls.
You have id, StartTimeUtc and EndTimeUtc changing between requests.
Initializing them before the loop should fix your problem.
Try something like this:
private static void Main(string[] args)
{
Console.Clear();
var creds = new StoredProfileAWSCredentials();
var c = new AmazonCloudWatchClient(creds, RegionEndpoint.EUCentral1);
Task<GetMetricDataResponse> t = null;
string nextToken = null;
var endTime = DateTime.UtcNow;
var startTime = endTime.AddHours(-1);
var id = "a" + Guid.NewGuid().ToString().Replace("-", "");
do
{
var req = new GetMetricDataRequest
{
EndTimeUtc = endTime,
MaxDatapoints = 10,
StartTimeUtc = startTime,
ScanBy = new ScanBy("TimestampDescending"),
NextToken = nextToken,
MetricDataQueries = new List<MetricDataQuery>
{
new MetricDataQuery
{
Id = id,
MetricStat = new MetricStat
{
Stat = "Maximum",
Metric = new Metric
{
MetricName = "CPUUtilization",
Dimensions = new List<Dimension>
{
new Dimension
{
Name = "InstanceId",
Value = "i-04f27d16c91c70119"
}
},
Namespace = "AWS/EC2"
},
Period = 60,
Unit = StandardUnit.Percent
}
}
}
};
t = c.GetMetricDataAsync(req);
t.Wait();
var usage = t.Result;
if (usage.MetricDataResults.Any())
foreach (var r in usage.MetricDataResults)
foreach (var rValue in r.Values)
Console.WriteLine(Math.Round(rValue * 100));
nextToken = t.Result.NextToken;
} while (!string.IsNullOrEmpty(nextToken));
Console.ReadKey();
}
I have some data in a csv file:
Survived Pclass Sex Age
0 3 male 22
1 1 female 38
1 3 male 26
1 1 female 35
...
I loaded the data using:
context.Data.LoadFromTextFile(path: dataPath,...);
Once I loaded the data, I need to add calculated column say, AgeName, so that the:
if (Age < 18)
AgeName ="Child"
else if(Age < 55)
AgeNAme = "Man"
else
AgeNAme = "Grandpa"
Is there builtin method in the ML.NET in order to add the column, or do I need to implement it manually?
I think you would want to use the CustomMapping transform.
Below is a sample. First, some input and output classes:
class InputData
{
public int Age { get; set; }
}
class CustomMappingOutput
{
public string AgeName { get; set; }
}
class TransformedData
{
public int Age { get; set; }
public string AgeName { get; set; }
}
Then, in the ML.NET program:
MLContext mlContext = new MLContext();
var samples = new List<InputData>
{
new InputData { Age = 16 },
new InputData { Age = 35 },
new InputData { Age = 60 },
new InputData { Age = 28 },
};
var data = mlContext.Data.LoadFromEnumerable(samples);
Action<InputData, CustomMappingOutput> mapping =
(input, output) =>
{
if (input.Age < 18)
{
output.AgeName = "Child";
}
else if (input.Age < 55)
{
output.AgeName = "Man";
}
else
{
output.AgeName = "Grandpa";
}
};
var pipeline = mlContext.Transforms.CustomMapping(mapping, contractName: null);
var transformer = pipeline.Fit(data);
var transformedData = transformer.Transform(data);
var dataEnumerable = mlContext.Data.CreateEnumerable<TransformedData>(transformedData, reuseRowObject: true);
foreach (var row in dataEnumerable)
{
Console.WriteLine($"{row.Age}\t {row.AgeName}");
}
I am testing one of my services - movieService. GetLatestMovies method should return all movies, ordered by date. I am using automapper to map the entity Movie to MovieViewModel.
* Question 1: * How am I supposed to test that, should I set the return collection in the test or what?
* Question 2: * I am filling the InMemory database with a few movies and I am expecting correct ordered result from the movieService, how am I supposed to check, If the service is returning correct result, If I set the return from the automapper?
TestUtils.FillContextWithActorsMoviesAndGenres(options) - just fills the context with a few movies.
This is the movieService method I am testing
public async Task<ICollection<MovieViewModel>> GetLatestMoviesAsync()
{
var movies = await this.context.Movies
.Include(um => um.ApplicationUserMovie)
.ThenInclude(u => u.User)
.Include(x => x.Genre)
.Include(x => x.MovieActor)
.ThenInclude(x => x.Actor)
.OrderByDescending(x => x.CreatedOn).ToListAsync();
var returnMovies = this.mappingProvider.MapTo<ICollection<MovieViewModel>>(movies);
return returnMovies;
}
[TestMethod]
public async Task Return_TwoMoviesWithHighestRating()
{
var dabataseName = nameof(Return_TwoMoviesWithHighestRating);
var options = TestUtils.GetOptions(dabataseName);
// We fill the context with data and save it.
TestUtils.FillContextWithActorsMoviesAndGenres(options);
var movieOne = new MovieViewModel()
{
Name = "BestRatedMovieTest",
Duration = 90,
Director = "TestDirector",
Storyline = "TestStoryline",
ImageUrl = "TestImageUrl",
Genre = "Comedy"
};
var movieTwo = new MovieViewModel()
{
Name = "SecondMovieTestName",
Duration = 90,
Director = "TestDirector",
Storyline = "TestStoryline",
ImageUrl = "TestImageUrl",
Genre = "Comedy"
};
var collectionMovieViewModels = new List<MovieViewModel>() { movieOne, movieTwo };
var mappingProviderMock = new Mock<IMappingProvider>();
mappingProviderMock
.Setup(x => x.MapTo<ICollection<MovieViewModel>>(It.IsAny<List<Movie>>()))
.Returns(collectionMovieViewModels);
using (var actAndAssertContext = new MovieManagementContext(options))
{
var sut = new MovieService(actAndAssertContext, mappingProviderMock.Object);
var movies = await sut.GetLatestMoviesAsync();
Assert.AreEqual(2, movies.Count());
Assert.AreEqual("BestRatedMovieTest", movies.FirstOrDefault().Name);
}
}
I created an empty collection and set the Callback of the method to fill that collection,
var collectionOfMovies = new List<Movie>();
var mappingProviderMock = new Mock<IMappingProvider>();
mappingProviderMock
.Setup(x => x.MapTo<ICollection<MovieViewModel>>(It.IsAny<List<Movie>>()))
.Callback<object>(inputargs => collectionOfMovies = inputargs as List<Movie>);
I need to display a different list each time I click on different links in my view. Help would be appreciated :)
My controller:
public class HomeController : Controller
{
Teams tm = new Teams();
Details det = new Details();
public ActionResult Index()
{
var model = new List<Teams>();
model.Add(new Teams { Name = "Manchester United", NickName = "The Red Devils", HomeGround = "Old Trafford", Founded = 1878 });
model.Add(new Teams { Name = "Liverpool", NickName = "The reds", HomeGround = "Anfield", Founded = 1870 });
return View(model);
}
public ActionResult About()
{
var title = new List<Details>();
title.Add(new Details { MajorHonours = 62, PremLeague = 20, FACup = 11, LeagueCup = 4, UEFA = 3 });
title.Add(new Details { MajorHonours = 60, PremLeague = 18, FACup = 7, LeagueCup = 8, UEFA = 5 });
return View();
}
My view with the links:
#model IEnumerable<Standings.Models.Teams>
#{
ViewBag.Title = "Standings";
}
<h1>List of teams</h1>
#foreach (var item in Model)
{
<div>
#Html.ActionLink(#item.Name, "About") (#item.NickName, #item.HomeGround, #item.Founded)
<hr />
</div>
}
My model:
public class Details
{
public int MajorHonours { get; set; }
public int PremLeague { get; set; }
public int FACup { get; set; }
public int LeagueCup { get; set; }
public int UEFA { get; set; }
}
And I have a clean View with the name About that the list needs to be displayed on