How to add calculated column into IDataView, and then included as a feature - ml.net

I have some data in a csv file:
Survived Pclass Sex Age
0 3 male 22
1 1 female 38
1 3 male 26
1 1 female 35
...
I loaded the data using:
context.Data.LoadFromTextFile(path: dataPath,...);
Once I loaded the data, I need to add calculated column say, AgeName, so that the:
if (Age < 18)
AgeName ="Child"
else if(Age < 55)
AgeNAme = "Man"
else
AgeNAme = "Grandpa"
Is there builtin method in the ML.NET in order to add the column, or do I need to implement it manually?

I think you would want to use the CustomMapping transform.
Below is a sample. First, some input and output classes:
class InputData
{
public int Age { get; set; }
}
class CustomMappingOutput
{
public string AgeName { get; set; }
}
class TransformedData
{
public int Age { get; set; }
public string AgeName { get; set; }
}
Then, in the ML.NET program:
MLContext mlContext = new MLContext();
var samples = new List<InputData>
{
new InputData { Age = 16 },
new InputData { Age = 35 },
new InputData { Age = 60 },
new InputData { Age = 28 },
};
var data = mlContext.Data.LoadFromEnumerable(samples);
Action<InputData, CustomMappingOutput> mapping =
(input, output) =>
{
if (input.Age < 18)
{
output.AgeName = "Child";
}
else if (input.Age < 55)
{
output.AgeName = "Man";
}
else
{
output.AgeName = "Grandpa";
}
};
var pipeline = mlContext.Transforms.CustomMapping(mapping, contractName: null);
var transformer = pipeline.Fit(data);
var transformedData = transformer.Transform(data);
var dataEnumerable = mlContext.Data.CreateEnumerable<TransformedData>(transformedData, reuseRowObject: true);
foreach (var row in dataEnumerable)
{
Console.WriteLine($"{row.Age}\t {row.AgeName}");
}

Related

wpf DataGrid set itemsource List<object>

I have a problem with DataGrid in wpf
this is a class :
class Superviser
{
public long Id = 0;
public string name = "";
public string father = "";
public string code = "";
}
and this is a function that build a list of this class object
public List<Superviser> allSuperviser()
{
return db.tbPersons.Where(i => i.level == StaticsObject.isSuperviser).Select(x => new Superviser
{
Id = x.Id,
name = x.firstName,
father = x.father,
code = x.code,
}).ToList();
}
and I use this code to set this list in datagrid
dgvPerson.ItemsSource = classPerson.allSuperviser();
but when run program datagrid is empty !
tip : The list is not empty.
Where is the problem?
How do I display this list on DataGrid?
hello agian i solved it
I changed class to :
class Superviser
{
public long Id { get; set; }
public string name { get; set; }
public string father { get; set; }
public string code { get; set; }
public Superviser() { }
public Superviser(long Id, string name, string father, string code)
{
this.Id = Id;
this.name = name;
this.father = father;
this.code = code;
}
}
and change function to :
public List<Superviser> allSuperviser()
{
return db.tbPersons.Where(i => i.level == StaticsObject.isSuperviser).Select(x => new Superviser { Id = x.Id, name = x.firstName + " " + x.lastName, father = x.father, code = x.code }).ToList();
}
And the problem was fixed :)

Joining 2 lists with condition

I want to join 2 lists. Students and Persons.
Both Student and Person has id and a name variable, plus Student has another variable called isStudying. I want to join all Students to Person with the isStudying variable.
var persons = new Person { id = 1, name = "John" };
var persons = new Person { id = 2, name = "Ace" };
var persons = new Person { id = 3, name = "Mike" };
var persons = new Person { id = 4, name = "Yob" };
var persons = new Person { id = 5, name = "Ken" };
var students = new Student { id = 2, name = "Ace", isStudying=true };
var students = new Student { id = 3, name = "Mike", isStudying = true };
var students = new Student { id = 5, name = "Ken", isStudying = true };
persons.Addrange(students.where(student.id.contain(persons.id)));
Here is full solution implemented in linq, used inheritance
class Person
{
public int id { get; set; }
public string name { get; set; }
}
class Student : Person
{
public bool isStudying { get; set; }
}
void Main()
{
var person1 = new Person { id = 1, name = "John" };
var person2 = new Person { id = 2, name = "Ace" };
var person3 = new Person { id = 3, name = "Mike" };
var person4 = new Person { id = 4, name = "Yob" };
var person5 = new Person { id = 5, name = "Ken" };
var persons = new List<Person> { person1, person2, person3, person4, person5 };
var student1 = new Student { id = 2, name = "Ace", isStudying = true };
var student2 = new Student { id = 3, name = "Mike", isStudying = true };
var student3 = new Student { id = 5, name = "Ken", isStudying = true };
var students = new List<Student> { student1, student2, student3 };
var personsRes = persons.Where(y => students.Select(x => x.id).Contains(y.id));
Console.WriteLine(personsRes);
}
Try This:
public class Person
{
public int id { get; set; }
public string name { get; set; }
}
public class Student
{
public int id { get; set; }
public string name { get; set; }
public bool isStudying { get; set; }
}
var persons = new List<Person> { new Person { id = 1, name = "John" },
new Person { id = 2, name = "Ace" },
new Person { id = 3, name = "Mike"},
new Person { id = 4, name = "Yob" },
new Person { id = 5, name = "Ken" } };
var students = new List<Student> { new Student { id = 2, name = "Ace", isStudying = true },
new Student { id = 3, name = "Mike", isStudying = true },
new Student { id = 5, name = "Ken", isStudying = true } };
var allPersons = (from p in persons
join s in students on new { first = p.id } equals new { first = s.id } into sjoin
from slj in sjoin.DefaultIfEmpty()
select new
{
id = p.id,
name = p.name,
isStudying = (slj != null ? (slj.isStudying ? "TRUE" : "FALSE") : string.Empty)
}).ToList();
For future readers' reference, one simple answer which I provided in comments using LINQ is:
With anonymous types:
persons.Select(p => new { p.id, p.name, students.Any(s => s.id == p.id && s.isStudying) });
Using a custom class (the Student class can actually be re-used):
persons.Select(p => new Student { id = p.id, name = p.name, isStudying = students.Any(s => s.id == p.id && s.isStudying) });

Traverse n-level parent child list C#

I have a parent-child list. It can be of n-level.
I need each item within this hierarchy.
I am using 'foreach' loop and recursive function but it's not working.
I have went through other solutions at 'stackoverflow' but none of them worked for me.
public class Item
{
public int Id { get; set; }
public string Name { get; set; }
public int ParentId { get; set; }
private List<Item> children = new List<Item>();
public List<Item> Children
{
get { return children; }
set { children = value; }
}
}
List<Item> test = new List<Item>();
test.Add(new Item { Id = 1, Name = "Root1", ParentId = -1 });
test[0].Children.Add(new Item { Id = 4, Name = "child-1-Root1", ParentId = 1 });
test[0].Children[0].Children.Add(new Item { Id = 10, Name = "grandchild-1-Root1", ParentId = 4 });
test[0].Children.Add(new Item { Id = 5, Name = "child-2-Root1", ParentId = 1 });
test.Add(new Item { Id = 2, Name = "Root2", ParentId = -1 });
test[1].Children.Add(new Item { Id = 6, Name = "child-1-Root2", ParentId = 2 });
test[1].Children.Add(new Item { Id = 7, Name = "child-2-Root2", ParentId = 2 });
test.Add(new Item { Id = 3, Name = "Root3", ParentId = -1 });
test[2].Children.Add(new Item { Id = 8, Name = "child-1-Root3", ParentId = 3 });
test[2].Children.Add(new Item { Id = 9, Name = "child-2-Root3", ParentId = 3 });
The solution is:
public static Find()
{
foreach (var item in test)
{
FindRecursively(item, item.Name);
}
}
public static Item FindRecursively(Item node, string name)
{
if (node == null)
return null;
if (node.Name == name)
Console.WriteLine(node.Name);
foreach (var child in node.Children)
{
var found = FindRecursively(child, child.Name);
}
return null;
}

RavenDB MultiMapReduce Sum not returning the correct value

Sorry for this lengthy query, I decided to add the whole test so that it will be easier for even newbies to help me with this total brain-melt.
The using directives are:
using System.Collections.Generic;
using System.Linq;
using NUnit.Framework;
using Raven.Client;
using Raven.Client.Embedded;
using Raven.Client.Indexes;
Please leave feedback if I'm too lengthy, but what could possibly go wrong if I add a complete test?
[TestFixture]
public class ClicksByScoreAndCardTest
{
private IDocumentStore _documentStore;
[SetUp]
public void SetUp()
{
_documentStore = new EmbeddableDocumentStore {RunInMemory = true}.Initialize();
_documentStore.DatabaseCommands.DisableAllCaching();
IndexCreation.CreateIndexes(typeof (ClicksBySearchAndProductCode).Assembly, _documentStore);
}
[TearDown]
public void TearDown()
{
_documentStore.Dispose();
}
[Test]
public void ShouldCountTotalLeadsMatchingPreference()
{
var userFirst = new User {Id = "users/134"};
var userSecond = new User {Id = "users/135"};
var searchFirst = new Search(userFirst)
{
Id = "searches/24",
VisitId = "visits/63"
};
searchFirst.Result = new Result();
searchFirst.Result.Rows = new List<Row>(
new[]
{
new Row {ProductCode = "CreditCards/123", Score = 6},
new Row {ProductCode = "CreditCards/124", Score = 4}
});
var searchSecond = new Search(userSecond)
{
Id = "searches/25",
VisitId = "visits/64"
};
searchSecond.Result = new Result();
searchSecond.Result.Rows = new List<Row>(
new[]
{
new Row {ProductCode = "CreditCards/122", Score = 9},
new Row {ProductCode = "CreditCards/124", Score = 4}
});
var searches = new List<Search>
{
searchFirst,
searchSecond
};
var click = new Click
{
VisitId = "visits/64",
ProductCode = "CreditCards/122",
SearchId = "searches/25"
};
using (var session = _documentStore.OpenSession())
{
foreach (var search in searches)
{
session.Store(search);
}
session.Store(click);
session.SaveChanges();
}
IList<ClicksBySearchAndProductCode.MapReduceResult> clicksBySearchAndProductCode = null;
using (var session = _documentStore.OpenSession())
{
clicksBySearchAndProductCode = session.Query<ClicksBySearchAndProductCode.MapReduceResult>(ClicksBySearchAndProductCode.INDEX_NAME)
.Customize(x => x.WaitForNonStaleResults()).ToArray();
}
Assert.That(clicksBySearchAndProductCode.Count, Is.EqualTo(4));
var mapReduce = clicksBySearchAndProductCode
.First(x => x.SearchId.Equals("searches/25")
&& x.ProductCode.Equals("CreditCards/122"));
Assert.That(mapReduce.Clicks,
Is.EqualTo(1));
}
}
public class ClicksBySearchAndProductCode :
AbstractMultiMapIndexCreationTask
<ClicksBySearchAndProductCode.MapReduceResult>
{
public const string INDEX_NAME = "ClicksBySearchAndProductCode";
public override string IndexName
{
get { return INDEX_NAME; }
}
public class MapReduceResult
{
public string SearchId { get; set; }
public string ProductCode { get; set; }
public string Score { get; set; }
public int Clicks { get; set; }
}
public ClicksBySearchAndProductCode()
{
AddMap<Search>(
searches =>
from search in searches
from row in search.Result.Rows
select new
{
SearchId = search.Id,
ProductCode = row.ProductCode,
Score = row.Score.ToString(),
Clicks = 0
});
AddMap<Click>(
clicks =>
from click in clicks
select new
{
SearchId = click.SearchId,
ProductCode = click.ProductCode,
Score = (string)null,
Clicks = 1
});
Reduce =
results =>
from result in results
group result by
new { SearchId = result.SearchId, ProductCode = result.ProductCode }
into g
select
new
{
SearchId = g.Key.SearchId,
ProductCode = g.Key.ProductCode,
Score = g.First(x => x.Score != null).Score,
Clicks = g.Sum(x => x.Clicks)
};
}
}
public class User
{
public string Id { get; set; }
}
public class Search
{
public string Id { get; set; }
public string VisitId { get; set; }
public User User { get; set; }
private Result _result = new Result();
public Result Result
{
get { return _result; }
set { _result = value; }
}
public Search(User user)
{
User = user;
}
}
public class Result
{
private IList<Row> _rows = new List<Row>();
public IList<Row> Rows
{
get { return _rows; }
set { _rows = value; }
}
}
public class Row
{
public string ProductCode { get; set; }
public int Score { get; set; }
}
public class Click
{
public string VisitId { get; set; }
public string SearchId { get; set; }
public string ProductCode { get; set; }
}
My problem here is that I expect Count to be one in that specific test, but it just doesn't seem to add the Clicks in the Click map and the result is 0 clicks. I'm totally confused, and I'm sure that there is a really simple solution to my problem, but I just can't find it..
..hope there is a week-end warrior out there who can take me under his wings.
Yes, it was a brain-melt, for me non-trivial, but still. The proper reduce should look like this:
Reduce =
results =>
from result in results
group result by
new { SearchId = result.SearchId, ProductCode = result.ProductCode }
into g
select
new
{
SearchId = g.Key.SearchId,
ProductCode = g.Key.ProductCode,
Score = g.Select(x=>x.Score).FirstOrDefault(),
Clicks = g.Sum(x => x.Clicks)
};
Not all Maps had the Score set to a non-null-value, and therefore my original version had a problem with:
Score = g.First(x => x.Score != null).Score
Mental note, use:
Score = g.Select(x=>x.Score).FirstOrDefault()
Don't use:
Score = g.First(x => x.Score != null).Score

MultiMap / Reduce - Counts = 0?

I want to create an index for a query, I want to return to my view a list of Audio items along with statistics for these items, which are TotalDownloads & TotalPlays.
Here are my relevant docs:
Audio
- Id
- ArtistName
- Name
AudioCounter
- AudioId
- Type
- DateTime
Here is my current Index:
public class AudioWithCounters : AbstractMultiMapIndexCreationTask<AudioWithCounters.AudioViewModel>
{
public class AudioViewModel
{
public string Id { get; set; }
public string ArtistName { get; set; }
public string Name { get; set; }
public int TotalDownloads { get; set; }
public int TotalPlays { get; set; }
}
public AudioWithCounters()
{
AddMap<Audio>(audios => from audio in audios
select new
{
Id = audio.Id,
ArtistName = audio.ArtistName,
Name = audio.Name,
TotalDownloads = 0,
TotalPlays = 0
});
AddMap<AudioCounter>(counters => from counter in counters
where counter.Type == Core.Enums.Audio.AudioCounterType.Download
select new
{
Id = counter.AudioId,
ArtistName = (string)null,
Name = (string)null,
TotalDownloads = 1,
TotalPlays = 0
});
AddMap<AudioCounter>(counters => from counter in counters
where counter.Type == Core.Enums.Audio.AudioCounterType.Download
select new
{
Id = counter.AudioId,
ArtistName = (string)null,
Name = (string)null,
TotalDownloads = 0,
TotalPlays = 1
});
Reduce = results => from result in results
group result by result.Id
into g
select new
{
Id = g.Key,
ArtistName = g.Select(x => x.ArtistName).Where(x => x != null).First(),
Name = g.Select(x => x.Name).Where(x => x != null).First(),
TotalDownloads = g.Sum(x => x.TotalDownloads),
TotalPlays = g.Sum(x => x.TotalPlays)
};
}
}
However, my TotalDownloads & TotalPlays are always 0 even though there should be data in there. What am I doing wrong?
In the reduce function, replace .First() with .FirstOrDefault(), then it works.
Besides that, there is a typo in the second map-function, because you are filtering on the same AudioCounterType.Download.