F# Nested lists immutability - list

Question

I am trying to pick F# again and I have a case where I find it very difficult to deal with immutability.
In most cases I find immutability very nice. However, I find it very hard to deal with nested collections.
I would like to write a single user GUI program.
For simplicity let’s say we have the model:
type Employee =
{
Name : string
}
type Company =
{
Name : string
Employees : Employee list
}
If I modify an existing Employee with
let myNewEmployee = { myEmployee with Name = "John Smith" }
I get a new Employee and that is fine. However, here comes the chain of problems:
I need to remove the old myEmployee and add myNewEmployee to the
Company’s Employees list.
That provokes a mutation in the list with yields a new list.
That forces me to create a new Company record in order to inject the new collection, having to rebuild entire Company list I have.
In other words, changing an employee’s name, makes me rebuild the entire data structure.
Every time I am stumped, I have found that F# has a different and creative way of doing things.
I assume that it is my ignorance on not knowing how to deal with this situation in a functional way, please enlighten me ;)
Should I be using other F# libraries, like F# Data ?
TIA,
David

Answer 1

I think the answer depends partly on the larger context - such as how is the user interface of your application implemented. You are right that modifying one name in a company requires you to produce a new list and a new company record. That is not too bad if you do it in a single call to List.map though.
To make the sample easier, I added ID of type int to each emplyee:
let updateName id name company =
let newEmployees = company.Employees |> List.map (fun emp ->
if emp.ID = id then { emp with Name = name } else emp)
{ company with Employees = newEmployees }
If you are using something like the Elm architecture, then this is probably a reasonable way to go.
In some cases, you can do a more clever thing (but it depends on your scenario). For example, you could create a new type to represent a company with a list of updates that have been applied to it:
type Update =
| Rename of id:int * newName:string
type UpdatedCompany =
{ Company : Company
Updates : Update list }
Now changing a name is just a matter of appending a new Rename update to the list of Updates. Of course, once you need to display the final Company, you will need to iterate over all the employees (as above) and apply the updates. However, if you do a lot of updates before you need to get a new final Company value, this might be a nice trick.

Answer 2

The functional-first F# style means that you keep your functions pure by default i the core of your program and moving the necessary side-effects towards the edge of your program.
I need to remove the old myEmployee and add myNewEmployee to the Company’s Employees list.
Do you really need to do this? I encourage you to think about this chain going in the opposite direction. Why do you need this new list? What effect does that ultimately have on the outside world? Maybe the complete list can be built up in one pass by restructuring your code to a functional style. Maybe you are responding to user input and it's fine to build a new list every time. Maybe it would be better to use a different data structure like a Map (immutable dictionary) with the employee ID as the key.

Answer 3

Long entry with no clear answers but some ideas on how to deal with nested immutable data
As the OP noted, when discussing immutability an area that is ignored how to update nested properties. While this is trivial in mutable structure (just navigate there and update the property) with immutable structures one need to navigate there, create a copy with the updated property and then recreate all parents.
Luckily people have been thinking about this already and the functional pattern to address this is called Lenses or Prisms.
Lenses have a reputation of being somewhat difficult.
But that IMO is because in Haskell they talk about polymorphic lenses which can be defined like this:
type Lens s t a b = forall f. Functor f => (a -> f b) -> (s -> f t) // Que?
Braver souls than me have implemented polymorphic lenses in F# (which has a more simplistic type system than Haskell): http://www.fssnip.net/7Pk
(The polymorphic lens implementation in F# is provided AFAIK by the guy behind Hopac. He's pretty decent I say)
Polymorphic lenses are cool but if you remove the polymorphic bit lenses become much simpler to understand and still very usable.
Basically a lens is a pair of a getter and a setter function. It could look like this in F#:
type Lens<'T, 'U> = ('T -> 'U)*('T -> 'U -> 'T)
Given a value the getter get a property of that value. Given a value and property value the setter creates a new copy of the value with the property updated.
It can be thought of as a functional composable properties remotely comparable to .NET properties.
Given that your example (and many real-world things) deals with maps and lists Prisms are typically more usable. Here is one proposal:
type [<Struct>] Prism<'T, 'U> = Prism of ('T -> 'U option)*('T -> 'U -> 'T)
The only difference here is that the getter might return None if the property don't exists, for example if the employee don't exist in the list.
For a prism we define the operator >-> which combines a two prisms into a new one that let's you get the focused property as well as updating it like so:
let p = PropertyA >-> PropertyB >-> Property C
// Updates the nested property c in b in a and returns a new instance
let newA = a |> set p c
Let's see how this could look for the example in OPs post.
type Company =
{
Name : string
Employees : Map<EmployeeNo, Employee>
}
// Define Prisms for properties of Company
static member _Name : Prism<Company, _> = prism' (fun t v -> { t with Name = v }) (fun t -> t.Name |> Some)
static member _Employees : Prism<Company, _> = prism' (fun t v -> { t with Employees = v }) (fun t -> t.Employees |> Some)
Unfortunately there is a bit of boiler-plate code surrounding prisms, but that can be mitigated with code-gen tools and possibly even type-providers.
We define Employee in a similar manner and we can start defining functions that allows us manipulating the nested immutable structure.
// Uses Prisms to update the email
let updateEmail company employeeNo newEmail =
company
// The path to the Employee email
|> set (Company._Employees >-> lookup employeeNo >-> Employee._Email) newEmail
Prisms are chainable useful when updating more than one property.
// Uses Prisms to update the position and salary
let updatePosition company employeeNo newPosition newSalary =
company
// The path to the Employee position
|> set (Company._Employees >-> lookup employeeNo >-> Employee._Position) newPosition
// The path to the Employee salary
|> set (Company._Employees >-> lookup employeeNo >-> Employee._Salary ) newSalary
While the example above works it's inefficient to construct two Company objects and throw away the first. Better would be to navigate to the correct Employee and update it before updating the Company object.
// Uses Prisms to update the position and salary
// Does so in a more efficient manner
let updatePosition' company employeeNo newPosition newSalary =
// The problem with updatePosition above is that it constructs a new company
// object with position updated and then another one with the salary updated
// A faster approach is to navigate to the employee and once found
// update both the position and the salary
// Updates an employee position & salary
let updater = function
| None -> None
| Some e -> { e with Position = newPosition; Salary = newSalary} |> Some
company
// The path to the employee
|> update (Company._Employees >-> lookup employeeNo) updater
In conclusion; the implementation given here is intended to be a starting point for perhaps a different way of thinking on how to work with nested immutable structures. There are some issues with the implementation (like knowing if and why an update failed). This is solvable but I didn't want to pollute the idea with lots of worries.
Hopefully you found it interesting.
Full sample code:
// A Prism consists of two parts
// a getter that gets a property of a value (might return None)
// a setter that sets a property of a value (returns a new instance)
type [<Struct>] Prism<'T, 'U> = Prism of ('T -> 'U option)*('T -> 'U -> 'T)
module Prism =
let inline prism g s = Prism (g, s)
let inline prism' s g = Prism (g, s)
// join joins two Prisms into a new Prism, this is how we navigate nested structures
// Note: Creates in addition to a nested getter also a nested setter so a Prism
// allows both getting and setting of nested properties
let inline join (Prism (tg, ts)) (Prism (ug, us)) =
let getter t =
match tg t with
| None -> None
| Some tv -> ug tv
let setter t v =
match tg t with
| None -> t
| Some tv -> ts t (us tv v)
prism getter setter
// Prism that allows us to navigate Maps
let inline lookup key =
let getter m = Map.tryFind key m
let setter m v = Map.add key v m
prism getter setter
// Given a Prism and a value returns the nested property pointed out by the prism
let get (Prism (tg, _)) t = tg t
// Given a Prism and a value sets the nested property pointed out by the prism
let set (Prism (_, ts)) v t = ts t v
// Given a Prism and a value allows an update function to see the nested property
// and return update it
let update (Prism (tg, ts)) u t =
match u (tg t) with
| None -> t
| Some tv -> ts t tv
type Prism<'T, 'U> with
static member inline ( >-> ) (t, u) = Prism.join t u
module Demo =
open System
open Prism
// Our Domain Model
type [<Struct>] EmployeeNo = EmployeeNo of int
type Position = Contractor | IndividualContributor | Manager
// So prisms enforces some measure of boiler plating.
// Can be mitigated by code generations and possibly type providers
type Employee =
{
No : EmployeeNo
Name : string
Email : string
Hired : DateTime
Salary : decimal
Position : Position
}
// Define Prisms for properties of Employee
static member _No = prism' (fun t v -> { t with No = v }) (fun t -> t.No |> Some)
static member _Name = prism' (fun t v -> { t with Name = v }) (fun t -> t.Name |> Some)
static member _Email = prism' (fun t v -> { t with Email = v }) (fun t -> t.Email |> Some)
static member _Hired = prism' (fun t v -> { t with Hired = v }) (fun t -> t.Hired |> Some)
static member _Salary = prism' (fun t v -> { t with Salary = v }) (fun t -> t.Salary |> Some)
static member _Position = prism' (fun t v -> { t with Position = v }) (fun t -> t.Position |> Some)
type Company =
{
Name : string
Employees : Map<EmployeeNo, Employee>
}
// Define Prisms for properties of Company
static member _Name : Prism<Company, _> = prism' (fun t v -> { t with Name = v }) (fun t -> t.Name |> Some)
static member _Employees : Prism<Company, _> = prism' (fun t v -> { t with Employees = v }) (fun t -> t.Employees |> Some)
open Prism
// Uses Prisms to update the email
let updateEmail company employeeNo newEmail =
company
// The path to the Employee email
|> set (Company._Employees >-> lookup employeeNo >-> Employee._Email) newEmail
// Uses Prisms to update the position and salary
let updatePosition company employeeNo newPosition newSalary =
company
// The path to the Employee position
|> set (Company._Employees >-> lookup employeeNo >-> Employee._Position) newPosition
// The path to the Employee salary
|> set (Company._Employees >-> lookup employeeNo >-> Employee._Salary ) newSalary
// Uses Prisms to update the position and salary
// Does so in a more efficient manner
let updatePosition' company employeeNo newPosition newSalary =
// The problem with updatePosition above is that it constructs a new company
// object with position updated and then another one with the salary updated
// A faster approach is to navigate to the employee and once found
// update both the position and the salary
// Updates an employee position & salary
let updater = function
| None -> None
| Some e -> { e with Position = newPosition; Salary = newSalary} |> Some
company
// The path to the employee
|> update (Company._Employees >-> lookup employeeNo) updater
let test () =
// The initial state of the company
let company : Company =
let e no name email year month day salary position =
let eno = EmployeeNo no
let e : Employee =
{
No = eno
Name = name
Email = email
Hired = DateTime (year, month, day)
Salary = salary
Position = position
}
eno, e
let es =
[|
e 1 "Bill Gates" "billg#microsoft.com" 1979 1 1 100000M Manager
e 2 "Melinda Gates" "melindag#microsoft.com" 1985 6 6 20000M IndividualContributor
|] |> Map.ofArray
{ Name = "Microsoft"; Employees = es}
// Does some organizational changes of the company
printfn "Initial: %A" company
let company = updateEmail company (EmployeeNo 1) "billg#hotmail.com"
printfn "Changing Bill Gates email: %A" company
let company = updatePosition company (EmployeeNo 2) Manager 200000M
printfn "Promoting Melinda Gates: %A" company
let company = updatePosition' company (EmployeeNo 1) IndividualContributor 10000M
printfn "Demoting Bill Gates: %A" company