Split list of objects by delimiter in Kotlin - list

Question

I have a List of objects I want to split by a delimiter into sublists, e.g:
val tokens = listOf(
Token(name = "lorem", val = "ipsum"),
Token(name = "dolor", val = "sit"),
Token(name = "newline", val = "\n"),
Token(name = "amet", val = "consectetur")
)
The delimiter should be any Token whose name is "newline", so after the split, tokens should become:
listOf(
listOf(
Token(name = "lorem", val = "ipsum"),
Token(name = "dolor", val = "sit")
),
listOf(
Token(name = "amet", val = "consectetur")
)
)
I've written my own function to do this already, but is there some elegant, built-in (preferably functional) way of doing it? I say this because I'm learning Kotlin and, coming from C++, find myself "reinventing the wheel" a lot with these types of things.

Answer 1

In such cases I suggest not going with too much functional transformations. We can for example do it by folding/reducing, we can also first find indices of all delimiters and then zipWithNext() between them to get ranges, etc. This way we get the solution in a very few lines of code, but this code will be very hard to read and understand.
Instead, I suggest going with a good old and very simple loop. However, to make it smoother and more performant, we can use sequences and subList():
fun main() {
val tokens = ...
tokens.splitBy { it.name == "newline" }
}
fun <T> List<T>.splitBy(predicate: (T) -> Boolean): Sequence<List<T>> = sequence {
if (isEmpty()) return#sequence
var last = 0
forEachIndexed { i, v ->
if (predicate(v)) {
yield(subList(last, i))
last = i + 1
}
}
yield(subList(last, size))
}
Please note this solution does not involve any data copying. It iteratively creates views of the original list, so it should be pretty fast. On the other hand, it should be used with care if the original list may change.
Also, you need to be aware of corner cases like: delimiter at the beginning and end, no delimiters in the list or empty list. There is no single answer to how splitting should work in these cases. Whatever solution you pick, I suggest checking it for these cases. My above solution mirrors the behavior of String.split().

Answer 2

I think there isn't any extension function in standard library for handling this case. You will have to write your own logic. You can do something like this:
val newList = mutableListOf<List<Token>>()
var subList = mutableListOf<Token>()
for (token in tokens) {
if (token.name == "newline") {
newList += subList
subList = mutableListOf()
} else {
subList += token
}
}
if (subList.isNotEmpty())
newList += subList
println(newList)
You can also extract this code out in the form of an extension function:
fun <T> List<T>.split(delimiter: (T) -> Boolean): List<List<T>> {
val newList = mutableListOf<List<T>>()
var subList = mutableListOf<T>()
for (token in this) {
if (delimiter(token)) {
newList += subList
subList = mutableListOf()
} else {
subList += token
}
}
if (subList.isNotEmpty())
newList += subList
return newList
}
// Usage
fun main() {
val tokens = listOf(
Token(name = "lorem", val = "ipsum"),
Token(name = "dolor", val = "sit"),
Token(name = "newline", val = "\n"),
Token(name = "amet", val = "consectetur")
)
println(tokens.split { it.name == "newline" })
}

Answer 3

You can use fold:
tokens
.fold(mutableListOf(mutableListOf<Token>())) { l, elem ->
if(elem.name=="newline") l.add(mutableListOf())
else l.last().add(elem)
l
}
The first parameter is the initial value, a list with a single list in it (if there isn't any newline, you still want to have a single list containing the elements).
The second parameter is a function that is executed for every element.
If the token name is newline, it adds a new list. If not, it adds the element to the last list.
The last line of fold containing l makes sure that the list is returned.