Small blog containing code and other posts by Lyndon Armitage
© 2025. All rights reserved.
This year I have decided to try and do the code challenges on the
Advent of Code website in Scala and possibly Spark
if needed (or an interesting solution arises).
These are simple little coding challenges given once per day like an Advent
Calendar before Christmas.
This year I have decided to try and do the code challenges on the
Advent of Code website in Scala and possibly Spark
if needed (or an interesting solution arises).
These are simple little coding challenges given once per day like an Advent
Calendar before Christmas.
This year I have decided to try and do the code challenges on the
Advent of Code website in Scala and possibly Spark
if needed (or an interesting solution arises).
These are simple little coding challenges given once per day like an Advent
Calendar before Christmas.
Sometimes when dealing with data in Spark you may find yourself needing to join
data against a large date range. I have encountered this when needing take a
sparsely populated table (in terms of the dates) and fill in any missing entries
with some sensible value, be it a default value (using the na functions) or a
previous dates value (using a windowing function).
In my previous post
I mentioned one possible solution for adding comments to my blog using the
built in support for data files in Jekyll. This approach was pioneered by
Damien Guard.
In this post I hope to have a crack at designing such a system myself and
implementing it.