Schedule
BackA full-time CRuby committer at Speee Inc. I'm currently focusing on making Ruby a data-science-ready programming language.
Reducing ActiveRecord memory consumption using Apache Arrow
The pluck
method provided in ActiveRecord is a platform to obtain one or few field values as an array or arrays from a database. The pluck
method, compared with finder methods, can considerably reduce memory consumption because it does not generate model instances.
In this talk, I would like to introduce my new approach to reduce the memory consumption of ActiveRecord. This approach employs Apache Arrow as the internal data representation of an ActiveRecord::Result
object. This approach can achieve a remarkable reduction of the memory consumption of the pluck
method; it is 2-12x efficient than the original implementation.