Pure Ruby Apache Arrow reader/writer

JA

Apache Arrow is the de fact standard data format in modern data processing systems. We can use the official Red Arrow gem to process Apache Arrow data. It's suitable for fast large data processing but it's over-performance for only low cost data exchange needs. Red Arrow is larger and a bit difficult to install than pure Ruby gems because Red Arrow is implemented as bindings.

I'm implementing the official pure Ruby Apache Arrow reader/writer for only low cost data exchange needs. I expect that more Ruby libraries and applications add support for Apache Arrow inputs/outputs by the pure Ruby Apache Arrow reader/writer. Ruby can be used more for data processing by it.

This talk describes how to implement fast pure Ruby binary data reader/writer and the future of data processing in Ruby.

This is a 2025 Ruby Association Grant project: https://www.ruby.or.jp/en/news/20251030

  • Sutou Kouhei
    Sutou Kouhei
    @ktou

    He is a free software programmer and the president of ClearCode Inc. He is also the namer of ClearCode Inc. The origin of the company name is "clear code". We will be programmers that code clear code as our company name suggests. He is interested in how to tell other programmers about how he codes clear code.