Thalweg · API reference ← Main docs

Module bridge

Module bridge 

Source
Expand description

Arrow ↔ DataFusion RecordBatch bridge for the remote UDF layer.

twg-udf links two arrow crate versions: arrow 59 (the latest, also used by arrow-flight 59) and the arrow 58 that datafusion 54 re-exports as [datafusion::arrow]. Their RecordBatch/DataType types are wire- compatible via the Arrow IPC format but are distinct Rust types, so batches crossing the boundary are round-tripped through an in-memory IPC stream.

Arrow Flight decodes into arrow-59 batches; DataFusion hands us arrow-58 batches — this module is the only place the two meet. If a future datafusion release re-exports the same arrow major as arrow-flight, this module and its two calls in super::udf_impl can be deleted.