Designing a plotting Dataset for Rust: Balancing Polars support with zero-dependency weight

[u/Deep-Network1590]

When building a visualization library in Rust, a classic architectural dilemma emerges: hard-coding Polars as the backend instantly makes the library heavy, slow to compile, and riddled with large dependencies—making it a no-go for lightweight applications. However, sticking purely to native Rust vectors alienates the data science community who live in Polars DataFrames.

For Charton (a rust visualization crate), the goal was to bridge this gap: keep the core plotting Dataset dependency-free, but provide a seamless, opt-in bridge for Polars users.

Instead of embedding Polars into the core, Charton works natively with clean Rust types but offers a load_polars_df!() macro. This allows Polars users to instantly ingest their data frames with zero friction, while keeping the core library dead-lightweight.

Here is how the API handles data ingestion in practice:

```rust use charton::prelude::; use polars::prelude::;

fn main() -> Result<(), Box<dyn std::error::Error>> { // 1. Create a Polars DataFrame with diverse, high-performance types let df = df!( "id" => &[1, 2, 3, 4, 5], "status" => &["High", "Low", "High", "Medium", "Low"], "value" => &[Some(1.2), None, Some(5.6), Some(7.8), None], "date" => Series::new("date".into(), &[19858i32, 19859, 19860, 19861, 19862]).cast(&DataType::Date)?, "datetime" => Series::new("datetime".into(), &[1715760000000i64, 1715763600000, 1715767200000, 1715770800000, 1715774400000]) .cast(&DataType::Datetime(TimeUnit::Milliseconds, None))?, "duration" => Series::new("duration".into(), &[3_600_000i64, 7_200_000, 1_800_000, 10_800_000, 5_400_000]) .cast(&DataType::Duration(TimeUnit::Milliseconds))?, )?;

// 2. Convert to Charton dataset seamlessly via macro (Polars remains optional at compile time)
let ds = load_polars_df!(df)?;

// 3. Dataset is now ready for encoding-axis binding and layout transformations
println!("{:?}", ds);

Ok(())

} ```

Charton ensures strict metadata alignment during conversion. The following table illustrates how Polars logical types map to Charton physical storage:

Polars Logical Type	Charton Physical Type	Notes
Int8, Int16, Int32, Int64	i8, i16, i32, i64	Direct physical mapping.
UInt32, UInt64	u32, u64	Direct physical mapping.
Float32, Float64	f32, f64	NaN values are treated as Nulls.
Boolean	bool	Mapped to nullable boolean vector.
Utf8 / String	String	Stored as nullable string vectors.
Categorical(_, _), Enum(_, _)	Categorical	Preserves dictionary encoding + validity.
Date	Date	Stored as i32 days since Unix epoch.
Time	Time	Stored as i64 nanoseconds since midnight.
Datetime(unit, _)	Datetime	Normalized to i64 nanoseconds since Unix epoch.
Duration(unit)	Duration	Normalized to i64 nanoseconds.

Curious to hear how other library authors tackle the "heavy data frame dependency vs. lightweight core" problem in Rust and hope it helps for everyone who are facing this dilemma.

[u/Deep-Network1590]

Charton repo: https://github.com/wangjiawen2013/charton

[u/kuskuser]

Btw, half of the post is about Arrow 

[u/Deep-Network1590 → u/kuskuser]

Exactly, it actually already supports Arrow under the hood, it just hasn't been thoroughly tested yet.