Indexed samples
Datasets expose stable size and indexed retrieval; each sample carries an input Matrix and an optional target Matrix.
Data · Rust
Data into models.
Contracts intact.
Index, split, collate, and batch semantic samples without inventing a second tensor system or claiming a hidden streaming runtime.
Data path
From sample to model. Every boundary explicit.
Index, partition, collate, and iterate through one deterministic contract. Batches stay OA matrices when they enter training.
Seeded partitions
Explicit seeds produce reproducible shuffle order and train, validation, and test splits.
Checked batches
Collation stacks samples only when their shapes, dtypes, storage, and target presence agree.
Model-ready values
DataLoader yields deterministic batches as ordinary OA matrices ready for the Ml path.
Current surface
Small enough to understand. Complete enough to train.
OA Data owns deterministic dataset composition and batch iteration. Domain transforms stay with Vision, Audio, and Ml; device execution remains with the shared engine.
Boundary
The old streaming experiments are not the product.
Earlier OA trees explored broad data infrastructure. The current public contract keeps only the parts with clear ownership and tests; advanced sources return when a real consumer can prove them.
No hidden I/O
The current Data surface does not claim an implicit filesystem, network, mmap, or io_uring backend.
No phantom workers
There is no invisible prefetch pool, background upload path, or infinite-streaming runtime behind DataLoader.
No second tensor system
Samples and batches use OA Matrix values rather than crossing into a parallel host array or framework contract.
Future work stays bounded
Async sources and prefetch return only as explicit sessions with cancellation, backpressure, completion, and workload evidence.
Oa Library
One library. Across the stack.
Follow the same Engine, values, and explicit execution contract into another module.