split_async

Function split_async 

Source
pub async fn split_async<ObjectID: FsVerityHashValue>(
    tar_stream: impl AsyncBufRead + Unpin,
    repo: Arc<Repository<ObjectID>>,
    content_type: u64,
) -> Result<ObjectID>
Expand description

Asynchronously splits a tar archive into a composefs split stream.

Similar to split() but processes the tar stream asynchronously with parallel object storage. Large files are streamed to O_TMPFILE via a channel, and their fs-verity digests are computed in background blocking tasks. This avoids blocking the async runtime while allowing multiple files to be processed concurrently.

Concurrency is limited to available_parallelism() to avoid overwhelming the system with too many concurrent I/O operations.

Files larger than INLINE_CONTENT_MAX are stored externally in the object store, while smaller files and metadata are stored inline in the split stream.

ยงArguments

  • tar_stream - The async buffered tar stream to read from
  • repo - The repository for creating tmpfiles and storing objects
  • content_type - The content type identifier for the splitstream

Returns the fs-verity object ID of the stored splitstream.