OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
RAG & Knowledge Systems

Vector Io

Vector Io is an open-source Python library for importing, exporting and managing vector dataset data across supported vector databases and repositories. It uses VDF, a shared format comprising JSON metadata and Parquet data files.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Vector Io — OpenViST project profile in RAG & Knowledge Systems.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Vector Io when a project needs a shared dataset format for moving vector data between supported repositories, or when evaluating tooling for vector-data export and import.

Who it is for

  • Developers managing vector data across supported databases.
  • RAG teams that need to transfer or export vector datasets.

Documented uses

  • Export vector data from a supported database into VDF.
  • Import VDF data into a supported database.
  • Plan vector dataset backups or re-embedding workflows.
Before you choose

Requirements & limits

The repository is public and specifies an Apache-2.0 license. Installation is documented through pip or from source.

  • The repository lists nine databases as fully supported for import and export.
  • The export and import command-line help lists a narrower backend set than the full-support table.
  • VDF stores metadata in JSON and vector data in Parquet files or folders.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Export vector data to VDF

Use the export CLI to prepare data from a documented database for export in the universal Vector Dataset Format.

Before you start

  • Install vdf-io using pip.
  • Data in one of the CLI-listed databases: Pinecone, Qdrant, KDB.AI, Milvus, or Vertex AI Vector Search.

Steps

  1. Run export_vdf --help to display the export command options.

  2. Select the matching database subcommand: pinecone, qdrant, kdbai, milvus, or vertexai_vectorsearch.

  3. Use the displayed options as needed: --model_name names the model used, --max_file_size sets a maximum in MB (default 1024), and --push_to_hub and --public control hub publishing and dataset visibility.

Expected outcome

The CLI provides an export path to VDF, whose documented structure includes `VDF_META.json` and Parquet files or folders for metadata and vectors.

Guide limitations

  • CLI export choices are narrower than the README's broader supported-database table; do not assume the CLI options apply to every listed database.
  • Publishing to a hub is optional, and public visibility defaults to false.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
13 October 2026
Original website title
AI-Northstar-Tech/vector-io · Official homepage title · retained in its source language
How we review sourcesSuggest a correction