Loading...

vastorbit.read_csv

vastorbit.read_csv(path: str, schema: str | None = None, table_name: str | None = None, catalog: str | None = None, dtype: dict | None = None, parse_nrows: int = 1000, insert: bool = False, temporary_table: bool = False, sep: str | None = None, header: bool = True, header_names: list | None = None, na_rep: str = '', infer_types: bool = True, genSQL: bool = False, batch_size: int = 800) → VastFrame | List[VastFrame] | list[str]

Read CSV file(s) and create table in database via Trino.

Supports wildcard patterns to load multiple CSV files into the same table.

Parameters:

  • path (str) – Path to the CSV file(s). Supports wildcards (e.g., 'data/*.csv', 'file_*.csv')

  • schema (str, optional) – Target schema. Supports formats: - ‘schema_name’ (uses default catalog from config) - ‘catalog.schema_name’ (catalog and schema) If None, uses ‘memory.default’

  • table_name (str, optional) – Target table name. If None, generates from filename For wildcards, you must provide a table_name

  • catalog (str, optional) – Target catalog (overrides catalog from schema parameter) Examples: ‘hive’, ‘postgresql’, ‘vast’, ‘memory’

  • dtype (dict, optional) – Column definitions {column_name: trino_type} If None, types are inferred from data

  • parse_nrows (int, optional) – Number of rows to use for type inference (default: 1000)

  • insert (bool, optional) – If True, insert into existing table. If False, create new table

  • temporary_table (bool, optional) – If True, create temporary table

  • sep (str, optional) – Column separator. If None, auto-detected

  • header (bool, optional) – Whether CSV has header row (default: True)

  • header_names (list, optional) – Custom column names (overrides CSV header)

  • na_rep (str, optional) – String representing missing values (default: ‘’)

  • infer_types (bool, optional) – Whether to infer types from data (default: True)

  • genSQL (bool, optional) – If True, return SQL statements without executing

  • batch_size (int, optional) – Batch size specifies the number of rows inserted into the VAST DataBase concurrently during a bulk operation.

Returns:

If genSQL=False and single file: VastFrame object If genSQL=False and multiple files: List of VastFrame objects If genSQL=True: List of SQL statements

Return type:

VastFrame or List[VastFrame] or list[str]

Examples

Single file:

from vastorbit.core.parsers.csv import read_csv

vdf = read_csv('data.csv')

Multiple files with wildcard:

# Load all CSV files in a directory into one table
vdf = read_csv('data/*.csv', table_name='combined_data')

# Load files matching pattern
vdf = read_csv('sales_2024_*.csv', table_name='sales_2024')