Loading...

vastorbit.read_json

vastorbit.read_json(path: str, schema: str | None = None, table_name: str | None = None, catalog: str | None = None, dtype: dict | None = None, orient: str = 'records', lines: bool = False, insert: bool = False, temporary_table: bool = False, flatten: bool = True, genSQL: bool = False) → VastFrame | List[VastFrame] | list[str]

Read JSON file(s) and create table in database via Trino.

Supports wildcard patterns to load multiple JSON files into the same table.

Parameters:

  • path (str) – Path to the JSON file(s). Supports wildcards (e.g., 'data/*.json', 'file_*.json')

  • schema (str, optional) – Target schema. Supports formats: - ‘schema_name’ (uses default catalog from config) - ‘catalog.schema_name’ (catalog and schema) If None, uses ‘memory.default’

  • table_name (str, optional) – Target table name. If None, generates from filename For wildcards, you must provide a table_name

  • catalog (str, optional) – Target catalog (overrides catalog from schema parameter) Examples: ‘hive’, ‘postgresql’, ‘vast’, ‘memory’

  • dtype (dict, optional) – Column definitions {column_name: trino_type} If None, types are inferred from data

  • orient (str, optional) – JSON orientation format (default: ‘records’) Options: ‘records’, ‘index’, ‘columns’, ‘values’, ‘split’

  • lines (bool, optional) – If True, read JSON lines format (one JSON object per line)

  • insert (bool, optional) – If True, insert into existing table. If False, create new table

  • temporary_table (bool, optional) – If True, create temporary table

  • flatten (bool, optional) – If True, flatten nested JSON structures (default: True)

  • genSQL (bool, optional) – If True, return SQL statements without executing

Returns:

If genSQL=False and single file: VastFrame object If genSQL=False and multiple files: VastFrame object If genSQL=True: List of SQL statements

Return type:

VastFrame or List[VastFrame] or list[str]

Examples

Single file:

from vastorbit.core.parsers.json import read_json

# Load to memory.default (temporary)
vdf = read_json('data.json')

# Load to specific schema
vdf = read_json('data.json', schema='public')

Multiple files with wildcard:

# Load all JSON files in a directory into one table
vdf = read_json('data/*.json', table_name='combined_data')

# Load files matching pattern
vdf = read_json('events_2024_*.json', table_name='events_2024')

# Load JSON lines files
vdf = read_json('logs/*.jsonl', table_name='logs', lines=True)

Notes

  • Complex nested structures are flattened or stored as JSON strings

  • Arrays are converted to JSON strings

  • NULL values are handled properly

  • Memory catalog is used for staging