Caution
anyschema is still in early development and possibly unstable.
Documentation | Source Code | Issue Tracker
anyschema allows you to convert from type specifications (such as Pydantic models, TypedDict, dataclasses, or plain
Python dicts) to any dataframe schema (by "any" we intend those supported by Narwhals).
Let's see how it works in practice with an example:
from anyschema import AnySchema
from pydantic import BaseModel
from pydantic import PositiveInt
class Student(BaseModel):
name: str
age: PositiveInt
classes: list[str]
schema = AnySchema(spec=Student)
# Convert to pyarrow schema
pa_schema = schema.to_arrow()
type(pa_schema)
# pyarrow.lib.Schema
pa_schema
# name: string
# age: uint64
# classes: list<item: string>
# child 0, item: string
pl_schema = schema.to_polars()
type(pl_schema)
# polars.schema.Schema
pl_schema
# Schema([('name', String), ('age', UInt64), ('classes', List(String))])To read more about anyschema functionalities and features consider checking out the
documentation website.
anyschema is available on pypi, and it can be installed directly via
any package manager. For instance:
uv pip install "anyschema[pydantic]"To allow interoperability with pydantic models.
anyschema is designed for scenarios where some type specifications (e.g. a Pydantic Model) want to be used as a single
source of truth for both validation and (dataframe) schema generation.
The typical use cases are: Data pipelines, API to database workflows, schema generation, type-safe data processing.
The project was inspired by a Talk Python podcast episode featuring the creator of LanceDB, who mentioned the need to convert from Pydantic models to PyArrow schemas.
This challenge led to a realization: such conversion could be generalized to many dataframe libraries by using Narwhals
as an intermediate representation. anyschema makes this conversion seamless and extensible.