[Enh]: Pydantic integration

[danielgafni]

We would like to learn about your use case. For example, if this feature is needed to adopt Narwhals in an open source project, could you please enter the link to it below?

https://anam-org.github.io/metaxy/main/

Please describe the purpose of the new feature or describe the problem to solve.

I would like to convert my Pydantic models into Narwhals schemas.
I am probably going to implement something myself right now, but I feel like this feature is general and useful enough to belong to Narwhals itself.

Suggest a solution if possible.

No response

If you have tried alternatives, please describe them below.

Something like https://github.com/FBruzzesi/anyschema, but staying in Narwhals land instead.

Additional information that may help us understand your needs.

I need to make sure DataFrames have the right schema before writing them to storage formats that support schema evolution (such as DeltaLake or LanceDB).

My dataframes are always associated with Pydantic models, so this functionality is exactly what I'd like to have.

[FBruzzesi]

Something like https://github.com/FBruzzesi/anyschema, but staying in Narwhals land instead.

I am flattered 🙏🏼

I hinted a few times to the possibility of moving it under the narwhals-dev organization eventually, but the current state is not really usable and I didn't have too much time/motivation to try developing it further

My dataframes are always associated with Pydantic models, so this functionality is exactly what I'd like to have.

Disclaimer: I am not very familiar with its API nor dependencies

Maybe Pandera can help?

1. They might eventually support narwhals [🐋 narwhals] create stub modules, classes, and functions for narwhals integration unionai-oss/pandera#2081
2. Maybe there is already a way to use what they built?

[danielgafni]

Nah, I want a very lightweight, dependency-free solution, just to do schema conversion. No data validation or anything like that. Sounds like this should belong to Narwhals (integration).

[FBruzzesi]

@danielgafni I will take a stab at getting anyschema to a somewhat usable state during the weekend.
I will start by targeting Pydantic models. Do you mind if I tag you there eventually?

[danielgafni]

Feel free to!