Skip to content

Array/list feature materialization in BQ crashes in type conversion #1839

@adchia

Description

@adchia

Currently, if you try to use BQ and materialize a feature that is a list (of numbers, strings, etc), Feast will crash because in BQ, the value type of the feature is a dictionary, such as

{'list': [{'item': 3}, {'item': 3}]}
In materialize, we convert the latest values retrieval job to a pyarrow table and then converts to ValueProtos to write. This calls

python_type_to_feast_value_type, which doesn't support dict types.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions