Skip to content

[MySQL] Avoid storing copies of row data #528

@rkistner

Description

@rkistner

Related to #398.

For Postgres, we handle partial updates (TOAST columns excluded) by storing a copy of each row in current_data.

For MySQL, we currently do the same, but don't actually need that. In MySQL, there is a binlog_row_image setting:

full (Log all columns)
minimal (Log only changed columns, and columns needed to identify rows)
noblob (Log all columns, except for unneeded BLOB and TEXT columns)

So for full (the default), we get all columns for every change, and don't need to store a copy of the data in current_data.

For minimal and noblob, some columns could be excluded from the binlog. Right now, we don't support that either way - those columns would end up as null.

So I propose:

  1. Always use storeCurrentData: false for MySQL.
  2. Add an additional validation check when connecting, ensuring that binlog_row_image=full.

The validation would not catch all cases: It is possible to change the value of binlog_row_image for a specific session. However, it should catch common cases of global misconfiguration.

I don't think it's worth specifically supporting minimal or noblob configuration: Any savings the source database gets from that would just be transferred to overhead in the PowerSync storage database. I prefer that we just document and check that it should always be full.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions