Related to #398.
For Postgres, we handle partial updates (TOAST columns excluded) by storing a copy of each row in current_data.
For MySQL, we currently do the same, but don't actually need that. In MySQL, there is a binlog_row_image setting:
full (Log all columns)
minimal (Log only changed columns, and columns needed to identify rows)
noblob (Log all columns, except for unneeded BLOB and TEXT columns)
So for full (the default), we get all columns for every change, and don't need to store a copy of the data in current_data.
For minimal and noblob, some columns could be excluded from the binlog. Right now, we don't support that either way - those columns would end up as null.
So I propose:
- Always use
storeCurrentData: false for MySQL.
- Add an additional validation check when connecting, ensuring that
binlog_row_image=full.
The validation would not catch all cases: It is possible to change the value of binlog_row_image for a specific session. However, it should catch common cases of global misconfiguration.
I don't think it's worth specifically supporting minimal or noblob configuration: Any savings the source database gets from that would just be transferred to overhead in the PowerSync storage database. I prefer that we just document and check that it should always be full.
Related to #398.
For Postgres, we handle partial updates (TOAST columns excluded) by storing a copy of each row in
current_data.For MySQL, we currently do the same, but don't actually need that. In MySQL, there is a binlog_row_image setting:
So for
full(the default), we get all columns for every change, and don't need to store a copy of the data incurrent_data.For
minimalandnoblob, some columns could be excluded from the binlog. Right now, we don't support that either way - those columns would end up asnull.So I propose:
storeCurrentData: falsefor MySQL.binlog_row_image=full.The validation would not catch all cases: It is possible to change the value of
binlog_row_imagefor a specific session. However, it should catch common cases of global misconfiguration.I don't think it's worth specifically supporting
minimalornoblobconfiguration: Any savings the source database gets from that would just be transferred to overhead in the PowerSync storage database. I prefer that we just document and check that it should always befull.