Support Union types - #54
Merged
Merged
Conversation
Instead of having the outer loop of the typed decoder switch on the *expected* type, we revert back to switching on the *received* type. This slightly slows down untyped decoding with the benefit of reducing code size (both LOC and binary) by unifying the typed & untyped decoders. It also opens up the opportunity for handling `Union` types in the decoder.
Now typenodes are implemented via a bitset of type values & a variable length array of extra info. This makes it possible to express type unions in an efficient manner (though union types beyond `Optional` aren't yet implemented). This has no performance measurable performance impact compared with the main branch, but does reduce slightly the untyped decoding degradations introduced in the previous commit. The untyped decoder is still slower than it was previously (at most ~7% slower by my measurements), but typed decoding measures the same as the main branch, and can now support more features going forward.
This adds support to `msgspec.Decoder` for parsing type unions. Limitations within a single type `Union` include: - Multiple array-like types are not supported - Multiple dict types are not supported - IntEnum and int cannot coexist - Enum and str cannot coexist - Struct and dict (or Struct and array-like types, if `asarray=True`) cannot coexist - If a custom type is present, the only other type supported in the union is `None`.
A bunch of micro-optimizations to remove most of the performance degradation caused by this refactor. Perf is basically the same as before now. Also fixes a bug around unions containing both asarray structs and dicts.
Also squash a segfault on windows.
jcrist
force-pushed
the
refactor-type-dispatch
branch
from
October 29, 2021 23:18
d463edd to
9f70856
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds support for
Uniontypes (with a few restrictions). Specifically only one of each of the following types may appear within a single union:This has a mild (negligible) performance degradation for typed decoders, and a slightly larger performance degradation for untyped decoding. I think this is fine, as most users should use the typed decoders anyway, since
Structtypes in general make things much faster.