-
Notifications
You must be signed in to change notification settings - Fork 7
Implement Typed Documents and TypeRegistry #282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jterapin
wants to merge
65
commits into
decaf
Choose a base branch
from
typed_documents
base: decaf
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
65 commits
Select commit
Hold shift + click to select a range
d33fcdf
Add type registry prototype class
jterapin 5ff40cc
Add type registry to codegenerated schema
jterapin 66a6285
Update projections
jterapin c5e45ed
Merge branch 'decaf' into typed_documents
jterapin 2bd15a2
Merge branch 'decaf' into typed_documents
jterapin 03bfa82
Merge branch 'decaf' into typed_documents
jterapin e6435d5
Update requires
jterapin 0830827
Add initial document implementation
jterapin 877654f
Merge decaf into branch
jterapin a61318f
Update to include cbor
jterapin 4edfae3
Expand on typed docs
jterapin ff959f1
Update file names
jterapin 8b9b560
Merge branch 'decaf' into typed_documents
jterapin 598db66
More refactoring
jterapin 3a4c0d1
Merge branch 'decaf' into typed_documents
jterapin 269b2b5
Remove scratches
jterapin 90c58ce
Fix rubocop
jterapin a1e46cc
Clean up document
jterapin 8b666cd
Clean document specs
jterapin 2ddf4bd
Update TypeRegistry
jterapin 112ddf4
Add documentation
jterapin 6283813
Add TypeRegistry specs
jterapin efbfa5e
Merge branch 'decaf' into typed_documents
jterapin 88ff845
Add TypeRegistry tests
jterapin 66b2cde
Update projections
jterapin 22998a0
Update syntax
jterapin 9afeacd
Update projections
jterapin 232844c
Merge branch 'decaf' into typed_documents
jterapin e8920fd
Change schema name to shapes to stay aligned
jterapin ef8c027
Remove as_typed method from TypeRegistry
jterapin 3854661
Create TimeHelper module
jterapin 48e1b0f
Fix timestamp failures
jterapin 04d0d5b
Refactor type registry per feedbacks
jterapin b35e09b
Update 1 projection
jterapin 66825be
Update weather projection
jterapin 4efe669
Use SchemaHelper for testing
jterapin 73e0e83
Update TypeRegistry to use SchemaHelper for testing
jterapin 60ef29c
Add TypeRegistry documentation
jterapin b67cf54
Update example
jterapin ea9b380
Merge decaf into branch
jterapin 381602b
Remove reference to type registry from client
jterapin 6d41f64
Update projections
jterapin 83fa2bb
Update projections
jterapin b23a2e9
Document now inherits SimpleDelegator
jterapin 306a97e
Expand on type registry docs
jterapin 8ef3055
Fix bug in timehelper
jterapin a297963
Slim down the sample shapes
jterapin 8e2d322
Update docs
jterapin 532ba8a
Merge decaf into branch
jterapin cb66cec
Only add structures to type registry
jterapin 9fa984c
Update TypeRegistry to limit to StructureShape
jterapin 166c8b1
Document revamp
jterapin 9cef7ee
Rename document test cases
jterapin d01a780
Merge branch 'decaf' into typed_documents
jterapin e283281
Improve Document Deserializer
jterapin a48f7a1
Update Document Serializer and its specs
jterapin 1913938
Update Document Data class
jterapin 8cc697c
Remove TimeHelper
jterapin 75141d1
Require delegate
jterapin d36a70b
Fix relative ordering
jterapin b2a6998
Fix type registry test
jterapin 9219b13
Remove unnecessary shape
jterapin 1f0b771
Rubocop fix
jterapin 74450a1
More changes
jterapin 68c94c1
Add docs
jterapin File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
# frozen_string_literal: true | ||
|
||
require_relative 'document/deserializer' | ||
require_relative 'document/serializer' | ||
require 'delegate' | ||
|
||
module Smithy | ||
module Schema | ||
# The module provides functionality for handling Smithy document types, | ||
# which represent protocol-agnostic data structures in the Smithy data model. | ||
# | ||
# This module includes capabilities for: | ||
# * Serialization and deserialization of document data | ||
# * Type-aware data handling | ||
# * Support for JSON document format | ||
# | ||
# @example Basic usage with a document | ||
# data = Document::Data.new({ name: "document" }) | ||
# data.data # => { "name" => "example" } | ||
# | ||
# @example Using with a shape | ||
# shape = Smithy::Schema::StructureShape.new | ||
# data = Document::Data.new({ "name" => "example" }, shape: shape) | ||
# | ||
module Document | ||
# A Smithy document, representing typed or untyped data from the Smithy data model. | ||
# The Data class delegates to the underlying data object while providing additional | ||
# document-specific functionality. | ||
class Data < ::SimpleDelegator | ||
# @param [Object] data document data that is in JSON-friendly format | ||
# @param [Hash] options | ||
# @option options [String] :discriminator This value is used to identify a specific | ||
# shape. This is equivalent of a Smithy shape ID. | ||
def initialize(data, options = {}) | ||
@data = data | ||
super(@data) | ||
@discriminator = options[:discriminator] || nil | ||
jterapin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
end | ||
|
||
# Returns the discriminator value for the document | ||
# | ||
# @return [String. nil] discriminator | ||
attr_reader :discriminator | ||
|
||
def data | ||
jterapin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
__getobj__ | ||
end | ||
end | ||
end | ||
end | ||
end |
182 changes: 182 additions & 0 deletions
182
gems/smithy-schema/lib/smithy-schema/document/deserializer.rb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,182 @@ | ||
# frozen_string_literal: true | ||
|
||
module Smithy | ||
module Schema | ||
module Document | ||
# Deserializes document data into runtime shape. | ||
class Deserializer | ||
include Shapes | ||
|
||
# @param [TypeRegistry] type_registry required to find shape based | ||
# on document discriminator. | ||
def initialize(type_registry) | ||
@type_registry = type_registry | ||
end | ||
|
||
# Deserializes a {Data} into a runtime shape. | ||
# | ||
# @param [Data] document The document to deserialize. Must have | ||
# a discriminator that maps to a shape in the type registry. | ||
# @param [StructureShape, nil] shape Optional shape to use for | ||
# deserialization. If provided, this shape takes precedence over the | ||
# document's discriminator. The shape must have a type. | ||
# @return [Object] deserialized runtime shape | ||
# | ||
# @example Standard Example | ||
# # create deserializer with an existing type registry | ||
# deserializer = Smithy::Schema::Document::Deserializer(type_registry) | ||
# | ||
# deserializer.deserialize(document) # passing document data | ||
# # => #<struct SampleService::Types::SampleShape....> | ||
# @example Providing a shape as input | ||
# # using the existing discriminator above | ||
# # given shape is a structure and has a type | ||
# deserializer.deserialize(document, shape: some_structure) | ||
# # => #<struct SampleService::Types::SomeStructure....> | ||
def deserialize(document, shape: nil) | ||
validate_input(document, shape) | ||
|
||
shape ||= resolve_shape(document) | ||
shape(ShapeRef.new(shape: shape), document.data, shape.type.new) | ||
end | ||
|
||
private | ||
|
||
def validate_input(document, shape) | ||
msg = 'document must be an instance of `Document::Data` class' | ||
raise ArgumentError, msg unless document.is_a?(Data) | ||
|
||
if shape | ||
msg = 'invalid shape - must be a structure shape with type' | ||
raise ArgumentError, msg unless valid_shape(shape) | ||
else | ||
msg = 'invalid document - must have a discriminator' | ||
raise ArgumentError, msg unless document.discriminator | ||
end | ||
end | ||
|
||
def valid_shape(shape) | ||
shape.is_a?(StructureShape) && shape.type | ||
end | ||
|
||
def resolve_shape(document) | ||
msg = 'document discriminator not found in type registry' | ||
raise ArgumentError, msg unless @type_registry.key?(document.discriminator) | ||
|
||
@type_registry[document.discriminator] | ||
end | ||
|
||
def shape(ref, value, target = nil) # rubocop:disable Metrics/CyclomaticComplexity | ||
case ref.shape | ||
when StructureShape then structure(ref, value, target) | ||
when UnionShape then union(ref, value, target) | ||
when ListShape then list(ref, value, target) | ||
when MapShape then map(ref, value, target) | ||
when TimestampShape then timestamp(value) | ||
when DocumentShape then document(value) | ||
when BlobShape then Base64.strict_decode64(value) | ||
when FloatShape then float(value) | ||
else value | ||
end | ||
end | ||
|
||
def document(values) | ||
return values unless values.is_a?(Hash) && values.key?('__type') | ||
|
||
msg = 'invalid document - document discriminator not found in type registry' | ||
raise ArgumentError, msg unless @type_registry.key?(values['__type']) | ||
|
||
shape_ref = ShapeRef.new(shape: @type_registry[values['__type']]) | ||
shape(shape_ref, values) | ||
end | ||
|
||
def float(value) | ||
case value | ||
when 'Infinity' then ::Float::INFINITY | ||
when '-Infinity' then -::Float::INFINITY | ||
when 'NaN' then ::Float::NAN | ||
when nil then nil | ||
else value.to_f | ||
end | ||
end | ||
|
||
def list(ref, values, target = nil) | ||
target = [] if target.nil? | ||
values.each do |value| | ||
next if value.nil? && !sparse?(ref.shape) | ||
|
||
target << (value.nil? ? nil : shape(ref.shape.member, value)) | ||
end | ||
target | ||
end | ||
|
||
def map(ref, values, target = nil) | ||
target = {} if target.nil? | ||
values.each do |key, value| | ||
next if value.nil? && !sparse?(ref.shape) | ||
|
||
target[key] = value.nil? ? nil : shape(ref.shape.value, value) | ||
end | ||
target | ||
end | ||
|
||
def structure(ref, values, target = nil) | ||
return Smithy::Schema::EmptyStructure.new if ref.shape == Prelude::Unit | ||
|
||
target = ref.shape.type.new if target.nil? | ||
ref.shape.members.each do |member_name, member_ref| | ||
name = member_ref.member_name | ||
next unless values.key?(name) | ||
|
||
target[member_name] = shape(member_ref, values[name]) | ||
end | ||
target | ||
end | ||
|
||
def timestamp(value) | ||
case value | ||
when nil then nil | ||
when Numeric | ||
Time.at(value).utc | ||
when /^[\d.]+$/ | ||
Time.at(value.to_f).utc | ||
else | ||
begin | ||
fractional_time = Time.parse(value).to_f | ||
Time.at(fractional_time).utc | ||
rescue ArgumentError | ||
raise "unhandled timestamp format `#{value}'" | ||
end | ||
end | ||
end | ||
|
||
def union(ref, values, target = nil) # rubocop:disable Metrics/AbcSize | ||
jterapin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
validate_union(values) | ||
|
||
key, value = values.first | ||
return if key.nil? | ||
|
||
ref.shape.members.each do |member_name, member_ref| | ||
name = member_ref.member_name | ||
next unless values.key?(name) | ||
|
||
target = ref.shape.member_type(member_name) if target.nil? | ||
return target.new(shape(member_ref, values[name])) | ||
end | ||
ref.shape.member_type(:unknown).new(key, value) | ||
end | ||
|
||
def validate_union(values) | ||
return unless values.size > 1 | ||
|
||
msg = "union value includes more than one key, received: #{values.keys}" | ||
raise ArgumentError, msg if values.size > 1 | ||
end | ||
|
||
def sparse?(shape) | ||
shape.traits.include?('smithy.api#sparse') | ||
end | ||
end | ||
end | ||
end | ||
end |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was expecting document to be a class and have it be the delegator. What was the intention of making another data subclass?