Skip to content

Commit 5c8a47e

Browse files
authored
Merge pull request grpc#9776 from apolcyn/add_http2_flow_control_interop_tests
add http2 testing interop server uses small data frames and padding
2 parents 209c41a + 50fdc8a commit 5c8a47e

14 files changed

+307
-77
lines changed

doc/negative-http2-interop-test-descriptions.md renamed to doc/http2-interop-test-descriptions.md

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,3 +193,79 @@ Server Procedure:
193193
1. Sets MAX_CONCURRENT_STREAMS to one after the connection is made.
194194
195195
*The assertion that the MAX_CONCURRENT_STREAMS limit is upheld occurs in the http2 library we used.*
196+
197+
### data_frame_padding
198+
199+
This test verifies that the client can correctly receive padded http2 data
200+
frames. It also stresses the client's flow control (there is a high chance
201+
that the sender will deadlock if the client's flow control logic doesn't
202+
correctly account for padding).
203+
204+
Client Procedure:
205+
(Note this is the same procedure as in the "large_unary" gRPC interop tests.
206+
Clients should use their "large_unary" gRPC interop test implementations.)
207+
Procedure:
208+
1. Client calls UnaryCall with:
209+
210+
```
211+
{
212+
response_size: 314159
213+
payload:{
214+
body: 271828 bytes of zeros
215+
}
216+
}
217+
```
218+
219+
Client asserts:
220+
* call was successful
221+
* response payload body is 314159 bytes in size
222+
* clients are free to assert that the response payload body contents are zero
223+
and comparing the entire response message against a golden response
224+
225+
Server Procedure:
226+
1. Reply to the client's request with a `SimpleResponse`, with a payload
227+
body length of `SimpleRequest.response_size`. But send it across specific
228+
http2 data frames as follows:
229+
* Each http2 data frame contains a 5 byte payload and 255 bytes of padding.
230+
231+
* Note the 5 byte payload and 255 byte padding are partly arbitrary,
232+
and other numbers are also ok. With 255 bytes of padding for each 5 bytes of
233+
payload containing actual gRPC message, the 300KB response size will
234+
multiply into around 15 megabytes of flow control debt, which should stress
235+
flow control accounting.
236+
237+
### no_df_padding_sanity_test
238+
239+
This test verifies that the client can correctly receive a series of small
240+
data frames. Note that this test is intentionally a slight variation of
241+
"data_frame_padding", with the only difference being that this test doesn't use data
242+
frame padding when the response is sent. This test is primarily meant to
243+
prove correctness of the http2 server implementation and highlight failures
244+
of the "data_frame_padding" test.
245+
246+
Client Procedure:
247+
(Note this is the same procedure as in the "large_unary" gRPC interop tests.
248+
Clients should use their "large_unary" gRPC interop test implementations.)
249+
Procedure:
250+
1. Client calls UnaryCall with:
251+
252+
```
253+
{
254+
response_size: 314159
255+
payload:{
256+
body: 271828 bytes of zeros
257+
}
258+
}
259+
```
260+
261+
Client asserts:
262+
* call was successful
263+
* response payload body is 314159 bytes in size
264+
* clients are free to assert that the response payload body contents are zero
265+
and comparing the entire response message against a golden response
266+
267+
Server Procedure:
268+
1. Reply to the client's request with a `SimpleResponse`, with a payload
269+
body length of `SimpleRequest.response_size`. But send it across series of
270+
http2 data frames that contain 5 bytes of "payload" and zero bytes of
271+
"padding" (the padding flags on the data frames should not be set).

test/http2_test/http2_base_server.py

Lines changed: 22 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939

4040
_READ_CHUNK_SIZE = 16384
4141
_GRPC_HEADER_SIZE = 5
42+
_MIN_SETTINGS_MAX_FRAME_SIZE = 16384
4243

4344
class H2ProtocolBaseServer(twisted.internet.protocol.Protocol):
4445
def __init__(self):
@@ -121,38 +122,46 @@ def on_request_received_default(self, event):
121122
)
122123
self.transport.write(self._conn.data_to_send())
123124

124-
def on_window_update_default(self, event):
125-
# send pending data, if any
126-
self.default_send(event.stream_id)
125+
def on_window_update_default(self, _, pad_length=None, read_chunk_size=_READ_CHUNK_SIZE):
126+
# try to resume sending on all active streams (update might be for connection)
127+
for stream_id in self._send_remaining:
128+
self.default_send(stream_id, pad_length=pad_length, read_chunk_size=read_chunk_size)
127129

128130
def send_reset_stream(self):
129131
self._conn.reset_stream(self._stream_id)
130132
self.transport.write(self._conn.data_to_send())
131133

132-
def setup_send(self, data_to_send, stream_id):
134+
def setup_send(self, data_to_send, stream_id, pad_length=None, read_chunk_size=_READ_CHUNK_SIZE):
133135
logging.info('Setting up data to send for stream_id: %d' % stream_id)
134136
self._send_remaining[stream_id] = len(data_to_send)
135137
self._send_offset = 0
136138
self._data_to_send = data_to_send
137-
self.default_send(stream_id)
139+
self.default_send(stream_id, pad_length=pad_length, read_chunk_size=read_chunk_size)
138140

139-
def default_send(self, stream_id):
141+
def default_send(self, stream_id, pad_length=None, read_chunk_size=_READ_CHUNK_SIZE):
140142
if not self._send_remaining.has_key(stream_id):
141143
# not setup to send data yet
142144
return
143145

144146
while self._send_remaining[stream_id] > 0:
145147
lfcw = self._conn.local_flow_control_window(stream_id)
146-
if lfcw == 0:
148+
padding_bytes = pad_length + 1 if pad_length is not None else 0
149+
if lfcw - padding_bytes <= 0:
150+
logging.info('Stream %d. lfcw: %d. padding bytes: %d. not enough quota yet' % (stream_id, lfcw, padding_bytes))
147151
break
148-
chunk_size = min(lfcw, _READ_CHUNK_SIZE)
152+
chunk_size = min(lfcw - padding_bytes, read_chunk_size)
149153
bytes_to_send = min(chunk_size, self._send_remaining[stream_id])
150-
logging.info('flow_control_window = %d. sending [%d:%d] stream_id %d' %
151-
(lfcw, self._send_offset, self._send_offset + bytes_to_send,
152-
stream_id))
154+
logging.info('flow_control_window = %d. sending [%d:%d] stream_id %d. includes %d total padding bytes' %
155+
(lfcw, self._send_offset, self._send_offset + bytes_to_send + padding_bytes,
156+
stream_id, padding_bytes))
157+
# The receiver might allow sending frames larger than the http2 minimum
158+
# max frame size (16384), but this test should never send more than 16384
159+
# for simplicity (which is always legal).
160+
if bytes_to_send + padding_bytes > _MIN_SETTINGS_MAX_FRAME_SIZE:
161+
raise ValueError("overload: sending %d" % (bytes_to_send + padding_bytes))
153162
data = self._data_to_send[self._send_offset : self._send_offset + bytes_to_send]
154163
try:
155-
self._conn.send_data(stream_id, data, False)
164+
self._conn.send_data(stream_id, data, end_stream=False, pad_length=pad_length)
156165
except h2.exceptions.ProtocolError:
157166
logging.info('Stream %d is closed' % stream_id)
158167
break
@@ -200,5 +209,5 @@ def parse_received_data(self, stream_id):
200209
req_proto_str = recv_buffer[5:5+grpc_msg_size]
201210
sr = messages_pb2.SimpleRequest()
202211
sr.ParseFromString(req_proto_str)
203-
logging.info('Parsed request for stream %d: response_size=%s' % (stream_id, sr.response_size))
212+
logging.info('Parsed simple request for stream %d' % stream_id)
204213
return sr

test/http2_test/http2_test_server.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@
4444
import test_rst_after_data
4545
import test_rst_after_header
4646
import test_rst_during_data
47+
import test_data_frame_padding
4748

4849
_TEST_CASE_MAPPING = {
4950
'rst_after_header': test_rst_after_header.TestcaseRstStreamAfterHeader,
@@ -52,6 +53,10 @@
5253
'goaway': test_goaway.TestcaseGoaway,
5354
'ping': test_ping.TestcasePing,
5455
'max_streams': test_max_streams.TestcaseSettingsMaxStreams,
56+
57+
# Positive tests below:
58+
'data_frame_padding': test_data_frame_padding.TestDataFramePadding,
59+
'no_df_padding_sanity_test': test_data_frame_padding.TestDataFramePadding,
5560
}
5661

5762
_exit_code = 0
@@ -73,6 +78,8 @@ def buildProtocol(self, addr):
7378

7479
if self._testcase == 'goaway':
7580
return t(self._num_streams).get_base_server()
81+
elif self._testcase == 'no_df_padding_sanity_test':
82+
return t(use_padding=False).get_base_server()
7683
else:
7784
return t().get_base_server()
7885

@@ -81,7 +88,8 @@ def parse_arguments():
8188
parser.add_argument('--base_port', type=int, default=8080,
8289
help='base port to run the servers (default: 8080). One test server is '
8390
'started on each incrementing port, beginning with base_port, in the '
84-
'following order: goaway,max_streams,ping,rst_after_data,rst_after_header,'
91+
'following order: data_frame_padding,goaway,max_streams,'
92+
'no_df_padding_sanity_test,ping,rst_after_data,rst_after_header,'
8593
'rst_during_data'
8694
)
8795
return parser.parse_args()
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Copyright 2016, Google Inc.
2+
# All rights reserved.
3+
#
4+
# Redistribution and use in source and binary forms, with or without
5+
# modification, are permitted provided that the following conditions are
6+
# met:
7+
#
8+
# * Redistributions of source code must retain the above copyright
9+
# notice, this list of conditions and the following disclaimer.
10+
# * Redistributions in binary form must reproduce the above
11+
# copyright notice, this list of conditions and the following disclaimer
12+
# in the documentation and/or other materials provided with the
13+
# distribution.
14+
# * Neither the name of Google Inc. nor the names of its
15+
# contributors may be used to endorse or promote products derived from
16+
# this software without specific prior written permission.
17+
#
18+
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
19+
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
20+
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
21+
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
22+
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
23+
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
24+
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
25+
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
26+
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
27+
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28+
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29+
30+
import http2_base_server
31+
import logging
32+
import messages_pb2
33+
34+
# Set the number of padding bytes per data frame to be very large
35+
# relative to the number of data bytes for each data frame sent.
36+
_LARGE_PADDING_LENGTH = 255
37+
_SMALL_READ_CHUNK_SIZE = 5
38+
39+
class TestDataFramePadding(object):
40+
"""
41+
In response to an incoming request, this test sends headers, followed by
42+
data, followed by a reset stream frame. Client asserts that the RPC failed.
43+
Client needs to deliver the complete message to the application layer.
44+
"""
45+
def __init__(self, use_padding=True):
46+
self._base_server = http2_base_server.H2ProtocolBaseServer()
47+
self._base_server._handlers['DataReceived'] = self.on_data_received
48+
self._base_server._handlers['WindowUpdated'] = self.on_window_update
49+
self._base_server._handlers['RequestReceived'] = self.on_request_received
50+
51+
# _total_updates maps stream ids to total flow control updates received
52+
self._total_updates = {}
53+
# zero window updates so far for connection window (stream id '0')
54+
self._total_updates[0] = 0
55+
self._read_chunk_size = _SMALL_READ_CHUNK_SIZE
56+
57+
if use_padding:
58+
self._pad_length = _LARGE_PADDING_LENGTH
59+
else:
60+
self._pad_length = None
61+
62+
def get_base_server(self):
63+
return self._base_server
64+
65+
def on_data_received(self, event):
66+
logging.info('on data received. Stream id: %d. Data length: %d' % (event.stream_id, len(event.data)))
67+
self._base_server.on_data_received_default(event)
68+
if len(event.data) == 0:
69+
return
70+
sr = self._base_server.parse_received_data(event.stream_id)
71+
stream_bytes = ''
72+
# Check if full grpc msg has been read into the recv buffer yet
73+
if sr:
74+
response_data = self._base_server.default_response_data(sr.response_size)
75+
logging.info('Stream id: %d. total resp size: %d' % (event.stream_id, len(response_data)))
76+
# Begin sending the response. Add ``self._pad_length`` padding to each
77+
# data frame and split the whole message into data frames each carrying
78+
# only self._read_chunk_size of data.
79+
# The purpose is to have the majority of the data frame response bytes
80+
# be padding bytes, since ``self._pad_length`` >> ``self._read_chunk_size``.
81+
self._base_server.setup_send(response_data , event.stream_id, pad_length=self._pad_length, read_chunk_size=self._read_chunk_size)
82+
83+
def on_request_received(self, event):
84+
self._base_server.on_request_received_default(event)
85+
logging.info('on request received. Stream id: %s.' % event.stream_id)
86+
self._total_updates[event.stream_id] = 0
87+
88+
# Log debug info and try to resume sending on all currently active streams.
89+
def on_window_update(self, event):
90+
logging.info('on window update. Stream id: %s. Delta: %s' % (event.stream_id, event.delta))
91+
self._total_updates[event.stream_id] += event.delta
92+
total = self._total_updates[event.stream_id]
93+
logging.info('... - total updates for stream %d : %d' % (event.stream_id, total))
94+
self._base_server.on_window_update_default(event, pad_length=self._pad_length, read_chunk_size=self._read_chunk_size)

tools/doxygen/Doxyfile.c++

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -780,11 +780,11 @@ doc/fail_fast.md \
780780
doc/g_stands_for.md \
781781
doc/health-checking.md \
782782
doc/http-grpc-status-mapping.md \
783+
doc/http2-interop-test-descriptions.md \
783784
doc/internationalization.md \
784785
doc/interop-test-descriptions.md \
785786
doc/load-balancing.md \
786787
doc/naming.md \
787-
doc/negative-http2-interop-test-descriptions.md \
788788
doc/server-reflection.md \
789789
doc/server_reflection_tutorial.md \
790790
doc/server_side_auth.md \

tools/doxygen/Doxyfile.c++.internal

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -780,11 +780,11 @@ doc/fail_fast.md \
780780
doc/g_stands_for.md \
781781
doc/health-checking.md \
782782
doc/http-grpc-status-mapping.md \
783+
doc/http2-interop-test-descriptions.md \
783784
doc/internationalization.md \
784785
doc/interop-test-descriptions.md \
785786
doc/load-balancing.md \
786787
doc/naming.md \
787-
doc/negative-http2-interop-test-descriptions.md \
788788
doc/server-reflection.md \
789789
doc/server_reflection_tutorial.md \
790790
doc/server_side_auth.md \

tools/doxygen/Doxyfile.core

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -780,11 +780,11 @@ doc/fail_fast.md \
780780
doc/g_stands_for.md \
781781
doc/health-checking.md \
782782
doc/http-grpc-status-mapping.md \
783+
doc/http2-interop-test-descriptions.md \
783784
doc/internationalization.md \
784785
doc/interop-test-descriptions.md \
785786
doc/load-balancing.md \
786787
doc/naming.md \
787-
doc/negative-http2-interop-test-descriptions.md \
788788
doc/server-reflection.md \
789789
doc/server_reflection_tutorial.md \
790790
doc/server_side_auth.md \

tools/doxygen/Doxyfile.core.internal

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -780,11 +780,11 @@ doc/fail_fast.md \
780780
doc/g_stands_for.md \
781781
doc/health-checking.md \
782782
doc/http-grpc-status-mapping.md \
783+
doc/http2-interop-test-descriptions.md \
783784
doc/internationalization.md \
784785
doc/interop-test-descriptions.md \
785786
doc/load-balancing.md \
786787
doc/naming.md \
787-
doc/negative-http2-interop-test-descriptions.md \
788788
doc/server-reflection.md \
789789
doc/server_reflection_tutorial.md \
790790
doc/server_side_auth.md \

tools/internal_ci/linux/grpc_interop_badserver_java.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,5 +37,5 @@ cd $(dirname $0)/../../..
3737

3838
git submodule update --init
3939

40-
tools/run_tests/run_interop_tests.py -l java --use_docker --http2_badserver_interop $@
40+
tools/run_tests/run_interop_tests.py -l java --use_docker --http2_server_interop $@
4141

tools/internal_ci/linux/grpc_interop_badserver_python.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,5 +37,5 @@ cd $(dirname $0)/../../..
3737

3838
git submodule update --init
3939

40-
tools/run_tests/run_interop_tests.py -l python --use_docker --http2_badserver_interop $@
40+
tools/run_tests/run_interop_tests.py -l python --use_docker --http2_server_interop $@
4141

0 commit comments

Comments
 (0)