Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Content-Range header length does not match the provided number of bytes. #747

Closed
RomanSter opened this issue Jul 15, 2024 · 1 comment
Labels
status:waiting-for-triage An issue that is yet to be reviewed or assigned type:bug A broken experience

Comments

@RomanSter
Copy link

Describe the bug

I am trying to upload a large file using the msgraph sdk.

taking examples from this issue and tweaking a bit to fit the updated SDK, still getting the "The Content-Range header length does not match the provided number of bytes" error.

when executing "SendNoContent", I get the error.

Upon investigation and debugging, I found that when the http library that you use is compressing the request body:

`

// Intercept is invoked by the middleware pipeline to either move the request/response
// to the next middleware in the pipeline
func (c *CompressionHandler) Intercept(pipeline Pipeline, middlewareIndex int, req *http.Request) (*http.Response, error) {
reqOption, ok := req.Context().Value(compressKey).(compression)
if !ok {
	reqOption = c.options
}

obsOptions := GetObservabilityOptionsFromRequest(req)
ctx := req.Context()
var span trace.Span
if obsOptions != nil {
	ctx, span = otel.GetTracerProvider().Tracer(obsOptions.GetTracerInstrumentationName()).Start(ctx, 
   "CompressionHandler_Intercept")
	span.SetAttributes(attribute.Bool("com.microsoft.kiota.handler.compression.enable", true))
	defer span.End()
	req = req.WithContext(ctx)
}

if !reqOption.ShouldCompress() || req.Body == nil {
	return pipeline.Next(req, middlewareIndex)
}
if span != nil {
	span.SetAttributes(attribute.Bool("http.request_body_compressed", true))
}

unCompressedBody, err := io.ReadAll(req.Body)
unCompressedContentLength := req.ContentLength
if err != nil {
	if span != nil {
		span.RecordError(err)
	}
	return nil, err
}

compressedBody, size, err := compressReqBody(unCompressedBody)
if err != nil {
	if span != nil {
		span.RecordError(err)
	}
	return nil, err
}

req.Header.Set("Content-Encoding", "gzip")
req.Body = compressedBody
req.ContentLength = int64(size)

if span != nil {
	span.SetAttributes(attribute.Int64("http.request_content_length", req.ContentLength))
}

// Sending request with compressed body
resp, err := pipeline.Next(req, middlewareIndex)
if err != nil {
	return nil, err
}

// If response has status 415 retry request with uncompressed body
if resp.StatusCode == 415 {
	delete(req.Header, "Content-Encoding")
	req.Body = io.NopCloser(bytes.NewBuffer(unCompressedBody))
	req.ContentLength = unCompressedContentLength

	if span != nil {
		span.SetAttributes(attribute.Int64("http.request_content_length", req.ContentLength),
			attribute.Int("http.request_content_length", 415))
	}

	return pipeline.Next(req, middlewareIndex)
}

return resp, nil
}

   func compressReqBody(reqBody []byte) (io.ReadCloser, int, error) {
var buffer bytes.Buffer
gzipWriter := gzip.NewWriter(&buffer)
if _, err := gzipWriter.Write(reqBody); err != nil {
	return nil, 0, err
}

if err := gzipWriter.Close(); err != nil {
	return nil, 0, err
}

return io.NopCloser(&buffer), buffer.Len(), nil
}

`

as you can see after the "compressReqBody" function is invoked, it updates the ContentLength header to a new length, but ignores the Content-Range header that is still based on the uncompressed body.

Expected behavior

The SDK should upload the file to the designated path in SharePoint.

How to reproduce

this is the code that I am using:

`

    // create upload session
driveItemId := "root:/test.txt:"

requestBody := drives.NewItemItemsItemCreateuploadsessionCreateUploadSessionPostRequestBody()
name = "name"
size := len(data)
requestBody.SetAdditionalData(map[string]any{"@microsoft.graph.conflictBehavior": "replace", "fileSize": &size, "name": &name})

uploadSession, err := client.Drives().ByDriveId("driveID").Items().ByDriveItemId(driveItemId).CreateUploadSession().Post(context.Background(), requestBody, nil)
if err != nil {
	log.Fatal(err)
}

requestInfo := abstractions.NewRequestInformation()
requestInfo.UrlTemplate = *uploadSession.GetUploadUrl()

requestInfo.Method = abstractions.PUT
bytes := fmt.Sprintf("bytes 0-%d/%d", len(data)-1, len(data))

requestInfo.Headers.Add("Content-Length", fmt.Sprintf("%d", len(data)))
requestInfo.Headers.Add("Content-Range", fmt.Sprintf("%s", bytes))
requestInfo.Headers.Add("Content-Type", "application/octet-stream")

requestInfo.Content = []byte(data)
// upload file
errorMapping := abstractions.ErrorMappings{
	"4XX": odataerrors.CreateODataErrorFromDiscriminatorValue,
	"5XX": odataerrors.CreateODataErrorFromDiscriminatorValue,
}
err = client.BaseRequestBuilder.RequestAdapter.SendNoContent(context.Background(), requestInfo, errorMapping)
if err != nil {
	log.Fatal(err)
}

`

SDK Version

1.45.0

Latest version known to work for scenario above?

No response

Known Workarounds

I managed to solve this by compressing the body with the same gzip library and setting the Content-Range header values with the new compressed bytes length.

`

var buf bytes2.Buffer
g := gzip.NewWriter(&buf)
if _, err = g.Write([]byte(data)); err != nil {
	slog.Error(err.Error())
	return
}
if err = g.Close(); err != nil {
	slog.Error(err.Error())
	return
}
bytes := fmt.Sprintf("bytes 0-%d/%d", len(buf.Bytes())-1, len(buf.Bytes()))

requestInfo := abstractions.NewRequestInformation()
requestInfo.UrlTemplate = *uploadSession.GetUploadUrl()
requestInfo.Headers.Add("Content-Length", fmt.Sprintf("%d", len(buf.Bytes())))
requestInfo.Headers.Add("Content-Range", fmt.Sprintf("%s", bytes))
requestInfo.Method = abstractions.PUT
requestInfo.Headers.Add("Content-Type", "application/octet-stream")

`

Debug output

Click to expand log ```
</details>


### Configuration

- OS MacOS Ventura
- Apple M2 Pro

- Do you know whether it is specific to that configuration? unaware.

### Other information

_No response_
@andrueastman
Copy link
Member

Thanks for raising this @RomanSter and looking into the root cause of this.

Looks like this is a duplicate of microsoftgraph/msgraph-sdk-go-core#295. We'll close this for now to track this there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status:waiting-for-triage An issue that is yet to be reviewed or assigned type:bug A broken experience
Projects
None yet
Development

No branches or pull requests

2 participants