Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multipart body length limit 16384 exceeded in NET CORE #50252

Closed
mohpais opened this issue Aug 22, 2023 · 8 comments · Fixed by #59646
Closed

Multipart body length limit 16384 exceeded in NET CORE #50252

mohpais opened this issue Aug 22, 2023 · 8 comments · Fixed by #59646
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions
Milestone

Comments

@mohpais
Copy link

mohpais commented Aug 22, 2023

What I am trying to do

I'm trying to access the API in my project (NET Core 6 WebApi) to get data from the response for the project purpose. The API response that I will access is XML data which consists of some data such as title, description, creation date, and also photo data in it.

Here's how I access the API using IHttpClientFactory in mycontroller:

[Route("Get")]
[HttpGet]
public async Task<IActionResult> Get()
{
   HttpResponseMessage response = await _httpClient.GetAsync('the_name_of_the_api_i_will_be_accessing');
   if (response.IsSuccessStatusCode)
   {
      // Check if the response is multipart/mixed
      var headers = response.Content.Headers;
      if (headers.ContentType?.MediaType == "multipart/mixed")
      {
         // Get the boundary from the content type header
         string boundary = headers.ContentType.Parameters.FirstOrDefault(p => p.Name == "boundary")?.Value ?? "";
         if (string.IsNullOrWhiteSpace(boundary) || (boundary.Length > new FormOptions().MultipartBoundaryLengthLimit))
         {
            throw new InvalidDataException("Boundary is missing or too long.");
         }
         Stream contentStream = await response.Content.ReadAsStreamAsync();
         // Create a new reader based on the boundary
         var reader = new MultipartReader(boundary, contentStream);

         // Start reading sections from the MultipartReader until there are no more
         var section = await reader.ReadNextSectionAsync();
         while (section != null)
         {
            // Check the content type of each section
            var contentType = new ContentType(section.ContentType);
            // Read and process the content of each section based on its content type
            if (contentType.MediaType == "application/xml")
            {
               // This section contains XML data, you can parse and process it as needed
               var xmlContent = await section.ReadAsStringAsync();
               // Process the XML content
            }
            else if (contentType.MediaType == "image/jpeg")
            {
               // This section contains an image (binary data), so i can save it or process it as needed
               using (var imageStream = File.Create("path/to/save/image.jpg"))
               {
                  await section.Body.CopyToAsync(imageStream);
               }
            }
            // Read the next section
            section = await reader.ReadNextSectionAsync();
         }
      }
      return Ok();
   }
   else
   {
      return StatusCode((int)response.StatusCode, "API request failed");
   }
}

However, when I try to read section from the MultipartReader in code:

var section = await reader.ReadNextSectionAsync();

I got an Error like this

System.IO.InvalidDataException: Multipart body length limit 16384 exceeded.

Things i've tried:

I've tried several ways to overcome this, one of which is to increase the MultipartBodyLengthLimit in Program.cs

services.Configure<FormOptions>(options =>
{
  options.MultipartBodyLengthLimit = 104857600; // Set to a higher value for the body
});

and I still get the same error

Postman response

In postman, it will return XML data and provides information with content-type multipart/mixed.

Here is the response result in postman:

<?xml version="1.0" encoding="utf-8"?>
<EventNotificationAlert version="2.0" xmlns="http://www.isapi.org/ver20/XMLSchema">
<dateTime>2023-08-22T09:47:30.486+07:00</dateTime>
<eventState>active</eventState>
<eventDescription>ANPR</eventDescription>
...
</EventNotificationAlert>
-----------------------------7daf10c20d06
Content-Disposition: form-data; name="detectionPicture"; filename="detectionPicture.jpg"
Content-Type: image/jpeg
Content-Length: 548804

How to solve this problem?

@wenz
Copy link
Contributor

wenz commented Aug 22, 2023

Maybe it's related to this TODO comment:

// TODO: HeadersLengthLimit can't be modified until after the constructor.
(the limit can only be updated after the constructor has run, so I assume it sticks with the default)

@mohpais
Copy link
Author

mohpais commented Aug 22, 2023

Maybe it's related to this TODO comment:

// TODO: HeadersLengthLimit can't be modified until after the constructor.

(the limit can only be updated after the constructor has run, so I assume it sticks with the default)

Do you mean like this

var reader = new MultipartReader(boundary, contentStream);
reader.HeadersLengthLimit = 32768; // Set to a higher value, if necessary
reader.BodyLengthLimit = 1048576;  // Set to a higher value for the body, e.g., 1 MB

@mohpais
Copy link
Author

mohpais commented Aug 22, 2023

Maybe it's related to this TODO comment:

// TODO: HeadersLengthLimit can't be modified until after the constructor.

(the limit can only be updated after the constructor has run, so I assume it sticks with the default)

Do you mean like this

var reader = new MultipartReader(boundary, contentStream);
reader.HeadersLengthLimit = 32768; // Set to a higher value, if necessary
reader.BodyLengthLimit = 1048576;  // Set to a higher value for the body, e.g., 1 MB

Its still not working btw

@wenz
Copy link
Contributor

wenz commented Aug 22, 2023

Yes, because in the MultipartReader constructor, the MultipartReaderStream instance is created, using the default HeadersLengthLimit value.

_currentStream = new MultipartReaderStream(_stream, _boundary) { LengthLimit = HeadersLengthLimit };

@Tratcher
Copy link
Member

Postman response

Is there a boundary and headers at the start of that XML? There should be.

It's also possible that you're not passing the correct boundary value to the reader. E.g. the boundary likely has quotes around it that you need to remove.

@mohpais
Copy link
Author

mohpais commented Aug 23, 2023

Postman response

Is there a boundary and headers at the start of that XML? There should be.

It's also possible that you're not passing the correct boundary value to the reader. E.g. the boundary likely has quotes around it that you need to remove.

Postman response

Is there a boundary and headers at the start of that XML? There should be.

It's also possible that you're not passing the correct boundary value to the reader. E.g. the boundary likely has quotes around it that you need to remove.

Do u mean this?
Header

@Tratcher
Copy link
Member

Tratcher commented Aug 23, 2023

No, the body should look like this:

-----------------------------7daf10c20d06
Content-Disposition: form-data; name="metadata"
Content-Type: text/xml

<?xml version="1.0" encoding="utf-8"?>
<EventNotificationAlert version="2.0" xmlns="http://www.isapi.org/ver20/XMLSchema">
<dateTime>2023-08-22T09:47:30.486+07:00</dateTime>
<eventState>active</eventState>
<eventDescription>ANPR</eventDescription>
...
</EventNotificationAlert>
-----------------------------7daf10c20d06
Content-Disposition: form-data; name="detectionPicture"; filename="detectionPicture.jpg"
Content-Type: image/jpeg
Content-Length: 548804
...

MultipartReader discards anything before the first boundary, but would throw if it's too long.

@amcasey amcasey added area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions and removed area-runtime labels Aug 25, 2023
@dotnet-policy-service dotnet-policy-service bot added the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Feb 6, 2024
@wtgodbe wtgodbe removed the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Feb 6, 2024
@dotnet-policy-service dotnet-policy-service bot added the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Feb 6, 2024
@wtgodbe wtgodbe removed the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Feb 13, 2024
@dotnet dotnet deleted a comment from dotnet-policy-service bot Feb 13, 2024
@dotnet dotnet deleted a comment from dotnet-policy-service bot Feb 13, 2024
@Azizjan-leo
Copy link

Still getting the same error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants