Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The header reads 261 bytes and then wants to use the full file data, so it's missing 261 bytes. How to handle this gracefully #80

Open
helloshaohua opened this issue Mar 31, 2020 · 4 comments
Labels

Comments

@helloshaohua
Copy link

helloshaohua commented Mar 31, 2020

The test case following:

func (o *OSSESSuite) Test_Upload_FileStream_SetContentType2() {
	file, err := os.Open("../../test/assets/HgWFAEPozfVdcst")
	assert.NoError(o.T(), err)

	// Gets the file size, equal to true.
	info, err := file.Stat()
	assert.Equal(o.T(), int64(87064), info.Size())

	head := make([]byte, 261)
	_, err = file.Read(head)
	assert.NoError(o.T(), err)
	match, err := filetype.Match(head)
	assert.Equal(o.T(), "image/png", match.MIME.Value)

	// Gets the file size, equal to true.
	info, err = file.Stat()
	assert.NoError(o.T(), err)
	assert.Equal(o.T(), int64(87064), info.Size())

	o.client.SetObjectContentType(match.MIME.Value)
	err = o.client.PutObject("HgWFAEPozfVdcst", file)
	assert.NoError(o.T(), err)
}

The test case result:

=== RUN   TestOSSESSuite
--- FAIL: TestOSSESSuite (0.03s)
=== RUN   TestOSSESSuite/Test_Upload_FileStream_SetContentType2
    --- FAIL: TestOSSESSuite/Test_Upload_FileStream_SetContentType2 (0.03s)
        osses_test.go:105: 
            	Error Trace:	osses_test.go:105
            	Error:      	Received unexpected error:
            	            	Put http://hello-world.oss-cn-beijing.aliyuncs.com/HgWFAEPozfVdcst: net/http: HTTP/1.x transport connection broken: http: ContentLength=87064 with Body length 86803
            	Test:       	TestOSSESSuite/Test_Upload_FileStream_SetContentType2
FAIL

The focus is on: ContentLength=87064 with Body length 86803

@h2non
Copy link
Owner

h2non commented Mar 31, 2020

file reader stream is stateful and when the stream is consumed, data will be drained. The simplest way to solve this is to reinitialize the reader by opening the file again or, alternatively, writing head data back into the reader buffer via file.WriteAt(head, 0)

@h2non h2non added the question label Mar 31, 2020
@helloshaohua helloshaohua changed the title The header reads 261 bytes and then wants to use the full file data, so it's missing 264 bytes. How to handle this gracefully The header reads 261 bytes and then wants to use the full file data, so it's missing 261 bytes. How to handle this gracefully Apr 1, 2020
@helloshaohua
Copy link
Author

@h2non I tried to write it, but it didn't work

@h2non
Copy link
Owner

h2non commented Apr 1, 2020

You could just close the first file reader stream, then open it again and read the full content.

@sergolius
Copy link

@helloshaohua
There are two options:

  1. As your case is a file you can use Seek method _, err := file.Seek(0, io.SeekStart). Next read start from zero;
  2. For any case is to use []byte buffer, like you did, but also chain readers with io.MultiReader
file, err := os.Open("../../test/assets/HgWFAEPozfVdcst")
// ...
head := make([]byte, 262)
_, _ = io.ReadFull(file, head)
match, err := filetype.Match(head)
// ...
reader := io.MultiReader(bytes.NewBuffer(head), file)
// ...
err = o.client.PutObject("HgWFAEPozfVdcst", reader) // put io.MultiReader

p.s. Don't forget to close the file/reader

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants
@h2non @sergolius @helloshaohua and others