Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resizing GIFs is failing on linux, hanging / breaking on macOS for larger GIFs #255

Open
AttilaTheFun opened this issue Jan 22, 2022 · 4 comments

Comments

@AttilaTheFun
Copy link
Contributor

Hi @Elad-Laufer and @davidbyttow! I wanted to try out the GIF resizing on my server in a Go container-based lambda function.

Unfortunately, while I was able to resize a GIF locally with the latest version of govips, when I tried to resize it in my lambda function I got this error:

"VipsOperation: class \"magicksave_buffer\" not found\n\n
Stack:\ngoroutine 1 [running]:\nruntime/debug.Stack()\n\t/usr/local/go/src/runtime/debug/stack.go:24 
+0x65\ngithub.com/davidbyttow/govips/v2/vips.handleVipsError()\n\t/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/error.go:39 
+0x57\ngithub.com/davidbyttow/govips/v2/vips.handleSaveBufferError(0xc000044000)\n\t/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/error.go:32 +0x25\ngithub.com/davidbyttow/govips/v2/vips.vipsSaveToBuffer({0x7ffbacea87f0, 0x0, 0x5, 0x0, 0x0, 0x4b, 0x0, 0x0, 0x1, 0x0, ...})\n\t/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/foreign.go:437 
+0xa5\ngithub.com/davidbyttow/govips/v2/vips.vipsSaveGIFToBuffer(0x7ffbacea87f0, {0x0, 0x0})\n\t/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/foreign.go:432 
+0xf8\ngithub.com/davidbyttow/govips/v2/vips.(*ImageRef).ExportGIF(0xc000244050, 0xc2)\n\t/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/image.go:860 
+0x59\ngithub.com/attilathefun/utils/imageutil.(*Image).EncodeToBytes(...

From the name of the missing class, I assumed that govips might be depending on imagemagick so I installed that but still got the same error.

Do you have any idea what could be causing this?

My Dockerfile is as follows:

###############################################################################
# STEP 1: Build Executable ####################################################
###############################################################################

FROM golang:1.17-alpine as builder

# Install packages:
# RUN apk add pkgconfig git gcc musl-dev vips-dev
RUN apk add pkgconfig git gcc musl-dev vips-dev

# Resolve the buildtime arguments:
ARG GITHUB_URL

# Configure git:
RUN git config --global url.${GITHUB_URL}.insteadOf "https://github.com/"
RUN go env -w GOPRIVATE=github.com/attilathefun

# Work in the /var/task directory:
WORKDIR /var/task

# Copy over the go module files and download the deps so they can be cached:
COPY go.* ./
RUN go mod download

# Copy the application directory and build the executable:
COPY ./app ./app
RUN GOOS=linux GOARCH=amd64 go build ./app/main.go

###############################################################################
# STEP 2: Build Small Image ###################################################
###############################################################################

FROM alpine

# Install libvips:
RUN apk add vips-dev imagemagick

# Copy our static executable.
COPY --from=builder /var/task/main /main

###############################################################################
# STEP 4: Run Container #######################################################
###############################################################################

# Set the entrypoint:
ENTRYPOINT [ "/main" ]

Additionally, I was able to resize this first GIF locally on macOS:

Before After
RESIZABLE-GIF RESIZED-GIF

But this second, larger GIF caused one core of my CPU (M1 Max) to be pegged at 100% for a couple of minutes while using ~800MB of RAM before it spat out the following broken GIF image which seems to just have placed all of the frames vertically in one super tall image.

These files were too large to upload to Github directly so I put them in an S3 bucket to share with you:

Original:
https://logan-shared-files.s3.us-west-2.amazonaws.com/FAILS-TO-RESIZE.gif

Broken resized:
https://logan-shared-files.s3.us-west-2.amazonaws.com/FAILED-TO-RESIZE.gif

@aksdb
Copy link

aksdb commented Feb 10, 2022

What version of libvips are you using? Native animated GIF support is only present since 8.12. Before that it likely falls back to imagemagick which is HORRIBLE for animated gifs.

@AttilaTheFun
Copy link
Contributor Author

Hi @aksdb! Sorry it's been a while 😅
I was moving across the country and starting a new job so I haven't had time to work on this until now.

Following @vansante's PR, I cleaned out all of my brew packages and did a clean install on my M1 Max MBP as follows:

  • Used the official Go macOS Apple Silicon installer to upgrade to go 1.18.1
  • Having uninstalled all of my brew casks and formulae, ran brew install pkg-config vips
  • Installed govips 2.11.0 via a go.mod file and ran go mod tidy -compat=1.17

I was then able to resize a few gifs successfully, including ones larger than the broken gif linked above. I can see that @vansante's gifsave PR substantially improved performance. However, for some reason that one gif is still failing to resize.

The error I'm getting is:

2022/05/08 14:29:44 [VIPS.info] g_getenv( "PATH" ) == "/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/Library/Apple/usr/bin:/Users/logan/go/bin:/Users/logan/Developer/Go/github.com/attilathefun/scripts"
2022/05/08 14:29:44 [VIPS.info] looking in "/opt/homebrew/bin" for "govips"
2022/05/08 14:29:44 [VIPS.info] looking in "/opt/homebrew/sbin" for "govips"
2022/05/08 14:29:44 [VIPS.info] looking in "/usr/local/bin" for "govips"
2022/05/08 14:29:44 [VIPS.info] looking in "/usr/bin" for "govips"
2022/05/08 14:29:44 [VIPS.info] looking in "/bin" for "govips"
2022/05/08 14:29:44 [VIPS.info] looking in "/usr/sbin" for "govips"
2022/05/08 14:29:44 [VIPS.info] looking in "/sbin" for "govips"
2022/05/08 14:29:44 [VIPS.info] looking in "/usr/local/go/bin" for "govips"
2022/05/08 14:29:44 [VIPS.info] looking in "/Library/Apple/usr/bin" for "govips"
2022/05/08 14:29:44 [VIPS.info] looking in "/Users/logan/go/bin" for "govips"
2022/05/08 14:29:44 [VIPS.info] looking in "/Users/logan/Developer/Go/github.com/attilathefun/scripts" for "govips"
2022/05/08 14:29:44 [VIPS.info] trying for dir = "/Users/logan/Developer/Go/github.com/attilathefun/imageutil/govips", name = "govips"
2022/05/08 14:29:44 [VIPS.info] canonicalised path = "/Users/logan/Developer/Go/github.com/attilathefun/imageutil"
2022/05/08 14:29:44 [VIPS.info] VIPS_PREFIX = /opt/homebrew/Cellar/vips/8.12.2_2
2022/05/08 14:29:44 [VIPS.info] VIPS_LIBDIR = /opt/homebrew/Cellar/vips/8.12.2_2/lib
2022/05/08 14:29:44 [VIPS.info] prefix = /opt/homebrew/Cellar/vips/8.12.2_2
2022/05/08 14:29:44 [VIPS.info] libdir = /opt/homebrew/Cellar/vips/8.12.2_2/lib
2022/05/08 14:29:44 [VIPS.info] searching "/opt/homebrew/Cellar/vips/8.12.2_2/lib/vips-plugins-8.12"
2022/05/08 14:29:44 [govips.info] vips 8.12.2 started with concurrency=1 cache_max_files=0 cache_max_mem=52428800 cache_max=100
2022/05/08 14:29:44 [govips.info] registered image type loader type=webp
2022/05/08 14:29:44 [govips.info] registered image type loader type=heif
2022/05/08 14:29:44 [govips.info] registered image type loader type=jp2k
2022/05/08 14:29:44 [govips.info] registered image type loader type=gif
2022/05/08 14:29:44 [govips.info] registered image type loader type=magick
2022/05/08 14:29:44 [govips.info] registered image type loader type=pdf
2022/05/08 14:29:44 [govips.info] registered image type loader type=svg
2022/05/08 14:29:44 [govips.info] registered image type loader type=jpeg
2022/05/08 14:29:44 [govips.info] registered image type loader type=png
2022/05/08 14:29:44 [govips.info] registered image type loader type=tiff
2022/05/08 14:29:44 [govips.info] registered image type loader type=heif
2022/05/08 14:29:44 [VIPS.info] residual reducev by 0.888889
2022/05/08 14:29:44 [VIPS.info] reducev: 7 point mask
2022/05/08 14:29:44 [VIPS.info] residual reduceh by 0.888889
2022/05/08 14:29:44 [VIPS.info] reduceh: 7 point mask
--- FAIL: TestResizeGIF (13.74s)
    resize_test.go:215: gifsave_buffer: frame too large
        
        Stack:
        goroutine 4 [running]:
        runtime/debug.Stack()
        	/usr/local/go/src/runtime/debug/stack.go:24 +0x64
        github.com/davidbyttow/govips/v2/vips.handleVipsError()
        	/Users/logan/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/error.go:38 +0x40
        github.com/davidbyttow/govips/v2/vips.handleSaveBufferError(0x0)
        	/Users/logan/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/error.go:31 +0x44
        github.com/davidbyttow/govips/v2/vips.vipsSaveToBuffer({0x14b8707f0, 0x0, 0x5, 0x0, 0x0, 0x4b, 0x0, 0x0, 0x1, 0x0, ...})
        	/Users/logan/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/foreign.go:440 +0xb0
        github.com/davidbyttow/govips/v2/vips.vipsSaveGIFToBuffer(0x14b8707f0, {0x0, 0x4b, 0x0, 0x7, 0x8})
        	/Users/logan/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/foreign.go:435 +0xd4
        github.com/davidbyttow/govips/v2/vips.(*ImageRef).ExportGIF(0x14000029810, 0x1400001a720)
        	/Users/logan/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/image.go:918 +0xb0
        github.com/attilathefun/imageutil.(*Image).EncodeToBytes(0x140000102f8, {0x10058af6e, 0x3})
        	/Users/logan/Developer/Go/github.com/attilathefun/imageutil/encode.go:74 +0xdc
        github.com/attilathefun/imageutil.(*Image).EncodeToFile(0x140000102f8, {0x10058af6e, 0x3}, 0x14000010358)
        	/Users/logan/Developer/Go/github.com/attilathefun/imageutil/encode.go:23 +0x98
        github.com/attilathefun/imageutil.TestResizeGIF(0x14000125860)
        	/Users/logan/Developer/Go/github.com/attilathefun/imageutil/resize_test.go:213 +0x82c
        testing.tRunner(0x14000125860, 0x1006718f0)
        	/usr/local/go/src/testing/testing.go:1439 +0x178
        created by testing.(*T).Run
        	/usr/local/go/src/testing/testing.go:1486 +0x548

Is there something wrong with the gif? I noticed that it also doesn't show its size in Finder which is a little odd.

@AttilaTheFun
Copy link
Contributor Author

Update @davidbyttow @aksdb @vansante -- I was able to resize the same GIF I'm unable to resize with this library using ezgif.com so it's definitely possible. I'm wondering if this is a bug with the gifsave library?

I've hosted the two gifs for you to test with in an S3 bucket because they're too big for Github:
Original (fails to resize with govips): https://logan-shared-files.s3.us-west-2.amazonaws.com/ORIGINAL.gif
Resized (using ezgif.com): https://logan-shared-files.s3.us-west-2.amazonaws.com/RESIZED.gif

The code I'm using to resize it is:

func TestResizeGIF(t *testing.T) {

	// Configure the test:
	inputFileType := mediautil.ImageFileTypeGIF
	inputICCProfileName := ICCProfileNamesRGBIEC6196621
	inputFileName := inputICCProfileName + "." + mediautil.ImageFileTypeToFileExtension(inputFileType).String()
	inputFilePath := filepath.Join(inputTestImagesFolder, inputFileName)

	// Open the input file:
	inputFile, err := fileutil.OpenFile(inputFilePath)
	if err != nil {
		t.Fatal(err)
	}
	defer inputFile.Close()

	// Decode the image:
	image, err := DecodeFromFile(inputFile)
	if err != nil {
		t.Fatal(err)
	}

	// Resize the image:
	var resizedWidth uint64 = 240
	var resizedHeight uint64 = 240
	err = image.Resize(resizedWidth, resizedWidth)
	if err != nil {
		t.Fatal(err)
	}

	// Create the output file:
	outputFileType := mediautil.ImageFileTypeGIF
	outputFileName := inputICCProfileName +
		" Resized to " +
		strconv.FormatUint(uint64(resizedWidth), 10) + "x" +
		strconv.FormatUint(uint64(resizedHeight), 10) + "." +
		mediautil.ImageFileTypeToFileExtension(outputFileType).String()
	outputFilePath := filepath.Join(outputTestImagesFolder, outputFileName)
	outputFile, err := fileutil.CreateFile(outputFilePath)
	if err != nil {
		t.Fatal(err)
	}
	defer outputFile.Close()

	// Encode the decoded image to a file:
	err = image.EncodeToFile(outputFileType, outputFile)
	if err != nil {
		t.Fatal(err)
	}

	log.Println("success")
}

Where Resize is defined:

// Resize - Resizes the input image via aspect-fill to completely cover the given width and height.
// E.g. if the input image has dimensions 640x480 (aspect ratio 1.5),
// and the given width and height are 240x240, the resized image will have dimensions 320x240.
func (img *Image) Resize(width uint64, height uint64) error {
	if width == 0 || height == 0 {
		return errors.New("you must specify values for both width and height")
	}

	// Determine the aspect ratios of the original image and the filled region:
	originalImageAspectRatio := img.AspectRatio()
	filledRegionAspectRatio := float64(width) / float64(height)

	// Determine the scale for the image:
	scale := float64(0)
	if filledRegionAspectRatio < originalImageAspectRatio {
		// The original image is wider than the filled region, so the scale is determined
		// by the ratio of the original image's height to the resized height.
		// E.g. if the original image has dimensions 1920x1080 and the filled region has dimensions 240x240,
		// the scale is 240/1080, i.e. 0.222...
		scale = float64(height) / float64(img.Height())
	} else {
		// The original image is narrower than the filled region, so the scale is determined
		// by the ratio of the original image's width to the resized width.
		// E.g. if the original image has dimensions 1080x1920 and the filled region has dimensions 240x240,
		// the scale is 240/1920, i.e. 0.125.
		scale = float64(width) / float64(img.Width())
	}

	// Resize the image:
	return img.ResizeScale(scale)
}

// Resize - Resizes the input image with the given scale, maintaining aspect ratio.
func (img *Image) ResizeScale(scale float64) error {

	// Resize the image:
	err := img.vipsImage.Resize(scale, vips.KernelAuto)
	if err != nil {
		return err
	}

	return nil
}

And EncodeToFile effectively calls this:

		// Export a GIF:
		ep := vips.NewGifExportParams()
		bytes, _, err := img.vipsImage.ExportGIF(ep)
		if err != nil {
			return nil, err
		}

		return bytes, nil

@hungtcs
Copy link

hungtcs commented Jul 22, 2024

I had the same problem when trying to resize a GIF

https://commons.wikimedia.org/wiki/Category:Animated_GIF_files_between_50_MP_and_100_MP#/media/File:01_Rechtwinkliges_Dreieck-inverser_Satz.gif

  • libvips version: 8.15.2
  • OS version: MacOS 14.5 (23F79)
......

2024/07/22 17:06:33 [VIPS.info] residual reducev by 0.809899
2024/07/22 17:06:33 [VIPS.info] reducev: 9 point mask
2024/07/22 17:06:33 [VIPS.info] residual reduceh by 0.809899
2024/07/22 17:06:33 [VIPS.info] reduceh: 9 point mask
2024/07/22 17:06:33 [VIPS.info] cgifsave: 0 frames
2024/07/22 17:06:33 [VIPS.info] cgifsave: 0 unique palettes
panic: gifsave_buffer: frame too large

Stack:
goroutine 1 [running]:
runtime/debug.Stack()
        /opt/homebrew/Cellar/go/1.22.2/libexec/src/runtime/debug/stack.go:24 +0x64
github.com/davidbyttow/govips/v2/vips.handleVipsError()
        /Users/hungtcs/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/error.go:38 +0x38
github.com/davidbyttow/govips/v2/vips.handleSaveBufferError(0x140001a0460?)
        /Users/hungtcs/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/error.go:31 +0x28
github.com/davidbyttow/govips/v2/vips.vipsSaveToBuffer({0x12e0750b0, 0x0, 0x5, 0x0, 0x0, 0x4b, 0x0, 0x0, 0x1, 0x0, ...})
        /Users/hungtcs/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/foreign.go:495 +0x98
github.com/davidbyttow/govips/v2/vips.vipsSaveGIFToBuffer(0x12e0750b0, {0x0, 0x4b, 0x0, 0x7, 0x8})
        /Users/hungtcs/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/foreign.go:475 +0xec
github.com/davidbyttow/govips/v2/vips.(*ImageRef).ExportGIF(0x140000a80f0, 0x140000122c0?)
        /Users/hungtcs/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/image.go:1002 +0x88
github.com/davidbyttow/govips/v2/vips.(*ImageRef).ExportNative(0x140001b3b58?)
        /Users/hungtcs/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/image.go:915 +0x14c
github.com/davidbyttow/govips/v2/vips.(*ImageRef).Export(0x140000a80f0?, 0x3fe9eab0cf55867b?)
        /Users/hungtcs/go/pkg/mod/github.com/davidbyttow/govips/[email protected]/vips/image.go:826 +0x398
govips-test/image_processor.Normalize({0x14000692000, 0x2b794e, 0x2b794f})
        /Users/hungtcs/Workspaces/@temp/govips-test/image_processor/normalize.go:54 +0x190
main.main()
        /Users/hungtcs/Workspaces/@temp/govips-test/main.go:24 +0xcc


goroutine 1 [running]:
main.main()
        /Users/hungtcs/Workspaces/@temp/govips-test/main.go:26 +0x1b0
exit status 2

This is my test code

func Normalize(buf []byte) (_ []byte, _ *vips.ImageMetadata, err error) {
	imageType := vips.DetermineImageType(buf)
	if imageType == vips.ImageTypeUnknown {
		return nil, nil, fmt.Errorf("unknown image type")
	}

	importParams := vips.NewImportParams()
	importParams.NumPages.Set(-1)
	imageRef, err := vips.LoadImageFromBuffer(buf, importParams)
	if err != nil {
		return nil, nil, err
	}
	defer imageRef.Close()

	err = imageRef.AutoRotate()
	if err != nil {
		return nil, nil, err
	}

	originWidth := imageRef.Width()
	if originWidth > MAX_WIDTH {
		scale := MAX_WIDTH / float64(originWidth)
		err = imageRef.Resize(scale, vips.KernelLanczos3)
		if err != nil {
			return nil, nil, err
		}
	}

	exportParams := vips.NewDefaultExportParams()
	exportParams.Quality = QUALITY
	exportParams.Lossless = false
	exportParams.Interlaced = true
	exportParams.Compression = 6
	exportParams.StripMetadata = true

	return imageRef.Export(vips.NewDefaultExportParams())
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants