45 lines
1.3 KiB
Diff
45 lines
1.3 KiB
Diff
Origin: https://github.com/webmproject/libvpx/commit/61c4d556bd03b97d84e3fa49180d14bde5a62baa
|
|
|
|
From 61c4d556bd03b97d84e3fa49180d14bde5a62baa Mon Sep 17 00:00:00 2001
|
|
From: Wan-Teh Chang <wtc@google.com>
|
|
Date: Fri, 12 Apr 2024 15:48:04 -0700
|
|
Subject: [PATCH] Fix a bug in alloc_size for high bit depths
|
|
|
|
I introduced this bug in commit 2e32276:
|
|
https://chromium-review.googlesource.com/c/webm/libvpx/+/5446333
|
|
|
|
I changed the line
|
|
|
|
stride_in_bytes = (fmt & VPX_IMG_FMT_HIGHBITDEPTH) ? s * 2 : s;
|
|
|
|
to three lines:
|
|
|
|
s = (fmt & VPX_IMG_FMT_HIGHBITDEPTH) ? s * 2 : s;
|
|
if (s > INT_MAX) goto fail;
|
|
stride_in_bytes = (int)s;
|
|
|
|
But I didn't realize that `s` is used later in the calculation of
|
|
alloc_size.
|
|
|
|
As a quick fix, undo the effect of s * 2 for high bit depths after `s`
|
|
has been assigned to stride_in_bytes.
|
|
|
|
Bug: chromium:332382766
|
|
Change-Id: I53fbf405555645ab1d7254d31aadabe4f426be8c
|
|
(cherry picked from commit 74c70af01667733483dc69298b8921779f5f6ff3)
|
|
---
|
|
vpx/src/vpx_image.c | 1 +
|
|
1 file changed, 1 insertion(+)
|
|
|
|
--- a/vpx/src/vpx_image.c
|
|
+++ b/vpx/src/vpx_image.c
|
|
@@ -95,6 +95,7 @@ static vpx_image_t *img_alloc_helper(vpx
|
|
s = (fmt & VPX_IMG_FMT_HIGHBITDEPTH) ? s * 2 : s;
|
|
if (s > INT_MAX) goto fail;
|
|
stride_in_bytes = (int)s;
|
|
+ s = (fmt & VPX_IMG_FMT_HIGHBITDEPTH) ? s / 2 : s;
|
|
|
|
/* Allocate the new image */
|
|
if (!img) {
|
|
|