packages/golang
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

!199 [Backport] fix some CVE

Browse Source 
From: @dayshappy 
Reviewed-by: @jing-rui 
Signed-off-by: @jing-rui
This commit is contained in:
openeuler-ci-bot 2023-05-11 03:19:18 +00:00 committed by Gitee
commit 1197529257
No known key found for this signature in database
GPG Key ID: 173E9B9CA92EEF8F
15 changed files with 4498 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

134
0003-release-branch.go1.19-path-filepath-do-not-Clean-a-..patch Normal file
View File

@ -0,0 +1,134 @@
From 3345ddca41f00f9ed6fc3c1a36f6e2bede02d7ff Mon Sep 17 00:00:00 2001
From: Damien Neil <dneil@google.com>
Date: Mon, 12 Dec 2022 16:43:37 -0800
Subject: [PATCH] [release-branch.go1.19] path/filepath: do not
Clean("a/../c:/b") into c:\b on Windows
Do not permit Clean to convert a relative path into one starting
with a drive reference. This change causes Clean to insert a .
path element at the start of a path when the original path does not
start with a volume name, and the first path element would contain
a colon.
This may introduce a spurious but harmless . path element under
some circumstances. For example, Clean("a/../b:/../c") becomes `.\c`.
This reverts CL 401595, since the change here supersedes the one
in that CL.
Thanks to RyotaK (https://twitter.com/ryotkak) for reporting this issue.
Updates #57274
Fixes #57275
Fixes CVE-2022-41722
Change-Id: I837446285a03aa74c79d7642720e01f354c2ca17
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1675249
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
TryBot-Result: Security TryBots <security-trybots@go-security-trybots.iam.gserviceaccount.com>
(cherry picked from commit 780dfa043ff5192c37de0d6fd1053a66b2b9f378)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1728206
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/468115
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Bypass: Michael Pratt <mpratt@google.com>
---
src/path/filepath/path.go | 27 +++++++++++++-------------
src/path/filepath/path_test.go | 7 +++++++
src/path/filepath/path_windows_test.go | 2 +-
3 files changed, 22 insertions(+), 14 deletions(-)
diff --git a/src/path/filepath/path.go b/src/path/filepath/path.go
index de7a2c758b..9b1f5ed7c0 100644
--- a/src/path/filepath/path.go
+++ b/src/path/filepath/path.go
@@ -15,6 +15,7 @@ import (
"errors"
"io/fs"
"os"
+ "runtime"
"sort"
"strings"
)
@@ -117,21 +118,9 @@ func Clean(path string) string {
case os.IsPathSeparator(path[r]):
// empty path element
r++
- case path[r] == '.' && r+1 == n:
+ case path[r] == '.' && (r+1 == n || os.IsPathSeparator(path[r+1])):
// . element
r++
- case path[r] == '.' && os.IsPathSeparator(path[r+1]):
- // ./ element
- r++
-
- for r < len(path) && os.IsPathSeparator(path[r]) {
- r++
- }
- if out.w == 0 && volumeNameLen(path[r:]) > 0 {
- // When joining prefix "." and an absolute path on Windows,
- // the prefix should not be removed.
- out.append('.')
- }
case path[r] == '.' && path[r+1] == '.' && (r+2 == n || os.IsPathSeparator(path[r+2])):
// .. element: remove to last separator
r += 2
@@ -157,6 +146,18 @@ func Clean(path string) string {
if rooted && out.w != 1 || !rooted && out.w != 0 {
out.append(Separator)
}
+ // If a ':' appears in the path element at the start of a Windows path,
+ // insert a .\ at the beginning to avoid converting relative paths
+ // like a/../c: into c:.
+ if runtime.GOOS == "windows" && out.w == 0 && out.volLen == 0 && r != 0 {
+ for i := r; i < n && !os.IsPathSeparator(path[i]); i++ {
+ if path[i] == ':' {
+ out.append('.')
+ out.append(Separator)
+ break
+ }
+ }
+ }
// copy element
for ; r < n && !os.IsPathSeparator(path[r]); r++ {
out.append(path[r])
diff --git a/src/path/filepath/path_test.go b/src/path/filepath/path_test.go
index a783d6be28..9a57920dd7 100644
--- a/src/path/filepath/path_test.go
+++ b/src/path/filepath/path_test.go
@@ -96,6 +96,13 @@ var wincleantests = []PathTest{
{`.\c:`, `.\c:`},
{`.\c:\foo`, `.\c:\foo`},
{`.\c:foo`, `.\c:foo`},
+
+ // Don't allow cleaning to move an element with a colon to the start of the path.
+ {`a/../c:`, `.\c:`},
+ {`a\..\c:`, `.\c:`},
+ {`a/../c:/a`, `.\c:\a`},
+ {`a/../../c:`, `..\c:`},
+ {`foo:bar`, `foo:bar`},
}
func TestClean(t *testing.T) {
diff --git a/src/path/filepath/path_windows_test.go b/src/path/filepath/path_windows_test.go
index 9e6c0ec81d..857f4d5c7c 100644
--- a/src/path/filepath/path_windows_test.go
+++ b/src/path/filepath/path_windows_test.go
@@ -542,7 +542,7 @@ func TestIssue52476(t *testing.T) {
}{
{`..\.`, `C:`, `..\C:`},
{`..`, `C:`, `..\C:`},
- {`.`, `:`, `:`},
+ {`.`, `:`, `.\:`},
{`.`, `C:`, `.\C:`},
{`.`, `C:/a/b/../c`, `.\C:\a\c`},
{`.`, `\C:`, `.\C:`},
--
2.37.1

644
0004-release-branch.go1.19-mime-multipart-limit-memory-in.patch Normal file
View File

@ -0,0 +1,644 @@
From 5c55ac9bf1e5f779220294c843526536605f42ab Mon Sep 17 00:00:00 2001
From: Damien Neil <dneil@google.com>
Date: Wed, 25 Jan 2023 09:27:01 -0800
Subject: [PATCH] [release-branch.go1.19] mime/multipart: limit memory/inode
consumption of ReadForm
Reader.ReadForm is documented as storing "up to maxMemory bytes + 10MB"
in memory. Parsed forms can consume substantially more memory than
this limit, since ReadForm does not account for map entry overhead
and MIME headers.
In addition, while the amount of disk memory consumed by ReadForm can
be constrained by limiting the size of the parsed input, ReadForm will
create one temporary file per form part stored on disk, potentially
consuming a large number of inodes.
Update ReadForm's memory accounting to include part names,
MIME headers, and map entry overhead.
Update ReadForm to store all on-disk file parts in a single
temporary file.
Files returned by FileHeader.Open are documented as having a concrete
type of *os.File when a file is stored on disk. The change to use a
single temporary file for all parts means that this is no longer the
case when a form contains more than a single file part stored on disk.
The previous behavior of storing each file part in a separate disk
file may be reenabled with GODEBUG=multipartfiles=distinct.
Update Reader.NextPart and Reader.NextRawPart to set a 10MiB cap
on the size of MIME headers.
Thanks to Jakob Ackermann (@das7pad) for reporting this issue.
Updates #58006
Fixes #58362
Fixes CVE-2022-41725
Change-Id: Ibd780a6c4c83ac8bcfd3cbe344f042e9940f2eab
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1714276
Reviewed-by: Julie Qiu <julieqiu@google.com>
TryBot-Result: Security TryBots <security-trybots@go-security-trybots.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
(cherry picked from commit ed4664330edcd91b24914c9371c377c132dbce8c)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1728949
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/468116
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
---
src/mime/multipart/formdata.go | 132 ++++++++++++++++++++-----
src/mime/multipart/formdata_test.go | 140 ++++++++++++++++++++++++++-
src/mime/multipart/multipart.go | 25 +++--
src/mime/multipart/readmimeheader.go | 14 +++
src/net/http/request_test.go | 2 +-
src/net/textproto/reader.go | 20 +++-
6 files changed, 295 insertions(+), 38 deletions(-)
create mode 100644 src/mime/multipart/readmimeheader.go
diff --git a/src/mime/multipart/formdata.go b/src/mime/multipart/formdata.go
index fca5f9e15f..a7d4ca97f0 100644
--- a/src/mime/multipart/formdata.go
+++ b/src/mime/multipart/formdata.go
@@ -7,6 +7,7 @@ package multipart
import (
"bytes"
"errors"
+ "internal/godebug"
"io"
"math"
"net/textproto"
@@ -33,23 +34,58 @@ func (r *Reader) ReadForm(maxMemory int64) (*Form, error) {
func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
form := &Form{make(map[string][]string), make(map[string][]*FileHeader)}
+ var (
+ file *os.File
+ fileOff int64
+ )
+ numDiskFiles := 0
+ multipartFiles := godebug.Get("multipartfiles")
+ combineFiles := multipartFiles != "distinct"
defer func() {
+ if file != nil {
+ if cerr := file.Close(); err == nil {
+ err = cerr
+ }
+ }
+ if combineFiles && numDiskFiles > 1 {
+ for _, fhs := range form.File {
+ for _, fh := range fhs {
+ fh.tmpshared = true
+ }
+ }
+ }
if err != nil {
form.RemoveAll()
+ if file != nil {
+ os.Remove(file.Name())
+ }
}
}()
- // Reserve an additional 10 MB for non-file parts.
- maxValueBytes := maxMemory + int64(10<<20)
- if maxValueBytes <= 0 {
+ // maxFileMemoryBytes is the maximum bytes of file data we will store in memory.
+ // Data past this limit is written to disk.
+ // This limit strictly applies to content, not metadata (filenames, MIME headers, etc.),
+ // since metadata is always stored in memory, not disk.
+ //
+ // maxMemoryBytes is the maximum bytes we will store in memory, including file content,
+ // non-file part values, metdata, and map entry overhead.
+ //
+ // We reserve an additional 10 MB in maxMemoryBytes for non-file data.
+ //
+ // The relationship between these parameters, as well as the overly-large and
+ // unconfigurable 10 MB added on to maxMemory, is unfortunate but difficult to change
+ // within the constraints of the API as documented.
+ maxFileMemoryBytes := maxMemory
+ maxMemoryBytes := maxMemory + int64(10<<20)
+ if maxMemoryBytes <= 0 {
if maxMemory < 0 {
- maxValueBytes = 0
+ maxMemoryBytes = 0
} else {
- maxValueBytes = math.MaxInt64
+ maxMemoryBytes = math.MaxInt64
}
}
for {
- p, err := r.NextPart()
+ p, err := r.nextPart(false, maxMemoryBytes)
if err == io.EOF {
break
}
@@ -63,16 +99,27 @@ func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
}
filename := p.FileName()
+ // Multiple values for the same key (one map entry, longer slice) are cheaper
+ // than the same number of values for different keys (many map entries), but
+ // using a consistent per-value cost for overhead is simpler.
+ maxMemoryBytes -= int64(len(name))
+ maxMemoryBytes -= 100 // map overhead
+ if maxMemoryBytes < 0 {
+ // We can't actually take this path, since nextPart would already have
+ // rejected the MIME headers for being too large. Check anyway.
+ return nil, ErrMessageTooLarge
+ }
+
var b bytes.Buffer
if filename == "" {
// value, store as string in memory
- n, err := io.CopyN(&b, p, maxValueBytes+1)
+ n, err := io.CopyN(&b, p, maxMemoryBytes+1)
if err != nil && err != io.EOF {
return nil, err
}
- maxValueBytes -= n
- if maxValueBytes < 0 {
+ maxMemoryBytes -= n
+ if maxMemoryBytes < 0 {
return nil, ErrMessageTooLarge
}
form.Value[name] = append(form.Value[name], b.String())
@@ -80,35 +127,45 @@ func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
}
// file, store in memory or on disk
+ maxMemoryBytes -= mimeHeaderSize(p.Header)
+ if maxMemoryBytes < 0 {
+ return nil, ErrMessageTooLarge
+ }
fh := &FileHeader{
Filename: filename,
Header: p.Header,
}
- n, err := io.CopyN(&b, p, maxMemory+1)
+ n, err := io.CopyN(&b, p, maxFileMemoryBytes+1)
if err != nil && err != io.EOF {
return nil, err
}
- if n > maxMemory {
- // too big, write to disk and flush buffer
- file, err := os.CreateTemp("", "multipart-")
- if err != nil {
- return nil, err
+ if n > maxFileMemoryBytes {
+ if file == nil {
+ file, err = os.CreateTemp(r.tempDir, "multipart-")
+ if err != nil {
+ return nil, err
+ }
}
+ numDiskFiles++
size, err := io.Copy(file, io.MultiReader(&b, p))
- if cerr := file.Close(); err == nil {
- err = cerr
- }
if err != nil {
- os.Remove(file.Name())
return nil, err
}
fh.tmpfile = file.Name()
fh.Size = size
+ fh.tmpoff = fileOff
+ fileOff += size
+ if !combineFiles {
+ if err := file.Close(); err != nil {
+ return nil, err
+ }
+ file = nil
+ }
} else {
fh.content = b.Bytes()
fh.Size = int64(len(fh.content))
- maxMemory -= n
- maxValueBytes -= n
+ maxFileMemoryBytes -= n
+ maxMemoryBytes -= n
}
form.File[name] = append(form.File[name], fh)
}
@@ -116,6 +173,17 @@ func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
return form, nil
}
+func mimeHeaderSize(h textproto.MIMEHeader) (size int64) {
+ for k, vs := range h {
+ size += int64(len(k))
+ size += 100 // map entry overhead
+ for _, v := range vs {
+ size += int64(len(v))
+ }
+ }
+ return size
+}
+
// Form is a parsed multipart form.
// Its File parts are stored either in memory or on disk,
// and are accessible via the *FileHeader's Open method.
@@ -133,7 +201,7 @@ func (f *Form) RemoveAll() error {
for _, fh := range fhs {
if fh.tmpfile != "" {
e := os.Remove(fh.tmpfile)
- if e != nil && err == nil {
+ if e != nil && !errors.Is(e, os.ErrNotExist) && err == nil {
err = e
}
}
@@ -148,15 +216,25 @@ type FileHeader struct {
Header textproto.MIMEHeader
Size int64
- content []byte
- tmpfile string
+ content []byte
+ tmpfile string
+ tmpoff int64
+ tmpshared bool
}
// Open opens and returns the FileHeader's associated File.
func (fh *FileHeader) Open() (File, error) {
if b := fh.content; b != nil {
r := io.NewSectionReader(bytes.NewReader(b), 0, int64(len(b)))
- return sectionReadCloser{r}, nil
+ return sectionReadCloser{r, nil}, nil
+ }
+ if fh.tmpshared {
+ f, err := os.Open(fh.tmpfile)
+ if err != nil {
+ return nil, err
+ }
+ r := io.NewSectionReader(f, fh.tmpoff, fh.Size)
+ return sectionReadCloser{r, f}, nil
}
return os.Open(fh.tmpfile)
}
@@ -175,8 +253,12 @@ type File interface {
type sectionReadCloser struct {
*io.SectionReader
+ io.Closer
}
func (rc sectionReadCloser) Close() error {
+ if rc.Closer != nil {
+ return rc.Closer.Close()
+ }
return nil
}
diff --git a/src/mime/multipart/formdata_test.go b/src/mime/multipart/formdata_test.go
index e3a3a3eae8..5cded7170c 100644
--- a/src/mime/multipart/formdata_test.go
+++ b/src/mime/multipart/formdata_test.go
@@ -6,8 +6,10 @@ package multipart
import (
"bytes"
+ "fmt"
"io"
"math"
+ "net/textproto"
"os"
"strings"
"testing"
@@ -208,8 +210,8 @@ Content-Disposition: form-data; name="largetext"
maxMemory int64
err error
}{
- {"smaller", 50, nil},
- {"exact-fit", 25, nil},
+ {"smaller", 50 + int64(len("largetext")) + 100, nil},
+ {"exact-fit", 25 + int64(len("largetext")) + 100, nil},
{"too-large", 0, ErrMessageTooLarge},
}
for _, tc := range testCases {
@@ -224,7 +226,7 @@ Content-Disposition: form-data; name="largetext"
defer f.RemoveAll()
}
if tc.err != err {
- t.Fatalf("ReadForm error - got: %v; expected: %v", tc.err, err)
+ t.Fatalf("ReadForm error - got: %v; expected: %v", err, tc.err)
}
if err == nil {
if g := f.Value["largetext"][0]; g != largeTextValue {
@@ -234,3 +236,135 @@ Content-Disposition: form-data; name="largetext"
})
}
}
+
+// TestReadForm_MetadataTooLarge verifies that we account for the size of field names,
+// MIME headers, and map entry overhead while limiting the memory consumption of parsed forms.
+func TestReadForm_MetadataTooLarge(t *testing.T) {
+ for _, test := range []struct {
+ name string
+ f func(*Writer)
+ }{{
+ name: "large name",
+ f: func(fw *Writer) {
+ name := strings.Repeat("a", 10<<20)
+ w, _ := fw.CreateFormField(name)
+ w.Write([]byte("value"))
+ },
+ }, {
+ name: "large MIME header",
+ f: func(fw *Writer) {
+ h := make(textproto.MIMEHeader)
+ h.Set("Content-Disposition", `form-data; name="a"`)
+ h.Set("X-Foo", strings.Repeat("a", 10<<20))
+ w, _ := fw.CreatePart(h)
+ w.Write([]byte("value"))
+ },
+ }, {
+ name: "many parts",
+ f: func(fw *Writer) {
+ for i := 0; i < 110000; i++ {
+ w, _ := fw.CreateFormField("f")
+ w.Write([]byte("v"))
+ }
+ },
+ }} {
+ t.Run(test.name, func(t *testing.T) {
+ var buf bytes.Buffer
+ fw := NewWriter(&buf)
+ test.f(fw)
+ if err := fw.Close(); err != nil {
+ t.Fatal(err)
+ }
+ fr := NewReader(&buf, fw.Boundary())
+ _, err := fr.ReadForm(0)
+ if err != ErrMessageTooLarge {
+ t.Errorf("fr.ReadForm() = %v, want ErrMessageTooLarge", err)
+ }
+ })
+ }
+}
+
+// TestReadForm_ManyFiles_Combined tests that a multipart form containing many files only
+// results in a single on-disk file.
+func TestReadForm_ManyFiles_Combined(t *testing.T) {
+ const distinct = false
+ testReadFormManyFiles(t, distinct)
+}
+
+// TestReadForm_ManyFiles_Distinct tests that setting GODEBUG=multipartfiles=distinct
+// results in every file in a multipart form being placed in a distinct on-disk file.
+func TestReadForm_ManyFiles_Distinct(t *testing.T) {
+ t.Setenv("GODEBUG", "multipartfiles=distinct")
+ const distinct = true
+ testReadFormManyFiles(t, distinct)
+}
+
+func testReadFormManyFiles(t *testing.T, distinct bool) {
+ var buf bytes.Buffer
+ fw := NewWriter(&buf)
+ const numFiles = 10
+ for i := 0; i < numFiles; i++ {
+ name := fmt.Sprint(i)
+ w, err := fw.CreateFormFile(name, name)
+ if err != nil {
+ t.Fatal(err)
+ }
+ w.Write([]byte(name))
+ }
+ if err := fw.Close(); err != nil {
+ t.Fatal(err)
+ }
+ fr := NewReader(&buf, fw.Boundary())
+ fr.tempDir = t.TempDir()
+ form, err := fr.ReadForm(0)
+ if err != nil {
+ t.Fatal(err)
+ }
+ for i := 0; i < numFiles; i++ {
+ name := fmt.Sprint(i)
+ if got := len(form.File[name]); got != 1 {
+ t.Fatalf("form.File[%q] has %v entries, want 1", name, got)
+ }
+ fh := form.File[name][0]
+ file, err := fh.Open()
+ if err != nil {
+ t.Fatalf("form.File[%q].Open() = %v", name, err)
+ }
+ if distinct {
+ if _, ok := file.(*os.File); !ok {
+ t.Fatalf("form.File[%q].Open: %T, want *os.File", name, file)
+ }
+ }
+ got, err := io.ReadAll(file)
+ file.Close()
+ if string(got) != name || err != nil {
+ t.Fatalf("read form.File[%q]: %q, %v; want %q, nil", name, string(got), err, name)
+ }
+ }
+ dir, err := os.Open(fr.tempDir)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer dir.Close()
+ names, err := dir.Readdirnames(0)
+ if err != nil {
+ t.Fatal(err)
+ }
+ wantNames := 1
+ if distinct {
+ wantNames = numFiles
+ }
+ if len(names) != wantNames {
+ t.Fatalf("temp dir contains %v files; want 1", len(names))
+ }
+ if err := form.RemoveAll(); err != nil {
+ t.Fatalf("form.RemoveAll() = %v", err)
+ }
+ names, err = dir.Readdirnames(0)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(names) != 0 {
+ t.Fatalf("temp dir contains %v files; want 0", len(names))
+ }
+}
diff --git a/src/mime/multipart/multipart.go b/src/mime/multipart/multipart.go
index aa05ac8f9c..fc50d35196 100644
--- a/src/mime/multipart/multipart.go
+++ b/src/mime/multipart/multipart.go
@@ -128,12 +128,12 @@ func (r *stickyErrorReader) Read(p []byte) (n int, _ error) {
return n, r.err
}
-func newPart(mr *Reader, rawPart bool) (*Part, error) {
+func newPart(mr *Reader, rawPart bool, maxMIMEHeaderSize int64) (*Part, error) {
bp := &Part{
Header: make(map[string][]string),
mr: mr,
}
- if err := bp.populateHeaders(); err != nil {
+ if err := bp.populateHeaders(maxMIMEHeaderSize); err != nil {
return nil, err
}
bp.r = partReader{bp}
@@ -149,12 +149,16 @@ func newPart(mr *Reader, rawPart bool) (*Part, error) {
return bp, nil
}
-func (p *Part) populateHeaders() error {
+func (p *Part) populateHeaders(maxMIMEHeaderSize int64) error {
r := textproto.NewReader(p.mr.bufReader)
- header, err := r.ReadMIMEHeader()
+ header, err := readMIMEHeader(r, maxMIMEHeaderSize)
if err == nil {
p.Header = header
}
+ // TODO: Add a distinguishable error to net/textproto.
+ if err != nil && err.Error() == "message too large" {
+ err = ErrMessageTooLarge
+ }
return err
}
@@ -311,6 +315,7 @@ func (p *Part) Close() error {
// isn't supported.
type Reader struct {
bufReader *bufio.Reader
+ tempDir string // used in tests
currentPart *Part
partsRead int
@@ -321,6 +326,10 @@ type Reader struct {
dashBoundary []byte // "--boundary"
}
+// maxMIMEHeaderSize is the maximum size of a MIME header we will parse,
+// including header keys, values, and map overhead.
+const maxMIMEHeaderSize = 10 << 20
+
// NextPart returns the next part in the multipart or an error.
// When there are no more parts, the error io.EOF is returned.
//
@@ -328,7 +337,7 @@ type Reader struct {
// has a value of "quoted-printable", that header is instead
// hidden and the body is transparently decoded during Read calls.
func (r *Reader) NextPart() (*Part, error) {
- return r.nextPart(false)
+ return r.nextPart(false, maxMIMEHeaderSize)
}
// NextRawPart returns the next part in the multipart or an error.
@@ -337,10 +346,10 @@ func (r *Reader) NextPart() (*Part, error) {
// Unlike NextPart, it does not have special handling for
// "Content-Transfer-Encoding: quoted-printable".
func (r *Reader) NextRawPart() (*Part, error) {
- return r.nextPart(true)
+ return r.nextPart(true, maxMIMEHeaderSize)
}
-func (r *Reader) nextPart(rawPart bool) (*Part, error) {
+func (r *Reader) nextPart(rawPart bool, maxMIMEHeaderSize int64) (*Part, error) {
if r.currentPart != nil {
r.currentPart.Close()
}
@@ -365,7 +374,7 @@ func (r *Reader) nextPart(rawPart bool) (*Part, error) {
if r.isBoundaryDelimiterLine(line) {
r.partsRead++
- bp, err := newPart(r, rawPart)
+ bp, err := newPart(r, rawPart, maxMIMEHeaderSize)
if err != nil {
return nil, err
}
diff --git a/src/mime/multipart/readmimeheader.go b/src/mime/multipart/readmimeheader.go
new file mode 100644
index 0000000000..6836928c9e
--- /dev/null
+++ b/src/mime/multipart/readmimeheader.go
@@ -0,0 +1,14 @@
+// Copyright 2023 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+package multipart
+
+import (
+ "net/textproto"
+ _ "unsafe" // for go:linkname
+)
+
+// readMIMEHeader is defined in package net/textproto.
+//
+//go:linkname readMIMEHeader net/textproto.readMIMEHeader
+func readMIMEHeader(r *textproto.Reader, lim int64) (textproto.MIMEHeader, error)
diff --git a/src/net/http/request_test.go b/src/net/http/request_test.go
index af35f17f7c..0ec8f2474a 100644
--- a/src/net/http/request_test.go
+++ b/src/net/http/request_test.go
@@ -1104,7 +1104,7 @@ func testMissingFile(t *testing.T, req *Request) {
t.Errorf("FormFile file = %v, want nil", f)
}
if fh != nil {
- t.Errorf("FormFile file header = %q, want nil", fh)
+ t.Errorf("FormFile file header = %v, want nil", fh)
}
if err != ErrMissingFile {
t.Errorf("FormFile err = %q, want ErrMissingFile", err)
diff --git a/src/net/textproto/reader.go b/src/net/textproto/reader.go
index 1f7afc5766..b37be54d67 100644
--- a/src/net/textproto/reader.go
+++ b/src/net/textproto/reader.go
@@ -7,8 +7,10 @@ package textproto
import (
"bufio"
"bytes"
+ "errors"
"fmt"
"io"
+ "math"
"strconv"
"strings"
"sync"
@@ -481,6 +483,12 @@ var colon = []byte(":")
// "Long-Key": {"Even Longer Value"},
// }
func (r *Reader) ReadMIMEHeader() (MIMEHeader, error) {
+ return readMIMEHeader(r, math.MaxInt64)
+}
+
+// readMIMEHeader is a version of ReadMIMEHeader which takes a limit on the header size.
+// It is called by the mime/multipart package.
+func readMIMEHeader(r *Reader, lim int64) (MIMEHeader, error) {
// Avoid lots of small slice allocations later by allocating one
// large one ahead of time which we'll cut up into smaller
// slices. If this isn't big enough later, we allocate small ones.
@@ -522,9 +530,19 @@ func (r *Reader) ReadMIMEHeader() (MIMEHeader, error) {
}
// Skip initial spaces in value.
- value := strings.TrimLeft(string(v), " \t")
+ value := string(bytes.TrimLeft(v, " \t"))
vv := m[key]
+ if vv == nil {
+ lim -= int64(len(key))
+ lim -= 100 // map entry overhead
+ }
+ lim -= int64(len(value))
+ if lim < 0 {
+ // TODO: This should be a distinguishable error (ErrMessageTooLarge)
+ // to allow mime/multipart to detect it.
+ return m, errors.New("message too large")
+ }
if vv == nil && len(strs) > 0 {
// More than likely this will be a single-element key.
// Most headers aren't multi-valued.
--
2.37.1

2408
0005-release-branch.go1.19-crypto-tls-replace-all-usages-.patch Normal file
View File

File diff suppressed because it is too large Load Diff

168
0006-release-branch.go1.19-net-http-update-bundled-golang.patch Normal file
View File

@ -0,0 +1,168 @@
From 5c3e11bd0b5c0a86e5beffcd4339b86a902b21c3 Mon Sep 17 00:00:00 2001
From: Roland Shoemaker <bracewell@google.com>
Date: Mon, 6 Feb 2023 10:03:44 -0800
Subject: [PATCH] [release-branch.go1.19] net/http: update bundled
golang.org/x/net/http2
Disable cmd/internal/moddeps test, since this update includes PRIVATE
track fixes.
Fixes CVE-2022-41723
Fixes #58355
Updates #57855
Change-Id: Ie870562a6f6e44e4e8f57db6a0dde1a41a2b090c
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1728939
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/468118
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
---
src/cmd/internal/moddeps/moddeps_test.go | 2 +-
.../golang.org/x/net/http2/hpack/hpack.go | 79 ++++++++++++-------
2 files changed, 50 insertions(+), 31 deletions(-)
diff --git a/src/cmd/internal/moddeps/moddeps_test.go b/src/cmd/internal/moddeps/moddeps_test.go
index e96edf1f30..1fc9f708bd 100644
--- a/src/cmd/internal/moddeps/moddeps_test.go
+++ b/src/cmd/internal/moddeps/moddeps_test.go
@@ -33,7 +33,7 @@ import (
// See issues 36852, 41409, and 43687.
// (Also see golang.org/issue/27348.)
func TestAllDependencies(t *testing.T) {
- t.Skip("TODO(#57009): 1.19.4 contains unreleased changes from vendored modules")
+ t.Skip("TODO(#58355): 1.19.4 contains unreleased changes from vendored modules")
goBin := testenv.GoToolPath(t)
diff --git a/src/vendor/golang.org/x/net/http2/hpack/hpack.go b/src/vendor/golang.org/x/net/http2/hpack/hpack.go
index 85f18a2b0a..02e80e30a4 100644
--- a/src/vendor/golang.org/x/net/http2/hpack/hpack.go
+++ b/src/vendor/golang.org/x/net/http2/hpack/hpack.go
@@ -359,6 +359,7 @@ func (d *Decoder) parseFieldLiteral(n uint8, it indexType) error {
var hf HeaderField
wantStr := d.emitEnabled || it.indexed()
+ var undecodedName undecodedString
if nameIdx > 0 {
ihf, ok := d.at(nameIdx)
if !ok {
@@ -366,15 +367,27 @@ func (d *Decoder) parseFieldLiteral(n uint8, it indexType) error {
}
hf.Name = ihf.Name
} else {
- hf.Name, buf, err = d.readString(buf, wantStr)
+ undecodedName, buf, err = d.readString(buf)
if err != nil {
return err
}
}
- hf.Value, buf, err = d.readString(buf, wantStr)
+ undecodedValue, buf, err := d.readString(buf)
if err != nil {
return err
}
+ if wantStr {
+ if nameIdx <= 0 {
+ hf.Name, err = d.decodeString(undecodedName)
+ if err != nil {
+ return err
+ }
+ }
+ hf.Value, err = d.decodeString(undecodedValue)
+ if err != nil {
+ return err
+ }
+ }
d.buf = buf
if it.indexed() {
d.dynTab.add(hf)
@@ -459,46 +472,52 @@ func readVarInt(n byte, p []byte) (i uint64, remain []byte, err error) {
return 0, origP, errNeedMore
}
-// readString decodes an hpack string from p.
+// readString reads an hpack string from p.
//
-// wantStr is whether s will be used. If false, decompression and
-// []byte->string garbage are skipped if s will be ignored
-// anyway. This does mean that huffman decoding errors for non-indexed
-// strings past the MAX_HEADER_LIST_SIZE are ignored, but the server
-// is returning an error anyway, and because they're not indexed, the error
-// won't affect the decoding state.
-func (d *Decoder) readString(p []byte, wantStr bool) (s string, remain []byte, err error) {
+// It returns a reference to the encoded string data to permit deferring decode costs
+// until after the caller verifies all data is present.
+func (d *Decoder) readString(p []byte) (u undecodedString, remain []byte, err error) {
if len(p) == 0 {
- return "", p, errNeedMore
+ return u, p, errNeedMore
}
isHuff := p[0]&128 != 0
strLen, p, err := readVarInt(7, p)
if err != nil {
- return "", p, err
+ return u, p, err
}
if d.maxStrLen != 0 && strLen > uint64(d.maxStrLen) {
- return "", nil, ErrStringLength
+ // Returning an error here means Huffman decoding errors
+ // for non-indexed strings past the maximum string length
+ // are ignored, but the server is returning an error anyway
+ // and because the string is not indexed the error will not
+ // affect the decoding state.
+ return u, nil, ErrStringLength
}
if uint64(len(p)) < strLen {
- return "", p, errNeedMore
- }
- if !isHuff {
- if wantStr {
- s = string(p[:strLen])
- }
- return s, p[strLen:], nil
+ return u, p, errNeedMore
}
+ u.isHuff = isHuff
+ u.b = p[:strLen]
+ return u, p[strLen:], nil
+}
- if wantStr {
- buf := bufPool.Get().(*bytes.Buffer)
- buf.Reset() // don't trust others
- defer bufPool.Put(buf)
- if err := huffmanDecode(buf, d.maxStrLen, p[:strLen]); err != nil {
- buf.Reset()
- return "", nil, err
- }
+type undecodedString struct {
+ isHuff bool
+ b []byte
+}
+
+func (d *Decoder) decodeString(u undecodedString) (string, error) {
+ if !u.isHuff {
+ return string(u.b), nil
+ }
+ buf := bufPool.Get().(*bytes.Buffer)
+ buf.Reset() // don't trust others
+ var s string
+ err := huffmanDecode(buf, d.maxStrLen, u.b)
+ if err == nil {
s = buf.String()
- buf.Reset() // be nice to GC
}
- return s, p[strLen:], nil
+ buf.Reset() // be nice to GC
+ bufPool.Put(buf)
+ return s, err
}
--
2.37.1

206
0007-release-branch.go1.19-crypto-internal-nistec-reduce-.patch Normal file
View File

@ -0,0 +1,206 @@
From 639b67ed114151c0d786aa26e7faeab942400703 Mon Sep 17 00:00:00 2001
From: Filippo Valsorda <filippo@golang.org>
Date: Mon, 13 Feb 2023 15:16:27 +0100
Subject: [PATCH] [release-branch.go1.19] crypto/internal/nistec: reduce P-256
scalar
Unlike the rest of nistec, the P-256 assembly doesn't use complete
addition formulas, meaning that p256PointAdd[Affine]Asm won't return the
correct value if the two inputs are equal.
This was (undocumentedly) ignored in the scalar multiplication loops
because as long as the input point is not the identity and the scalar is
lower than the order of the group, the addition inputs can't be the same.
As part of the math/big rewrite, we went however from always reducing
the scalar to only checking its length, under the incorrect assumption
that the scalar multiplication loop didn't require reduction.
Added a reduction, and while at it added it in P256OrdInverse, too, to
enforce a universal reduction invariant on p256OrdElement values.
Note that if the input point is the infinity, the code currently still
relies on undefined behavior, but that's easily tested to behave
acceptably, and will be addressed in a future CL.
Updates #58647
Fixes #58719
Fixes CVE-2023-24532
(Filed with the "safe APIs like complete addition formulas are good" dept.)
Change-Id: I7b2c75238440e6852be2710fad66ff1fdc4e2b24
Reviewed-on: https://go-review.googlesource.com/c/go/+/471255
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
(cherry picked from commit 203e59ad41bd288e1d92b6f617c2f55e70d3c8e3)
Reviewed-on: https://go-review.googlesource.com/c/go/+/471696
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
---
src/crypto/internal/nistec/nistec_test.go | 81 +++++++++++++++++++
src/crypto/internal/nistec/p256_asm.go | 17 ++++
src/crypto/internal/nistec/p256_asm_ordinv.go | 1 +
3 files changed, 99 insertions(+)
diff --git a/src/crypto/internal/nistec/nistec_test.go b/src/crypto/internal/nistec/nistec_test.go
index 1903f19af3..1cedebc8ac 100644
--- a/src/crypto/internal/nistec/nistec_test.go
+++ b/src/crypto/internal/nistec/nistec_test.go
@@ -8,6 +8,7 @@ import (
"bytes"
"crypto/elliptic"
"crypto/internal/nistec"
+ "fmt"
"math/big"
"math/rand"
"os"
@@ -166,6 +167,86 @@ func testEquivalents[P nistPoint[P]](t *testing.T, newPoint, newGenerator func()
}
}
+func TestScalarMult(t *testing.T) {
+ t.Run("P224", func(t *testing.T) {
+ testScalarMult(t, nistec.NewP224Point, nistec.NewP224Generator, elliptic.P224())
+ })
+ t.Run("P256", func(t *testing.T) {
+ testScalarMult(t, nistec.NewP256Point, nistec.NewP256Generator, elliptic.P256())
+ })
+ t.Run("P384", func(t *testing.T) {
+ testScalarMult(t, nistec.NewP384Point, nistec.NewP384Generator, elliptic.P384())
+ })
+ t.Run("P521", func(t *testing.T) {
+ testScalarMult(t, nistec.NewP521Point, nistec.NewP521Generator, elliptic.P521())
+ })
+}
+
+func testScalarMult[P nistPoint[P]](t *testing.T, newPoint func() P, newGenerator func() P, c elliptic.Curve) {
+ G := newGenerator()
+ checkScalar := func(t *testing.T, scalar []byte) {
+ p1, err := newPoint().ScalarBaseMult(scalar)
+ fatalIfErr(t, err)
+ p2, err := newPoint().ScalarMult(G, scalar)
+ fatalIfErr(t, err)
+ if !bytes.Equal(p1.Bytes(), p2.Bytes()) {
+ t.Error("[k]G != ScalarBaseMult(k)")
+ }
+
+ d := new(big.Int).SetBytes(scalar)
+ d.Sub(c.Params().N, d)
+ d.Mod(d, c.Params().N)
+ g1, err := newPoint().ScalarBaseMult(d.FillBytes(make([]byte, len(scalar))))
+ fatalIfErr(t, err)
+ g1.Add(g1, p1)
+ if !bytes.Equal(g1.Bytes(), newPoint().Bytes()) {
+ t.Error("[N - k]G + [k]G != ∞")
+ }
+ }
+
+ byteLen := len(c.Params().N.Bytes())
+ bitLen := c.Params().N.BitLen()
+ t.Run("0", func(t *testing.T) { checkScalar(t, make([]byte, byteLen)) })
+ t.Run("1", func(t *testing.T) {
+ checkScalar(t, big.NewInt(1).FillBytes(make([]byte, byteLen)))
+ })
+ t.Run("N-1", func(t *testing.T) {
+ checkScalar(t, new(big.Int).Sub(c.Params().N, big.NewInt(1)).Bytes())
+ })
+ t.Run("N", func(t *testing.T) { checkScalar(t, c.Params().N.Bytes()) })
+ t.Run("N+1", func(t *testing.T) {
+ checkScalar(t, new(big.Int).Add(c.Params().N, big.NewInt(1)).Bytes())
+ })
+ t.Run("all1s", func(t *testing.T) {
+ s := new(big.Int).Lsh(big.NewInt(1), uint(bitLen))
+ s.Sub(s, big.NewInt(1))
+ checkScalar(t, s.Bytes())
+ })
+ if testing.Short() {
+ return
+ }
+ for i := 0; i < bitLen; i++ {
+ t.Run(fmt.Sprintf("1<<%d", i), func(t *testing.T) {
+ s := new(big.Int).Lsh(big.NewInt(1), uint(i))
+ checkScalar(t, s.FillBytes(make([]byte, byteLen)))
+ })
+ }
+ // Test N+1...N+32 since they risk overlapping with precomputed table values
+ // in the final additions.
+ for i := int64(2); i <= 32; i++ {
+ t.Run(fmt.Sprintf("N+%d", i), func(t *testing.T) {
+ checkScalar(t, new(big.Int).Add(c.Params().N, big.NewInt(i)).Bytes())
+ })
+ }
+}
+
+func fatalIfErr(t *testing.T, err error) {
+ t.Helper()
+ if err != nil {
+ t.Fatal(err)
+ }
+}
+
func BenchmarkScalarMult(b *testing.B) {
b.Run("P224", func(b *testing.B) {
benchmarkScalarMult(b, nistec.NewP224Generator(), 28)
diff --git a/src/crypto/internal/nistec/p256_asm.go b/src/crypto/internal/nistec/p256_asm.go
index bc443ba323..14713b0406 100644
--- a/src/crypto/internal/nistec/p256_asm.go
+++ b/src/crypto/internal/nistec/p256_asm.go
@@ -365,6 +365,21 @@ func p256PointDoubleAsm(res, in *P256Point)
// Montgomery domain (with R 2²⁵⁶) as four uint64 limbs in little-endian order.
type p256OrdElement [4]uint64
+// p256OrdReduce ensures s is in the range [0, ord(G)-1].
+func p256OrdReduce(s *p256OrdElement) {
+ // Since 2 * ord(G) > 2²⁵⁶, we can just conditionally subtract ord(G),
+ // keeping the result if it doesn't underflow.
+ t0, b := bits.Sub64(s[0], 0xf3b9cac2fc632551, 0)
+ t1, b := bits.Sub64(s[1], 0xbce6faada7179e84, b)
+ t2, b := bits.Sub64(s[2], 0xffffffffffffffff, b)
+ t3, b := bits.Sub64(s[3], 0xffffffff00000000, b)
+ tMask := b - 1 // zero if subtraction underflowed
+ s[0] ^= (t0 ^ s[0]) & tMask
+ s[1] ^= (t1 ^ s[1]) & tMask
+ s[2] ^= (t2 ^ s[2]) & tMask
+ s[3] ^= (t3 ^ s[3]) & tMask
+}
+
// Add sets q = p1 + p2, and returns q. The points may overlap.
func (q *P256Point) Add(r1, r2 *P256Point) *P256Point {
var sum, double P256Point
@@ -394,6 +409,7 @@ func (r *P256Point) ScalarBaseMult(scalar []byte) (*P256Point, error) {
}
scalarReversed := new(p256OrdElement)
p256OrdBigToLittle(scalarReversed, (*[32]byte)(scalar))
+ p256OrdReduce(scalarReversed)
r.p256BaseMult(scalarReversed)
return r, nil
@@ -408,6 +424,7 @@ func (r *P256Point) ScalarMult(q *P256Point, scalar []byte) (*P256Point, error)
}
scalarReversed := new(p256OrdElement)
p256OrdBigToLittle(scalarReversed, (*[32]byte)(scalar))
+ p256OrdReduce(scalarReversed)
r.Set(q).p256ScalarMult(scalarReversed)
return r, nil
diff --git a/src/crypto/internal/nistec/p256_asm_ordinv.go b/src/crypto/internal/nistec/p256_asm_ordinv.go
index 86a7a230bd..1274fb7fd3 100644
--- a/src/crypto/internal/nistec/p256_asm_ordinv.go
+++ b/src/crypto/internal/nistec/p256_asm_ordinv.go
@@ -25,6 +25,7 @@ func P256OrdInverse(k []byte) ([]byte, error) {
x := new(p256OrdElement)
p256OrdBigToLittle(x, (*[32]byte)(k))
+ p256OrdReduce(x)
// Inversion is implemented as exponentiation by n - 2, per Fermat's little theorem.
//
--
2.37.1

2
0005-release-branch.go1.19-net-textproto-avoid-overpredic.patch → 0008-release-branch.go1.19-net-textproto-avoid-overpredic.patch
View File

@ -184,5 +184,5 @@ index d11d40f1cf..f3c372ce03 100644
in string
inCode int
--
2.33.0
2.37.1

133
0009-release-branch.go1.19-mime-multipart-avoid-excessive.patch Normal file
View File

@ -0,0 +1,133 @@
From ef41a4e2face45e580c5836eaebd51629fc23f15 Mon Sep 17 00:00:00 2001
From: Damien Neil <dneil@google.com>
Date: Thu, 16 Mar 2023 14:18:04 -0700
Subject: [PATCH] [release-branch.go1.19] mime/multipart: avoid excessive copy
buffer allocations in ReadForm
When copying form data to disk with io.Copy,
allocate only one copy buffer and reuse it rather than
creating two buffers per file (one from io.multiReader.WriteTo,
and a second one from os.File.ReadFrom).
Thanks to Jakob Ackermann (@das7pad) for reporting this issue.
For CVE-2023-24536
For #59153
For #59269
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802453
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802395
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Change-Id: Ie405470c92abffed3356913b37d813e982c96c8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/481983
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
---
src/mime/multipart/formdata.go | 15 +++++++--
src/mime/multipart/formdata_test.go | 49 +++++++++++++++++++++++++++++
2 files changed, 61 insertions(+), 3 deletions(-)
diff --git a/src/mime/multipart/formdata.go b/src/mime/multipart/formdata.go
index a7d4ca97f0..975dcb6b26 100644
--- a/src/mime/multipart/formdata.go
+++ b/src/mime/multipart/formdata.go
@@ -84,6 +84,7 @@ func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
maxMemoryBytes = math.MaxInt64
}
}
+ var copyBuf []byte
for {
p, err := r.nextPart(false, maxMemoryBytes)
if err == io.EOF {
@@ -147,14 +148,22 @@ func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
}
}
numDiskFiles++
- size, err := io.Copy(file, io.MultiReader(&b, p))
+ if _, err := file.Write(b.Bytes()); err != nil {
+ return nil, err
+ }
+ if copyBuf == nil {
+ copyBuf = make([]byte, 32*1024) // same buffer size as io.Copy uses
+ }
+ // os.File.ReadFrom will allocate its own copy buffer if we let io.Copy use it.
+ type writerOnly struct{ io.Writer }
+ remainingSize, err := io.CopyBuffer(writerOnly{file}, p, copyBuf)
if err != nil {
return nil, err
}
fh.tmpfile = file.Name()
- fh.Size = size
+ fh.Size = int64(b.Len()) + remainingSize
fh.tmpoff = fileOff
- fileOff += size
+ fileOff += fh.Size
if !combineFiles {
if err := file.Close(); err != nil {
return nil, err
diff --git a/src/mime/multipart/formdata_test.go b/src/mime/multipart/formdata_test.go
index 5cded7170c..f5b56083b2 100644
--- a/src/mime/multipart/formdata_test.go
+++ b/src/mime/multipart/formdata_test.go
@@ -368,3 +368,52 @@ func testReadFormManyFiles(t *testing.T, distinct bool) {
t.Fatalf("temp dir contains %v files; want 0", len(names))
}
}
+
+func BenchmarkReadForm(b *testing.B) {
+ for _, test := range []struct {
+ name string
+ form func(fw *Writer, count int)
+ }{{
+ name: "fields",
+ form: func(fw *Writer, count int) {
+ for i := 0; i < count; i++ {
+ w, _ := fw.CreateFormField(fmt.Sprintf("field%v", i))
+ fmt.Fprintf(w, "value %v", i)
+ }
+ },
+ }, {
+ name: "files",
+ form: func(fw *Writer, count int) {
+ for i := 0; i < count; i++ {
+ w, _ := fw.CreateFormFile(fmt.Sprintf("field%v", i), fmt.Sprintf("file%v", i))
+ fmt.Fprintf(w, "value %v", i)
+ }
+ },
+ }} {
+ b.Run(test.name, func(b *testing.B) {
+ for _, maxMemory := range []int64{
+ 0,
+ 1 << 20,
+ } {
+ var buf bytes.Buffer
+ fw := NewWriter(&buf)
+ test.form(fw, 10)
+ if err := fw.Close(); err != nil {
+ b.Fatal(err)
+ }
+ b.Run(fmt.Sprintf("maxMemory=%v", maxMemory), func(b *testing.B) {
+ b.ReportAllocs()
+ for i := 0; i < b.N; i++ {
+ fr := NewReader(bytes.NewReader(buf.Bytes()), fw.Boundary())
+ form, err := fr.ReadForm(maxMemory)
+ if err != nil {
+ b.Fatal(err)
+ }
+ form.RemoveAll()
+ }
+
+ })
+ }
+ })
+ }
+}
--
2.37.1

183
0010-release-branch.go1.19-net-textproto-mime-multipart-i.patch Normal file
View File

@ -0,0 +1,183 @@
From 7a359a651c7ebdb29e0a1c03102fce793e9f58f0 Mon Sep 17 00:00:00 2001
From: Damien Neil <dneil@google.com>
Date: Thu, 16 Mar 2023 16:56:12 -0700
Subject: [PATCH] [release-branch.go1.19] net/textproto, mime/multipart:
improve accounting of non-file data
For requests containing large numbers of small parts,
memory consumption of a parsed form could be about 250%
over the estimated size.
When considering the size of parsed forms, account for the size of
FileHeader structs and increase the estimate of memory consumed by
map entries.
Thanks to Jakob Ackermann (@das7pad) for reporting this issue.
For CVE-2023-24536
For #59153
For #59269
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802454
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802396
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Change-Id: I31bc50e9346b4eee6fbe51a18c3c57230cc066db
Reviewed-on: https://go-review.googlesource.com/c/go/+/481984
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
---
src/mime/multipart/formdata.go | 9 +++--
src/mime/multipart/formdata_test.go | 55 ++++++++++++-----------------
src/net/textproto/reader.go | 8 ++++-
3 files changed, 37 insertions(+), 35 deletions(-)
diff --git a/src/mime/multipart/formdata.go b/src/mime/multipart/formdata.go
index 975dcb6b26..3f6ff697ca 100644
--- a/src/mime/multipart/formdata.go
+++ b/src/mime/multipart/formdata.go
@@ -103,8 +103,9 @@ func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
// Multiple values for the same key (one map entry, longer slice) are cheaper
// than the same number of values for different keys (many map entries), but
// using a consistent per-value cost for overhead is simpler.
+ const mapEntryOverhead = 200
maxMemoryBytes -= int64(len(name))
- maxMemoryBytes -= 100 // map overhead
+ maxMemoryBytes -= mapEntryOverhead
if maxMemoryBytes < 0 {
// We can't actually take this path, since nextPart would already have
// rejected the MIME headers for being too large. Check anyway.
@@ -128,7 +129,10 @@ func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
}
// file, store in memory or on disk
+ const fileHeaderSize = 100
maxMemoryBytes -= mimeHeaderSize(p.Header)
+ maxMemoryBytes -= mapEntryOverhead
+ maxMemoryBytes -= fileHeaderSize
if maxMemoryBytes < 0 {
return nil, ErrMessageTooLarge
}
@@ -183,9 +187,10 @@ func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
}
func mimeHeaderSize(h textproto.MIMEHeader) (size int64) {
+ size = 400
for k, vs := range h {
size += int64(len(k))
- size += 100 // map entry overhead
+ size += 200 // map entry overhead
for _, v := range vs {
size += int64(len(v))
}
diff --git a/src/mime/multipart/formdata_test.go b/src/mime/multipart/formdata_test.go
index f5b56083b2..8ed26e0c34 100644
--- a/src/mime/multipart/formdata_test.go
+++ b/src/mime/multipart/formdata_test.go
@@ -192,10 +192,10 @@ func (r *failOnReadAfterErrorReader) Read(p []byte) (n int, err error) {
// TestReadForm_NonFileMaxMemory asserts that the ReadForm maxMemory limit is applied
// while processing non-file form data as well as file form data.
func TestReadForm_NonFileMaxMemory(t *testing.T) {
- n := 10<<20 + 25
if testing.Short() {
- n = 10<<10 + 25
+ t.Skip("skipping in -short mode")
}
+ n := 10 << 20
largeTextValue := strings.Repeat("1", n)
message := `--MyBoundary
Content-Disposition: form-data; name="largetext"
@@ -203,38 +203,29 @@ Content-Disposition: form-data; name="largetext"
` + largeTextValue + `
--MyBoundary--
`
-
testBody := strings.ReplaceAll(message, "\n", "\r\n")
- testCases := []struct {
- name string
- maxMemory int64
- err error
- }{
- {"smaller", 50 + int64(len("largetext")) + 100, nil},
- {"exact-fit", 25 + int64(len("largetext")) + 100, nil},
- {"too-large", 0, ErrMessageTooLarge},
- }
- for _, tc := range testCases {
- t.Run(tc.name, func(t *testing.T) {
- if tc.maxMemory == 0 && testing.Short() {
- t.Skip("skipping in -short mode")
- }
- b := strings.NewReader(testBody)
- r := NewReader(b, boundary)
- f, err := r.ReadForm(tc.maxMemory)
- if err == nil {
- defer f.RemoveAll()
- }
- if tc.err != err {
- t.Fatalf("ReadForm error - got: %v; expected: %v", err, tc.err)
- }
- if err == nil {
- if g := f.Value["largetext"][0]; g != largeTextValue {
- t.Errorf("largetext mismatch: got size: %v, expected size: %v", len(g), len(largeTextValue))
- }
- }
- })
+ // Try parsing the form with increasing maxMemory values.
+ // Changes in how we account for non-file form data may cause the exact point
+ // where we change from rejecting the form as too large to accepting it to vary,
+ // but we should see both successes and failures.
+ const failWhenMaxMemoryLessThan = 128
+ for maxMemory := int64(0); maxMemory < failWhenMaxMemoryLessThan*2; maxMemory += 16 {
+ b := strings.NewReader(testBody)
+ r := NewReader(b, boundary)
+ f, err := r.ReadForm(maxMemory)
+ if err != nil {
+ continue
+ }
+ if g := f.Value["largetext"][0]; g != largeTextValue {
+ t.Errorf("largetext mismatch: got size: %v, expected size: %v", len(g), len(largeTextValue))
+ }
+ f.RemoveAll()
+ if maxMemory < failWhenMaxMemoryLessThan {
+ t.Errorf("ReadForm(%v): no error, expect to hit memory limit when maxMemory < %v", maxMemory, failWhenMaxMemoryLessThan)
+ }
+ return
}
+ t.Errorf("ReadForm(x) failed for x < 1024, expect success")
}
// TestReadForm_MetadataTooLarge verifies that we account for the size of field names,
diff --git a/src/net/textproto/reader.go b/src/net/textproto/reader.go
index 9a21777df8..c1284fde25 100644
--- a/src/net/textproto/reader.go
+++ b/src/net/textproto/reader.go
@@ -503,6 +503,12 @@ func readMIMEHeader(r *Reader, lim int64) (MIMEHeader, error) {
m := make(MIMEHeader, hint)
+ // Account for 400 bytes of overhead for the MIMEHeader, plus 200 bytes per entry.
+ // Benchmarking map creation as of go1.20, a one-entry MIMEHeader is 416 bytes and large
+ // MIMEHeaders average about 200 bytes per entry.
+ lim -= 400
+ const mapEntryOverhead = 200
+
// The first line cannot start with a leading space.
if buf, err := r.R.Peek(1); err == nil && (buf[0] == ' ' || buf[0] == '\t') {
line, err := r.readLineSlice()
@@ -538,7 +544,7 @@ func readMIMEHeader(r *Reader, lim int64) (MIMEHeader, error) {
vv := m[key]
if vv == nil {
lim -= int64(len(key))
- lim -= 100 // map entry overhead
+ lim -= mapEntryOverhead
}
lim -= int64(len(value))
if lim < 0 {
--
2.37.1

344
0011-release-branch.go1.19-mime-multipart-limit-parsed-mi.patch Normal file
View File

@ -0,0 +1,344 @@
From 7917b5f31204528ea72e0629f0b7d52b35b27538 Mon Sep 17 00:00:00 2001
From: Damien Neil <dneil@google.com>
Date: Mon, 20 Mar 2023 10:43:19 -0700
Subject: [PATCH] [release-branch.go1.19] mime/multipart: limit parsed mime
message sizes
The parsed forms of MIME headers and multipart forms can consume
substantially more memory than the size of the input data.
A malicious input containing a very large number of headers or
form parts can cause excessively large memory allocations.
Set limits on the size of MIME data:
Reader.NextPart and Reader.NextRawPart limit the the number
of headers in a part to 10000.
Reader.ReadForm limits the total number of headers in all
FileHeaders to 10000.
Both of these limits may be set with with
GODEBUG=multipartmaxheaders=<values>.
Reader.ReadForm limits the number of parts in a form to 1000.
This limit may be set with GODEBUG=multipartmaxparts=<value>.
Thanks for Jakob Ackermann (@das7pad) for reporting this issue.
For CVE-2023-24536
For #59153
For #59269
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802455
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1801087
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Change-Id: If134890d75f0d95c681d67234daf191ba08e6424
Reviewed-on: https://go-review.googlesource.com/c/go/+/481985
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
---
src/mime/multipart/formdata.go | 19 ++++++++-
src/mime/multipart/formdata_test.go | 61 ++++++++++++++++++++++++++++
src/mime/multipart/multipart.go | 31 ++++++++++----
src/mime/multipart/readmimeheader.go | 2 +-
src/net/textproto/reader.go | 19 +++++----
5 files changed, 115 insertions(+), 17 deletions(-)
diff --git a/src/mime/multipart/formdata.go b/src/mime/multipart/formdata.go
index 3f6ff697ca..4f26aab2cf 100644
--- a/src/mime/multipart/formdata.go
+++ b/src/mime/multipart/formdata.go
@@ -12,6 +12,7 @@ import (
"math"
"net/textproto"
"os"
+ "strconv"
)
// ErrMessageTooLarge is returned by ReadForm if the message form
@@ -41,6 +42,15 @@ func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
numDiskFiles := 0
multipartFiles := godebug.Get("multipartfiles")
combineFiles := multipartFiles != "distinct"
+ maxParts := 1000
+ multipartMaxParts := godebug.Get("multipartmaxparts")
+ if multipartMaxParts != "" {
+ if v, err := strconv.Atoi(multipartMaxParts); err == nil && v >= 0 {
+ maxParts = v
+ }
+ }
+ maxHeaders := maxMIMEHeaders()
+
defer func() {
if file != nil {
if cerr := file.Close(); err == nil {
@@ -86,13 +96,17 @@ func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
}
var copyBuf []byte
for {
- p, err := r.nextPart(false, maxMemoryBytes)
+ p, err := r.nextPart(false, maxMemoryBytes, maxHeaders)
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
+ if maxParts <= 0 {
+ return nil, ErrMessageTooLarge
+ }
+ maxParts--
name := p.FormName()
if name == "" {
@@ -136,6 +150,9 @@ func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
if maxMemoryBytes < 0 {
return nil, ErrMessageTooLarge
}
+ for _, v := range p.Header {
+ maxHeaders -= int64(len(v))
+ }
fh := &FileHeader{
Filename: filename,
Header: p.Header,
diff --git a/src/mime/multipart/formdata_test.go b/src/mime/multipart/formdata_test.go
index 8ed26e0c34..c78eeb7a12 100644
--- a/src/mime/multipart/formdata_test.go
+++ b/src/mime/multipart/formdata_test.go
@@ -360,6 +360,67 @@ func testReadFormManyFiles(t *testing.T, distinct bool) {
}
}
+func TestReadFormLimits(t *testing.T) {
+ for _, test := range []struct {
+ values int
+ files int
+ extraKeysPerFile int
+ wantErr error
+ godebug string
+ }{
+ {values: 1000},
+ {values: 1001, wantErr: ErrMessageTooLarge},
+ {values: 500, files: 500},
+ {values: 501, files: 500, wantErr: ErrMessageTooLarge},
+ {files: 1000},
+ {files: 1001, wantErr: ErrMessageTooLarge},
+ {files: 1, extraKeysPerFile: 9998}, // plus Content-Disposition and Content-Type
+ {files: 1, extraKeysPerFile: 10000, wantErr: ErrMessageTooLarge},
+ {godebug: "multipartmaxparts=100", values: 100},
+ {godebug: "multipartmaxparts=100", values: 101, wantErr: ErrMessageTooLarge},
+ {godebug: "multipartmaxheaders=100", files: 2, extraKeysPerFile: 48},
+ {godebug: "multipartmaxheaders=100", files: 2, extraKeysPerFile: 50, wantErr: ErrMessageTooLarge},
+ } {
+ name := fmt.Sprintf("values=%v/files=%v/extraKeysPerFile=%v", test.values, test.files, test.extraKeysPerFile)
+ if test.godebug != "" {
+ name += fmt.Sprintf("/godebug=%v", test.godebug)
+ }
+ t.Run(name, func(t *testing.T) {
+ if test.godebug != "" {
+ t.Setenv("GODEBUG", test.godebug)
+ }
+ var buf bytes.Buffer
+ fw := NewWriter(&buf)
+ for i := 0; i < test.values; i++ {
+ w, _ := fw.CreateFormField(fmt.Sprintf("field%v", i))
+ fmt.Fprintf(w, "value %v", i)
+ }
+ for i := 0; i < test.files; i++ {
+ h := make(textproto.MIMEHeader)
+ h.Set("Content-Disposition",
+ fmt.Sprintf(`form-data; name="file%v"; filename="file%v"`, i, i))
+ h.Set("Content-Type", "application/octet-stream")
+ for j := 0; j < test.extraKeysPerFile; j++ {
+ h.Set(fmt.Sprintf("k%v", j), "v")
+ }
+ w, _ := fw.CreatePart(h)
+ fmt.Fprintf(w, "value %v", i)
+ }
+ if err := fw.Close(); err != nil {
+ t.Fatal(err)
+ }
+ fr := NewReader(bytes.NewReader(buf.Bytes()), fw.Boundary())
+ form, err := fr.ReadForm(1 << 10)
+ if err == nil {
+ defer form.RemoveAll()
+ }
+ if err != test.wantErr {
+ t.Errorf("ReadForm = %v, want %v", err, test.wantErr)
+ }
+ })
+ }
+}
+
func BenchmarkReadForm(b *testing.B) {
for _, test := range []struct {
name string
diff --git a/src/mime/multipart/multipart.go b/src/mime/multipart/multipart.go
index fc50d35196..fab676a70a 100644
--- a/src/mime/multipart/multipart.go
+++ b/src/mime/multipart/multipart.go
@@ -16,11 +16,13 @@ import (
"bufio"
"bytes"
"fmt"
+ "internal/godebug"
"io"
"mime"
"mime/quotedprintable"
"net/textproto"
"path/filepath"
+ "strconv"
"strings"
)
@@ -128,12 +130,12 @@ func (r *stickyErrorReader) Read(p []byte) (n int, _ error) {
return n, r.err
}
-func newPart(mr *Reader, rawPart bool, maxMIMEHeaderSize int64) (*Part, error) {
+func newPart(mr *Reader, rawPart bool, maxMIMEHeaderSize, maxMIMEHeaders int64) (*Part, error) {
bp := &Part{
Header: make(map[string][]string),
mr: mr,
}
- if err := bp.populateHeaders(maxMIMEHeaderSize); err != nil {
+ if err := bp.populateHeaders(maxMIMEHeaderSize, maxMIMEHeaders); err != nil {
return nil, err
}
bp.r = partReader{bp}
@@ -149,9 +151,9 @@ func newPart(mr *Reader, rawPart bool, maxMIMEHeaderSize int64) (*Part, error) {
return bp, nil
}
-func (p *Part) populateHeaders(maxMIMEHeaderSize int64) error {
+func (p *Part) populateHeaders(maxMIMEHeaderSize, maxMIMEHeaders int64) error {
r := textproto.NewReader(p.mr.bufReader)
- header, err := readMIMEHeader(r, maxMIMEHeaderSize)
+ header, err := readMIMEHeader(r, maxMIMEHeaderSize, maxMIMEHeaders)
if err == nil {
p.Header = header
}
@@ -330,6 +332,19 @@ type Reader struct {
// including header keys, values, and map overhead.
const maxMIMEHeaderSize = 10 << 20
+func maxMIMEHeaders() int64 {
+ // multipartMaxHeaders is the maximum number of header entries NextPart will return,
+ // as well as the maximum combined total of header entries Reader.ReadForm will return
+ // in FileHeaders.
+ multipartMaxHeaders := godebug.Get("multipartmaxheaders")
+ if multipartMaxHeaders != "" {
+ if v, err := strconv.ParseInt(multipartMaxHeaders, 10, 64); err == nil && v >= 0 {
+ return v
+ }
+ }
+ return 10000
+}
+
// NextPart returns the next part in the multipart or an error.
// When there are no more parts, the error io.EOF is returned.
//
@@ -337,7 +352,7 @@ const maxMIMEHeaderSize = 10 << 20
// has a value of "quoted-printable", that header is instead
// hidden and the body is transparently decoded during Read calls.
func (r *Reader) NextPart() (*Part, error) {
- return r.nextPart(false, maxMIMEHeaderSize)
+ return r.nextPart(false, maxMIMEHeaderSize, maxMIMEHeaders())
}
// NextRawPart returns the next part in the multipart or an error.
@@ -346,10 +361,10 @@ func (r *Reader) NextPart() (*Part, error) {
// Unlike NextPart, it does not have special handling for
// "Content-Transfer-Encoding: quoted-printable".
func (r *Reader) NextRawPart() (*Part, error) {
- return r.nextPart(true, maxMIMEHeaderSize)
+ return r.nextPart(true, maxMIMEHeaderSize, maxMIMEHeaders())
}
-func (r *Reader) nextPart(rawPart bool, maxMIMEHeaderSize int64) (*Part, error) {
+func (r *Reader) nextPart(rawPart bool, maxMIMEHeaderSize, maxMIMEHeaders int64) (*Part, error) {
if r.currentPart != nil {
r.currentPart.Close()
}
@@ -374,7 +389,7 @@ func (r *Reader) nextPart(rawPart bool, maxMIMEHeaderSize int64) (*Part, error)
if r.isBoundaryDelimiterLine(line) {
r.partsRead++
- bp, err := newPart(r, rawPart, maxMIMEHeaderSize)
+ bp, err := newPart(r, rawPart, maxMIMEHeaderSize, maxMIMEHeaders)
if err != nil {
return nil, err
}
diff --git a/src/mime/multipart/readmimeheader.go b/src/mime/multipart/readmimeheader.go
index 6836928c9e..25aa6e2092 100644
--- a/src/mime/multipart/readmimeheader.go
+++ b/src/mime/multipart/readmimeheader.go
@@ -11,4 +11,4 @@ import (
// readMIMEHeader is defined in package net/textproto.
//
//go:linkname readMIMEHeader net/textproto.readMIMEHeader
-func readMIMEHeader(r *textproto.Reader, lim int64) (textproto.MIMEHeader, error)
+func readMIMEHeader(r *textproto.Reader, maxMemory, maxHeaders int64) (textproto.MIMEHeader, error)
diff --git a/src/net/textproto/reader.go b/src/net/textproto/reader.go
index c1284fde25..1ad4416ee1 100644
--- a/src/net/textproto/reader.go
+++ b/src/net/textproto/reader.go
@@ -483,12 +483,12 @@ var colon = []byte(":")
// "Long-Key": {"Even Longer Value"},
// }
func (r *Reader) ReadMIMEHeader() (MIMEHeader, error) {
- return readMIMEHeader(r, math.MaxInt64)
+ return readMIMEHeader(r, math.MaxInt64, math.MaxInt64)
}
// readMIMEHeader is a version of ReadMIMEHeader which takes a limit on the header size.
// It is called by the mime/multipart package.
-func readMIMEHeader(r *Reader, lim int64) (MIMEHeader, error) {
+func readMIMEHeader(r *Reader, maxMemory, maxHeaders int64) (MIMEHeader, error) {
// Avoid lots of small slice allocations later by allocating one
// large one ahead of time which we'll cut up into smaller
// slices. If this isn't big enough later, we allocate small ones.
@@ -506,7 +506,7 @@ func readMIMEHeader(r *Reader, lim int64) (MIMEHeader, error) {
// Account for 400 bytes of overhead for the MIMEHeader, plus 200 bytes per entry.
// Benchmarking map creation as of go1.20, a one-entry MIMEHeader is 416 bytes and large
// MIMEHeaders average about 200 bytes per entry.
- lim -= 400
+ maxMemory -= 400
const mapEntryOverhead = 200
// The first line cannot start with a leading space.
@@ -538,16 +538,21 @@ func readMIMEHeader(r *Reader, lim int64) (MIMEHeader, error) {
continue
}
+ maxHeaders--
+ if maxHeaders < 0 {
+ return nil, errors.New("message too large")
+ }
+
// Skip initial spaces in value.
value := string(bytes.TrimLeft(v, " \t"))
vv := m[key]
if vv == nil {
- lim -= int64(len(key))
- lim -= mapEntryOverhead
+ maxMemory -= int64(len(key))
+ maxMemory -= mapEntryOverhead
}
- lim -= int64(len(value))
- if lim < 0 {
+ maxMemory -= int64(len(value))
+ if maxMemory < 0 {
// TODO: This should be a distinguishable error (ErrMessageTooLarge)
// to allow mime/multipart to detect it.
return m, errors.New("message too large")
--
2.37.1

2
0003-release-branch.go1.19-go-scanner-reject-large-line.patch → 0012-release-branch.go1.19-go-scanner-reject-large-line-a.patch
View File

@ -87,5 +87,5 @@ index 07e07581f7..6a7e30bb2f 100644
return
}
--
2.33.0
2.37.1

2
0004-release-branch.go1.19-html-template-disallow-actions.patch → 0013-release-branch.go1.19-html-template-disallow-actions.patch
View File

@ -365,5 +365,5 @@ index 06df679330..92eb351906 100644
specials = `\/[]`
}
--
2.33.0
2.37.1

63
0014-release-branch.go1.19-html-template-disallow-angle-b.patch Normal file
View File

@ -0,0 +1,63 @@
From e49282327b05192e46086bf25fd3ac691205fe80 Mon Sep 17 00:00:00 2001
From: Roland Shoemaker <bracewell@google.com>
Date: Thu, 13 Apr 2023 15:40:44 -0700
Subject: [PATCH] [release-branch.go1.19] html/template: disallow angle
brackets in CSS values
Angle brackets should not appear in CSS contexts, as they may affect
token boundaries (such as closing a <style> tag, resulting in
injection). Instead emit filterFailsafe, matching the behavior for other
dangerous characters.
Thanks to Juho Nurminen of Mattermost for reporting this issue.
For #59720
Fixes #59811
Fixes CVE-2023-24539
Change-Id: Iccc659c9a18415992b0c05c178792228e3a7bae4
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1826636
Reviewed-by: Julie Qiu <julieqiu@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1851496
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/491335
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
---
src/html/template/css.go | 2 +-
src/html/template/css_test.go | 2 ++
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/src/html/template/css.go b/src/html/template/css.go
index 890a0c6b22..f650d8b3e8 100644
--- a/src/html/template/css.go
+++ b/src/html/template/css.go
@@ -238,7 +238,7 @@ func cssValueFilter(args ...any) string {
// inside a string that might embed JavaScript source.
for i, c := range b {
switch c {
- case 0, '"', '\'', '(', ')', '/', ';', '@', '[', '\\', ']', '`', '{', '}':
+ case 0, '"', '\'', '(', ')', '/', ';', '@', '[', '\\', ']', '`', '{', '}', '<', '>':
return filterFailsafe
case '-':
// Disallow <!-- or -->.
diff --git a/src/html/template/css_test.go b/src/html/template/css_test.go
index a735638b03..2b76256a76 100644
--- a/src/html/template/css_test.go
+++ b/src/html/template/css_test.go
@@ -231,6 +231,8 @@ func TestCSSValueFilter(t *testing.T) {
{`-exp\000052 ession(alert(1337))`, "ZgotmplZ"},
{`-expre\0000073sion`, "-expre\x073sion"},
{`@import url evil.css`, "ZgotmplZ"},
+ {"<", "ZgotmplZ"},
+ {">", "ZgotmplZ"},
}
for _, test := range tests {
got := cssValueFilter(test.css)
--
2.37.1

89
0015-release-branch.go1.19-html-template-handle-all-JS-wh.patch Normal file
View File

@ -0,0 +1,89 @@
From ce7bd33345416e6d8cac901792060591cafc2797 Mon Sep 17 00:00:00 2001
From: Roland Shoemaker <bracewell@google.com>
Date: Tue, 11 Apr 2023 16:27:43 +0100
Subject: [PATCH] [release-branch.go1.19] html/template: handle all JS
whitespace characters
Rather than just a small set. Character class as defined by \s [0].
Thanks to Juho Nurminen of Mattermost for reporting this.
For #59721
Fixes #59813
Fixes CVE-2023-24540
[0] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Character_Classes
Change-Id: I56d4fa1ef08125b417106ee7dbfb5b0923b901ba
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1821459
Reviewed-by: Julie Qiu <julieqiu@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1851497
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/491355
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Bypass: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
---
src/html/template/js.go | 8 +++++++-
src/html/template/js_test.go | 11 +++++++----
2 files changed, 14 insertions(+), 5 deletions(-)
diff --git a/src/html/template/js.go b/src/html/template/js.go
index fe7054efe5..4e05c14557 100644
--- a/src/html/template/js.go
+++ b/src/html/template/js.go
@@ -13,6 +13,11 @@ import (
"unicode/utf8"
)
+// jsWhitespace contains all of the JS whitespace characters, as defined
+// by the \s character class.
+// See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_expressions/Character_classes.
+const jsWhitespace = "\f\n\r\t\v\u0020\u00a0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200a\u2028\u2029\u202f\u205f\u3000\ufeff"
+
// nextJSCtx returns the context that determines whether a slash after the
// given run of tokens starts a regular expression instead of a division
// operator: / or /=.
@@ -26,7 +31,8 @@ import (
// JavaScript 2.0 lexical grammar and requires one token of lookbehind:
// https://www.mozilla.org/js/language/js20-2000-07/rationale/syntax.html
func nextJSCtx(s []byte, preceding jsCtx) jsCtx {
- s = bytes.TrimRight(s, "\t\n\f\r \u2028\u2029")
+ // Trim all JS whitespace characters
+ s = bytes.TrimRight(s, jsWhitespace)
if len(s) == 0 {
return preceding
}
diff --git a/src/html/template/js_test.go b/src/html/template/js_test.go
index e07c695f7a..e52180cc11 100644
--- a/src/html/template/js_test.go
+++ b/src/html/template/js_test.go
@@ -81,14 +81,17 @@ func TestNextJsCtx(t *testing.T) {
{jsCtxDivOp, "0"},
// Dots that are part of a number are div preceders.
{jsCtxDivOp, "0."},
+ // Some JS interpreters treat NBSP as a normal space, so
+ // we must too in order to properly escape things.
+ {jsCtxRegexp, "=\u00A0"},
}
for _, test := range tests {
- if nextJSCtx([]byte(test.s), jsCtxRegexp) != test.jsCtx {
- t.Errorf("want %s got %q", test.jsCtx, test.s)
+ if ctx := nextJSCtx([]byte(test.s), jsCtxRegexp); ctx != test.jsCtx {
+ t.Errorf("%q: want %s got %s", test.s, test.jsCtx, ctx)
}
- if nextJSCtx([]byte(test.s), jsCtxDivOp) != test.jsCtx {
- t.Errorf("want %s got %q", test.jsCtx, test.s)
+ if ctx := nextJSCtx([]byte(test.s), jsCtxDivOp); ctx != test.jsCtx {
+ t.Errorf("%q: want %s got %s", test.s, test.jsCtx, ctx)
}
}
--
2.37.1

95
0016-release-branch.go1.19-html-template-emit-filterFails.patch Normal file
View File

@ -0,0 +1,95 @@
From 9db0e74f606b8afb28cc71d4b1c8b4ed24cabbf5 Mon Sep 17 00:00:00 2001
From: Roland Shoemaker <bracewell@google.com>
Date: Thu, 13 Apr 2023 14:01:50 -0700
Subject: [PATCH] [release-branch.go1.19] html/template: emit filterFailsafe
for empty unquoted attr value
An unquoted action used as an attribute value can result in unsafe
behavior if it is empty, as HTML normalization will result in unexpected
attributes, and may allow attribute injection. If executing a template
results in a empty unquoted attribute value, emit filterFailsafe
instead.
Thanks to Juho Nurminen of Mattermost for reporting this issue.
For #59722
Fixes #59815
Fixes CVE-2023-29400
Change-Id: Ia38d1b536ae2b4af5323a6c6d861e3c057c2570a
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1826631
Reviewed-by: Julie Qiu <julieqiu@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1851498
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/491357
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
---
src/html/template/escape.go | 5 ++---
src/html/template/escape_test.go | 15 +++++++++++++++
src/html/template/html.go | 3 +++
3 files changed, 20 insertions(+), 3 deletions(-)
diff --git a/src/html/template/escape.go b/src/html/template/escape.go
index 3d4cc19b5d..bcba8db4aa 100644
--- a/src/html/template/escape.go
+++ b/src/html/template/escape.go
@@ -380,9 +380,8 @@ func normalizeEscFn(e string) string {
// for all x.
var redundantFuncs = map[string]map[string]bool{
"_html_template_commentescaper": {
- "_html_template_attrescaper": true,
- "_html_template_nospaceescaper": true,
- "_html_template_htmlescaper": true,
+ "_html_template_attrescaper": true,
+ "_html_template_htmlescaper": true,
},
"_html_template_cssescaper": {
"_html_template_attrescaper": true,
diff --git a/src/html/template/escape_test.go b/src/html/template/escape_test.go
index 972b00b921..a1a6c1cd16 100644
--- a/src/html/template/escape_test.go
+++ b/src/html/template/escape_test.go
@@ -678,6 +678,21 @@ func TestEscape(t *testing.T) {
`<img srcset={{",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"}}>`,
`<img srcset=,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,>`,
},
+ {
+ "unquoted empty attribute value (plaintext)",
+ "<p name={{.U}}>",
+ "<p name=ZgotmplZ>",
+ },
+ {
+ "unquoted empty attribute value (url)",
+ "<p href={{.U}}>",
+ "<p href=ZgotmplZ>",
+ },
+ {
+ "quoted empty attribute value",
+ "<p name=\"{{.U}}\">",
+ "<p name=\"\">",
+ },
}
for _, test := range tests {
diff --git a/src/html/template/html.go b/src/html/template/html.go
index 46e9d93151..6fb9237bda 100644
--- a/src/html/template/html.go
+++ b/src/html/template/html.go
@@ -14,6 +14,9 @@ import (
// htmlNospaceEscaper escapes for inclusion in unquoted attribute values.
func htmlNospaceEscaper(args ...any) string {
s, t := stringify(args...)
+ if s == "" {
+ return filterFailsafe
+ }
if t == contentTypeHTML {
return htmlReplacer(stripTags(s), htmlNospaceNormReplacementTable, false)
}
--
2.37.1

32
golang.spec
View File

@ -62,16 +62,28 @@
Name: golang
Version: 1.19.4
Release: 5
Release: 6
Summary: The Go Programming Language
License: BSD and Public Domain
URL: https://golang.org/
Source0: https://dl.google.com/go/go1.19.4.src.tar.gz
Patch0: 0001-Enable-go-plugin-support-for-riscv64.patch
Patch1: 0002-runtime-support-riscv64-SV57-mode.patch
Patch2: 0003-release-branch.go1.19-go-scanner-reject-large-line.patch
Patch3: 0004-release-branch.go1.19-html-template-disallow-actions.patch
Patch4: 0005-release-branch.go1.19-net-textproto-avoid-overpredic.patch
Patch2: 0003-release-branch.go1.19-path-filepath-do-not-Clean-a-..patch
Patch3: 0004-release-branch.go1.19-mime-multipart-limit-memory-in.patch
Patch4: 0005-release-branch.go1.19-crypto-tls-replace-all-usages-.patch
Patch5: 0006-release-branch.go1.19-net-http-update-bundled-golang.patch
Patch6: 0007-release-branch.go1.19-crypto-internal-nistec-reduce-.patch
Patch7: 0008-release-branch.go1.19-net-textproto-avoid-overpredic.patch
Patch8: 0009-release-branch.go1.19-mime-multipart-avoid-excessive.patch
Patch9: 0010-release-branch.go1.19-net-textproto-mime-multipart-i.patch
Patch10: 0011-release-branch.go1.19-mime-multipart-limit-parsed-mi.patch
Patch11: 0012-release-branch.go1.19-go-scanner-reject-large-line-a.patch
Patch12: 0013-release-branch.go1.19-html-template-disallow-actions.patch
Patch13: 0014-release-branch.go1.19-html-template-disallow-angle-b.patch
Patch14: 0015-release-branch.go1.19-html-template-handle-all-JS-wh.patch
Patch15: 0016-release-branch.go1.19-html-template-emit-filterFails.patch
%if !%{golang_bootstrap}
BuildRequires: gcc-go >= 5
%else
@ -391,6 +403,18 @@ fi
%files devel -f go-tests.list -f go-misc.list -f go-src.list
%changelog
* Wed May 10 2023 zhangzhihui<zhangzhihui@xfusion.com> - 1.19.4-6
- fix CVE-2024-29400
- fix CVE-2023-24540
- fix CVE-2023-24539
- fix CVE-2023-24536
- fix CVE-2023-24532
- fix CVE-2022-41723
- fix CVE-2022-41724
- fix CVE-2022-41725
- fix CVE-2022-41722
* Sat May 6 2023 sunchendong<sunchendong@xfusion.com> - 1.19.4-5
- fix CVE-2023-24534