From 2cb09e14639bd0c6d55f796336ce4728607fd4b3 Mon Sep 17 00:00:00 2001 From: Assaf Gordon Date: Fri, 27 Jul 2018 01:56:26 -0600 Subject: [PATCH 05/61] sed: fix extraneous NUL in s///n command Under certain conditions sed would add an extraneous NUL: $ echo 0 | sed -e 's/$/a/2' | od -tx1 -An 30 00 0a This would happen when the regex is an empty (zero-length) match at the end of the line (e.g. '$' and 'a*$') and the substitute number flag ('n' in s///n) is higher than the number of actual matches (multiple EOL matches are possible with multiline match, e.g. 's/$/a/3m'). Details: The comment at the top of 'execute.c:do_subst()' says: /* The first part of the loop optimizes s/xxx// when xxx is at the start, and s/xxx$// */ Which refers to lines 1051-3: 1051 /* Copy stuff to the left of this match into the output string. */ 1052 if (start < offset) 1053 str_append(&s_accum, line.active + start, offset - start); The above code appends text to 's_accum' but does *not* update 'start'. Later on, if the s/// command includes 'n' flag, and if 'matched == 0' (an empty match), this comparison will be incorrect: 1081 if (start < line.length) 1082 matched = 1; Will in turn will set 'matched' to 1, and the 'str_append' call that follows (line 1087) will append an additional character. Because the empty match is EOL, the appended character is NUL. More examples that trigger the bug: echo 0 | sed -e 's/a*$/X/3' printf "%s\n" 0 0 0 | sed -e 'N;N;s/a*$/X/4m' Examples that do not trigger the bug: # The 'a*' empty regex matches at the beginning of the line (in # addition to the end of the line), and the optimization in line # 1052 is skipped. echo 0 | sed -e 's/a*/X/3' # There are 3 EOLs in the pattern space, s///3 is not too large. printf "%s\n" 0 0 0 | sed -e 'N;N;s/a*$/X/3m' This was discovered while investigating bug#32271 reported by bugs@feusi.co in https://lists.gnu.org/r/bug-sed/2018-07/msg00018.html . * NEWS: Mention the fix. * sed/execute.c (do_subst): Update 'start' as needed. * testsuite/bug-32271-1.sh: New test. * testsuite/local.mk (T): Add test. --- NEWS | 3 +++ sed/execute.c | 5 ++++- testsuite/bug32271-1.sh | 45 +++++++++++++++++++++++++++++++++++++++++ testsuite/local.mk | 1 + 4 files changed, 53 insertions(+), 1 deletion(-) create mode 100755 testsuite/bug32271-1.sh diff --git a/sed/execute.c b/sed/execute.c index 1cc1d3f..c1d656a 100644 --- a/sed/execute.c +++ b/sed/execute.c @@ -1050,7 +1050,10 @@ do_subst(struct subst *sub) /* Copy stuff to the left of this match into the output string. */ if (start < offset) - str_append(&s_accum, line.active + start, offset - start); + { + str_append(&s_accum, line.active + start, offset - start); + start = offset; + } /* If we're counting up to the Nth match, are we there yet? And even if we are there, there is another case we have to -- 2.19.1