[Chapter 34] 34.24 Quick Reference: sed
Author: Jerry Peek, Tim O'Reilly & Mike Loukides
Date: 1998-08-04T21:47:38Z

#

| Chapter 34 The sed Stream Editor |

## 34.24 Quick Reference: sed

How sed operates:

- Each line of input is copied into a pattern space.

- Editing commands may be given on the command line (if more than one, use a -e option before each command) and/or in script files named after -f options. All editing commands are applied in order to each line of input.

- Editing commands are applied to all lines (globally) unless line addressing restricts the lines affected.

- If a command changes the input, subsequent command-addresses will be applied to the current line in the pattern space, not the original input line.

- The original input file is unchanged; editing commands modify a copy of the original input line. The copy is sent to standard output (13.1) unless the -n option was used; standard output can be redirected to a file (13.1, 34.3).

### 34.24.1 Syntax of sed Commands

sed commands have the general form:

> [`address`][`,``address`][`!`]`command` [`arguments`]

sed commands consist of `addresses` and editing `commands`. `commands` consist of a single letter or symbol; they are described later, alphabetically and by group. `arguments` include the label supplied to b or t, the filename supplied to r or w, and the substitution flags for s. `addresses` are described below. Elements in `[`brackets`]` are optional; don't type the brackets.

Braces (`{}`) are used in sed to nest one address inside another or to apply multiple commands at the same address:

>     [address][,address]{
>        command1
>        command2
>     }

The left curly brace (`{`) is a command that starts a group of other sed commands. The group ends with a right curly brace (`}`). Commands within the braces may be spread across multiple lines, as shown above. Or commands may be on the same line, with a semicolon (`;`) after each command (including the last command on a line) - as in:

> [`address`][`,``address`]`{``command1``;` ...`commandN``; }`

### 34.24.2 Pattern Addressing

A sed command can specify zero, one, or two addresses. An address can be a line number, the symbol `$` (for last line), or a regular expression enclosed in slashes (`/``pattern``/`). Regular expressions are described in Chapter 26, Regular Expressions (Pattern Matching). Additionally, `\n` can be used to match any newline in the pattern space (resulting from the N command), but not the newline at the end of the pattern space. See article 34.4.

If the command specifies: | Then it is applied to:
--- | ---
No address | Each input line.
One address | Any line matching the address. Some commands accept only one address: a, i, r, q, and `=`.
Two comma-separated addresses | First matching line and all succeeding lines up to and including a line matching the second address. Repeat for each matching range in the text.
An address followed by `!` | All lines that do not match the address.

#### 34.24.2.1 Examples

Substitute on all lines (all occurrences):

>     s/xx/yy/g

Delete lines containing `BSD`:

>     /BSD/d

Print the lines between each pair of `BEGIN` and `END`, inclusive:

>     /^BEGIN/,/^END/p

Delete any line that doesn't contain `SAVE`:

>     /SAVE/!d

Substitute on all lines, except between `BEGIN` and `END`:

>     /BEGIN/,/END/!s/xx/yy/g

### 34.24.3 Alphabetical Summary of sed Commands

`#`

  Begin a comment in a sed script. If the first such line is exactly `#n`, sed sets its -n command-line option.

`:`

  `:``label`

  Label a line in the script for the transfer of control by b or t. `label` may contain up to seven characters.

`=`

  [`address`]`=`

  Write to standard output the line number of each line addressed.

`a`

  > `[``address``]a\`
  > `text`

  Append `text` following each line matched by `address`. If there is more than one line of `text`, all newlines except the last must be "hidden" by preceding them with a backslash. `text` will be terminated by the first newline that is not hidden in this way. `text` is not available in the pattern space, and subsequent commands cannot be applied to it. The results of this command are sent to standard output when the list of editing commands is finished, regardless of what happens to the current line in the pattern space. (There's an example in article 43.22, among others.)

  Example

  >     $a\
  >     This goes after the last line in the file\
  >     (marked by $). This text is escaped at the\
  >     end of each line, except for the last one.

`b`

  `[``address1``][,``address2`]`b[``label``]`

  Transfer control unconditionally to `:``label` elsewhere in script. That is, the command following the `label` is the next command applied to the current line. If no `label` is specified, control falls through to the end of the script, so no more commands are applied to the current line. See articles 34.19 and 34.17.

  Example

  >     # Ignore tbl tables; resume script after TE:
  >     /^\.TS/,/^\.TE/b

`c`

  > `[``address1`][`,``address2``]c\`
  > `text`

  Replace the lines selected by the address with `text`. When a range of lines is specified, all lines as a group are replaced by a single copy of `text`. The newline following each line of `text` must be escaped by a backslash, except the last line. The contents of the pattern space are, in effect, deleted and no subsequent editing commands can be applied to it (or `text`).

  Example

  >     # Replace first 100 lines in a file:
  >     1,100c\
  >     ...first replacement line\
  >     ...second replacement line\
  >     ...\
  >     ...last replacement line

`d`

  `[``address1``][,``address2``]d`

  Delete the addressed line (or lines) from the pattern space. Thus, the line is not passed to standard output. A new line of input is read, and editing resumes with the first command in the script. See articles 34.4 and 34.18.

  Example

  >     # delete all blank lines:
  >     /^$/d

`D`

  [`address1`][`,``address2`]`D`

  Delete first part (up to embedded newline) of multiline pattern space created by N command and resume editing with first command in script. If this command empties the pattern space, then a new line of input is read, as if d had been executed. See article 34.18.

  Example

  >     # Strip multiple blank lines, leaving only one:
  >     /^$/{
  >         N
  >         /^\n$/D
  >     }

`g`

  `[``address1``][,``address2``]g`

  Paste the contents of the hold space (see h or H) back into the pattern space, wiping out the previous contents of the pattern space. See articles 34.13 and 34.16. The example shows a simple way to copy lines.

  Example

  This script collects all lines containing the word `Item:` and copies them to a place marker later in the file. The place marker is overwritten.

  >     /Item:/H
  >     /<Replace this line with the item list>/g

`G`

  [`address1`][`,``address2`]`G`

  Same as g, except that the hold space is pasted below the address instead of overwriting it. The example shows a simple way to "cut and paste" lines. See articles 34.13 and 34.16.

  Example

  This script collects all lines containing the word `Item:` and moves them after a place marker later in the file. The original `Item:` lines are deleted.

  >     /Item:/{
  >         H
  >         d
  >     }
  >     /Summary of items:/G

`h`

  [`address1`][`,``address2`]`h` Copy the pattern space into the hold space, a special temporary buffer. The previous contents of the hold space are obliterated. You can use h to save a line before editing it. See articles 34.13 and 34.16.

  Example

  >     # Edit a line; print the change; replay the original
  >     /UNIX/{
  >         h
  >         s/.* UNIX \(.*\) .*/\1:/
  >         p
  >         x
  >     }

  Sample input:

  >     This describes the UNIX ls command.
  >     This describes the UNIX cp command.

  Sample output:

  >     ls:
  >     This describes the UNIX ls command.
  >     cp:
  >     This describes the UNIX cp command.

`H`

  [`address1`][`,``address2`]`H`

  Append the contents of the pattern space (preceded by a newline) to the contents of the hold space. Even if the hold space is empty, H still appends a newline. H is like an incremental copy. See examples under g and G, also articles 34.13 and 34.16.

`i`

  > `[``address``]i\`
  > `text`

  Insert `text` before each line matched by `address`. (See a for details on `text`.) Article 43.20 shows a script that uses i.

  Example

  >     /Item 1/i\
  >     The five items are listed below:

`l`

  [`address1`][`,``address2`]`l`

  List the contents of the pattern space, showing non-printing characters as ASCII codes (51.3, 25.7). Long lines are wrapped.

`n`

  [`address1`][`,``address2`]`n`

  Read next line of input into pattern space. The current line is sent to standard output, and the next line becomes the current line. Control passes to the command following n instead of resuming at the top of the script.

  Example

  In the ms macros (43.14), a section header occurs on the line below an `.NH` macro. To print all lines of header text, invoke this script with sed -n:

  >     /^\.NH/{
  >         n
  >         p
  >     }

`N`

  [`address1`][`,``address2`]`N`

  Append next input line to contents of pattern space; the two lines are separated by an embedded newline. (This command is designed to allow pattern matches across two lines.) Using `\n` to match the embedded newline, you can match patterns across multiple lines. See example under D, also article 34.15.

  Examples

  Like previous example, but print `.NH` line as well as header title:

  >     /^\.NH/{
  >         N
  >         p
  >     }

  Join two lines (replace newline with space):

  >     /^\.NH/{
  >         N
  >         s/\n/ /
  >         p
  >     }

`p`

  [`address1`][`,``address2`]`p`

  Print the addressed line(s). Unless the -n command-line option is used, this command will cause duplicate lines to be output. Also, it is typically used before commands that change flow control (d, N, b) and that might prevent the current line from being output. See examples under h, n, and N.

`P`

  [`address1`][`,``address2`]`P`

  Print first part (up to embedded newline) of multiline pattern created by N command. Same as p if N has not been applied to a line.

  Example

  The following script prints each line containing `word` and also the line before it:

  >     N
  >     /word/P
  >     D

`q`

  [`address`]`q`

  Quit when `address` is encountered. The addressed line is first written to output (if default output is not suppressed), along with any text appended to it by previous a or r commands. See articles 34.21 and 34.22.

  Example

  Delete everything after the addressed line:

  >     /Garbled text follows:/q

  Print only the first 50 lines of a file:

  >     50q

`r`

  [`address`]`r` `file`

  Read contents of `file` and append after the contents of the pattern space. Exactly one space must be put between the `r` and `file`.

  Example

  >     /The list of items follows:/r item_file

`s`

  [`address1`][`,``address2`]`s/``pattern``/``replacement``/`[`flags`]

  Substitute `replacement` for `pattern` on each addressed line. If pattern addresses are used, the pattern `//` represents the last pattern address specified. The following flags can be specified:

  `n`

    Replace `n`th instance of `/``pattern``/` on each addressed line. `n` is any number in the range 1 to 512 (default is 1). See article 34.11.

  `g`

    Replace all instances of `/``pattern``/` on each addressed line, not just the first instance.

  `p`

    Print the line if a successful substitution is done. If several successful substitutions are done, multiple copies of the line will be printed. Often used in scripts with the -n command-line option (34.2).

  `w` `file`

    Write the line to a `file` if a replacement was done. A maximum of ten different files can be opened in a script. See articles 34.7 through 34.10.

    Examples

    Here are some short, commented scripts:

    >     # Change third and fourth quote to ( and ):
    >     /function/{
    >         s/"/)/4
    >         s/"/(/3
    >     }

    >     # Remove all quotes on a given line:
    >     /Title/s/"//g

    >     # Remove first colon or all quotes; print resulting lines:
    >     s/://p
    >     s/"//gp

    >     # Change first "if" but leave "ifdef" alone:
    >     /ifdef/!s/if/   if/

`t`

  [`address1`][`,``address2`]`t` [`label`]

  Test if any substitutions have been made on addressed lines, and if so, branch to line marked by `:``label`. (See b and `:`.) If `label` is not specified, control falls through to bottom of script. See article 34.20.

  The t command can be used like a case statement (44.5) in the Bourne shell. You test each case: when it's true, you exit the construct.

  Example

  Suppose you want to fill empty fields of a database. You have this:

  >     ID: 1   Name: greg   Rate: 45
  >     ID: 2   Name: dale
  >     ID: 3

  You want this:

  >     ID: 1   Name: greg   Rate: 45   Phone: ??
  >     ID: 2   Name: dale   Rate: ??   Phone: ??
  >     ID: 3   Name: ????   Rate: ??   Phone: ??

  You need to test the number of fields already there. Here's the script (fields are tab-separated):

  >     /ID/{
  >         s/ID: .* Name: .* Rate: .*/&   Phone: ??/p
  >         t
  >         s/ID: .* Name: .*/&   Rate: ??   Phone: ??/p
  >         t
  >         s/ID: .*/&   Name: ??     Rate: ??   Phone: ??/p
  >     }

`w`

  [`address1`][`,``address2`]`w` `file`

  Append, contents of pattern space to `file`. This action occurs when the command is encountered rather than when the pattern space is output. Exactly one space must separate the `w` and `file`. A maximum of ten different files can be opened in a script. This command will create the file if it does not exist; if the file exists, its contents will be overwritten each time the script is executed. Multiple write commands that direct output to the same file append to the end of the file.

  Example

  >     # Store tbl and eqn blocks in a file:
  >     /^\.TS/,/^\.TE/w troff_stuff
  >     /^\.EQ/,/^\.EN/w troff_stuff

`x`

  [`address1`][`,``address2`]`x`

  Exchange contents of the pattern space with the contents of the hold space. For examples, see h and articles 34.13 and 34.16.

`y`

  [`address1`][`,``address2`]`y/``abc``/``xyz``/`

  Translate characters. Change every instance of `a` to `x`, `b` to `y`, `c` to `z`, etc. See articles 34.12 and 34.14.

  Example

  >     # Change item 1, 2, 3 to Item A, B, C ...
  >     /^item [1-9]/y/i123456789/IABCDEFGHI/

- DG from O'Reilly & Associates' UNIX in a Nutshell (SVR4/Solaris)

34.23 sed Newlines, Quoting, and Backslashes in a Shell Script |  | 35. You Can't Quite Call This Editing