[Chapter 2] 2.3 Using sed
Author: Dale Dougherty & Arnold Robbins
Date: 1998-08-03T20:49:57Z

#

| Chapter 2 Understanding Basic Operations |

## 2.3 Using sed

There are two ways to invoke sed: either you specify your editing instructions on the command line or you put them in a file and supply the name of the file.

### 2.3.1 Specifying Simple Instructions

You can specify simple editing commands on the command line.

> `sed` [`-e`] '`instruction`' `file`

The -e option is necessary only when you supply more than one instruction on the command line. It tells sed to interpret the next argument as an instruction. When there is a single instruction, sed is able to make that determination on its own. Let's look at some examples.

Using the sample input file, list, the following example uses the `s` command for substitution to replace "MA" with "Massachusetts."

>     $ sed 's/MA/Massachusetts/' list
>     John Daggett, 341 King Road, Plymouth Massachusetts
>     Alice Ford, 22 East Broadway, Richmond VA
>     Orville Thomas, 11345 Oak Bridge Road, Tulsa OK
>     Terry Kalkas, 402 Lans Road, Beaver Falls PA
>     Eric Adams, 20 Post Road, Sudbury Massachusetts
>     Hubert Sims, 328A Brook Road, Roanoke VA
>     Amy Wilde, 334 Bayshore Pkwy, Mountain View CA
>     Sal Carpenter, 73 6th Street, Boston Massachusetts

Three lines are affected by the instruction but all lines are displayed.

Enclosing the instruction in single quotes is not required in all cases but you should get in the habit of always doing it. The enclosing single quotes prevent the shell from interpreting special characters or spaces found in the editing instruction. (The shell uses spaces to determine individual arguments submitted to a program; characters that are special to the shell are expanded before the command is invoked.)

For instance, the first example could have been entered without them but in the next example they are required, since the substitution command contains spaces:

>     $ sed 's/ MA/, Massachusetts/' list
>     John Daggett, 341 King Road, Plymouth, Massachusetts
>     Alice Ford, 22 East Broadway, Richmond VA
>     Orville Thomas, 11345 Oak Bridge Road, Tulsa OK
>     Terry Kalkas, 402 Lans Road, Beaver Falls PA
>     Eric Adams, 20 Post Road, Sudbury, Massachusetts
>     Hubert Sims, 328A Brook Road, Roanoke VA
>     Amy Wilde, 334 Bayshore Pkwy, Mountain View CA
>     Sal Carpenter, 73 6th Street, Boston, Massachusetts

In order to place a comma between the city and state, the instruction replaced the space before the two-letter abbreviation with a comma and a space.

There are three ways to specify multiple instructions on the command line:

1. Separate instructions with a semicolon.

   >     sed 's/ MA/, Massachusetts/; s/ PA/, Pennsylvania/' list

2. Precede each instruction by -e.

   >     sed -e 's/ MA/, Massachusetts/' -e 's/ PA/, Pennsylvania/' list

3. Use the multiline entry capability of the Bourne shell.[1] Press RETURN after entering a single quote and a secondary prompt (>) will be displayed for multiline input.

   > [1] These days there are many shells that are compatible with the Bourne shell, and work as described here: `ksh`, `bash`, `pdksh`, and `zsh`, to name a few.

   >     $ sed '
   >     > s/ MA/, Massachusetts/
   >     > s/ PA/, Pennsylvania/
   >     > s/ CA/, California/' list
   >     John Daggett, 341 King Road, Plymouth, Massachusetts
   >     Alice Ford, 22 East Broadway, Richmond VA
   >     Orville Thomas, 11345 Oak Bridge Road, Tulsa OK
   >     Terry Kalkas, 402 Lans Road, Beaver Falls, Pennsylvania
   >     Eric Adams, 20 Post Road, Sudbury, Massachusetts
   >     Hubert Sims, 328A Brook Road, Roanoke VA
   >     Amy Wilde, 334 Bayshore Pkwy, Mountain View, California
   >     Sal Carpenter, 73 6th Street, Boston, Massachusetts

   This technique will not work in the C shell. Instead, use semicolons at the end of each instruction, and you can enter commands over multiple lines by ending each line with a backslash. (Or, you could temporarily go into the Bourne shell by entering `sh` and then type the command.)

In the example above, changes were made to five lines and, of course, all lines were displayed. Remember that nothing has changed in the input file.

#### 2.3.1.1 Command garbled

The syntax of a sed command can be detailed, and it's easy to make a mistake or omit a required element. Notice what happens when incomplete syntax is entered:

>     $ sed -e 's/MA/Massachusetts' list
>     sed: command garbled: s/MA/Massachusetts

Sed will usually display any line that it cannot execute, but it does not tell you what is wrong with the command.[2] In this instance, a slash, which marks the search and replacement portions of the command, is missing at the end of the substitute command.

> [2] Some vendors seem to have improved things. For instance, on SunOS 4.1.x, sed reports "sed: Ending delimiter missing on substitution: s/MA/Massachusetts".

GNU sed is more helpful:

>     $ gsed -e 's/MA/Massachusetts' list
>     gsed: Unterminated `s' command

### 2.3.2 Script Files

It is not practical to enter longer editing scripts on the command line. That is why it is usually best to create a script file that contains the editing instructions. The editing script is simply a list of sed commands that are executed in the order in which they appear. This form, using the -f option, requires that you specify the name of the script file on the command line.

> `sed` `-f` `scriptfile file`

All the editing commands that we want executed are placed in a file. We follow a convention of creating temporary script files named sedscr.

>     $ cat sedscr
>     s/ MA/, Massachusetts/
>     s/ PA/, Pennsylvania/
>     s/ CA/, California/
>     s/ VA/, Virginia/
>     s/ OK/, Oklahoma/

The following command reads all of the substitution commands in sedscr and applies them to each line in the input file list:

>     $ sed -f sedscr list
>     John Daggett, 341 King Road, Plymouth, Massachusetts
>     Alice Ford, 22 East Broadway, Richmond, Virginia
>     Orville Thomas, 11345 Oak Bridge Road, Tulsa, Oklahoma
>     Terry Kalkas, 402 Lans Road, Beaver Falls, Pennsylvania
>     Eric Adams, 20 Post Road, Sudbury, Massachusetts
>     Hubert Sims, 328A Brook Road, Roanoke, Virginia
>     Amy Wilde, 334 Bayshore Pkwy, Mountain View, California
>     Sal Carpenter, 73 6th Street, Boston, Massachusetts

Once again, the result is ephemeral, displayed on the screen. No change is made to the input file.

If a sed script can be used again, you should rename the script and save it. Scripts of proven value can be maintained in a personal or system-wide library.

#### 2.3.2.1 Saving output

Unless you are redirecting the output of sed to another program, you will want to capture the output in a file. This is done by specifying one of the shell's I/O redirection symbols followed by the name of a file:

>     $ sed -f sedscr list > newlist

Do not redirect the output to the file you are editing or you will clobber it. (The ">" redirection operator truncates the file before the shell does anything else.) If you want the output file to replace the input file, you can do that as a separate step, using the `mv` command. But first make very sure your editing script has worked properly!

In Chapter 4, Writing sed Scripts, we will look at a shell script named `runsed` that automates the process of creating a temporary file and using `mv` to overwrite the original file.

#### 2.3.2.2 Suppressing automatic display of input lines

The default operation of sed is to output every input line. The -n option suppresses the automatic output. When specifying this option, each instruction intended to produce output must contain a print command, `p`. Look at the following example.

>     $ sed -n -e 's/MA/Massachusetts/p' list
>     John Daggett, 341 King Road, Plymouth Massachusetts
>     Eric Adams, 20 Post Road, Sudbury Massachusetts
>     Sal Carpenter, 73 6th Street, Boston Massachusetts

Compare this output to the first example in this section. Here, only the lines that were affected by the command were printed.

#### 2.3.2.3 Mixing options (POSIX)

You can build up a script by combining both the -e and -f options on the command line. The script is the combination of all the commands in the order given. This appears to be supported in UNIX versions of sed, but this feature is not clearly documented in the manpage. The POSIX standard explicitly mandates this behavior.

#### 2.3.2.4 Summary of options

Table 2.1 summarizes the sed command-line options.

Table 2.1: Command-Line Options for sed
Option | Description
--- | ---
-e | Editing instruction follows.
-f | Filename of script follows.
-n | Suppress automatic output of input lines.

2.2 Command-Line Syntax |  | 2.4 Using awk