[Chapter 8] 8.5 An Acronym Processor
Author: Dale Dougherty & Arnold Robbins
Date: 1998-08-03T20:59:26Z

#

| Chapter 8 Conditionals, Loops, and Arrays |

## 8.5 An Acronym Processor

Now let's look at a program that scans a file for acronyms. Each acronym is replaced with a full text description, and the acronym in parentheses. If a line refers to "BASIC," we'd like to replace it with the description "Beginner's All-Purpose Symbolic Instruction Code" and put the acronym in parentheses afterwards. (This is probably not a useful program in and of itself, but the techniques used in the program are general and have many such uses.)

We can design this program for use as a filter that prints all lines, regardless of whether a change has been made. We'll call it `awkro`.

>     awk '# awkro - expand acronyms
>     # load acronyms file into array "acro"
>     FILENAME == "acronyms" {
>     	split($0, entry, "\t")
>     	acro[entry[1]] = entry[2]
>     	next
>     }

>     # process any input line containing caps
>     /[A-Z][A-Z]+/ {
>     	# see if any field is an acronym
>     	for (i = 1; i <= NF; i++)
>     		if ( $i in acro ) {
>     			# if it matches, add description
>     			$i = acro[$i] " (" $i ")"
>     		}
>     }

>     {
>     	# print all lines
>     	print $0
>     }' acronyms  $*

Let's first see it in action. Here's a sample input file.

>     $ cat sample
>     The USGCRP is a comprehensive
>     research effort that includes applied
>     as well as basic research.
>     The NASA program Mission to Planet Earth
>     represents the principal space-based component
>     of the USGCRP and includes new initiatives
>     such as EOS and Earthprobes.

And here is the file acronyms:

>     $ cat acronyms
>     USGCRP	U.S. Global Change Research Program
>     NASA	National Aeronautic and Space Administration
>     EOS	Earth Observing System

Now we run the program on the sample file.

>     $ awkro sample
>     The U.S. Global Change Research Program (USGCRP) is a comprehensive
>     research effort that includes applied
>     as well as basic research.
>     The National Aeronautic and Space Administration (NASA) program
>     Mission to Planet Earth
>     represents the principal space-based component
>     of the U.S. Global Change Research Program (USGCRP) and includes new
>     initiatives
>     such as Earth Observing System (EOS) and Earthprobes.

We'll look at this program in two parts. The first part reads records from the acronyms file.

>     # load acronyms file into array "acro"
>     FILENAME == "acronyms" {
>     	split($0, entry, "\t")
>     	acro[entry[1]] = entry[2]
>     	next
>     }

The two fields from these records are loaded into an array using the first field as the subscript and assigning the second field to an element of the array. In other words, the acronym itself is the index to its description.

Note that we did not change the field separator, but instead used the `split()` function to create the array `entry`. This array is then used in creating an array named `acro`.

Here is the second half of the program:

>     # process any input line containing caps
>     /[A-Z][A-Z]+/ {
>     	# see if any field is an acronym
>     	for (i = 1; i <= NF; i++)
>     		if ( $i in acro ) {
>     			# if it matches, add description
>     			$i = acro[$i] " (" $i ")"
>     		}
>     }

>     {
>     	# print all lines
>     	print $0
>     }

Only lines that contain more than one consecutive capital letter are processed by the first of the two actions shown here. This action loops through each field of the record. At the heart of this section is the conditional statement that tests if the current field (`$i`) is a subscript of the array (`acro`). If the field is a subscript, we replace the original value of the field with the array element and the original value in parentheses. (Fields can be assigned new values, just like regular variables.) Note that the insertion of the description of the acronym results in lines that may be too long. See the next chapter for a discussion of the `length()` function, which can be used to determine the length of a string so you can divide it up if it is too long.

Now we're going to change the program so it makes a replacement only the first time an acronym appears. After we've found it, we don't want to search for that acronym any more. This is easy to do; we simply delete that acronym from the array.

>     if ( $i in acro ) {
>     	# if it matches, add description
>     	$i = acro[$i] " (" $i ")"
>     	# only expand the acronym once
>     	delete acro[acronym]
>     }

There are other changes that would be good to make. In running the `awkro` program, we soon discovered that it failed to match the acronym if it was followed by a punctuation mark. Our initial solution was not to handle it in awk at all. Instead, we used two sed scripts, one before processing:

>     sed 's/\([^.,;:!][^.,;:!]*\)\([.,;:!]\)/\1 @@@\2/g'

and one after:

>     sed 's/ @@@\([.,;:!]\)/\1/g'

A sed script, run prior to invoking awk, could simply insert a space before any punctuation mark, causing it to be interpreted as a separate field. A string of garbage characters (`@@@`) was also added so we'd be able to easily identify and restore the punctuation mark. (The complicated expression used in the first sed command makes sure that we catch the case of more than one punctuation mark on a line.)

This kind of solution, using another tool in the UNIX toolbox, demonstrates that not everything needs to be done as an awk procedure. Awk is all the more valuable because it is situated in the UNIX environment.

However, with POSIX awk, we can implement a different solution, one that uses a regular expression to match the acronym. Such a solution can be implemented with the `match()` and `sub()` functions described in the next chapter.

### 8.5.1 Multidimensional Arrays

Awk supports linear arrays in which the index to each element of the array is a single subscript. If you imagine a linear array as a row of numbers, a two-dimensional array represents rows and columns of numbers. You might refer to the element in the second column of the third row as "array[3, 2]." Two- and three-dimensional arrays are examples of multidimensional arrays. Awk does not support multidimensional arrays but instead offers a syntax for subscripts that simulate a reference to a multidimensional array. For instance, you could write the following expression:

>     file_array[NR, i] = $i

where each field of an input record is indexed by its record number and field number. Thus, the following reference:

>     file_array[2, 4]

would produce the value of the fourth field of the second record.

This syntax does not create a multidimensional array. It is converted into a string that uniquely identifies the element in a linear array. The components of a multidimensional subscript are interpreted as individual strings ("2" and "4," for instance) and concatenated together separated by the value of the system variable `SUBSEP`. The subscript-component separator is defined as `"`\034`"` by default, an unprintable character rarely found in ASCII text. Thus, awk maintains a one-dimensional array and the subscript for our previous example would actually be `"`2\0344`"` (the concatenation of "`2`," the value of `SUBSEP`, and "`4`"). The main consequence of this simulation of multidimensional arrays is that the larger the array, the slower it is to access individual elements. However, you should time this, using your own application, with different awk implementations (see Chapter 11, A Flock of awks).

Here is a sample awk script named `bitmap.awk` that shows how to load and output the elements of a multidimensional array. This array represents a two-dimensional bitmap that is 12 characters in width and height.

>     BEGIN { FS = ","   # comma-separated fields
>     	# assign width and height of bitmap
>     	WIDTH = 12
>     	HEIGHT = 12
>     	# loop to load entire array with "O"
>     	for (i = 1; i <= WIDTH; ++i)
>     		for (j = 1; j <= HEIGHT; ++j)
>     			bitmap[i, j] = "O"
>     }
>     # read input of the form x,y.
>     {
>     	# assign "X" to that element of array
>     	bitmap[$1, $2] = "X"
>     }
>     # at end output multidimensional array
>     END {
>     	for (i = 1; i <= WIDTH; ++i){
>     		for (j = 1; j <= HEIGHT; ++j)
>     			printf("%s", bitmap[i, j] )
>     		# after each row, print newline
>     		printf("\n")
>     	}
>     }

Before any input is read, the `bitmap` array is loaded with O's. This array has 144 elements. The input to this program is a series of coordinates, one per line.

>     $ cat bitmap.test
>     1,1
>     2,2
>     3,3
>     4,4
>     5,5
>     6,6
>     7,7
>     8,8
>     9,9
>     10,10
>     11,11
>     12,12
>     1,12
>     2,11
>     3,10
>     4,9
>     5,8
>     6,7
>     7,6
>     8,5
>     9,4
>     10,3
>     11,2
>     12,1

For each coordinate, the program will put an "X" in place of an "O" as that element of the array. At the end of the script, the same kind of loop that loaded the array, now outputs it. The following example reads the input from the file bitmap.test.

>     $ awk -f bitmap.awk bitmap.test
>     XOOOOOOOOOOX
>     OXOOOOOOOOXO
>     OOXOOOOOOXOO
>     OOOXOOOOXOOO
>     OOOOXOOXOOOO
>     OOOOOXXOOOOO
>     OOOOOXXOOOOO
>     OOOOXOOXOOOO
>     OOOXOOOOXOOO
>     OOXOOOOOOXOO
>     OXOOOOOOOOXO
>     XOOOOOOOOOOX

The multidimensional array syntax is also supported in testing for array membership. The subscripts must be placed inside parentheses.

>     if ((i, j) in array)

This tests whether the subscript `i,j` (actually, `i SUBSEP j`) exists in the specified array.

Looping over a multidimensional array is the same as with one-dimensional arrays.

>     for (item in array)

You must use the `split()` function to access individual subscript components. Thus:

>     split(item, subscr, SUBSEP)

creates the array `subscr` from the subscript `item`.

Note that we needed to use the loop-within-a-loop to output the two-dimensional bitmap array in the previous example because we needed to maintain rows and columns.

8.4 Arrays |  | 8.6 System Variables That Are Arrays