ehmatthes.com

code, politics, and life

A sed one-liner, from my father

Feb 2, 2012

I have known about sed and awk for a long time now, but I haven’t used either of them much at all. I recently got my hands on a copy of sed & awk by Dougherty and Robbins, so the next time I needed to replace some text, I decided to learn how to do it using sed. This simple one-liner replaces every occurrence of ‘cat’ in animals.txt with the word ‘dog’:

sed -i 's/cat/dog/g' animals/animals.txt

The -i argument causes sed to write the changes to the file directly, rather than outputting the new text to the screen. The s flag at the beginning of the regular expression tells it to substitute ‘dog’ for ‘cat’, and the g flag at the end makes the command act globally on the whole file, rather than just finding the first occurrence of ‘cat’. To make the command act recursively on every file in the directory, we can pipe the output of grep to sed:

grep -rl cat animals/ | xargs sed -i 's/cat/dog/g'

The -r flag makes grep act recursively on the directory, and the -l flag causes grep to output the names of the files that contain the word cat. The xargs command passes these filenames to the sed command. We can exclude some directories, such as directories associated with version control, using the –exclude-dir option for grep:

grep -rl --exclude-dir='.git' cat animals/ | xargs sed -i 's/cat/dog/g'

It’s also a good idea to run the grep command by itself first to make sure you are only affecting the files you want to change. These simple one-liners are good examples of why many of us will always use the command line for some tasks.

My father's copy of "sed & awk".

There’s a bit more to the story of how I came to use sed myself, though. My copy of sed & awk came from my father, who learned to program using punch cards in the 1950s and 1960s. He worked as a software engineer for Digital Equipment Corporation in the 1970s and 1980s when I was growing up, and the first terminal he brought home from work had a printer for output instead of a monitor. I learned about programming from my father, and I enjoyed his stories about the people he worked with and the culture of software development as much as I enjoyed our technical conversations.

When my father passed away last spring, it was gut-wrenching for all of us. He died of a sudden heart attack, and I flew home to be with my mom and my brother. The week at home was a blur of heartache, trying to focus on good memories, and helping each other through everything that needed to be taken care of. Whenever I had the chance I went upstairs to his office, where I could be surrounded by his work and his books. My father had built bookshelves around two walls of his office, and one whole wall was a closet full of bookshelves. There were probably sixty or eighty feet of shelving in all, and I think he had the entire O’Reilly catalog on those shelves. I always loved visiting my parents, and finding books I was interested in but had not bought yet on my father’s bookshelf. I don’t know if every programmer owns that many books, but I sure got the impression it was a normal thing from my father.

At first I didn’t want to touch any of the books, and move them from where he had last placed them on the shelf. But I slowly started to pick a few out and look through them. I began to find his bookmarks, and his notes jotted in some of the margins. I saw that he highlighted the copyright date in most books. He stamped his name on the bottom of each book, and many books had gold seals which said, “This book belongs to Fred Matthes.” I found his receipts in many of the books; bookstore receipts in the older books, and Amazon invoices in the newer books.

My father bought his copy of Kernighan and Ritchie's "The C Programming Language" on November 16, 1985.

Over the next few days I got more comfortable looking through my father’s work as well. It started to feel less like prying, and more like hearing his voice one more time. It was sad, though, knowing he’d never finish the projects he was working on. Toward the end of my time at home, I realized it would be a good thing to go through his books. I wanted the good books to be kept in use, and I didn’t want the outdated books to end up in a library where someone would be tempted to learn HTML practices that were current in 1998. I took my time picking the books I wanted to keep. I found a couple cardboard boxes and filled them with some MySQL references, some Python books, some JavaScript texts, and others. I kept a few books that I would probably never read, but which made me think of my father, such as the copy of K&R he bought in 1985.

I cleared a shelf and made space to leave the books that I didn’t want, but which might be useful to someone else. My mom could let people look through them when they came to visit. Then I started carrying crates of old books down the stairs to the garage. I stacked them in a long row in the middle of the garage, where some of my mom’s friends could easily load them into a truck and take them to the recycling center. I weighed one crate before carrying it downstairs, just to get a sense of how many books I was actually moving. I thought it would weigh about sixty pounds, but the crate weighed ninety pounds. By the end, I had moved almost a thousand pounds of books out to the garage. It was very satisfying to finish such a concrete, physical task for my mom before leaving for Alaska again. It was also very grounding to put my hands on every book my father had in his office.

I had a few conversations with my mom and our neighbor to make sure the books would be recycled and not donated to a library, letting them know that my father would not want people trying to learn from the truly outdated books. I had kept my eye out through it all, though, for one or two books to leave behind. I wanted one programming book to leave in the house, in with my Mom’s books, that would keep my father’s technical spirit alive. I knew it as soon as I picked it up off the shelf: Donald Knuth’s three-volume series The Art of Computer Programming. Old-school like my father, but timeless. I sleep better these days knowing those three books are still sitting on a bookshelf in my mom’s house. I look forward to thumbing through them late at night the next time I visit my mom, probably while hanging out in my father’s old office space.

My father’s passing was rough on all of us, but his passing was also one last lesson for me on technology and software development. I always valued technical conversations with my father because he had a long-term, big-picture perspective on the field. Looking at his unfinished work made me realize how little the programs we are working on matter if they only live on our own hard drives. They are fun, just like the little programs we write for things like Project Euler. But the projects we are working on because we think they will solve problems for end users are useless if they never leave our own computers. My father’s final lesson was a reminder to release what we build, and not wait for it to be perfect. Thanks dad, I miss you.