sed: stream editing basics
Editing text without opening an editor: substitutions with s///, selecting lines by number or pattern, deleting lines, and changing files in place with -i.
What you will learn
- Replace text with `s/old/new/` and `s/old/new/g`.
- Address lines by number, range, pattern and `
TempMV (TempVM) — Temporary Virtual Machines in your browser · Máquinas virtuales temporales , and use `p` and `d`. - Edit a file in place with `-i`, safely.
sed is a stream editor. It reads its input line by line, applies a small script to each line, and prints the result. The original file is not touched unless you ask, which makes sed safe to experiment with: run it, look at the output, adjust, and only when you are happy write the result somewhere. Most of what people do with sed comes down to one command, s for substitute, and two more, p and d.
Substitution: s/old/new/
sed 's/red/blue/' file replaces the first red on each line with blue. Add the g flag (*global*) to replace every occurrence on the line: s/red/blue/g. The old part is a regular expression, exactly as in lessons 33 and 34, so s/[0-9]*// removes leading digits and s/^/> / puts > at the start of every line. In the new part, & stands for whatever was matched: s/[0-9][0-9]*/(&)/ wraps the first number in parentheses.
The slash is only a convention. Any character after s becomes the delimiter, which saves a forest of backslashes when the text itself contains slashes: sed 's#/usr/local#/opt#' is far easier to read than the escaped version.
~% cd /root/lab/l39
l39% cat colors.txt
color=red
color=red red
l39% sed 's/red/blue/' colors.txt
color=blue
color=blue red
l39% sed 's/red/blue/g' colors.txt
color=blue
color=blue blue
l39% sed -n '2p' colors.txt
color=red redChoosing lines: addresses, p and d
Any sed command can be preceded by an address saying which lines it applies to. Without one it applies to all lines. d deletes the selected lines from the output; p prints them, and is almost always used with -n, which turns off the automatic printing of every line so that you see only what p prints. sed -n '5,8p' is the quickest way to view lines 5 to 8, and sed '/^#/d' hides comments.
| Address | Selects | Example |
|---|---|---|
N | line N | sed '1d' drops a header |
N,M | lines N to M | sed -n '5,8p' |
$ | the last line | sed -n '$p' |
/regex/ | lines matching regex | sed '/^$/d' drops empty lines |
/regex/s/a/b/ | substitute only on matching lines | sed '/^port/s/80/8080/' |
Several commands can be given with -e each, or separated by ; in one script: sed -e '/^#/d' -e '/^$/d' and sed '/^#/d;/^$/d' both strip comments and blank lines. -E switches on extended regular expressions, with groups you can reuse as \1: sed -E 's/(.*)=(.*)/\2=\1/' swaps both sides of an =.
Changing the file: -i
To save the result you can redirect it to a new file: sed 's/a/b/' in.txt > out.txt. Never redirect onto the same file you are reading; > empties it before sed reads a single line. For editing in place, sed -i 's/a/b/' file rewrites the file itself, and sed -i.bak 's/a/b/' file keeps the original as file.bak. A good routine is: run without -i, check the output, then press the up arrow and add -i.
Commands in this lesson
| Command | What it does |
|---|---|
sed 's/old/new/' FILE | Replace the first match on each line. |
sed 's/old/new/g' FILE | Replace every match. |
sed -n '5,8p' FILE | Print only lines 5 to 8. |
sed '/^#/d' FILE | Delete lines matching a regex. |
sed -e CMD1 -e CMD2 | Several commands (or `CMD1;CMD2`). |
sed -i 's/a/b/g' FILE | Edit the file in place (`-i.bak` keeps a backup). |
sed -E 's/(x)(y)/\2\1/' | Extended regex with groups. |
Quiz
A line reads `red red red`. What does `sed 's/red/blue/'` turn it into?
- `blue blue blue`
- `blue red red`
- `red red blue`
After `sed 's/a/b/g' notes.txt`, what has happened to `notes.txt`?
- Nothing; the result was only printed
- It was edited
- It was backed up and edited
Why is `sed -n '3p'` written with `-n`?
- `-n` means "line number"
- Without it every line is printed anyway, and line 3 twice
- It is required for `p` to work at all
Which command removes empty lines?
- `sed '/^$/d' file`
- `sed 's/ //g' file`
- `sed -n '/^$/p' file`
Why is `sed 's/a/b/' f.txt > f.txt` a mistake?
- `sed` refuses to read a file it writes
- The shell empties `f.txt` before `sed` reads it, so the file ends up empty
- It is fine, it is the same as `-i`
Practice
/root/lab/l39/links.txt still uses `http:`. Edit the file **in place** so that every `http:` becomes `https:`, including when a line contains it twice.
Write into /root/lab/l39/clean.conf a version of /root/lab/l39/app.conf without its comment lines (starting with `#`) and without its empty lines. Leave app.conf unchanged.
Save lines 3 to 5 of /root/lab/l39/poem.txt, and nothing else, into /root/lab/l39/middle.txt using `sed`.
Open this lesson in the app to do the tasks in a real Linux machine and have them checked.