cut, paste and tr

A lot of the text you meet on a Unix system is tabular: /etc/passwd has colon-separated fields, CSV exports have commas, ls -l has space-aligned columns. Three small filters cover most of what you need to do with such text before handing it to sort, uniq or grep: cut picks columns, paste joins them back, and tr rewrites characters.

cut: one column, please

cut -d SEP -f N splits each line on the single character SEP and prints field number N. Fields are numbered from 1, and the list can be a range or several numbers: -f1,3, -f2-4, -f3- (from 3 to the end). The default separator is a tab, which is why cut -f1 on a space-separated file prints whole lines: there was no tab to split on. cut -c 1-3 works on character positions instead of fields, useful for fixed-width output.

~% head -3 /etc/passwd
root:x:0:0:root:/root:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/false
bin:x:2:2:bin:/bin:/bin/false
~% cut -d: -f1 /etc/passwd | head -3
root
daemon
bin
~% cut -d: -f1,7 /etc/passwd | head -3
root:/bin/sh
daemon:/bin/false
bin:/bin/false
~% echo abcdef | cut -c2-4
bcd

paste: columns from files

paste a b prints line 1 of a, a tab, line 1 of b, then line 2 of each, and so on. -d chooses the separator: paste -d, a b makes a two-column CSV, paste -d' ' a b separates with a space. paste -s file goes the other way, joining all lines of one file into a single line, which is a neat way to turn a list into a b c.

tr: character by character

tr SET1 SET2 reads stdin and replaces every character in SET1 with the character at the same position in SET2. It only reads stdin, so use < or a pipe: tr 'a-z' 'A-Z' < file uppercases, and the named classes [:lower:] and [:upper:] do the same more portably. tr -d SET deletes the characters instead of translating them, tr -s SET squeezes repeated characters into one, and -c complements the set (everything *not* in it).

~% echo hello | tr 'a-z' 'A-Z'
HELLO
~% echo 'too    many   spaces' | tr -s ' '
too many spaces
~% echo 'ph0n3 nu3b3r' | tr -d '0-9'
phn nubr
~% echo 'a,b,c' | tr ',' '\n'
a
b
c
~% printf 'one\ntwo\n' > x; printf 'uno\ndos\n' > y
~% paste x y
one\tuno
two\tdos
~% paste -d= x y
one=uno
two=dos

tr is also how you turn one shape of text into another so the next tool can work: tr ' ' '\n' breaks a sentence into one word per line, ready for sort | uniq -c; tr -d '\r' strips the carriage returns from a file saved on Windows; tr -cs 'a-zA-Z' '\n' keeps only letters and puts each run on its own line.