awk: columns and simple reports

grep picks lines and sed rewrites them; awk understands that lines often have columns. It reads each line, splits it into fields on runs of spaces or tabs, and names them $1, $2, and so on; $0 is the whole line, NF is the number of fields and $NF the last one. An awk program is a list of pattern { action } pairs: for every line that matches the pattern, the action runs. Leave out the pattern and the action runs for every line; leave out the action and matching lines are printed.

~% cd /root/lab/l40
l40% cat sales.txt
ana madrid 1200
luis sevilla 950
maria bilbao 2100
pepe madrid 600
l40% awk '{print $1, $3}' sales.txt
ana 1200
luis 950
maria 2100
pepe 600
l40% awk '$3 > 1000' sales.txt
ana madrid 1200
maria bilbao 2100
l40% awk '$2 == "madrid" {print $1}' sales.txt
ana
pepe
l40% awk '{sum += $3} END {print "total:", sum}' sales.txt
total: 4850

Fields and separators

print $1, $3 prints two fields separated by a space (the comma inserts the output separator; without it the values are glued together). For files that use another separator, -F sets it: awk -F: '{print $1}' /etc/passwd lists user names, and -F, reads simple CSV. Always wrap the program in single quotes, because $1 inside double quotes would be expanded by the shell before awk ever saw it.

Patterns, variables, BEGIN and END

Patterns can be regular expressions, /madrid/, or comparisons on fields: $3 > 1000, $2 == "madrid", $1 ~ /^m/ (field matches a regex), joined with && and ||. NR is the current line number, so NR > 1 skips a header. Variables need no declaration and start at zero, which makes totals one line long: {sum += $3} END {print sum}. BEGIN { } runs before the first line, handy for printing a heading, and END { } after the last, where totals and averages (sum/NR) belong. printf aligns columns: printf "%-8s %6d\n", $1, $3.

Module review: which tool?