Read
SettingsMount machine

Module 10 · Putting it all together

A troubleshooting checklist

A calm, repeatable method for when something fails: read the message, check the exit status, the command, the path, the permissions, the resources and the script itself, one step at a time.

Lesson 99 of 10035 min

What you will learn

  • Map the most common error messages to their usual causes.
  • Work through a fixed checklist instead of guessing.
  • Debug scripts with `sh -n` and `sh -x`, and find and stop a runaway process.

Things will break: a script that worked yesterday, a command that says not found, a machine that suddenly feels slow. Experienced administrators are not people who never see errors; they are people who have a method. This lesson collects the tools of the previous ninety-eight lessons into a checklist you can follow when you have no idea where to start. Then the three exercises let you practise it on real, broken things.

The checklist

  1. Read the whole message. It usually names the program that complained, the file and the reason. cat: can't open 'notes.txt': No such file or directory already tells you who, what and why.
  2. Check the exit status with echo $? right after the failure: 1 is a general error, 2 often a usage mistake, 126 found but not executable, 127 not found.
  3. Is it the command you think? type name, which name, echo $PATH. An alias, a typo or a missing PATH entry explains most not found errors.
  4. Does the path exist, exactly? ls -l path, then each parent with ls -ld. Watch for relative paths run from the wrong directory (pwd), uppercase letters, and spaces without quotes.
  5. Permissions and owner. ls -l and id. A script needs x; a directory needs x to be entered. Root ignores most read and write bits, but never the execute bit.
  6. Resources. df -h (full disk), du -s (who is filling it), free (memory), uptime (load).
  7. Processes. ps and top to find what is running or stuck, kill PID to stop it, kill -9 only as a last resort.
  8. Kernel messages. dmesg | tail for hardware, drivers, mounts and out-of-memory kills.
  9. Change one thing at a time, keep a copy first (cp f f.bak), and use diff to see what you changed.

Common messages and where they point

MessageUsual causeFirst command
not found (127)Typo, not in PATH, missing ./, or a shebang naming a missing interpretertype cmd, head -1 script
Permission denied (126)No execute bit, or a non-root user without rightsls -l file
No such file or directoryWrong path or wrong current directorypwd; ls -l path
No space left on deviceFull filesystemdf -h, du -s /dir/*
syntax error: unexpected …Missing fi, done, ; or quote in a scriptsh -n script

Debugging scripts

sh -n script.sh reads the script without running it and reports syntax errors with a line number. sh -x script.sh runs it and prints every command, after expansion, with a + in front, so you see the real values of your variables. A variable that expands to nothing is a typical clue: here $dri should have been $dir, so ls listed the current directory instead.

~% sh -x lab/l99/count.sh
+ dir=/root/lab/l99/logs
+ ls
+ wc -l
+ count=2
+ echo 2
~% grep -n dri lab/l99/count.sh
3:count=$(ls $dri | wc -l)

Commands in this lesson

CommandWhat it does
echo $?Exit status of the last command.
type cmdWhat the shell will run for `cmd`.
ls -ld /path /path/subCheck each level of a path.
df -h; freeDisk and memory at a glance.
sh -n scriptSyntax check without running.
sh -x scriptTrace every command as it runs.
ps | grep '[n]ame'Find a process by name.
dmesg | tailLatest kernel messages.

Quiz

  1. A script starts with `#!/bin/bash` and running it gives `not found`, although the file exists and is executable. Why?

    • The interpreter named in the shebang does not exist on this system.
    • The script is too long.
    • The disk is full.

  2. Which command shows each line of a script, with variables expanded, as it runs?

    • `sh -n script`
    • `sh -x script`
    • `cat -n script`

  3. Writing a file fails with `No space left on device`. First command?

    • `df -h`
    • `chmod 777`
    • `reboot`

  4. Why does `ps | grep '[h]og'` not show the grep line itself?

    • grep hides itself automatically.
    • The grep's own command line contains `[h]og`, which the pattern `hog` does not match.
    • ps never lists grep.

  5. What is the best habit before editing a config file to fix a problem?

    • Make a copy (`cp f f.bak`) and change one thing at a time
    • Change everything that looks suspicious at once
    • Delete it and start over

Practice

  1. `/root/lab/l99/backup.sh` should create `/root/lab/l99/backup.tar`, but `./backup.sh` fails. There are **two** problems (hint: permissions and the first line). Fix both so that running `/root/lab/l99/backup.sh` directly works.

  2. `/root/lab/l99/count.sh` should write the number of files in `/root/lab/l99/logs` (there are 4) into `/root/lab/l99/count.txt`, but it writes a wrong number. Trace it with `sh -x`, find the bug and fix the script.

  3. Something is running a script called `hog.sh` in an endless loop. Find its process with `ps` and stop it with `kill`.

Open this lesson in the app to do the tasks in a real Linux machine and have them checked.