Shell Scripting Complete 12: Debugging with bash -x, ShellCheck and Logging

By Ravindra Bagale, trainer · LinkedIn · Last updated: 25 October 2026

1. An everyday example: a doctor's check-up

Why do we need this?

When you feel unwell, a good doctor in Pune does not guess. She checks temperature, BP and pulse, asks what you ate, and looks at old reports. Then she finds the cause. Debugging a script is the same: check syntax, watch each step with tracing, read the logs, and let ShellCheck do a routine check-up before problems start.

What is it?

Routine check-up = shellcheck. Quick look at the tongue = bash -n. Watching the pulse live = set -x. Patient's old reports = logs with timestamps. Writing down symptoms = log levels INFO, WARN, ERROR.

How to do it (step by step)

2. Syntax check: bash -n

Why do we need this?

A missing fi or done in a deploy script should be caught before it runs on production.

What is it?

bash -n script.sh reads the script and reports syntax errors without running anything. It catches missing fi, done, esac, quotes and brackets. It does not catch wrong commands or logic, so use it as the first quick step, often in CI or a git hook.

How to do it (step by step)

1. Clean script.

bash -n deploy.sh && echo 'syntax ok'

Output:

syntax ok

What it means: Nothing ran.

2. Missing fi.

bash -n broken.sh

Output:

broken.sh: line 9: syntax error: unexpected end of file

What it means: Line points near the end.

3. Check many.

for f in scripts/*.sh; do bash -n "$f" || echo "BAD $f"; done

Output:

BAD scripts/old-import.sh

What it means: Batch check.

3. Tracing: bash -x and set -x

Why do we need this?

You see the wrong result but not why. Tracing shows each command with variables already filled in.

What is it?

bash -x script.sh prints every command with + before it, after expansion. set -x turns it on inside a script and set +x turns it off, so you can trace only the hard part. Trace goes to stderr. bash -v prints lines as read, before expansion. BASH_XTRACEFD=3 sends trace to another file descriptor.

How to do it (step by step)

1. Trace all.

bash -x greet.sh Asha

Output:

+ name=Asha
+ echo 'Hello Asha'
Hello Asha

What it means: Values shown.

2. Trace a part.

set -x
files=$(find /data -mtime +7 | wc -l)
set +x

Output:

++ find /data -mtime +7
++ wc -l
+ files=12

What it means: Only this block.

3. Trace to a file.

exec 5> /tmp/trace.log; BASH_XTRACEFD=5; set -x

What it means: Screen stays clean.

4. Trace only with DEBUG=1.

[[ ${DEBUG:-0} == 1 ]] && set -x

What it means: Run as DEBUG=1 ./job.sh.

4. Better traces with PS4

Why do we need this?

In a 500-line script, + echo done does not tell you where you are.

What is it?

PS4 is the prefix printed before each trace line. Set it to show file, line and function: export PS4='+ ${BASH_SOURCE##*/}:${LINENO}:${FUNCNAME[0]:-main}: '. Use single quotes so the values are filled in at each line. Add $(date +%T) for timing, though it slows tracing.

How to do it (step by step)

1. Line numbers.

PS4='+ ${LINENO}: ' bash -x backup.sh

Output:

+ 4: src=/data
+ 5: tar czf /backup/data.tgz /data

What it means: Line of each step.

2. File and function.

export PS4='+ ${BASH_SOURCE##*/}:${LINENO}:${FUNCNAME[0]:-main}: '; bash -x deploy.sh

Output:

+ deploy.sh:21:health_check: curl -fs http://localhost/health

What it means: Know the function.

3. With time.

PS4='+ $(date +%T) ${LINENO}: ' bash -x slow.sh

Output:

+ 10:02:01 7: rsync -a src/ dst/
+ 10:04:39 8: echo synced

What it means: rsync took 2.5 min.

5. ShellCheck: find bugs before they run

Why do we need this?

Missing quotes, wrong tests and useless cats are easy to miss in review and painful in production.

What is it?

shellcheck script.sh reads the script and warns about common bugs, each with a code like SC2086 (unquoted variable) and a wiki page explaining it. Install with dnf, apt or brew. -S warning hides style notes. A # shellcheck disable=SC2034 comment above a line silences one warning when you are sure. Run it in your editor and in CI so every change is checked.

How to do it (step by step)

1. Install.

sudo dnf install -y ShellCheck   # Ubuntu: sudo apt install -y shellcheck

Output:

Complete!

What it means: Package name differs.

2. Check.

shellcheck clean.sh

Output:

In clean.sh line 3:
rm -rf $dir/*
       ^--^ SC2086 (info): Double quote to prevent globbing and word splitting.

What it means: Real risk found.

3. Only warnings and errors.

shellcheck -S warning *.sh

Output:

In run.sh line 7:
cd /opt/app
^---------^ SC2164 (warning): Use 'cd ... || exit' in case cd fails.

What it means: Style hidden.

4. In CI.

find . -name '*.sh' -print0 | xargs -0 shellcheck -S warning

What it means: Non-zero fails the build.

6. A log function with time and level

Why do we need this?

echo 'done' tells you nothing next morning. A log line should say when, how serious, and what.

What is it?

Write one log function: printf '%(%F %T)T [%s] %s\n' -1 "$level" "$*" >&2. Add info, warn and error wrappers. Send logs to stderr so stdout stays clean for data. Tee them into a file with exec 2> >(tee -a "$LOG" >&2). Add a LOG_LEVEL variable so DEBUG lines show only when needed.

How to do it (step by step)

1. Log function.

log(){ local l=$1; shift; printf '%(%F %T)T [%s] %s\n' -1 "$l" "$*" >&2; }
log INFO 'backup started'

Output:

2026-10-25 02:00:01 [INFO] backup started

What it means: When, level, what.

2. Wrappers.

info(){ log INFO "$@"; }; warn(){ log WARN "$@"; }; err(){ log ERROR "$@"; }
warn 'disk at 82%'

Output:

2026-10-25 02:00:03 [WARN] disk at 82%

What it means: Easy to grep.

3. Debug level.

debug(){ [[ ${LOG_LEVEL:-INFO} == DEBUG ]] && log DEBUG "$@"; return 0; }
LOG_LEVEL=DEBUG; debug "files=$n"

Output:

2026-10-25 02:00:04 [DEBUG] files=12

What it means: Hidden by default.

4. Also to a file.

exec 2> >(tee -a /var/log/backup.log >&2)

What it means: All logs saved.

7. logger and journald

Why do we need this?

On servers, ops teams search logs centrally with journalctl or a log shipper, not in random files.

What is it?

logger -t mytag 'message' writes to syslog/journald with a tag. -p user.err sets priority. Read with journalctl -t mytag, add --since today or -f to follow. Scripts run by systemd already send stdout and stderr to the journal, so plain echo shows in journalctl -u name.

How to do it (step by step)

1. Write.

logger -t nightly-backup 'backup ok: 2.1G'

What it means: Goes to journal.

2. Error priority.

logger -t nightly-backup -p user.err 'backup failed: disk full'

What it means: Shows red in journalctl.

3. Read.

journalctl -t nightly-backup --since today

Output:

Oct 25 02:00:05 ip-10-0-1-5 nightly-backup[811]: backup ok: 2.1G

What it means: Tagged lines.

4. Errors only.

journalctl -t nightly-backup -p err

Output:

Oct 25 03:00:02 ip-10-0-1-5 nightly-backup[912]: backup failed: disk full

What it means: Filter by priority.

8. Debugging step by step

Why do we need this?

Random changes make things worse. A fixed method finds the cause faster.

What is it?

1. Read the exact message and line. 2. bash -n and shellcheck. 3. Run with bash -x and a PS4 with line numbers. 4. Print suspicious values as printf '[%s]\n' "$v" and od -c to see hidden spaces or \r. 5. Check the environment: cron and systemd have a small PATH and no profile. 6. Make the smallest example that fails. 7. Change one thing, test, repeat.

How to do it (step by step)

1. Hidden characters.

printf '%s' "$host" | od -c

Output:

0000000   w   e   b   1  \r

What it means: A \r from Windows.

2. Env difference.

env -i PATH=/usr/bin:/bin bash -c 'command -v aws || echo missing'

Output:

missing

What it means: Cron-like PATH.

3. Stop at a point.

read -rp 'paused, press Enter ' _

Output:

paused, press Enter

What it means: Inspect files before going on.

4. Who called?

echo "called from ${FUNCNAME[1]} line ${BASH_LINENO[0]}"

Output:

called from main line 42

What it means: Call stack info.

9. Real examples from many industries

Why do we need this?

The same Linux skill shows up in every kind of company. Seeing it in many places helps you remember it.

What is it?

Each row is a made-up team. It shows who uses this and how.

How to do it (step by step)

10. Common errors and how to fix them

Why do we need this?

Everyone hits these errors. Knowing the cause saves hours.

What is it?

Read the exact message first. Then match it to a row below.

How to do it (step by step)

11. Troubleshooting step by step

Why do we need this?

Use the same order every time so you do not miss a simple cause.

What is it?

Go top to bottom. Stop at the first step that shows the problem.

How to do it (step by step)

1. Syntax.

bash -n s.sh

2. Lint.

shellcheck s.sh

3. Trace with lines.

PS4='+ ${LINENO}: ' bash -x s.sh

4. Hidden chars.

cat -A s.sh | head

5. Env.

env | sort > /tmp/env.txt

6. Logs.

journalctl -t mytag -n 50

12. Lab 1: Fix a broken script with ShellCheck and bash -x

Why do we need this?

Take a buggy cleanup script, find the problems with bash -n, ShellCheck and tracing, and fix them one by one.

What is it?

A full hands-on lab. Run every step on a test Linux machine (an EC2 instance, a VM, or WSL).

How to do it (step by step)

1. Buggy script.

cat > ~/clean.sh <<'X'
#!/bin/bash
dir=$1
cd $dir
for f in $(ls *.tmp); do
  rm $f
done
echo "removed from $dir"
X

2. Syntax ok?

bash -n ~/clean.sh && echo ok

Output:

ok

3. Lint.

shellcheck ~/clean.sh | grep -o 'SC[0-9]*' | sort -u

Output:

SC2045
SC2086
SC2164

4. Fixed version.

cat > ~/clean.sh <<'X'
#!/usr/bin/env bash
set -euo pipefail
dir=${1:?usage: clean.sh DIR}
cd -- "$dir" || exit 1
for f in ./*.tmp; do [[ -e $f ]] || continue; rm -- "$f"; done
echo "removed from $dir"
X
shellcheck ~/clean.sh && echo clean

Output:

clean

5. Trace a test.

mkdir -p /tmp/t && touch '/tmp/t/a b.tmp' && PS4='+ ${LINENO}: ' bash -x ~/clean.sh /tmp/t

Output:

+ 3: dir=/tmp/t
+ 4: cd -- /tmp/t
+ 5: for f in ./*.tmp
+ 5: [[ -e ./a b.tmp ]]
+ 5: rm -- './a b.tmp'
+ 6: echo 'removed from /tmp/t'
removed from /tmp/t

13. Lab 2: Add logging to a script

Why do we need this?

Add a log function with levels, write to a file and journald, and turn on debug only with LOG_LEVEL=DEBUG.

What is it?

A full hands-on lab. Run every step on a test Linux machine (an EC2 instance, a VM, or WSL).

How to do it (step by step)

1. Script.

cat > ~/job.sh <<'X'
#!/usr/bin/env bash
LOG=/tmp/job.log
log(){ local l=$1; shift; printf '%(%F %T)T [%s] %s\n' -1 "$l" "$*" | tee -a "$LOG" >&2; logger -t job -- "[$l] $*"; }
debug(){ [[ ${LOG_LEVEL:-INFO} == DEBUG ]] && log DEBUG "$@"; return 0; }
log INFO start
n=$(ls /etc | wc -l); debug "etc entries=$n"
log INFO done
X
chmod +x ~/job.sh

2. Normal.

~/job.sh

Output:

2026-10-25 11:00:01 [INFO] start
2026-10-25 11:00:01 [INFO] done

3. Debug.

LOG_LEVEL=DEBUG ~/job.sh

Output:

2026-10-25 11:00:09 [INFO] start
2026-10-25 11:00:09 [DEBUG] etc entries=212
2026-10-25 11:00:09 [INFO] done

4. File.

wc -l < /tmp/job.log

Output:

5

5. Journal.

journalctl -t job -n 3 --no-pager

Output:

Oct 25 11:00:09 host job[2201]: [INFO] start
Oct 25 11:00:09 host job[2203]: [DEBUG] etc entries=212
Oct 25 11:00:09 host job[2205]: [INFO] done

14. Frequently asked questions

15. Interview questions and answers

Beginner level

Mid level level

Senior level

16. Related chapters

• Linux Ops Refresh for DevOps
• Bash Scripting for Automation
• Linux grep and find Commands with Examples
• How to Set Up Passwordless SSH Between Linux Servers

Revision, comparison and interview cards

Revise this lesson in a few minutes. Save the cards on your phone or download them as PNG files. The QR code on each card opens this lesson.

More cards: Ravi Sir's Cards.