Linux cut, sort, uniq, tr, paste and diff Commands Explained with Examples

By Ravindra Bagale, trainer · LinkedIn · Last updated: 25 October 2026

1. An everyday example: a class result sheet

Why do we need this?

Mr. Kulkarni has a result list: roll number, name, city and marks. He wants only the names (pick a column), toppers first (sort by marks), how many students from each city (count duplicates), all names in capitals for the certificate (change characters), the old list next to the new list, and what changed after re-checking (compare).

What is it?

Pick a column = cut. Toppers first = sort -k4 -nr. Count per city = sort | uniq -c. Capitals = tr a-z A-Z. Side by side = paste. What changed = diff.

How to do it (step by step)

2. cut: pick columns or characters

Why do we need this?

CSV files, /etc/passwd and many logs have fields split by a character. Often you need just one field.

What is it?

cut -d',' -f2 picks field 2 split by commas. -f1,3 picks fields 1 and 3, -f2- from 2 to the end. -c1-5 picks characters 1 to 5. --complement gives everything except the chosen fields. The default delimiter is TAB. cut cannot treat many spaces as one; for space-aligned output use awk or tr -s ' ' first.

How to do it (step by step)

1. Usernames from /etc/passwd.

cut -d: -f1 /etc/passwd | head -3

Output:

root
bin
daemon

What it means: Fields are split by colons.

2. User and shell.

cut -d: -f1,7 /etc/passwd | tail -2

Output:

ravi:/bin/bash
nginx:/sbin/nologin

What it means: Output keeps the same delimiter.

3. City column from a CSV.

cut -d, -f2 sales.csv | tail -n +2 | head -3

Output:

Pune
Nashik
Pune

What it means: tail -n +2 drops the header.

4. First 10 characters (the date) of each log line.

cut -c1-10 app.log | head -2

Output:

2026-10-25
2026-10-25

What it means: -c works by character position.

5. Everything except the password column.

cut -d, -f3 --complement users.csv | head -2

Output:

id,name,city
1,Aditi,Pune

What it means: --complement inverts the pick (GNU cut).

6. Space-aligned output: squeeze first.

ps aux | tr -s ' ' | cut -d' ' -f2,11 | head -2

Output:

PID COMMAND
1 /sbin/init

What it means: tr -s turns many spaces into one so cut can split.

3. sort: put lines in order

Why do we need this?

Sorted data is easy to read, and tools like uniq, comm and join need sorted input.

What is it?

sort orders lines as text by default. -n numeric, -r reverse, -h human sizes (2K, 5M, 1G), -k2 by field 2, -t, sets the field separator, -u unique, -f ignore case, -V version order (v1.2 < v1.10), -o file writes output (safe even onto the same file), -c checks if sorted. -k2,2 means only field 2; -k2 means from field 2 to the end of the line.

How to do it (step by step)

1. Text sort vs number sort.

printf '10\n9\n100\n' | sort; printf '10\n9\n100\n' | sort -n

Output:

10
100
9
9
10
100

What it means: Without -n, 100 comes before 9 because '1' < '9'.

2. Students by marks, highest first.

sort -t, -k4,4nr marks.csv | head -3

Output:

7,Sneha,Pune,98
3,Imran,Mumbai,95
9,Anna,Goa,91

What it means: Field 4, numeric, reverse.

3. Biggest folders.

du -sh /var/log/* | sort -rh | head -3

Output:

1.2G /var/log/journal
300M /var/log/nginx
12M /var/log/dnf.log

What it means: -h understands K, M and G.

4. Version order.

printf 'v1.10\nv1.2\nv1.9\n' | sort -V

Output:

v1.2
v1.9
v1.10

What it means: -V sorts like humans read versions.

5. Unique sorted cities.

cut -d, -f3 marks.csv | tail -n +2 | sort -u

Output:

Goa
Mumbai
Pune

What it means: Same as sort | uniq.

6. Sort a file in place safely.

sort -o names.txt names.txt

Output:

(no output)

What it means: sort names.txt > names.txt would empty the file; -o is safe.

4. uniq: remove and count duplicates

Why do we need this?

Who logged in most? Which error repeats? Counting repeats is the heart of log analysis.

What is it?

uniq joins neighbouring equal lines. So you almost always sort first. -c adds a count, -d shows only repeated lines, -u only lines that appear once, -i ignores case, -f1 skips the first field. The famous pipeline is sort | uniq -c | sort -nr | head.

How to do it (step by step)

1. Why sort first.

printf 'a\nb\na\n' | uniq; printf 'a\nb\na\n' | sort | uniq

Output:

a
b
a
a
b

What it means: Without sort, the two a's are not neighbours.

2. Count students per city.

cut -d, -f3 marks.csv | tail -n +2 | sort | uniq -c

Output:

      2 Goa
      4 Mumbai
      6 Pune

What it means: The number is the count.

3. Top 3 IPs in an access log.

cut -d' ' -f1 access.log | sort | uniq -c | sort -nr | head -3

Output:

    812 10.0.0.7
    430 10.0.0.2
     95 49.36.12.4

What it means: 10.0.0.7 sent 812 requests.

4. Only duplicate emails.

sort emails.txt | uniq -d

Output:

aditi@example.com

What it means: These appear more than once.

5. Only unique lines.

sort emails.txt | uniq -u | wc -l

Output:

148

What it means: 148 emails appear only once.

6. Count error types.

grep -o 'ERROR [A-Za-z]*' app.log | sort | uniq -c | sort -nr

Output:

     41 ERROR timeout
      9 ERROR auth

What it means: grep -o prints only the match (next chapter).

5. tr: translate, squeeze and delete characters

Why do we need this?

You need uppercase names, to remove Windows line ends, turn commas into new lines or squeeze many spaces.

What is it?

tr reads only stdin (no file name argument) and works on single characters. tr 'a-z' 'A-Z' changes case, tr -d '\r' deletes characters, tr -s ' ' squeezes repeats into one, tr ',' '\n' changes commas to new lines, tr -cd '[:digit:]' keeps only digits (-c = complement). Classes: [:upper:] [:lower:] [:digit:] [:space:] [:alnum:] [:punct:].

How to do it (step by step)

1. Uppercase.

echo 'pune city' | tr 'a-z' 'A-Z'

Output:

PUNE CITY

What it means: Each letter maps to its capital.

2. Remove Windows line ends.

tr -d '\r' < win.sh > unix.sh && cat -A unix.sh | head -1

Output:

#!/bin/bash$

What it means: No more ^M. Same job as dos2unix.

3. Squeeze spaces.

echo 'a    b   c' | tr -s ' '

Output:

a b c

What it means: Many spaces become one.

4. Split a comma list into lines.

echo 'web1,web2,db1' | tr ',' '\n'

Output:

web1
web2
db1

What it means: Now you can sort or loop over it.

5. Keep only digits from a phone number.

echo '+91 (98220) 12-345' | tr -cd '[:digit:]'

Output:

919822012345

What it means: -c means 'not these', -d deletes them.

6. Show PATH one per line.

echo "$PATH" | tr ':' '\n'

Output:

/usr/local/bin
/usr/bin
/bin

What it means: Easy to read and check.

6. paste: join files side by side

Why do we need this?

Names in one file and emails in another must become one list.

What is it?

paste joins line 1 with line 1, line 2 with line 2, using a TAB. -d, uses a comma. -s joins all lines of one file into one line. paste -sd+ builds a sum for bc.

How to do it (step by step)

1. Names and emails side by side.

paste -d, names.txt emails.txt

Output:

Aditi,aditi@ex.com
John,john@ex.com

What it means: Line 1 with line 1, line 2 with line 2.

2. All lines into one comma list.

paste -sd, servers.txt

Output:

web1,web2,db1

What it means: Handy for building arguments.

3. Add numbers with paste and bc.

paste -sd+ amounts.txt | bc

Output:

12750

What it means: Makes 4500+3200+5050 and bc adds it.

7. join: merge two files on a key

Why do we need this?

ids with names in one file and ids with cities in another need one combined table, like a database join.

What is it?

join FILE1 FILE2 joins on field 1 by default. -t, sets the separator, -1 2 -2 3 picks key fields, -a1 keeps unmatched lines from file 1 (left join). Both files must be sorted on the key.

How to do it (step by step)

1. Join two files on id.

join -t, <(sort ids_names.csv) <(sort ids_city.csv)

Output:

1,Aditi,Pune
2,John,London

What it means: Both files must be sorted on the key.

2. Left join: keep unmatched.

join -t, -a1 <(sort ids_names.csv) <(sort ids_city.csv)

Output:

1,Aditi,Pune
2,John,London
3,Mei

What it means: Mei has no city but is kept.

8. comm: compare two sorted lists

Why do we need this?

Old server list vs new list: what was added, removed, or kept?

What is it?

comm prints three columns: only in file 1, only in file 2, in both. -1 -2 -3 hide columns: -12 common, -23 only in file 1, -13 only in file 2. Inputs must be sorted.

How to do it (step by step)

1. Servers in both lists.

comm -12 <(sort old.txt) <(sort new.txt)

Output:

web1
web2

What it means: Common to both.

2. Servers removed.

comm -23 <(sort old.txt) <(sort new.txt)

Output:

db-old

What it means: Only in the old list.

3. Servers added.

comm -13 <(sort old.txt) <(sort new.txt)

Output:

web3

What it means: Only in the new list.

9. diff: see exactly what changed

Why do we need this?

A config change broke the site. You must find which line changed between the backup and the live file.

What is it?

diff a b shows the lines to change to turn a into b. -u gives unified format (the one Git uses): lines with - are removed, + are added, @@ shows line numbers. -y side by side, -q only says if they differ, -r compares folders, -i ignores case, -w ignores spaces. Exit code: 0 same, 1 different, 2 trouble. patch applies a diff -u file. cmp compares binary files byte by byte; vimdiff shows colours.

How to do it (step by step)

1. Unified diff of two configs.

diff -u nginx.conf.bak nginx.conf

Output:

--- nginx.conf.bak
+++ nginx.conf
@@ -12,3 +12,3 @@
     listen 80;
-    root /var/www/shop;
+    root /var/www/shopp;
     index index.html;

What it means: Someone typed shopp. That broke the site.

2. Just ask: are they different?

diff -q a.txt b.txt; echo $?

Output:

Files a.txt and b.txt differ
1

What it means: Exit code 1 means different. Good in scripts.

3. Side by side.

diff -y --width=50 old.txt new.txt

Output:

web1          web1
web2        | web3
db1           db1

What it means: | marks a changed line.

4. Compare two folders.

diff -rq /etc/nginx /backup/nginx

Output:

Files /etc/nginx/nginx.conf and /backup/nginx/nginx.conf differ
Only in /etc/nginx/conf.d: new.conf

What it means: Shows changed and extra files.

5. Make and apply a patch.

diff -u app.conf.orig app.conf > fix.patch && patch app.conf.orig < fix.patch

Output:

patching file app.conf.orig

What it means: Now the orig file has the same change.

6. Compare output of two servers.

diff <(ssh web1 rpm -qa | sort) <(ssh web2 rpm -qa | sort)

Output:

< openssl-3.0.7-1
> openssl-3.0.8-1

What it means: Process substitution <( ) lets diff compare commands.

10. cmp: compare files byte by byte

Why do we need this?

For binaries, images or archives, line diffs make no sense. You just need to know if they are identical and where they differ.

What is it?

cmp a b prints the first differing byte and line. -s is silent (exit code only: 0 same, 1 different). -l lists every differing byte. For checking downloads, sha256sum is often better.

How to do it (step by step)

1. Compare two binaries.

cmp app-v1.bin app-v2.bin

Output:

app-v1.bin app-v2.bin differ: byte 1025, line 3

What it means: First difference at byte 1025.

2. Silent check in a script.

cmp -s backup.tar live.tar && echo identical || echo changed

Output:

identical

What it means: Uses only the exit code.

11. Real examples from many industries

Why do we need this?

The same Linux skill shows up in every kind of company. Seeing it in many places helps you remember it.

What is it?

Each row is a made-up team. It shows who uses this and how.

How to do it (step by step)

12. Common errors and how to fix them

Why do we need this?

Everyone hits these errors. Knowing the cause saves hours.

What is it?

Read the exact message first. Then match it to a row below.

How to do it (step by step)

13. Troubleshooting step by step

Why do we need this?

When a text pipeline gives a wrong answer, run it one step at a time and look at each output.

What is it?

Go top to bottom. Stop at the first step that shows the problem.

How to do it (step by step)

1. Look at the raw input and its hidden characters.

head -3 data.csv | cat -A

2. Check the delimiter and field you cut.

head -3 data.csv | cut -d, -f2

3. Check sorting is numeric where needed.

cut -d, -f4 data.csv | sort -n | tail -3

4. Check uniq got sorted input.

cut -d, -f3 data.csv | sort | uniq -c

5. Fix locale surprises.

LC_ALL=C sort file | head

6. Check that diff is not just line ends.

diff --strip-trailing-cr -q a b

14. Lab 1: Top visitors report from a web log

Why do we need this?

Build the classic count pipeline step by step.

What is it?

A full hands-on lab. Run every step on a test Linux machine (an EC2 instance, a VM, or WSL).

How to do it (step by step)

1. Make a sample log.

mkdir -p ~/lab8 && cd ~/lab8 && for i in $(seq 1 300); do echo "10.0.0.$((i%4)) - GET /p$((i%5)) $((i%7==0?500:200))"; done > access.log; head -2 access.log

Output:

10.0.0.1 - GET /p1 200
10.0.0.2 - GET /p2 200

2. Pick the IP field.

cut -d' ' -f1 access.log | head -2

Output:

10.0.0.1
10.0.0.2

3. Count each IP.

cut -d' ' -f1 access.log | sort | uniq -c

Output:

     75 10.0.0.0
     75 10.0.0.1
     75 10.0.0.2
     75 10.0.0.3

4. Top pages with 500 errors.

grep ' 500$' access.log | cut -d' ' -f4 | sort | uniq -c | sort -nr | head -3

Output:

     9 /p0
     9 /p1
     8 /p4

5. Save the report.

cut -d' ' -f1 access.log | sort | uniq -c | sort -nr > top-ips.txt && wc -l top-ips.txt

Output:

4 top-ips.txt

15. Lab 2: Find a broken config change with diff

Why do we need this?

Make a backup, change a line, find it and undo it with a patch.

What is it?

A full hands-on lab. Run every step on a test Linux machine (an EC2 instance, a VM, or WSL).

How to do it (step by step)

1. Create a config and a backup.

cd ~/lab8 && printf 'port=8080\nroot=/srv/shop\nmode=prod\n' > app.conf && cp app.conf app.conf.bak

Output:

(no output)

2. Make a mistake.

sed -i 's#/srv/shop#/srv/shopp#' app.conf

Output:

(no output)

3. Find it.

diff -u app.conf.bak app.conf

Output:

-root=/srv/shop
+root=/srv/shopp

4. Make a reverse patch and apply it.

diff -u app.conf app.conf.bak > undo.patch; patch app.conf < undo.patch

Output:

patching file app.conf

5. Confirm they match.

diff -q app.conf app.conf.bak && echo same

Output:

same

16. Frequently asked questions

17. Interview questions and answers

Beginner level

Mid level level

Senior level

18. Related chapters

• Linux Ops Refresh for DevOps
• Bash Scripting for Automation
• Linux grep and find Commands with Examples
• How to Set Up Passwordless SSH Between Linux Servers

Revision, comparison and interview cards

Revise this lesson in a few minutes. Save the cards on your phone or download them as PNG files. The QR code on each card opens this lesson.

More cards: Ravi Sir's Cards.