Ravindra BagaleCourses & study guides मराठी Track your progress

Chapter 4: Linux Advanced Commands

4.3 Text processing: awk and sed

By Last updated:

Friends, awk is the king of columns and sed is find-and-replace. Whether you need the top 10 IPs from Mauli Dairy's access log or want to change a port in a config, these two tools do the job.

awk — column-based processing

awk splits each line into fields $1, $2, ... (space-separated by default; -F sets the separator).

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head
#  ^ top 10 client IPs hitting your web server

awk -F: '{print $1, $7}' /etc/passwd          # user name and shell
awk -F: '$3 >= 1000 {print $1}' /etc/passwd   # normal (non-system) users
df -h | awk 'NR>1 {print $5, $6}'             # usage % and mount point
awk '{sum += $10} END {print sum/1024/1024 " MB"}' /var/log/nginx/access.log  # bytes served

sed — stream editor (find & replace)

sed 's/http/https/' file.txt              # replace first match per line (prints result)
sed 's/http/https/g' file.txt             # replace all matches
sed -i 's/Listen 80/Listen 8080/' /etc/httpd/conf/httpd.conf   # edit file in place
sed -i.bak 's/old/new/g' config.ini       # in place, keep backup config.ini.bak
sed -n '10,20p' file.txt                  # print only lines 10-20
sed '/^#/d' file.txt                      # delete comment lines

Test sed before using -i

Run the sed command without -i first and check the output. Once you are happy, add -i (or -i.bak to keep a backup).

Ravindra Bagale's Tip

A common mistake is running sed -i directly on a live config – one wrong pattern and the whole file is ruined. First run it without -i and look at the output, then use -i.bak so a backup remains. After changing the config, be sure to run sudo nginx -t or sudo apachectl configtest.

Lab

Chala, awk -F: '{print $1, $7}' /etc/passwd chalva aani konta user konta shell vaprto te bagha. Mag cp /etc/hosts ~/hosts.test karun sed 's/localhost/mauli-local/g' ~/hosts.test aadhi -i shivay chalva, output bagha, aani mag sed -i.bak 's/localhost/mauli-local/g' ~/hosts.test ne badla.