How to Set Up Passwordless SSH Between Linux Servers

By Ravindra Bagale, trainer · LinkedIn · Last updated: 18 October 2026

1. The lab: three servers, and the checks before you log in

Why: Passwordless SSH only makes sense between several servers. Building three identical ones, and checking they can reach each other first, means any later failure is in the keys, not in the network.

What: three t3.micro servers with Amazon Linux 2023, in the same Region and VPC, launched with the same key pair and security group. Name them linux1, linux2 and linux3 in the Name tag.

How:

1. Launch one instance with Number of instances: 3, or launch one and then two more. The 7 launch steps are in Your first EC2 instance.
2. Rename each one in the console (click the pencil in the Name column).
3. Wait until Status check shows all checks passed. There is a system check (the AWS host and its network) and an instance check (your OS). Newer, Nitro-based types such as t3.micro also run an attached EBS check, so they show 3/3; older types show 2/2. Details: Status checks: 2/2 or 3/3?.
4. Log in with SSH and your key, or with EC2 Instance Connect in the browser.

Two things that help in a multi-server lab:

1. .pem or .ppk: .pem is the OpenSSH format, used by the ssh command on Windows, Mac and Linux. .ppk is PuTTY's format; PuTTYgen converts one into the other. More in Step 4: Key pair.
2. Which server am I on? The prompt shows the host name, which is ip- plus the private IP by default, not the Name tag. To see the Name in the prompt, run this once on each server, then log out and back in:

   sudo hostnamectl set-hostname linux1
   

2. Interview question: move an EC2 server to another Region

Why: you can't drag an instance from Mumbai to Singapore. An instance, its volumes, its key pair and its security group all belong to one Region. You copy the image instead.

How:

1. In Mumbai, select the instance, then Actions > Image and templates > Create image. Wait until the AMI is Available. (AMIs are explained in AWS Elastic IP, AMI and EBS Snapshots.)
2. In AMIs, select it, then Actions > Copy AMI, and choose Asia Pacific (Singapore) as the destination Region.
3. Switch the console to Singapore. The copy has a new AMI ID; wait until it is Available.
4. Launch instance from AMI. Pick a key pair that exists in Singapore (create one, or import your existing public key) and a security group with the ports you need.
5. Test the new server. Then stop or terminate the old one, and deregister AMIs and delete snapshots you no longer need, because each Region bills its own copy.

Anything written on the old server after step 1 is not in the copy, so plan a short cut-over. If an interviewer then asks you to rate yourself out of 10, be honest: a 6 you can defend is better than a 9 you can't.

3. Quick recap: cat, tac, stat and touch

Why: These commands come back in this chapter: cat to read the public key, stat and touch while checking files. A short refresher keeps the setup steps fast.

These come from earlier chapters, so here is only the short version:

cat > file1
cat >> file1
cat file1 file2 > file3
tac file3
stat file3
touch f1 f2 f3

1. cat > file1 writes what you type; press Ctrl+D on an empty line to save. Ctrl+C cancels the command and can leave you with an empty or half-written file.
2. >> appends at the end; a single > replaces the content.
3. cat file1 file2 > file3 joins two files into a third; tac prints the lines in reverse.
4. stat shows Access, Modify, Change and Birth. On EC2 the clock is in UTC, so add 5:30 to read it in India time.
5. touch on an existing file sets Access and Modify to now (Change follows); on a new name it creates an empty file. It never touches the content.

The full explanations: cat and the > trap, the three timestamps and cat and tac.

4. The vi and nano survival kit

Why: you will edit authorized_keys in section 14, and on many servers vi is the only editor installed.

How, in vi:

1. vi notes.txt opens the file in normal mode: keys are commands, not text.
2. Press i to type at the cursor, or o to open a new line below. The bottom shows -- INSERT --.
3. Type your text, then press Esc to go back to normal mode.
4. Type :wq and Enter to save and quit, :q! to quit without saving, or :wq! to force the save (for example on a read-only file you own).
5. Useful in normal mode: G jumps to the last line, dd deletes the current line.

In nano: type straight away; Ctrl+O saves, Ctrl+X exits (answer Y and Enter if it asks to save).

The browser terminal and zoom: in EC2 Instance Connect, zooming the browser while vi is open can scramble the screen, because the browser terminal doesn't pass the new size to vi cleanly. vi itself is fine. Set the zoom before you open vi, press Ctrl+L to redraw, or use a normal SSH terminal. More on both editors: nano and vi.

5. Reading ls -l

Why: SSH refuses keys and folders with loose permissions. Reading ls -l correctly is how you spot that before SSH silently ignores your key.

What: ls -l prints one item per line with its type, permissions, owner, size and time. The first character alone tells file from folder (more on it).

1. Type: - file, d directory, l link.
2. Permissions: three groups of rwx for owner, group and others. The . after them on Amazon Linux means the file has an SELinux label.
3. Links, owner, group.
4. Size in bytes. Add -h (ls -lh) to see K, M and G.
5. Last modified time, then the name.

6. Sorting by time: ls -lt and ls -ltr

Why: "which file changed last?" is the first question when something breaks.

How:

touch file7
touch file8
ls -lt
ls -ltr

1. ls -lt sorts by Modify time, newest first: file8 is at the top.
2. ls -ltr reverses it, newest last: file8 is at the bottom, right above your prompt, where your eyes already are. ls -lrt is the same command; the letters can go in any order.
3. On a server, ls -ltr /var/log shows the log that was written most recently at the bottom.

7. Hidden files: ls -a, ls -la and ll

Why: The SSH files live in the hidden folder ~/.ssh. A plain ls never shows it, so you need -a to see what you are about to change.

What: a name that starts with a dot is hidden. ls skips it; ls -a shows it.

ls -a
ls -la
ll

1. ls -a lists everything, including . (this folder), .. (the folder above), .bashrc, .bash_profile and .ssh.
2. ls -la gives the same list in long format: use it to check permissions on .ssh.
3. ll is a shortcut (an alias). On Amazon Linux it means ls -l with colours; on Ubuntu it means ls -alF. Type alias ll to see what it means on your server.
4. A typo such as la -la gives command not found: read the first word of the error.

More on hidden files: Hidden files: a dot at the start of the name.

8. Folders: mkdir -p, cd and pwd

Why: You will create and move between folders constantly while setting up servers. -p and relative cd paths save many commands.

How:

mkdir dir1
mkdir -p dir2/dir3/dir4/dir5
cd dir2/dir3/dir4/dir5
pwd
cd ../..
cd

1. mkdir -p makes the whole chain at once. Without -p, mkdir dir9/dir8 fails with No such file or directory because dir9 doesn't exist. With -p, running it again is not an error.
2. cd dir2/dir3/dir4/dir5 goes down four levels in one step; pwd prints the full path.
3. cd ../.. goes up two levels. cd alone always takes you home (/home/ec2-user).
4. cd dir1/dir2 fails with No such file or directory when that path doesn't exist. Read the error, then check the path with ls and pwd.

The details of paths and cd are in Linux directory commands.

9. du: how big is this file or folder?

Why: When a disk fills up you need to know which folder is eating it. du answers that.

What: du (disk usage) shows the space that the things you name take on disk.

du -sh file7
du -sh dir2
du -sh *

1. -s means summarize: one total per item, instead of a line for every folder inside.
2. -h means human-readable: K, M and G instead of blocks.
3. du -sh * gives one line for each item in the current folder (hidden items are skipped by *).
4. An empty file shows 0. A small file shows 4.0K, because disk space is used in 4 KB blocks; ls -l shows the exact bytes.
5. For folders you can't read, such as /var/log, use sudo du -sh /var/log.

10. df: how full is the disk?

Why: A full disk stops databases and websites. df is the first check, and with passwordless SSH you can run it on every server from one place.

What: df (disk free) shows each mounted filesystem: its size, how much is used, how much is free, and where it is mounted.

df -h
df -h /

1. The line mounted on / is your root EBS volume: 8 GiB by default on Amazon Linux.
2. Use% is the number to watch. Near 100%, programs start failing with No space left on device.
3. The tmpfs and devtmpfs lines live in RAM, not on the disk.

Worked example: the disk is full.

1. An app fails with No space left on device.
2. df -h / shows Use% at 100%.
3. sudo du -sh /var/* shows that /var/log is the biggest.
4. sudo du -sh /var/log/* points to one huge log file. Clean or rotate it, or grow the EBS volume.

11. mv: move and rename

Why: Moving and renaming files is part of every setup, including moving a copied key into place.

mv file3 dir1
mv file2 notes.txt
mv dir2 dir1

1. mv file3 dir1 moves file3 into dir1.
2. mv file2 notes.txt renames it, because notes.txt is not an existing folder.
3. mv moves folders without -r.
4. A wrong name gives mv: cannot stat 'file99': No such file or directory.
5. mv overwrites an existing file without asking; mv -i asks first.

The full chapter: Linux mv and rm.

12. Passwordless SSH: why servers need it

Why: in a real setup, one server sends work to the others. Ansible connects from a controller to every managed server over SSH. A Jenkins controller starts builds on agent servers over SSH. (Older documents call them master and slave.) If a person had to type a password for every login, nothing could be automated.

Think of a team of four where three are on leave and one does all the work. A controller that can log in to every server spreads the work out, without a person in between.

The goal: linux1 logs in to linux2 and linux3 as ec2-user with no password, using a key pair that linux1 owns.

13. What goes where: keys, authorized_keys and port 22

Why: Most failed setups put a key on the wrong server or in the wrong file. Knowing exactly which file sits where avoids that.

1. The key pair is made on linux1: id_ed25519 (private) and id_ed25519.pub (public), in ~/.ssh.
2. The private key never leaves linux1. Anyone who has it can log in wherever its public key is trusted.
3. The public key is one line. It goes at the end of ~/.ssh/authorized_keys of the user you will log in as on linux2 and linux3: /home/ec2-user/.ssh/authorized_keys.
4. Permissions: ~/.ssh must be 700 and authorized_keys 600, owned by that user. SSH ignores the file when they are more open.
5. Port 22: the security groups of linux2 and linux3 must allow inbound SSH, TCP 22 from linux1: its private IP with /32, or better, linux1's security group as the source. The launch wizard's default rule allows 22 from anywhere, so the lab works with it, but in real setups you narrow it.
6. Log in as ec2-user, not root. On Amazon Linux 2023, root's authorized_keys already contains your launch key, wrapped in a command that only prints Please login as the user "ec2-user" rather than the user "root". and closes the connection, and password logins are off. Use ec2-user and sudo for admin work.

14. How: set it up step by step as ec2-user

Why: Tools like Ansible and Jenkins log in to servers with no one there to type a password. These steps are exactly what they need.

On linux1, make the key pair:

ssh-keygen -t ed25519
ls -la ~/.ssh
cat ~/.ssh/id_ed25519.pub

1. Press Enter three times: the default file name, then an empty passphrase twice. (An empty passphrase lets scripts use the key. On a laptop, set a passphrase.)
2. ls -la ~/.ssh now shows id_ed25519 with -rw------- and id_ed25519.pub with -rw-r--r--.
3. cat prints the public key: one line starting ssh-ed25519 AAAA... and ending with a comment such as ec2-user@linux1. Select the whole line and copy it. (ssh-keygen with no -t on Amazon Linux 2023 makes an RSA key, id_rsa.pub, which works the same way.)

On linux2, add it (log in there as usual with your .pem):

vi ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

1. In vi, press G to go to the last line, o to open a new line below it, paste, press Esc, and type :wq.
2. Don't delete the line that is already there. It is the public half of your .pem key; without it you lock yourself out of linux2.
3. The chmod lines make sure the permissions are right. On EC2 they already are, but run them anyway; they are harmless.
4. Repeat on linux3.

Check the security groups of linux2 and linux3: an inbound rule for SSH (22) from linux1's private IP or security group.

Back on linux1, test it:

ssh ec2-user@LINUX2_PRIVATE_IP
hostname
exit
ssh ec2-user@LINUX3_PRIVATE_IP 'df -h /'

1. Use the private IP shown in the console (or hostname -I on linux2): the servers are in the same VPC, so the traffic stays inside AWS.
2. The first time, SSH asks Are you sure you want to continue connecting (yes/no/[fingerprint])?. Type yes; the server's key is saved in ~/.ssh/known_hosts and you won't be asked again.
3. No password prompt appears, and the prompt changes to linux2. exit brings you back to linux1.
4. The last line runs one command on linux3 and prints the result on linux1: this is how tools like Ansible work.

ssh-copy-id can install a key for you, but only when you can already log in to the target. Between fresh EC2 servers you can't yet, so the copy and paste above is the way in.

15. The trust is one way

What: linux1 can log in to linux2 because linux1's public key is in linux2's authorized_keys. Nothing allows the opposite: ssh from linux2 to linux1 ends with Permission denied (publickey).

1. The one-way part is the permission to log in. Once you are in, the session carries data both ways: you type, the output comes back.
2. For linux2 to log in to linux1 too, run ssh-keygen on linux2 and put its public key into linux1's authorized_keys.

16. Task: four servers, both ways

The task: four servers, linux-1 to linux-4. Every server must log in to every other one without a password.

Worked example: count the paths.

1. Each server logs in to the other three, not to itself: 3 paths per server.
2. 4 servers × 3 = 12 one-way paths, not 4 × 4 = 16.
3. On each server: run ssh-keygen once, and put the other three public keys in its authorized_keys (3 lines added per server, 12 in total).
4. Test from each server, for example from linux-1: ssh ec2-user@IP_OF_LINUX_2 hostname, then linux-3 and linux-4.
5. The count grows fast: 10 servers both ways need 10 × 9 = 90 paths. That is why real setups use one controller (9 paths for 10 servers) and tools to hand out keys.

17. Ports you should know

Why: Every security group rule and every connection error names a port. Knowing the common ones tells you at once which service is involved.

A port number runs from 0 to 65,535 (65,536 ports). Ports 0 to 1023 are the well-known ports for standard services (0 itself is reserved). How ports work is in Ports: the IP reaches the server, the port reaches the app.

18. When something looks wrong

Why: Each error points to one link in the chain: network, user, key or permissions. Reading it correctly saves you from redoing the whole setup.

1. Connection timed out on port 22: the target's security group doesn't allow 22 from this server. Add the rule.
2. Permission denied (publickey):
   1. You used the wrong user: ec2-user on Amazon Linux, ubuntu on Ubuntu, not root.
   2. The key line lost its first characters, or was pasted across two lines. It must be one complete line.
   3. ~/.ssh is more open than 700, or authorized_keys is writable by others (such as 666): SSH refuses it.
   4. You created authorized_keys with sudo, so root owns it. Fix it with sudo chown ec2-user:ec2-user ~/.ssh/authorized_keys.
3. REMOTE HOST IDENTIFICATION HAS CHANGED: a new server got an IP you used before. Run ssh-keygen -R THAT_IP on linux1, then connect again.
4. Still stuck: ssh -v ec2-user@IP prints every step, including which key was offered.
5. vi looks scrambled after zooming: press Ctrl+L, or quit with :q! and reopen.
6. du shows 0 for a file: the file is empty. ls -l shows the size in bytes.

19. The whole class on one page

1. Launch three t3.micro servers with the same key pair and security group; wait for 3/3 status checks (system, instance, attached EBS).
2. To move a server to another Region: create an AMI, copy it, launch from the copy, and set up the key pair and security group again there.
3. cat > with Ctrl+D, >> to append, cat a b > c to join, tac to reverse, stat for the times (UTC on EC2), touch to create or update.
4. vi: i or o to type, Esc, then :wq or :q!. nano: Ctrl+O, Ctrl+X.
5. ls -l shows type, permissions, owner, size and time; ls -ltr puts the newest at the bottom; ls -la shows hidden files; ll is ls -l on Amazon Linux.
6. mkdir -p makes a whole chain; cd ../.. goes up two levels; cd alone goes home; pwd tells you where you are.
7. du -sh sizes what you name (-s summarize, -h human-readable); df -h shows how full each disk is.
8. mv moves and renames, folders included.
9. Passwordless SSH: ssh-keygen on linux1, its .pub line appended to ec2-user's authorized_keys on the others, 700/600, and port 22 open from linux1.
10. The private key never leaves its server. Log in as ec2-user, not root.
11. Trust is one way. Four servers both ways need 4 × 3 = 12 paths.
12. Ports run from 0 to 65,535: 22 SSH, 53 DNS, 80 HTTP, 443 HTTPS.

20. Hands-on lab: passwordless SSH from linux1 to linux2

Why this lab. This lab sets up one-way trust from linux1 to linux2, checks every file and permission on the way, and proves it by running a command on linux2 without typing anything.

1. Name the servers
   • Command: sudo hostnamectl set-hostname linux1
   • You should see: Nothing.
   • Check: After you log out and in again, the prompt shows ec2-user@linux1. Do the same on the second server with linux2.
   • Why: With identical servers, the prompt is the only way to know where you are. Running a key command on the wrong server is the most common mistake here.
2. Check that linux1 reaches port 22 on linux2
   • Command: ssh ec2-user@LINUX2_PRIVATE_IP
   • You should see: Permission denied (publickey...), quickly, not a timeout.
   • Check: A fast "denied" means the network is fine; Connection timed out means the security group blocks port 22.
   • Why: Use the private IP: the servers talk inside the VPC. Testing the network first separates network problems from key problems.
3. Make a key pair on linux1
   • Command: ssh-keygen -t ed25519
   • You should see: Three questions; press Enter for each. Then Your identification has been saved in /home/ec2-user/.ssh/id_ed25519.
   • Check: ls -la ~/.ssh shows id_ed25519 (-rw-------) and id_ed25519.pub.
   • Why: An empty passphrase is what makes it passwordless. The private key never leaves linux1; only the .pub file is shared.
4. Copy the public key
   • Command: cat ~/.ssh/id_ed25519.pub
   • You should see: One line starting with ssh-ed25519 AAAA and ending with ec2-user@linux1.
   • Check: Select and copy the whole line.
   • Why: The whole line, in one piece, is the public key. A line broken in two doesn't work.
5. Add it on linux2
   • Command: vi ~/.ssh/authorized_keys
   • You should see: The file opens with the key you launched with. Press o, paste the line, Esc, :wq.
   • Check: wc -l ~/.ssh/authorized_keys → 2
   • Why: Do this on linux2, using your laptop login to it. Keep the existing line, or your own laptop key stops working.
6. Check the permissions on linux2
   • Command: chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys
   • You should see: Nothing.
   • Check: ls -ld ~/.ssh starts with drwx------; ls -l ~/.ssh/authorized_keys with -rw-------.
   • Why: SSH ignores authorized_keys if others can write to it or its folder. These are usually already right; setting them rules it out.
7. Log in from linux1
   • Command: ssh ec2-user@LINUX2_PRIVATE_IP
   • You should see: The first time: Are you sure you want to continue connecting (yes/no/[fingerprint])? Type yes. Then the prompt ec2-user@linux2.
   • Check: No password asked. Type exit to return to linux1.
   • Why: The yes saves linux2's identity in ~/.ssh/known_hosts, so later logins can detect a server that has been swapped.
8. Run a command remotely
   • Command: ssh ec2-user@LINUX2_PRIVATE_IP 'df -h /'
   • You should see: linux2's disk usage, and you stay on linux1.
   • Check: The prompt still says linux1.
   • Why: This is what automation tools do: run commands on many servers from one, with no one typing.
9. Prove the trust is one way
   • Do: On linux2, run ssh ec2-user@LINUX1_PRIVATE_IP.
   • You should see: Permission denied (publickey...)
   • Check: linux2 has no key of its own in linux1's authorized_keys.
   • Why: Each direction needs its own key pair and its own authorized_keys line. Terminate the servers when you finish.

21. Try at home

Recap

Quick revision cards

Revise this lesson in two minutes. Save the cards on your phone or download them as PNG files. The QR code on each card opens its lesson.

More cards: Ravi Sir's Cards.