Using Serial-Over-Lan(SOL) to Debug Linux Kernels

Currently, I am in a research project that requires me to debug linux kernels. I found a way to use gdb using serial-over-lan(SOL) to debug my linux machines. I got hints from many related web pages. I am writing this because I also found that there isn’t a web page that explains how to use gdb using SOL as a debugging channel.

For the readers who do not need detailed explanation, I’ll explain it briefly first. To use SOL as a debugging channel, what you have to do is redirecting your SOL session to a remote debugger.

When you use SOL, your client console will be attached like this.
(Target Machine) – (Console Client)

If you want to use gdb over SOL, you have to redirect I/O to remote gdb.
(Target Machine) – (Console Client) – (I/O Redirecting Server) – (GDB)

I’ll explain how to setup debugging environments step-by-step. First, I will explain how to enable SOL in linux machines. After that, enabling KDB will be explained. Lastly, I’ll describe how to use gdb from a remote machine. Through this post, I’ll use comments(#) to specify the machine that should be handled.

1. How to Setup SOL
Please modify /etc/default/grub at your debuggee machine. Of course, you have to type “update-grub && reboot” after modifying it.

#at your debuggee machine
GRUB_CMDLINE_LINUX="console=tty1 console=ttyS1,115200n8"

At your debugger machine, you can connect to the debuggee’s serial console by typing the command below.

#at your debugger machine
ipmitool -I lanplus -H <IP Address to IPMI> -U <Your ID> -P <Your Password> sol activate

2. How to Setup KDB over SOL
To enable kdb, you have to give an additional command at your debuggee machine.

#at your debuggee machine
GRUB_CMDLINE_LINUX="console=tty1 console=ttyS1,115200n8 kgdboc=ttyS1,115200"

At this point, you can use kdb using serial console. Let’s see how it works.

#at your debuggee machine
echo g > /proc/sysrq-trigger

You’ll see a simple command line like this at your debuggee machine.

Entering kdb (current=0xffff88085a4e3900, pid 5202) on processor 2 due to Keyboard Entry

Just type “go” to escape the kdb command line.

3. How to Setup KGDB over SOL
Now let’s see how we can use kgdb through IPMI. To use it as a gdb server, you have to redirect I/O from/to local/remote machines. You can use the following command to run a simple I/O server. It will open a server with port number 1000.

#at your debugger machine
socat TCP4-LISTEN:1000,reuseaddr,fork EXEC:"ipmitool -I lanplus -H <IP Address to IPMI> -U <Your ID> -P <Your Password> sol activate"

Now you can connect to the serial console using telnet.

#at your debugger machine
telnet localhost 1000

Let’s activate KDB with the following command.

#at your debuggee machine
echo g > /proc/sysrq-trigger

In your kdb, you have to convert debugger mode to kgdb.

#at your debugger machine
[2] kdb> kgdb

Please run gdb at your debugger machine and connect to the target

#at your debugger machine
$gdb /path/to/vmlinux
gdb)target remote localhost:1000

Using Serial Over LAN to do remote gdb or some sort of kernel debugger, creatiwit,

Posted in 2) Computer Engineering

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Recent Posts
누적 방문자 수
  • 113,748 hits
%d bloggers like this: