A while back I posted a Netconsole tutorial for how to capture Linux kernel debugging messages from a crashing machine. I’ve refined the instructions down to three scripts and three commands, which are after the break.
The scripts are meant to run on Ubuntu Gnome boxes, and they use specific paths to kill NetworkManager. If you’re adapting these scripts for another distro or KDE, you’re going to have to change them a bit. You will also have to modify the scripts to have the correct ethernet interface (e.g. eth0).
The first script, netconsole-target.sh, is meant to be run on the box that is going to be receiving the debugging messages:
#!/bin/sh
sudo killall nm-applet
sudo /etc/init.d/NetworkManager stop
sudo /etc/init.d/networking stop
sudo ip addr add 10.0.0.1/8 dev eth0
sudo ip link set eth0 up
This sets up the target box to have an IP address of 10.0.0.1. Now we set up the source box to have an IP address of 10.0.0.2 with the second script, netconsole.sh:
#!/bin/sh
sudo /etc/init.d/NetworkManager stop
sudo /etc/init.d/networking stop
sudo killall nm-applet
sudo ip link set eth0 up
sudo ip addr add 10.0.0.2/8 dev eth0
/sbin/ifconfig eth0
Now, we connect the crossover cable (or straight-through cable if one of the boxes can auto-negociate, like Thinkpads can). On the target box, we make sure we can ping the source box:
ping 10.0.0.2
On the source box, we run a similar command:
ping 10.0.0.1
If the ping doesn’t work, you have issues with either NetworkManager or the wrong ethernet interface.
Once we have both boxes configured, we set up the target box to receive the messages and pipe them to standard out and a log file, like so:
nc -u -l -p 6666 | tee logfile.txt
Now we run the third script, netconsole-ending.sh, on the source machine to make it start sending messages:
#!/bin/sh
sudo dmesg -n 8
sudo modprobe netconsole netconsole=@/eth0,@10.0.0.1/
On the target box, you should see lines like this appear on standard out:
[ 632.180045] console [netcon0] enabled
[ 632.180056] netconsole: network logging started
Now you can do whatever you want to crash your source machine and still be able to get the log file out! Happy hacking.
Hi Sarah. That’s a lot of messing about with NetworkManager. If you use netconsole a lot it is worthwhile buying a ethernet/USB dongle. Then configure that dongle in NetworkManager with a fixed IP address and tick “Advanced | Use this connection only for resources on its network” (ie, no default route is to point to this interface when the interface becomes active).
When you need to debug, plug the “debugging cable” (UTP patch and dongle) into the computers, don’t touch NetworkManager, but as usual run nc and modprobe netconsole. What is particularly nice is that the listening machine’s other interface stays up. I find continued Internet access whilst trying to debug an issue is worth the $20 for the dongle.
Best wishes, and congratulations on the success of the Linux Kernel Internships (I came here via the LWN article), glen.