A few days ago knarf noticed something strange on one of his boxes:
A quick test with
telnet confirmed that the port was indeed open.
The port number did not ring a bell - although it was below
1024 (so it was a privileged port) - it was not listed in
Trying to find out where that socket came from we discussed a few ideas - starting with the obvious: A rootkit. It would be unlikely that both sockstat and lsof would be patched to skip that socket and netstat would've been forgotten, but it was still an option.
The second idea was, that lsof/sockstat didn't show anything, because there was no process there. – As the socket did react to connection attempts, this meant that the kernel had to be involved.
Checking the system include headers, the
struct socket was quickly located and contained an interesting looking field named
To look at the socket in question, we first need to find it in memory. The easiest way is to get
netstat -A. This command gives us the address of the tcp control block (tcpcb) for tcp sockets or the address of the internet protocol control block (inpcb) for udp sockets:
netstat -Aan | grep ' \*\.portno .* LISTEN'
So, lets fire up kgdb and look at the so_upcall element:
p ((struct tcpcb*)0xc1a6bde0)->t_inpcb->inp_socket->so_upcall (for tcp)
p ((struct inpcb*)0xc1a6bde0)->inp_socket->so_upcall (for udp)
A quick test on the ssh socket returned:
$1 = (void (*)(struct socket *, void *, int)) 0
Which is the expected result, as there is an actual process listening, and no
Our mysterious socket returned:
$2 = (void (*)(struct socket *, void *, int)) 0xc0974250 <svc_vc_soupcall>
So now we can be sure that it is the kernel listening here.