How to know witch process is running in a particular port on AIX using KDB

Fellas, here is the gig, please, don’t cry! 😀

Suppose your application is not working well (or crashed, or stops and cannot goes online again) because someone install a new software that uses the same port as the application and leave the server, simple like this – believe (it happens). What a cool thing right?

Well, In this example, i will use the port 1334, but, in this question itself we will discuss when you need to remove a socket from a determined port but the rmsock command itself cannot tell you – WITCH PROCESS is being held the process, something like this:

# netstat -Aan |grep *.1334
f1000e0000180bb8 tcp4       0      0  *.1334                *.*                   LISTEN

Normal case:

# rmsock f1000e0000180bb8 tcpcb
The socket 0xf1000e0000180808 is being held by proccess 3211434 (writesrv).

Note that, sometimes AIX can give you all the job listing the PID 3211434, the process writesrv but, sometimes…

Worst case (and that’s what we gonna discuss it now):

# rmsock f1000e0000180bb8 tcpcb
The socket 0xf1000e0000180808 is being held by Kernel/Kernel Extension.

And now i ask you: WTF?

Now, let’s use KDB to starting playing around. Note that the output is too detailed, so you need to take your time to check it out what you need!

PS: If the protocol used by the socket is UDP, you need to change to “inpcb” instead of “tcbcp”.

Back to Life!

# kdb
           START              END 
0000000000001000 00000000058A0000 start+000FD8
F00000002FF47600 F00000002FFDF9C8 __ublock+000000
000000002FF22FF4 000000002FF22FF8 environ+000000
000000002FF22FF8 000000002FF22FFC errno+000000
F1000F0A00000000 F1000F0A10000000 pvproc+000000
F1000F0A10000000 F1000F0A18000000 pvthread+000000
read vscsi_scsi_ptrs OK, ptr = 0x5AB0380
(0)>

Then, you type: sockinfo f1000e0000180bb8 tcpcb

(0)> sockinfo f1000e0000180bb8 tcpcb
---- TCPCB ----(@ F1000E0000180BB8)----
    seg_next......@F1000E0000180BB8  seg_prev......@F1000E0000180BB8  
    t_softerror... 00000000 t_state....... 00000001 (LISTEN)
    t_timer....... 00000000 (TCPT_REXMT)
    t_timer....... 00000000 (TCPT_PERSIST)
    t_timer....... 00000000 (TCPT_KEEP)
    t_timer....... 00000000 (TCPT_2MSL)
    t_rxtshift.... 00000000 t_rxtcur...... 00000006 t_dupacks..... 00000000 
    t_maxseg...... 000005B4 t_force....... 00000000 
    t_flags....... 00000000 ()
    t_oobflags.... 00000000 ()
    t_template....@0000000000000000  t_inpcb.......@F1000E0000180AA0  
    t_iobc........ 00000000 t_timestamp... 22B9A601 snd_una....... 00000000 
    snd_nxt....... 00000000 snd_up........ 00000000 snd_wl1....... 00000000 
    snd_wl2....... 00000000 iss........... 00000000 
    snd_wnd....... 0000000000000000 rcv_wnd....... 0000000000000000 
    rcv_nxt....... 00000000 rcv_up........ 00000000 irs........... 00000000 
    snd_wnd_scale. 00000000 rcv_wnd_scale. 00000000 req_scale_sent 00000000 
    req_scale_rcvd 00000000 last_ack_sent. 00000000 timestamp_rec. 00000000 
    timestamp_age. 00000006 rcv_adv....... 00000000 snd_max....... 00000000 
    snd_cwnd...... 000000003FFFC000        snd_ssthresh.. 000000003FFFC000 
    t_idle........ 00000006 t_rtt......... 00000000 t_rtseq....... 00000000 
    t_srtt........ 00000000 t_rttvar...... 00000006 t_rttmin...... 00000002 
    max_rcvd...... 0000000000000000        max_sndwnd.... 0000000000000000 
    t_peermaxseg.. 000005B4 snd_in_pipe... 00000000 
    sack_data.....@0000000000000000         snd_recover... 00000000 
    snd_high...... 00000000 snd_ecn_max... 00000000 snd_ecn_clear. 00000000 
    t_splice_with.@0000000000000000         t_splice_flags 00000000 


-------- TCB --------- INPCB  INFO ----(@ F1000E0000180AA0)----
    next........@0000000000000000  prev........@0000000000000000  
    head........@00000000061DDC00  faddr_6.....@F1000E0000180AC0  
    iflowinfo... 00000000 fport....... 00000000 fatype...... 00000000 
    oflowinfo... 00000000 lport....... 00000536 latype...... 00000000 
    laddr_6.....@F1000E0000180AD8  socket......@F1000E0000180808  
    ppcb........@F1000E0000180BB8  route_6.....@F1000E0000180AF8  
    ifa.........@0000000000000000  flags....... 00000400 
    proto....... 00000000 tos......... 00000000 ttl......... 0000003C 
(0)> more (^C to quit) ?

Let’s hit ENTER

rcvttl...... 00000000 rcvif.......@0000000000000000  
    options.....@0000000000000000  refcnt...... 00000000 
    lock........ 0000000000000000  rc_lock..... 0000000000000000 
    moptions....@0000000000000000  hash.next...@F1000A002C047D10  
    hash.prev...@F1000A002C047D10  timewait.nxt@0000000000000000  
    timewait.prv@0000000000000000  inp_v6opts  @0000000000000000  
    inp_pmtu....@0000000000000000  inp_fastlo..@0000000000000000  

---- SOCKET INFO ----(@ F1000E0000180808)----
    type........ 0001 (STREAM)
    opts........ 0002 (ACCEPTCONN)
    linger...... 0000 state....... FFFF8080 (PRIV)
    pcb.....@F1000E0000180AA0  proto...@00000000061B5688  
    lock....@F1000E0000166A80  head....@0000000000000000  
    q0......@0000000000000000  q.......@0000000000000000  
    q0len....... 0000 qlen........ 0000 qlimit...... 0005 
    timeo....... 0000 error....... 0000 special..... 0E08 
    pgid.... 0000000000000000  oobmark. 0000000000000000 

snd:cc...... 0000000000000000  hiwat... 0000000000004000 
    mbcnt... 0000000000000000  mbmax... 0000000000010000 
    lowat... 0000000000001000  mb......@0000000000000000  
    sel.....@0000000000000000  events...... 0000 
    iodone.. 00000000          ioargs..@0000000000000000  
    lastpkt.@0000000000000000  wakeone. FFFFFFFFFFFFFFFF 
    timer...@0000000000000000  timeo... 00000000 
    flags....... 0000 ()
    wakeup.. 00000000          wakearg.@0000000000000000  
    lockwtg. FFFFFFFFFFFFFFFF 

MBUF LIST

rcv:cc...... 0000000000000000  hiwat... 0000000000004000 
    mbcnt... 0000000000000000  mbmax... 0000000000010000 
    lowat... 0000000000000001  mb......@0000000000000000  
    sel.....@0000000000000000  events...... 0000 
    iodone.. 00000000          ioargs..@0000000000000000  
    lastpkt.@0000000000000000  wakeone. FFFFFFFFFFFFFFFF 
    timer...@0000000000000000  timeo... 00000000 
    flags....... 0000 ()
    wakeup.. 00000000          wakearg.@0000000000000000

Hit ENTER again

lockwtg. FFFFFFFFFFFFFFFF  

MBUF LIST

    tpcb....@0000000000000000  fdev_ch.@0000000000000000  
    sec_info@0000000000000000  qos.....@0000000000000000  
    gidlist.@0000000000000000  private.@0000000000000000  
    uid..... 00000000 bufsize. 00000000 threadcnt00000000 
    nextfree@0000000000000000  
    siguid.. 00000000 sigeuid. 00000000 sigpriv. 00000000 
    sndtime. 0000000000000000  sec  0000000000000000  usec 
    rcvtime. 0000000000000000  sec  0000000000000000  usec 
    saioq...@0000000000000000  saioqhd.@0000000000000000  
    accept.. 00000000008F001F  frcatime 00000000 
    isnoflgs 00000000 ()
    rcvlen.. 0000000000000000  frcaback@0000000000000000  
    frcassoc@0000000000000000  frcabckt 0000000000000000 
    iodone.. 00000000          iodonefl 00000000 ()
    ioarg...@0000000000000000  refcnt.. 0000000000000001 
    trclev........... 0001 

proc/fd:  49/3
proc/fd: fd: 3
              SLOT NAME     STATE      PID    PPID          ADSPACE  CL #THS

pvproc+00C400   49*writesrv ACTIVE 03100AA 02E0078 000000081C327400   0 0001



(0)> 

The complete command show this (for this particular process):

PS: Interesting information that you gonna use are in bold.

---- TCPCB ----(@ F1000E0000180BB8)----
    seg_next......@F1000E0000180BB8  seg_prev......@F1000E0000180BB8  
    t_softerror... 00000000 t_state....... 00000001 (LISTEN)
    t_timer....... 00000000 (TCPT_REXMT)
    t_timer....... 00000000 (TCPT_PERSIST)
    t_timer....... 00000000 (TCPT_KEEP)
    t_timer....... 00000000 (TCPT_2MSL)
    t_rxtshift.... 00000000 t_rxtcur...... 00000006 t_dupacks..... 00000000 
    t_maxseg...... 000005B4 t_force....... 00000000 
    t_flags....... 00000000 ()
    t_oobflags.... 00000000 ()
    t_template....@0000000000000000  t_inpcb.......@F1000E0000180AA0  
    t_iobc........ 00000000 t_timestamp... 22B9A601 snd_una....... 00000000 
    snd_nxt....... 00000000 snd_up........ 00000000 snd_wl1....... 00000000 
    snd_wl2....... 00000000 iss........... 00000000 
    snd_wnd....... 0000000000000000 rcv_wnd....... 0000000000000000 
    rcv_nxt....... 00000000 rcv_up........ 00000000 irs........... 00000000 
    snd_wnd_scale. 00000000 rcv_wnd_scale. 00000000 req_scale_sent 00000000 
    req_scale_rcvd 00000000 last_ack_sent. 00000000 timestamp_rec. 00000000 
    timestamp_age. 00000006 rcv_adv....... 00000000 snd_max....... 00000000 
    snd_cwnd...... 000000003FFFC000        snd_ssthresh.. 000000003FFFC000 
    t_idle........ 00000006 t_rtt......... 00000000 t_rtseq....... 00000000 
    t_srtt........ 00000000 t_rttvar...... 00000006 t_rttmin...... 00000002 
    max_rcvd...... 0000000000000000        max_sndwnd.... 0000000000000000 
    t_peermaxseg.. 000005B4 snd_in_pipe... 00000000 
    sack_data.....@0000000000000000         snd_recover... 00000000 
    snd_high...... 00000000 snd_ecn_max... 00000000 snd_ecn_clear. 00000000 
    t_splice_with.@0000000000000000         t_splice_flags 00000000 


-------- TCB --------- INPCB  INFO ----(@ F1000E0000180AA0)----
    next........@0000000000000000  prev........@0000000000000000  
    head........@00000000061DDC00  faddr_6.....@F1000E0000180AC0  
    iflowinfo... 00000000 fport....... 00000000 fatype...... 00000000 
    oflowinfo... 00000000 lport....... 00000536 latype...... 00000000 
    laddr_6.....@F1000E0000180AD8  socket......@F1000E0000180808  
    ppcb........@F1000E0000180BB8  route_6.....@F1000E0000180AF8  
    ifa.........@0000000000000000  flags....... 00000400 
    proto....... 00000000 tos......... 00000000 ttl......... 0000003C 
    rcvttl...... 00000000 rcvif.......@0000000000000000  
    options.....@0000000000000000  refcnt...... 00000000 
    lock........ 0000000000000000  rc_lock..... 0000000000000000 
    moptions....@0000000000000000  hash.next...@F1000A002C047D10  
    hash.prev...@F1000A002C047D10  timewait.nxt@0000000000000000  
    timewait.prv@0000000000000000  inp_v6opts  @0000000000000000  
    inp_pmtu....@0000000000000000  inp_fastlo..@0000000000000000  

---- SOCKET INFO ----(@ F1000E0000180808)----
    type........ 0001 (STREAM)
    opts........ 0002 (ACCEPTCONN)
    linger...... 0000 state....... FFFF8080 (PRIV)
    pcb.....@F1000E0000180AA0  proto...@00000000061B5688  
    lock....@F1000E0000166A80  head....@0000000000000000  
    q0......@0000000000000000  q.......@0000000000000000  
    q0len....... 0000 qlen........ 0000 qlimit...... 0005 
    timeo....... 0000 error....... 0000 special..... 0E08 
    pgid.... 0000000000000000  oobmark. 0000000000000000 

snd:cc...... 0000000000000000  hiwat... 0000000000004000 
    mbcnt... 0000000000000000  mbmax... 0000000000010000 
    lowat... 0000000000001000  mb......@0000000000000000  
    sel.....@0000000000000000  events...... 0000 
    iodone.. 00000000          ioargs..@0000000000000000  
    lastpkt.@0000000000000000  wakeone. FFFFFFFFFFFFFFFF 
    timer...@0000000000000000  timeo... 00000000 
    flags....... 0000 ()
    wakeup.. 00000000          wakearg.@0000000000000000  
    lockwtg. FFFFFFFFFFFFFFFF 

MBUF LIST

rcv:cc...... 0000000000000000  hiwat... 0000000000004000 
    mbcnt... 0000000000000000  mbmax... 0000000000010000 
    lowat... 0000000000000001  mb......@0000000000000000  
    sel.....@0000000000000000  events...... 0000 
    iodone.. 00000000          ioargs..@0000000000000000  
    lastpkt.@0000000000000000  wakeone. FFFFFFFFFFFFFFFF 
    timer...@0000000000000000  timeo... 00000000 
    flags....... 0000 ()
    wakeup.. 00000000          wakearg.@0000000000000000  
    lockwtg. FFFFFFFFFFFFFFFF  

MBUF LIST

    tpcb....@0000000000000000  fdev_ch.@0000000000000000  
    sec_info@0000000000000000  qos.....@0000000000000000  
    gidlist.@0000000000000000  private.@0000000000000000  
    uid..... 00000000 bufsize. 00000000 threadcnt00000000 
    nextfree@0000000000000000  
    siguid.. 00000000 sigeuid. 00000000 sigpriv. 00000000 
    sndtime. 0000000000000000  sec  0000000000000000  usec 
    rcvtime. 0000000000000000  sec  0000000000000000  usec 
    saioq...@0000000000000000  saioqhd.@0000000000000000  
    accept.. 00000000008F001F  frcatime 00000000 
    isnoflgs 00000000 ()
    rcvlen.. 0000000000000000  frcaback@0000000000000000  
    frcassoc@0000000000000000  frcabckt 0000000000000000 
    iodone.. 00000000          iodonefl 00000000 ()
    ioarg...@0000000000000000  refcnt.. 0000000000000001 
    trclev........... 0001 

proc/fd:  49/3
proc/fd: fd: 3
              SLOT NAME     STATE      PID    PPID          ADSPACE  CL #THS

pvproc+00C400   49*writesrv ACTIVE 03100AA 02E0078 000000081C327400   0 0001



(0)>

Where:

writesrv is the process itself;
ACTIVE is the state (of course);

And, the MOST IMPORTANT ONE:

03100AA, that is the PID in Hex.

If you stills on KDB, you can convert it using kdb function hcal, like this:

(0)> hcal 03100AA
Value hexa: 003100AA          Value decimal: 3211434

(0)> 

Or, You can also use perl to convert it to decimal:

# perl -le 'print hex("03100AA");'
3211434

Tell me if you have a smart way to find the process and pid using a different way, i want to know, and of course, love to share it! Joao Bosco Cortez Filho

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s