I experienced a strange issue after making some configuration changes to a Juniper MX104 device with a MS-MIC-16G installed. After removing some NAT rules the services interface that the NAT service was attached to started doing a large amount of traffic. The same change was made on a second router and it had the same results. The devices were active/standby, and I confirmed that no traffic was going in/out of the standby router, the traffic was all internal to the MS-MIC-16G. The CPU usage of the card was also showing at a constant 96% when it usually is around 15%.
The services interface on the router was installed into FPC 1 PIC 0 and named ms-1/0/0. I thought I could request the chassis take the pic offline but after trying that it does not seem to support the command:
root@router> request chassis pic offline fpc-slot 1 pic-slot 0 FPC slot 1 does not support this command
I didn’t want to reboot the entire router to fix this issue as it was only localized just to the MS-MIC-16G. Juniper supplies a guide on how to get a shell on the MS-MPC and MS-MIC devices which was useful, I followed the guide and got the shell and had a poke around. I couldn’t find anything useful in the logs so I entered the debug CLI (mspdbg-cli -ps). From there the command testcrash can be sent which will force a crash on the device causing it to reboot. After a few minutes I verified the device came back online and everything was working as expected.
The output from my session looked like this:
root@router> show tnp addresses
Name TNPaddr MAC address IF MTU E H R
master 0x1 02:00:00:00:00:04 em0 1500 0 0 3
re0 0x4 02:00:00:00:00:04 em0 1500 0 0 3
afeb0 0x32 02:00:00:00:00:10 em0 1500 4 0 3
fpc1.pic0 0x111 02:00:10:00:01:11 em0 1500 2 0 3
bcast 0xffffffff ff:ff:ff:ff:ff:ff em0 1500 0 0 3
root@router> start shell user root
Password:
root@border2% telnet -Ji fpc1.pic0
Trying 128.0.1.17...
Connected to core1-bucharest.web365.ro.
Escape character is '^]'.
login: root
--- JUNOS 18.1R1.9 built 2018-03-23 23:36:46 UTC
root@ms10% mspdbg-cli -ps
MSPMAND-CLI>
clear clear command hierarchy
debug setup debug info in the system
lib MSPMAND Library Information
plugin plugin information
quit quit from this cli
request request services in the system
set set information in the system
show show information about the system
test enable test framework
testcrash command to crash the pic and reboot
MSPMAND-CLI> testcrash
<carriage return> Completes command
MSPMAND-CLI> testcrash
2 thoughts on “Juniper MX – Rebooting a MS-MIC-16G”
The below would do the magic (Please confirm your mic and fpc slot before going ahead)
request chassis mic offline mic-slot 0 fpc-slot 1
request chassis mic online mic-slot 0 fpc-slot 1
Nice tip, thank you for that.