Crazy Console AKA Brown Effect

At the 2017-02-08 Operation Meeting I recounted a recent occurrence of odd serial over LAN console behaviour, but a more extreme than previous events. See, specifically, the services-unit report. At the meeting I was to create a wiki page so we can log similar events in case a pattern could be spotted.

So here it is.

date machine type os notes
15/11/16 norrington PowerEdge R320 SL6 Item 7 of the inf-unit report to the following Operational meeting
1/2/2017 huldra PowerEdge R720 SL7 BIOS 2.5.4, Lifecycle controller,2.40.40.40, iDRAC 2.40.40.40
15/2/2017 blatiere PowerEdge R320 SL7 physical console affected, soft power cycle didn't fix it. Ian's post to COs
25/10/2017 knussen PowerEdge R320 SL7 2017-11-08 inf-unit Operational meeting report has details
12/2/2018 bahookie PowerEdge R620 SL7 apache stopped responding, ping was OK, couldn't login, went to console, doing the crazy thing, after hit return a few times, apache recovered
24/12/2018 hazlewood PE R330 SL7.5 details

Some notes

On Monday afternoon 20/2/2017 I deliberately triggered the same scsi/device error messages on crocotta (an SL7 PE1950), to see if it would trigger the serial console problem. As of Thursday 23/2/2017 morning, the error messages were still going to the console, but the login prompt was still available and working. So I've decided to stop this attempt to trigger the problem on crocotta, and stopped the error messages.

The plan is to vacate yeti (same spec'd machine as huldra, SL7 R720), and then try the same thing. yeti vacated of active data. scsi lun mask removed 14:50 Thursday 23/2/2017, and console messages appearing. 2/3/2017 messages still going to yeti console and console working frown

2/3/2017 For the record, here are the outputs of "dsu -i" for huldra and yeti. The same bar the FC cards, ordering of network devices, and the uEFI diags

   YETI                                                       |    HULDRA
1. BIOS  ( Version : 2.5.4 )                                    1. BIOS  ( Version : 2.5.4 )
2. Lifecycle Controller  ( Version : 2.40.40.40 )               2. Lifecycle Controller  ( Version : 2.40.40.40 )
3. Dell 64 Bit uEFI Diagnostics  ( Version : 4247A1 )         | 3. Dell 64 Bit uEFI Diagnostics, version 4247, 4247A1, 4247.2
4. OS Collector  ( Version : 0 )                              | 4. OS COLLECTOR 1.1, OSC_1.1, X10-00  ( Version : OSC_1.1 )
5. Power Supply  ( Version : 07.2B.7D )                         5. Power Supply  ( Version : 07.2B.7D )
6. Power Supply  ( Version : 07.2B.7D )                         6. Power Supply  ( Version : 07.2B.7D )
7. CPLD  ( Version : 1.0.3 )                                    7. CPLD  ( Version : 1.0.3 )
8. PERC H710P Mini Controller 0 Firmware  ( Version : 21.3.2-   8. PERC H710P Mini Controller 0 Firmware  ( Version : 21.3.2-
9. Firmware for  - Disk 0 in Backplane 1 of PERC H710P Mini C   9. Firmware for  - Disk 0 in Backplane 1 of PERC H710P Mini C
10. Firmware for  - Disk 2 in Backplane 1 of PERC H710P Mini    10. Firmware for  - Disk 2 in Backplane 1 of PERC H710P Mini 
11. 12G Backplane Expander FW (Firmware)   ( Version : 1.03 )   11. 12G Backplane Expander FW (Firmware)   ( Version : 1.03 )
12.  iDRAC  ( Version : 2.40.40.40 )                            12.  iDRAC  ( Version : 2.40.40.40 )
13. NetXtreme BCM5720 Gigabit Ethernet PCIe (em2)  ( Version  | 13. NetXtreme BCM5720 Gigabit Ethernet PCIe (em1)  ( Version 
14. NetXtreme BCM5720 Gigabit Ethernet PCIe (em3)  ( Version    14. NetXtreme BCM5720 Gigabit Ethernet PCIe (em3)  ( Version 
15. NetXtreme BCM5720 Gigabit Ethernet PCIe (em1)  ( Version  | 15. NetXtreme BCM5720 Gigabit Ethernet PCIe (em4)  ( Version 
16. NetXtreme BCM5720 Gigabit Ethernet PCIe (em4)  ( Version  | 16. NetXtreme BCM5720 Gigabit Ethernet PCIe (em2)  ( Version 
17. QLogic QLE2462 Adapter  ( Version : 02.23.06 )            | 17. QLogic QLE2562 Adapter  ( Version : 03.22.08 )
18. QLogic QLE2462 Adapter  ( Version : 02.23.06 )            | 18. QLogic QLE2562 Adapter  ( Version : 03.22.08 )

7/3/2017 - errors still going to yeti's console. console still working frown

-- NeilBrown - 15 Feb 2017


This topic: DICE > CrazyConsoleAKABrownEffect
Topic revision: r11 - 26 Nov 2018 - 11:52:25 - GeorgeRoss
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies