2013-14 upgrade of the Forum switches

  • 2013: we're doing the Forum [0123][AB] and [45]A closets.
  • 2014: we're doing the remainder.

Here's what we have to do...

General "setup" plan

For each switch:

  • Pick a new switch from the correct pile. Note that in each closet there will usually be one PoE+ switch and one or more non-PoE+ switch. They're not interchangeable, so make sure you have the correct model.
  • Unbox and check over
  • rfe lcfg/FSinstall2920 and set the MAC address
  • Install the mounting lugs. Install modules in the back, per the diagrams and table below. There will usually be one 10GbaseT module to go in slot A (left). There may be one SFP+ module to go in slot B (right). There may be one or more SFP+ to install in the SFP+ module. There may be an SFP to install in one of the front dual-personality ports.
  • Rack up and patch in port 47 to the "install" uplink
  • Plug in, turn on, and watch while the self-test goes through.
  • Set up, per the instructions. Note that there is a firmware upgrade required.
  • To download the configuration, rfe core/install2920.pc and change the name of the portfile to that of the switch you are installing. Then ssh to abbado, and run "make install2920.ets" in /disk/home/abbado/Forum/core/ as root. Fix and repeat as necessary until this goes through cleanly.
  • Unrack, label the switch, box up again and label the box.

Here's what should be in each of the switches:

Switch PoE+ 10GbaseT SFP+ Other Serial no MAC
0a0 yes no B1, DA Existing SX from 0a3 in 48 SG34FLZ26C 2C:59:E5:18:0C:40
0a3 no no DA, B2 Move p0a0 link to 48 SG34FLY110 2C:59:E5:0B:D0:C0
1a0 yes yes B1, B2   SG34FLZ1TW 2C:59:E5:18:F5:40
1a1 no yes no   SG33FLY12T 2C:59:E5:07:13:C0
1a2 no yes B1   SG33FLY13J 2C:59:E5:16:DF:80
2a0 yes yes no   SG34FLZ264 2C:59:E5:18:6B:40
2a1 no yes B1   SG33FLY002 74:46:A0:5A:54:C0
3a0 yes yes no   SG34FLZ268 2C:59:E5:18:8B:00
3a1 no yes B1, B2 Recover B2 after 4a0 done SG33FLY12R 2C:59:E5:14:1F:C0
4a0 yes yes no   SG37FLZ050 D4:C9:EF:B7:38:00
4a1 no yes B1   SG36FLY00V D4:C9:EF:C0:69:00
5a0 yes yes no   SG37FLZ07Z D4:C9:EF:B7:80:80
5a1 no yes B1   SG36FLY013 D4:C9:EF:C0:79:00
5a2 no yes no   SG36FLY05J D4:C9:EF:BE:80:40
0b0 yes yes no   SG34FLZ260 2C:59:E5:18:FB:40
0b4 no yes B1   SG33FLY13C 2C:59:E5:07:A4:40
1b0 yes yes B1   SG34FLZ1W2 2C:59:E5:18:1E:80
1b3 no yes no   SG33FLY13B 2C:59:E5:07:F2:40
2b0 yes yes B1   SG34FLZ269 2C:59:E5:18:3B:00
2b3 no yes no   SG33FLY014 74:46:A0:5A:AF:40
3b0 yes yes B1   SG34FLZ23R 2C:59:E5:18:99:00
3b1 no yes B2 Recover B2 and B module (and install in 2c0) after 4b0 done SG33FLY134 2C:59:E5:14:EE:C0
***** ALL THE ABOVE DONE AUGUST 2013 *****
4b0 yes yes no   SG3AFLZ2BS 9C:B6:54:D3:2F:00
4b1 no yes B1   SG41FLY0SG 9C:B6:54:CD:EE:40
5b0 yes yes no   SG3AFLZ2CC 9C:B6:54:D3:FD:40
5b1 no yes B1   SG41FLY0S0 9C:B6:54:CD:7E:40
***** ABOVE FOUR DONE APRIL 3 2014 *****
0c0 no yes B1   SG41FLY0RQ 9C:B6:54:CD:1F:00
0c1 no yes no Note 1000baseSX link in 48 SG41FLY0RX 9C:B6:54:CD:9E:00
***** ABOVE TWO DONE APRIL 18 2014 *****
1c0 yes yes B1   SG3AFLZ2C0 9C:B6:54:D3:3E:80
1c1 no yes no   SG41FLY0PT 9C:B6:54:CD:D7:C0
2c0 yes yes B1 Before installation, needs B module and B1 SFP+ from 3b1 SG3BFLZ1VK F0:92:1C:C5:B5:C0
2c2 no yes no   SG41FLY0P0 9C:B6:54:CD:59:80
3c0 yes yes no   SG3AFLZ28Z 9C:B6:54:D3:0C:40
3c2 no yes B1   SG41FLY0Q0 9C:B6:54:CD:48:80
***** ABOVE SIX DONE APRIL 10 2014 *****

General "install" plan

In order to maintain as much redundancy and connectivity as possible through the upgrade, it's important to do the upgrade in the right order. Pick either "column A" or "column B" (it doesn't matter which). Make sure the file abbado:/disk/home/abbado/netman/install2920.dsd does not exist (rm it if it does). Then, starting at the bottom, do the switches within each closet ONE AT A TIME in numerical order, as follows:

  • Power-down the old switch
  • rfe lcfg/FSwhatever, and update the MAC address to be that of the new switch. Update the sysinfo.model while you're at it.
  • Remove all the patch leads from the front of the old switch, making sure that you have a way to re-patch them all in the correct places.
  • Remove any fibre patch leads.
  • Remove the old switch from the rack, and replace it with the new one in the same slot.
  • Re-patch the inter-switch links. Note that there may be new 10GbaseT connections to make at the back. Leave the edge connections off for now.
  • Reinstate any fibre patch leads. Note that you might have to change an entire lead as the connectors at the switch end have changed. REMEMBER THE DUST CAPS.
  • Power up the switch. Watch as it does its self test. Check that the inter-switch links come up. Adjust any fibre patching as needed.
  • rfe core/whatever.pc and add the "dhcp-snooping enable" line. Leave arp-protection off for now. Change the 10Gbps uplinks/interlinks to match the new switch hardware, and make sure that they are "trusted". Check that the updates are pushed to the switch correctly.
  • Reinstate the edge connections.

Note that there will be a transitional arrangement required for the 3A and 3B closets, and corresponding 4A and 4B closets. This requires installing a displaced X2 module in each of the 4[ab]0 switches, configuring appropriately, and then patching with fibre vla the panel in the server room comms rack. This will go away next year, when we upgrade the remaining closets.

Then do the other column (B or A).

-- GeorgeRoss - 06 Jun 2013

Topic revision: r28 - 18 Apr 2014 - 15:02:41 - IanDurkacz
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies