Dell System Update (dsu)

Warning

(01/2021) Do not install iDRAC update version 4.40.00.00 for PowerEdge R440, R340 and R240 - it seems to break IPMI support.

Dell System Update or dsu automates the updating of firmware on Dell servers. It works on 12G servers and newer. It works on SL7 and (after a fashion) on Ubuntu. There's also a Windows version, which we haven't tried.

Safety first

Please note that using dsu (or any tool) to apply firmware updates can never be a zero-risk approach. MPU recommends a safety-first approach - we do not have access to all possible server models to test these updates. Common sense suggests that you will want to spread your firmware updates over several days/weeks; doing all your critical servers at the same time may end in tears at some point. MPU will make a special announcement for any firmware updates which they consider to be of critical importance and thus requiring immediate application.

What it does

dsu finds out the current firmware levels of the server it's run on. It compares these with what's available in Dell's firmware repository, and offers relevant updates. There are command line options to control whether the updates are installed immediately or downloaded for installation later.

When installation happens, most updates are installed there and then, one at a time. Others - BIOS and PSU updates at least - are staged using the machine's Lifecycle Controller. This means that they will apply at the next reboot, rather than immediately.

OS-specific notes

On SL7

You should find that dsu has been installed and configured automatically on suitable Dell/DICE servers. (The dice/hw/ header pulls in dice/options/dsu.h.) If dsu is not available on your SL7 Dell server, include dice/options/dsu.h and please also tell the MPU so that they can fix the problem. Note that your server needs to be Dell PowerEdge generation 12 or younger for dsu to work.

Unless you want to learn about dsu on Ubuntu, you should now skip ahead to Import public keys.

On Ubuntu

Until Dell gets around to producing a version of dsu which supports Ubuntu 20.04 and which Just Works, you'll need to use the version for 18.04. This has to be reinstalled each time you want to use it, but it's otherwise like the SL7 version.

To get it going, run these commands as root:

cd /afs/inf.ed.ac.uk/group/mp-unit/dsu
./Systems-Management_Application_8CTK7_LN64_1.9.0_A00.BIN
dpkg -i libssl1.0.0_1.0.2n-1ubuntu5.4_amd64.deb
export LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0"
dsu

Assuming that you've said "yes" to the question about importing public keys, it should then work as normal.

Import public keys

The first time you run dsu 1.9 it will ask you to import public keys. Answer "y". (And please tell MPU if it fails.)
dsu will not do anything else until this has been done.
[amarela]root: dsu
DELL EMC System Update 1.9.0
Copyright (C) 2014 DELL EMC Proprietary.
Do you want to import public key(s) on the system (Y/N)? : y
Import Public Key(s)
Importing key 1019CED6 is successful
Importing key 23B66A9D is successful
Importing key 34D8786F is successful
Importing key 46770C59 is successful

How to use dsu

dsu can take several minutes to start up. Be patient. It can be used in several ways:
Just list current firmware levels
dsu -i
Download to a directory
To download updates to a directory for later installation, for example to /var/tmp/dsu, use
dsu --destination-type=CBD --destination-location=/var/tmp/dsu
The update files are downloaded to the named directory, and a script (apply_updates.sh) is provided to install them with later. Having downloaded the updates, you can then boot the machine single user, and run the apply_updates.sh script to install the updates. That way, they can't be interfered with or interrupted by another user or process. This process seems the safest way to use 'dsu' - though it is more time-consuming. This method worked when tested.
Interactive update
dsu lists the firmware updates which can be installed. In interactive mode you choose which updates to install, then start the installation process. Nothing irretrievable happens until you press c for commit.
This method worked when tested. It's probably safe enough to use this method on a stable, quiet machine - but it's not the very safest way of using dsu, because it involves updating firmware while the machine is in multi-user mode, so the firmware updater stands a chance of being interrupted or otherwise interfered with, which could be bad. Also, this method is a good way to get a list of what needs doing.
Non-interactive update
dsu --non-interactive --preview lists all applicable firmware updates then exits.
dsu --non-interactive finds all applicable firmware updates then installs them with no further user input. Only use this if you are absolutely sure of what you are doing.
Make an ISO
Updates can be downloaded and made into an ISO image. The machine can then be booted from that ISO image to install the updates. To make the ISO,
dsu --destination-type=ISO --destination-location=/var/tmp/dsu.iso
In a test, a PowerEdge R710 would not boot from a USB key flashed with such an ISO. If you try this, please let us know how you get on.

Monthly mail report

Dell DICE servers will mail out a report once a month listing all available firmware and BIOS updates for that machine. The report is just the output of the command dsu --non-interactive --preview. This can be prevented by defining DICE_OPTIONS_DSU_INHIBIT_MAIL at the top of a machine's profile.

How to download an update manually

If you ever need to download an update yourself - perhaps you need an older version? - you can download it using the web:
  1. Point your web browser at the Dell support site.
  2. In the "Search Support" search box, enter the serial number of your server.
  3. Choose "Drivers & Downloads".
  4. Correct the "Operating System" choice, then browse the list of downloads to find the one you want.
  5. Download the .BIN version of the update. It may be helpfully labelled "Update Package for Red Had Linux" - but it should work on Ubuntu too. Click the download link to download the .BIN update file.
  6. Once downloaded, run md5sum on the update, and check that its MD5 checksum is the same as the one on the Dell webpage.
  7. chmod +x the .BIN file.
  8. As root on your chosen server, run the .BIN file. This installs the update in the usual way.
  9. Remember that some updates need the server to be rebooted as part of the install process.

Quirks and Oddities

dsu can be a little weird at times. Here are some tips and observations. If you come across a new one, let the MPU know. Thanks!
Import public keys
The first time you run dsu on a server, it will ask you if you want to import public keys. The answer is yes, you do! These are the keys which give access to the Dell repository, so dsu will not do anything else until this has been done.
Neil has pointed out the equivalent command line switch - thanks Neil:
dsu --import-public-key -i
Weird messages on the console
When you run dsu you will get messages appearing on the console saying that USB devices are being attached and detached or that disks are being mounted and unmounted. This is a normal part of dsu, don't panic.
BIOS updates and kernel module loading
BIOS updates involve installing a kernel module. This is fine unless your machine has disabled kernel module loading. (You do this by including dice/options/kernel-disable-module-loading.h.) If kernel module loading has been disabled on your machine, any attempt to update the BIOS will fail. You'll need to re-enable kernel module loading before you run dsu. To do this, add something like the following to the top of the machine's LCFG file; wait for the change to reach the machine; then reboot. Then run dsu.
/* REMINDER: kernel module loading has been re-enabled temporarily to help dsu */
#define DICE_OPTIONS_KERNEL_DISABLE_MODULE_LOADING
Please remember to undo this afterwards - remove the lines above then reboot again.
iDRAC updates
When updating iDRAC firmware, disconnect from the serial console. If you don't, the update will fail to apply. You can do such an update using dsu while remotely logged in by ssh. You can also do them using the Lifecycle Controller instead, but only if the Lifecycle Controller offers the same version of update (sometimes it's a version behind dsu).
Doesn't run on the console
dsu has been seen to not work when logged in directly to a server's console. It churns away happily for a few minutes then exits with the message Could not parse the inventory. (This has been observed on a PowerEdge R220 and on a PowerEdge R620.) However dsu worked perfectly well on the same machines when run from a shell obtained via ssh (then nsu). Running its "inventory collector" binary directly produced error messages such as Can't open display and DISPLAY is not set.
Watch out for downgrades!
You may occasionally be offered a downgrade instead of an upgrade. Either be careful - don't just blindly type "a" then "c"! - or try adding the -u or --apply-upgrades-only switch to your dsu command.
RPM installation
dsu installs a couple of RPMs when it runs. They seem harmless, but feel free to remove them afterwards, for example by running updaterpms. They have names like dsucatalog-16.12.00-684PR.noarch and invcol_WF06C_LN64_16.12.200.896_A00-16.12.200.896-WF06C.x86_64.
Safety
The very safest way to do firmware updates is probably to download them to a local directory (see above) then boot the server into single user mode and install the updates. This ensures that only one Dell updater will be running at any one time (reducing the chance of bricking hardware) and that the hardware will be as close to idle as it can be. However, nobody has yet suffered from just running dsu on an idle machine, so that seems fairly safe too.
Logs
An update may occasionally fail with the recommendation that you look in the log for further details. The logs can be found in /var/log/dell.
"Failed to update to standby image"
If an update fails with "Failed to update to standby image" it may be because you're trying to jump too big a gap in firmware versions. In this case upgrading to one or more intermediate firmware versions may help. Firmware updaters can be downloaded form dell.com. Look for .BIN packages for Red Hat Enterprise Linux. Ask Chris or the MPU for help if you have a problem with this.
OS Collector
TLDR: This doesn't seem to be necessary unless Dell asks for it. This isn't firmware, it's a piece of software. When you contact Dell with a problem, and they respond by asking you to send a SupportAssist report to them, the "OS Collector" will fill in the optional "OS and Application Data" part of the report. If it's not installed, the "OS and Application Data" option will be greyed out in the SupportAssist screen. Personal experience suggests that it's not necessary to have it installed unless Dell specifically asks for it.
Failed updates
Did your update fail shortly after a "modprobe" command appeared on the screen? If so, maybe your server has kernel module loading disabled with dice/options/kernel-disable-module-loading.h. You can fix this by defining DICE_OPTIONS_KERNEL_DISABLE_MODULE_LOADING at the top of its LCFG file, then rebooting it. Don't forget to undo this (then reboot again) afterwards! Failing that - if an update fails to apply using dsu, it may still apply cleanly via the Lifecycle Controller instead.
Order of applying updates
Dell advises that iDRAC and Lifecycle Controller updates should be applied first, then BIOS, then firmware updates. See iDRAC and Lifecycle Controller - A Recommended Workflow for Performing Firmware Updates on PowerEdge Servers for more details.

Where to find help

  • man dsu provides brief help on the command line options.
  • The homepage explains what's what and provides links to further sources of help.
Topic attachments
I Attachment Action Size Date Who Comment
pngpng Dell_firmware_-_choose_the_BIN_file.png manage 76.1 K 02 Dec 2020 - 09:22 ChrisCooke When downloading Dell firmware choose the BIN file
pngpng Dell_firmware_-_here_is_the_download_link.png manage 42.7 K 02 Dec 2020 - 09:19 ChrisCooke Here is the Dell firmware download link to click on
pngpng Dell_firmware_-_search_Dell_support.png manage 45.2 K 02 Dec 2020 - 09:12 ChrisCooke Search Dell support for the firmware filename you found out earlier
Topic revision: r32 - 07 Jan 2021 - 12:08:28 - ChrisCooke
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies